Defect detection method and device for remaining needle packaging needle tube assembly based on machine vision

Abstract

The invention relates to the field of machine vision, in particular to an indwelling needle packaging needle tube assembly defect detection method and device based on machine vision, positioning and defect detection of an indwelling needle needle tube assembly are carried out through a machine vision and image processing method, the problem of missing detection caused by manual detection fatigue is avoided, and meanwhile the defect detection accuracy is improved. The remaining needle product can be shot through a paperboard in a packaging box in a special imaging mode, and missing detection of curved needle damage caused in the packaging process is avoided; a strategy and a method for needle tube positioning under the condition that shielding and interference exist between the assemblies are innovatively provided, and high-precision curved needle detection is further achieved by reasonably selecting a reference straight line. The defect of needle bending missing detection of the indwelling needle tube from production to packaging is overcome. And compared with manual detection, the omission ratio can be greatly reduced, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to a method and device for detecting defects of an indwelling needle packaging needle tube assembly based on machine vision. Background Art

[0002] Regarding the inspection solutions for indwelling needle products, the currently disclosed technologies only mention the automatic inspection method of the indwelling needle bushing burrs by the machine vision system, or the needle tip positioning method during use. No solutions for the inspection of the needle tube assembly after packaging have been found. The quality inspection after packaging currently still relies on manual visual inspection.

[0003] Manual quality inspection is prone to missed inspections and is inefficient. In addition, the syringes are inspected before assembly, but the multiple steps in the assembly and packaging process will introduce new defects, which are very easy to miss. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a method and device for detecting defects of an indwelling needle packaging needle tube assembly based on machine vision to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The method for detecting defects of an indwelling needle packaging needle tube assembly based on machine vision of the present invention comprises the following steps:

[0007] Acquire an original image of the indwelling needle package, wherein the original image is a grayscale image;

[0008] extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline;

[0009] Based on the minimum circumscribed rectangle of the hemostat, the middle needle tube is located from the original image to obtain a middle needle tube location result; and the middle needle tube location result is used to correct the inclination of the minimum circumscribed rectangle of the hemostat;

[0010] Based on the minimum circumscribed rectangle of the modified hemostat, the main needle tube and the needle handle are positioned respectively to obtain the main needle tube positioning result and the needle handle positioning result;

[0011] Based on the main needle tube positioning result and the needle handle positioning result, defect detection is performed on the needle tube assembly in the package to obtain a detection result.

[0012] In one embodiment of the present application, extracting a preselected contour of the hemostat from the original image includes:

[0013] Performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image;

[0014] Performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat;

[0015] Performing multiple opening operations on the mask image to obtain a mask image with the needle tube portion removed;

[0016] Contour extraction is performed on the mask image with the needle tube portion removed, and a preselected contour is screened out using a preset area threshold.

[0017] In one embodiment of the present application, extracting the minimum circumscribed rectangle of the hemostat based on the preselected contour, and performing defect detection on the hemostat based on the preselected contour include:

[0018] Fitting the minimum circumscribed rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle;

[0019] Calculating the straight-line distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculating the longitudinal distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, wherein the center coordinates of the packaging box are preset;

[0020] The preselected contour is screened based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first screening parameter to obtain a first target contour, wherein the first screening parameter includes a width screening parameter, a straight-line distance screening parameter, and a longitudinal distance screening parameter;

[0021] When the first target contour does not exist, it is determined that the hemostat component in the indwelling needle package is missing;

[0022] When the first target contour exists, the first target contour is drawn in the blank image to obtain a hemostat preselected binary image;

[0023] performing a morphological closing operation on the hemostat pre-selected binary image to obtain a processed hemostat pre-selected binary image;

[0024] Extracting the binary contour in the processed pre-selected binary image of the hemostat, calculating the distance between the binary contour and the center coordinate of the packaging box, and taking the binary contour with the smallest distance from the center coordinate of the packaging box as the contour of the hemostat;

[0025] The minimum circumscribed rectangle of the outline of the hemostat is calculated to obtain the minimum circumscribed rectangle of the hemostat.

[0026] In an embodiment of the present application, central needle tube positioning is performed on the original image based on the minimum circumscribed rectangle of the hemostat to obtain a central needle tube positioning result, including:

[0027] Extract the initial contour of the central needle tube, and calculate the reference distance between the center of the initial contour of the central needle tube and the center of the minimum circumscribed rectangle of the hemostat;

[0028] Obtain the center of the minimum circumscribed rectangle of the hemostat, and move the center of the minimum circumscribed rectangle of the hemostat along the long side direction by the reference distance to obtain the reference position of the central needle tube;

[0029] Set a minimum circumscribed rectangle with a length equal to the actual length of the central needle tube and a width equal to the actual width of the central needle tube at the reference position of the central needle tube to obtain the initial rectangle of the central needle tube;

[0030] Extract the central needle tube image from the original image based on the initial rectangle of the central needle tube;

[0031] Perform contour extraction on the central needle tube image to obtain the contours within the central needle tube image; and screen the contours within the central needle tube image based on a preset second screening parameter to obtain the second target contour within the central needle tube image;

[0032] Fit the second target contour into a straight line, and extract the slope k, intercept b, maximum ordinate y t , minimum ordinate y b ; and calculate the maximum abscissa x t , minimum abscissa x b of the second target contour based on the slope k, intercept b, maximum ordinate y b , minimum ordinate y t : and determine the minimum circumscribed rectangle of the second target contour based on the maximum ordinate y b , minimum ordinate y t , maximum abscissa and minimum abscissa x t and maximum abscissa x b ;

[0033] Calculate the strategy value of the minimum circumscribed contour of each second target contour, and use the second target contour with the minimum strategy value as the contour of the central needle tube.

[0034] In an embodiment of the present application, the mathematical expression of the strategy value is: angle m +|d 0 -h m |-w m ;

[0035] In the formula,

[0036]

[0037] angle m is the included angle between the straight line ml and the connecting line hl, where the straight line ml is the straight line obtained by fitting the second target contour, and the connecting line hl is the connecting line between the center of the hemostat and the vertex of the straight line ml;

[0038] d 0 is the actual length of the middle syringe tube, h m is the length of the minimum circumscribed rectangle of the second target contour, w m is the width of the minimum circumscribed rectangle of the second target contour; is the vector expression symbol of the straight line ml, is the vector expression symbol of the connecting line hl; (x 1 , y 1 ), (x 2 , y 2 ) are the starting point and ending point coordinates of the middle syringe tube respectively, and (x 3 , y 3 ) is the center coordinate of the hemostat.

[0039] In an embodiment of the present application, based on the minimum circumscribed rectangle of the corrected hemostat, the main syringe tube is positioned from the original image, and the positioning result of the main syringe tube is obtained, including:

[0040] Obtain the center of the minimum circumscribed rectangle of the corrected hemostat, and move the center of the minimum circumscribed rectangle of the corrected hemostat a first preset distance along the long side direction to obtain the reference position of the main syringe tube;

[0041] Set a minimum circumscribed rectangle with a length equal to the actual length of the main syringe tube and a width equal to the actual width of the main syringe tube at the reference position of the middle syringe tube to obtain the initial rectangle of the main syringe tube;

[0042] Extract the main syringe tube image from the original image based on the initial rectangle of the main syringe tube;

[0043] Perform contour extraction on the main syringe tube image to obtain the contours within the main syringe tube image; and screen the contours within the main syringe tube image based on a preset third screening parameter to obtain the third target contour within the main syringe tube image;

[0044] Fit the third target contour into a straight line, and use the straight line whose distance from the end to the center of the minimum circumscribed rectangle of the corrected hemostat is not greater than a preset distance threshold as the candidate straight line;

[0045] Determine the length of the candidate straight line and the angle with the abscissa. When the length of the candidate straight line is greater than the preset length threshold and the angle of the target straight line is greater than the preset angle threshold, use the candidate straight line as the target straight line, and extract the average gray value I of the contour region corresponding to the target straight line a ;

[0046] Based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than the preset length threshold, use the points in the contour corresponding to the target straight line that meet the target conditions as the points at the bottom of the main syringe, and construct the bottom straight line based on the points at the bottom of the main syringe, where the target conditions include that the difference between the ordinate and the end coordinate of the corresponding fitting straight line is less than the preset difference l b ;

[0047] Put the bottom straight line, the target straight line, the contour region corresponding to the target straight line, and the average gray value I of the contour region corresponding to the target straight line a into the candidate list;

[0048] Score the target straight lines in the candidate list, and determine the minimum circumscribed rectangle of the main syringe based on the scores of the target straight lines

[0049] In an embodiment of the present application, scoring the target straight lines in the candidate list and determining the minimum circumscribed rectangle of the main syringe based on the scores of the target straight lines includes:

[0050] Score the target straight lines in the candidate list, where the mathematical expression of the score score is:

[0051] score = W 1 l - d hn - α hn - W 2 I a

[0052] In the formula, W 1 is the first weight, W 2 is the second weight, l is the length of the target straight line, d hn is the distance from the center of the hemostat after correction to the target straight line, α hn is the angle between the target straight line and the vertical axis;

[0053] Use the target straight line with the maximum score as the standard straight line;

[0054] When there is only one target straight line in the candidate list, obtain the minimum circumscribed rectangle of the main syringe based on the minimum circumscribed rectangle of the contour of the standard straight line;

[0055] When there are multiple target lines in the candidate list, other target lines in the candidate list are screened based on preset overlapping screening conditions to obtain target lines that do not overlap with the standard line; the target lines that do not overlap with the standard line are screened based on preset geometric screening conditions to obtain a fitting line at the bottom of the main syringe;

[0056] Among them, the overlapping screening conditions include: the average gray level differs from the average gray level within the area of the standard line by no more than a preset gray threshold, and the starting vertical coordinate is not less than the ending vertical coordinate of the standard line or the ending vertical coordinate is not less than the starting vertical coordinate of the standard line;

[0057] The geometric screening conditions include: the distance from the end point to the center of the corrected hemostat, the angle with the standard line, and the angle between the derived line and the standard line all fall into the corresponding screening ranges respectively, where the derived line is the line formed by the end point and the end point of the standard line;

[0058] Based on the minimum circumscribed rectangle of the fitting line at the bottom of the main syringe, and taking the minimum circumscribed rectangle of the contour of the standard line and the minimum circumscribed rectangle of the fitting line at the bottom of the main syringe as the minimum circumscribed rectangle of the main syringe.

[0059] In an embodiment of the present application, the needle handle is positioned based on the minimum circumscribed rectangle of the corrected hemostat to obtain a needle handle positioning result, including:

[0060] Obtain the center of the minimum circumscribed rectangle of the corrected hemostat, and move the center of the minimum circumscribed rectangle of the corrected hemostat a second preset distance along the long side direction to obtain a needle handle reference position;

[0061] Set a minimum circumscribed rectangle with a length equal to the true length of the needle handle and a width equal to the true width of the needle handle at the needle handle reference position to obtain an initial rectangle of the needle handle;

[0062] Extract the needle handle image from the original image based on the initial rectangle of the needle handle;

[0063] Extract the contour from the needle handle image, and take the contour with the largest area as the standard needle handle contour;

[0064] Calculate the minimum circumscribed rectangle Rect of the standard needle handle contour NeedleBase and the upper edge center point pt of the minimum circumscribed rectangle of the standard needle handle contour std ; and connect the center of the minimum circumscribed rectangle Rect NeedleBase with the upper edge center point pt std to obtain a new line Line std ;

[0065] Extract the center point of the upper edge of the minimum circumscribed rectangle of other contours in the needle handle image, and connect the center point of the upper edge of the minimum circumscribed rectangle of other contours with the upper edge center point pt std to obtain a new straight line Line test ;

[0066] Determine the new straight line Line std and the new straight line Line test to obtain the included angle therebetween. When the included angle between the new straight line Line std and the new straight line Line test is less than a preset reference threshold, connect the other contours to the standard needle handle contour to obtain an updated standard needle handle contour;

[0067] Calculate the minimum circumscribed rectangle of the updated standard needle handle contour, and use the minimum circumscribed rectangle of the updated standard needle handle contour as the minimum circumscribed rectangle of the needle handle.

[0068] In an embodiment of the present application, based on the main body needle tube positioning result and the needle handle positioning result, defect detection is performed on the needle tube assembly in the package to obtain a detection result, including:

[0069] When there is no minimum circumscribed rectangle of the main body needle tube in the main body needle tube positioning result, it is determined that there is a needle tube missing defect;

[0070] When there is a minimum circumscribed rectangle of the main body needle tube in the main body needle tube positioning result, form a first reference straight line by connecting the needle handle center with the end point of the topmost straight line of the main body needle tube, and form a second reference straight line by connecting the starting point of the topmost straight line of the main body needle tube with the end point of the bottommost straight line of the main body needle tube. Calculate the included angle between the first reference straight line and the second reference straight line, and perform bent needle detection based on the included angle between the first reference straight line and the second reference straight line.

[0071] The present application further provides a defect detection device for a detaining needle package needle tube assembly based on machine vision, which is characterized by including:

[0072] An acquisition module, configured to acquire an original image of the detaining needle package, wherein the original image is a grayscale image;

[0073] A hemostat positioning module, configured to extract a preselected contour of the hemostat from the original image, and based on the preselected contour, extract the minimum circumscribed rectangle of the hemostat, and perform defect detection on the hemostat based on the preselected contour;

[0074] The middle syringe positioning module is used to perform middle syringe positioning on the original image based on the minimum circumscribed rectangle of the hemostat to obtain the middle syringe positioning result; and use the middle syringe positioning result to correct the inclination of the minimum circumscribed rectangle of the hemostat.

[0075] The main syringe positioning module is used to perform main syringe positioning and needle handle positioning respectively based on the corrected minimum circumscribed rectangle of the hemostat to obtain the main syringe positioning result and the needle handle positioning result.

[0076] The defect detection module is used to perform defect detection on the syringe assembly in the package based on the main syringe positioning result and the needle handle positioning result to obtain the detection result.

[0077] The beneficial effects of the present invention are as follows: The method and device for defect detection of the indwelling needle package syringe assembly based on machine vision of the present invention perform positioning and defect detection of the indwelling needle syringe assembly through machine vision and image processing methods, avoiding the missed detection problem caused by manual detection fatigue. At the same time, a special imaging method can photograph the indwelling needle product through the cardboard in the packaging box, avoiding the missed detection of the bent needle damage caused during the packaging process; innovatively proposing a strategy and method for syringe positioning in the case of occlusion and interference between components, and achieving high-precision bent needle detection by reasonably selecting the reference straight line. This application makes up for the missed detection of bent needle defects caused during the production to packaging stage of the indwelling needle syringe. Compared with manual detection, the missed detection rate can be significantly reduced, improving the detection efficiency. Description of the Drawings

[0078] The present invention will be further described below in conjunction with the drawings and embodiments:

[0079] Figure 1 It is a schematic diagram of an indwelling needle sample in an embodiment of the present application;

[0080] Figure 2 It is an imaging effect diagram of an indwelling needle in an embodiment of the present application;

[0081] Figure 3 It is a schematic structural diagram of each part of the indwelling needle assembly in an embodiment of the present application;

[0082] Figure 4 It is a schematic diagram of a bent needle defect image in an embodiment of the present application;

[0083] Figure 5 It is a schematic diagram of the full process of needle assembly positioning and detection in an embodiment of the present application;

[0084] Figure 6 It is a flowchart of the method for defect detection of the indwelling needle package syringe assembly based on machine vision shown in an embodiment of the present application;

[0085] Figure 7 Schematic diagram of the positioning of the middle syringe needle and the hemostat in an embodiment of the present application;

[0086] Figure 8 Schematic diagram of the effect of the protective cap blocking the syringe needle in an embodiment of the present application;

[0087] Figure 9 Structural diagram of a defect detection device for a cannula assembly of an indwelling needle based on machine vision shown in an embodiment of the present application. Detailed implementation manners

[0088] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other.

[0089] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the layers related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the layers in actual implementation. The type, quantity, and ratio of each layer in actual implementation can be arbitrarily changed, and the layer layout type may also be more complex.

[0090] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details.

[0091] Figure 1 Schematic diagram of an indwelling needle sample in an embodiment of the present application. The indwelling needle package for detection in the present application is as Figure 1 shown.

[0092] Figure 2 Imaging effect diagram of an indwelling needle in an embodiment of the present application. The present application performs detection based on the Figure 2 original image shown.

[0093] Figure 3 Schematic diagram of the structure of each part of the indwelling needle assembly in an embodiment of the present application. As Figure 3 shown, the needle assembly of the indwelling needle product includes three parts: a hemostat, a syringe needle, and a needle handle. Among them, a sheath is sleeved on the syringe needle, and needle handle glue is injected into the needle handle.

[0094] Figure 4 Schematic diagram of the defective image of a bent needle in an embodiment of the present application. AsFigure 4 As shown, it is necessary to locate the various parts of the needle assembly through the plastic packaging shell to ultimately detect the bent needle defect.

[0095] In order to avoid the reflection problem of the plastic packaging shell, backlight imaging is used. At the same time, since the packaging paper on the back of the indwelling needle packaging is opaque, in order to image the indwelling needle component more clearly, a more penetrating red light source is used to backlight the indwelling needle product. The resulting image is as follows Figure 2 As shown in the figure, it can be seen that in the process of manually placing the indwelling needle into the packaging box, the posture of the indwelling needle varies greatly. Therefore, the occlusion rules of the needle tube assembly are likely to be different, and the component positioning algorithm needs to be adapted.

[0096] Figure 5 FIG. 1 is a schematic diagram of the entire process of positioning and detecting the needle assembly in an embodiment of the present application. The overall detection process is as follows: Figure 5 In fact, the needle assembly consists of three parts: the hemostat, the needle tube, and the needle handle. The needle handle is dotted with needle handle glue. The needle assembly part is the part with a low grayscale value in the image. Since the ultimate goal of the detection is to determine whether the needle tube is bent, it is necessary to first accurately locate the needle assembly, and then determine the comparison baseline of the needle tube. Since the entire needle tube is inserted into the hemostat and the needle handle is short, there is basically no possibility of bending, so the two can be used as benchmarks.

[0097] Figure 6 FIG. 1 is a specific implementation flow chart of a method for detecting defects of an indwelling needle packaging needle tube assembly based on machine vision in an embodiment of the present application. Figure 6 As shown, the method for detecting defects of an indwelling needle packaging needle tube assembly based on machine vision in this embodiment may include the following steps:

[0098] S610, acquiring an original image of the indwelling needle package, wherein the original image is a grayscale image;

[0099] S620, extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline;

[0100] In this embodiment, the hemostat needs to be positioned first, and the subsequent positioning detection is performed based on the hemostat. The specific process is as follows:

[0101] S621, performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image;

[0102] S622, performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat;

[0103] In steps S621 - S622, due to a lot of interference in the background, after combining two threshold segmentation methods, a mask image of the hemostat is obtained. The first binary image B1 is obtained by using fixed threshold segmentation, and then the second binary image B2 is obtained by using adaptive threshold segmentation. An image AND operation is performed on B1 and B2 to obtain the mask image H_img of the hemostat.

[0104] S623, perform multiple opening operations on the mask image to obtain a mask image with the syringe part removed;

[0105] Since the gray values of the syringe and the hemostat are close, it is necessary to remove the remaining syringe part on H_img. Multiple opening operations are performed on H_img. The kernel used here is set as {k1, k2,...}, and the size of the kernel in the column direction of the image is much larger than that in the row direction of the image, to obtain the updated mask image H_img of the hemostat.

[0106] S624, perform contour extraction on the mask image with the syringe part removed, and screen out preselected contours through a preset area threshold;

[0107] S625, fit the minimum bounding rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle;

[0108] S626, calculate the straight-line distance between the center point coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculate the longitudinal distance between the center point coordinates of the candidate rectangle and the center coordinates of the packaging box, where the center coordinates of the packaging box are preset in advance;

[0109] In steps S624 - S626, contour extraction is performed on H_img, the preselected contours are screened out through the area threshold, and the minimum bounding rectangle is fitted to obtain the width w of the candidate rectangle and the center point coordinates (pt1_x, pt1_y). We need to finally screen out the hemostat according to the basically fixed position of the hemostat in the packaging box. Let the center point coordinates of the minimum bounding rectangle of the packaging box be (pt0_x, pt0_y), then the distance dist between the candidate rectangle and the center of the packaging box H can be calculated according to the following formula:

[0110] dist H = √((pt1 x - pt0 x )) 2 +(pt1 y - pt0 y )) 2

[0111] The longitudinal distance dH between the candidate rectangle and the center of the packaging box y= pt1 y - pt0 y 。

[0112] S627. Screen the preselected contour based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first screening parameter to obtain a first target contour, where the first screening parameter includes a width screening parameter, a straight-line distance screening parameter, and a longitudinal distance screening parameter;

[0113] By restricting w, dist H , dH y within a certain range, further obtain the screened contours, and place these contours in the list cnt_list.

[0114] S628. When there is no first target contour, determine that the hemostat component in the indwelling needle package is missing; specifically, if cnt_list does not exist, directly return the defect of missing hemostat component and end the detection.

[0115] S629. When there is a first target contour, draw the first target contour in a blank image to obtain a preselected binary image of the hemostat;

[0116] Specifically, if cnt_list exists, initialize an image with all values equal to 0 and the same size as the grayscale image, and draw all the contours in cnt_lis on it to obtain a preselected binary image of the hemostat.

[0117] S6291. Perform a morphological closing operation on the preselected binary image of the hemostat to obtain a processed preselected binary image of the hemostat;

[0118] S6292. Extract the binary contours from the processed preselected binary image of the hemostat, calculate the distance between the binary contours and the center coordinates of the packaging box, and use the binary contour with the smallest distance from the center coordinates of the packaging box as the contour of the hemostat;

[0119] S6293. Calculate the minimum bounding rectangle of the contour of the hemostat to obtain the minimum bounding rectangle of the hemostat.

[0120] In steps S6292 - S6293, sort the contours. The sorting rule is to sort them in ascending order of the horizontal distance between the contour center and the center of the packaging box, select the contour with the smallest distance as the contour of the hemostat, and at the same time calculate the minimum bounding rectangle h_rect of the hemostat, so as to obtain the initial angle h_angle of the hemostat and the width w of the hemostat h , however, due to a series of morphological operations, the inclination angle of the hemostat has been distorted, and it is necessary to use the angle of the middle needle tube at the connection with the hemostat to determine the inclination angle of the hemostat.

[0121] S630, perform central needle tube positioning on the original image based on the minimum circumscribed rectangle of the hemostat to obtain the central needle tube positioning result; and correct the inclination of the minimum circumscribed rectangle of the hemostat using the central needle tube positioning result.

[0122] The purpose of central needle tube positioning is to correct the inclination angle of the minimum circumscribed rectangle of the hemostat. The specific process is as follows:

[0123] S631, extract the initial contour of the central needle tube, and calculate the reference distance between the center of the initial contour of the central needle tube and the center of the minimum circumscribed rectangle of the hemostat.

[0124] Specifically, first initialize the angle α of the central needle tube mn to h_angle. First, it is necessary to obtain the initial contour of the central needle. The distance from the center point of the minimum circumscribed rectangle of the initial contour to the center of the hemostat is dm. Let the actual length of the central needle tube be d 0 .

[0125] S632, obtain the center of the minimum circumscribed rectangle of the hemostat, and move the center of the minimum circumscribed rectangle of the hemostat along the long side direction by the reference distance to obtain the reference position of the central needle tube.

[0126] Move the hemostat rect h along the long side direction by a distance of dm. That is, taking the hemostat as a reference, the center position of the central needle tube can be found.

[0127] S633, set a minimum circumscribed rectangle with a length of the actual length d of the central needle tube 0 and a width of the actual width w of the central needle tube h at the reference position of the central needle tube to obtain the initial rectangle of the central needle tube.

[0128] S634, extract the central needle tube image from the original image based on the initial rectangle of the central needle tube.

[0129] The specific process includes: draw the initial rectangle of the central needle tube on a blank image to obtain the central needle tube mask image; and perform an AND operation on the central needle tube mask image and the original image to obtain the central needle tube image.

[0130] S635, perform contour extraction on the central needle tube image to obtain the contours within the central needle tube image; and screen the contours within the central needle tube image based on a preset second screening parameter to obtain the second target contour within the central needle tube image.

[0131] Specifically, perform adaptive threshold segmentation on the middle syringe needle image, and perform opening operation and closing operation respectively, extract the contour. Let the initial minimum bounding rectangle of the middle syringe needle be mn_rect. During the process of traversing the contour, filter out the contours that make up the middle syringe needle by restricting the thresholds of area, length, and aspect ratio.

[0132] S636, fit the second target contour into a straight line, and extract the slope k, intercept b, and maximum ordinate y of the straight line t , minimum ordinate y b ; and based on the slope k, intercept b, maximum ordinate y t , minimum ordinate y b calculate the maximum abscissa x of the second target contour b and minimum abscissa x t : and based on the maximum ordinate y b of the second target contour, minimum ordinate y t , maximum abscissa and minimum abscissa x t and maximum abscissa x b determine the minimum bounding rectangle of the second target contour;

[0133] Specifically, fit the above-mentioned contour into a straight line by the least square method respectively to obtain the slope k and intercept b of the straight line, and find the minimum and maximum y coordinates y t , y b in the set Y of contour ordinates, corresponding to the highest and lowest positions of the contour, and calculate the abscissa x b and x t corresponding to the contour ordinate by the following formula:

[0134]

[0135] The minimum bounding rectangles obtained from the filtered contours and the fitted straight lines are stored in the lists MRect and MLine respectively. It is necessary to find the topmost minimum bounding rectangle in MRect that meets the syringe needle characteristics. First, it is necessary to calculate the angle between the fitted straight line ml of the candidate middle syringe needle in MLine and the straight line hl connecting the vertex of the hemostat center to ml. Considering that the rectangle with a smaller angle is closest to the middle syringe needle. Second, the length h m of the selected rectangle should be as close as possible to the true length d 0 of the middle syringe needle. Finally, the width w m of the selected rectangle should be as large as possible to exclude the possibility of misjudging the peripheral shadow thin line as the middle syringe needle. In summary, the following strategy is adopted to judge the rectangle closest to the middle syringe needle:

[0136] S637, calculate the strategy value of the minimum bounding contour of each second target contour, and use the second target contour with the minimum strategy value as the contour of the middle syringe needle;

[0137] The mathematical expression of the strategy value is: angle m +|d 0 -h m |-w m ; In this application, the minimum strategy value needs to be found, which is written as min{angle m +|d 0 -h m |-w m}.

[0138] The rectangle that satisfies the minimum strategy value calculated from the above formula is the minimum circumscribed rectangle rect of the middle syringe mn , so the angle of the middle syringe is the angle of rect mn , which is set as α mn .

[0139] Among them, the straight-line equation of ml is set as The corresponding vectorized expression is The straight-line equation of hl is set as The corresponding vectorized expression is Among them, (x 1 , y 1 ), (x 2 , y 2 ) are the starting and ending coordinates of the middle syringe respectively, and (x 3 , y 3 ) is the center coordinate of the hemostat. The included angle between the two vectors is calculated according to the following formula:

[0140]

[0141] The process connection relationship between the middle syringe and the hemostat determines that there will be no relative inclination between the two. Therefore, the angle of the hemostat rect h is corrected to the angle α of the middle syringe mn , and rect h is updated.

[0142] Figure 7 This is a schematic diagram of the positioning of the middle syringe and the hemostat in an embodiment of this application. The positioning results of the middle syringe and the hemostat are as Figure 7 shown.

[0143] S640. Based on the minimum circumscribed rectangle of the corrected hemostat, position the main syringe and the syringe handle respectively to obtain the positioning results of the main syringe and the syringe handle;

[0144] This application uses the positioning results of the main syringe and the syringe handle to perform defect detection. The defects mainly include the defect of missing the main syringe and the defect of bent needle.

[0145] Specifically, based on the minimum bounding rectangle of the corrected hemostat, the positioning of the main body syringe needle is performed on the original image to obtain the positioning result of the main body syringe needle, including:

[0146] First, read the specifications of the indwelling needle from the configuration file. Since the lengths, widths of the syringe needles of different specifications and the distances from the hemostat are different, let the distance between the main body syringe needle and the center of the hemostat be d nh , and the length and width of the main body syringe needle be h n and w n .

[0147] S6401. Obtain the center of the minimum bounding rectangle of the corrected hemostat, and move the center of the minimum bounding rectangle of the corrected hemostat a first preset distance along the long side direction to obtain the reference position of the main body syringe needle;

[0148] S6402. Set a minimum bounding rectangle with a length equal to the actual length of the main body syringe needle and a width equal to the actual width of the main body syringe needle at the reference position of the middle syringe needle to obtain the initial rectangle of the main body syringe needle;

[0149] S6403. Extract the main body syringe needle image from the original image based on the initial rectangle of the main body syringe needle;

[0150] S6404. Perform contour extraction on the main body syringe needle image to obtain the contours within the main body syringe needle image; and filter the contours within the main body syringe needle image based on a preset third screening parameter to obtain the third target contour within the main body syringe needle image;

[0151] Steps S6401 - S6404 are similar to the previous ones. First, move the center of the rect of the hemostat h a distance d nh , expand the length and width to h n and w n respectively, and the angle is α mn , the initialized minimum bounding rectangle rect of the main body syringe needle can be obtained n . Fill and draw rect n on the blank image to obtain the mask image. Perform an AND operation on the mask image and the original grayscale image to obtain the initial main body syringe needle image. Perform adaptive threshold segmentation and morphological opening operation on it to obtain the binary image. Perform contour extraction on the binary image to obtain the third target contour. Then initialize an empty list Needles c for storing the components of the candidate main body syringe needle, and traverse and filter each contour contour according to the area, length, and aspect ratio.

[0152] S6405. Fit the third target contour into a straight line, and use the straight line whose distance from the center of the minimum circumscribed rectangle of the corrected hemostat to its end is not greater than a preset distance threshold as the candidate straight line;

[0153] Fit the needle tube component contour that meets the requirements into a straight line needle, calculate the distance from the end of the straight line to the center of the hemostat, and select the straight line whose distance is not greater than a given threshold as the candidate straight line.

[0154] S6406. Determine the length of the candidate straight line and the angle with the abscissa. When the length of the candidate straight line is greater than a preset length threshold and the angle of the target straight line is greater than a preset angle threshold, use the candidate straight line as the target straight line, and extract the average gray value I of the contour area corresponding to the target straight line a ;

[0155] In this embodiment, calculate the angle β between the candidate straight line and the horizontal axis, and obtain the length l of the line by calculating the distance between two points. If both β and l are greater than the given threshold, it is used as the target straight line to further calculate the average gray value I of the corresponding contour area a 。

[0156] S6407. Based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than a preset length threshold, use the points that meet the target conditions in the contour corresponding to the target straight line as the points at the bottom of the main needle tube, and construct a bottom straight line based on the points at the bottom of the main needle tube, where the target conditions include that the difference between the ordinate and the end coordinate of the corresponding fitted straight line is less than a preset difference l b ;

[0157] Figure 8 This is a schematic diagram of the effect of the protective cap on the needle tube in an embodiment of the present application. As Figure 8 shown, since the protective cap of the indwelling needle will block part of the needle tube, it may divide the needle tube into several discontinuous parts, or even deform the extracted needle tube, seriously interfering with the correctness of subsequent detection. Therefore, it is necessary to separately locate the bottom area of the needle tube. Let the theoretical length of the bottom of the main needle tube be l 0 , if the length of the minimum circumscribed rectangle of the contour is greater than the given threshold, the y coordinate of the points located at the bottom of the main needle tube in the contour needs to satisfy: the difference between the end coordinate of the fitted line and the y coordinate is less than l b , place the points that meet the above conditions in a newly initialized contour separately, and fit a straight line to it, then obtain the straight line needle at the bottom of the main body of the needle tube b 。

[0158] S6408, store the bottom straight line, the target straight line, the contour region corresponding to the target straight line, and the average gray value I of the contour region corresponding to the target straight line into the candidate list; a Deposit into the candidate list;

[0159] In this embodiment, store needle, contour, needle b , I a in the dictionary, and loop through to store each dictionary into the list Needles c .

[0160] Next, perform a descending order sorting on the group of straight lines fitted by the screened potential main body syringes. The sorting rule is: the length l of the fitted straight line should be as long as possible, the distance d from the center of the hemostat to the fitted straight line hn , the angle α between the longitudinal axis of the hemostat and the fitted straight line hn , and the average gray value I of the main body syringe region a should all be as small as possible, and a score score is generated accordingly.

[0161] S6409, score the target straight lines in the candidate list. Among them, the mathematical expression of the score score is:

[0162] score = w 1 l - d hn - α hn - w 2 I a

[0163] In the formula, W 1 is the first weight, W 2 is the second weight, l is the length of the target straight line, d hn is the distance from the center of the corrected hemostat to the target straight line, α hn is the angle between the target straight line and the longitudinal axis;

[0164] In the specific embodiment of this application, W 1 can take 0.2, W 2 can take 0.5.

[0165] S64010, take the target straight line with the maximum score as the standard straight line;

[0166] Specifically, the fitted straight line with the maximum score is set as the standard straight line needle std , and deposit it into the initialized new list Needles.

[0167] S64011, when there is only one target straight line in the candidate list, obtain the minimum circumscribed rectangle of the main body syringe based on the minimum circumscribed rectangle of the contour of the standard straight line;

[0168] S64012. When there are multiple target lines in the candidate list, other target lines in the candidate list are screened based on a preset overlapping screening condition to obtain target lines that do not overlap with the standard line; the target lines that do not overlap with the standard line are screened based on a preset geometric screening condition to obtain a fitted line at the bottom of the main syringe tube;

[0169] Among them, the overlapping screening condition includes: the average gray level differs from the average gray level within the area of the standard line by no more than a preset gray level threshold, and the starting vertical coordinate is not less than the ending vertical coordinate of the standard line or the ending vertical coordinate is not less than the starting vertical coordinate of the standard line;

[0170] The geometric screening condition includes: the distance from the end point to the center of the corrected hemostat, the angle with the standard line, and the angle between the derived line and the standard line all fall into the corresponding screening ranges respectively, where the derived line is the line formed by the end point and the standard line;

[0171] S64013. Based on the minimum circumscribed rectangle of the fitted line at the bottom of the main syringe tube, and the minimum circumscribed rectangle of the contour of the standard line and the minimum circumscribed rectangle of the fitted line at the bottom of the main syringe tube are used as the minimum circumscribed rectangle of the main syringe tube.

[0172] In steps S64012 - S64013, if there is more than one component in the candidate list Needles c , a second screening needs to be performed according to whether they are collinear.

[0173] The screening process is as follows: Traverse the components except needle std . If the average gray level differs from the average gray level of needle std by no more than 30, and the starting y - coordinate of the component line is not less than the ending y - coordinate of needle std or the ending y - coordinate of the component line is not greater than the starting y - coordinate of needle std , it means that the component does not overlap with the standard line;

[0174] When there is no overlap, it is necessary to further calculate the distance d 0 from its end point to the center of the hemostat, its angle γ std with needle 0 . A new line is formed by the end point of the component line and needle std , and the angle γ std between the new line and needle 1 is calculated. When d 0 , γ 0 , γ 1When all satisfy a given threshold, add this component straight line to Needles. After the traversal ends, sort the straight line components in Needles in ascending order of the y - coordinate of the starting point. At the same time, needle b is assigned to the component straight line with the largest y - coordinate of the end point. Fit the minimum bounding rectangle for all component contours contour that meet the conditions to obtain rect needle .

[0175] In an embodiment of the present application, based on the minimum bounding rectangle of the modified hemostat, position the needle handle to obtain the needle handle positioning result, including:

[0176] S6411, obtain the center of the minimum bounding rectangle of the modified hemostat, and move the center of the minimum bounding rectangle of the modified hemostat a second preset distance along the long - side direction to obtain the needle - handle reference position;

[0177] S6412, set a minimum bounding rectangle with a length equal to the true length of the needle handle and a width equal to the true width of the needle handle at the needle - handle reference position to obtain the initial rectangle of the needle handle;

[0178] S6413, extract the needle - handle image from the original image based on the initial rectangle of the needle handle;

[0179] Steps S6411 - S6413 are similar to those described above. Perform adaptive threshold segmentation on the input grayscale image. Let the distance from the center of the needle handle to the center of the lower edge of the hemostat be d bh , move the center of the minimum bounding rectangle of the hemostat by d bh , expand the length and width by the true length and width of the needle handle respectively to obtain the minimum bounding rectangle at the needle - handle position, draw it on a blank image, and perform an AND operation with the grayscale image to obtain the image of the needle - handle area.

[0180] S6414, extract the contours from the needle - handle image, and use the contour with the largest area as the standard needle - handle contour;

[0181] In an embodiment of the present application, due to the interference of the needle - handle glue shadow, the obtained needle - handle area may be discontinuous. It is necessary to perform morphological opening and closing operations on the needle - handle image and then extract the contours.

[0182] Traverse the contours, put the contours filtered by area and length thresholds into the list NeedleBases, sort NeedleBases in descending order of area, and set the component ranked first as the standard needle - handle contour NeedleBase std .

[0183] S6415, calculate the minimum bounding rectangle Rect of the standard needle - handle contour NeedleBaseand the center point pt of the upper edge of the minimum circumscribed rectangle of the standard needle handle profile std ; and connect the center of the minimum circumscribed rectangle Rect NeddleBase to the center point pt of the upper edge std to obtain a new straight line Line std ;

[0184] S6416. Extract the center points of the upper edges of the minimum circumscribed rectangles of other contours in the needle handle image, and connect the center points of the upper edges of the minimum circumscribed rectangles of other contours to the center point pt of the upper edge std to obtain a new straight line Line test ;

[0185] S6417. Determine the included angle between the new straight line Line std and the new straight line Line test . When the included angle between the new straight line Line std and the new straight line Line test is less than a preset reference threshold, connect the other contours to the standard needle handle profile to obtain an updated standard needle handle profile;

[0186] When the included angle is less than 10 degrees, it indicates that the other contours also follow the trend of the standard needle handle profile, and it can be inferred that they belong to the standard needle handle profile. Therefore, connect the other contours that meet the conditions to the standard needle handle profile. Judging each other contour in a traversal form can gradually extract the complete needle handle profile.

[0187] S6418. Calculate the minimum circumscribed rectangle of the updated standard needle handle profile, and use the minimum circumscribed rectangle of the updated standard needle handle profile as the minimum circumscribed rectangle of the needle handle.

[0188] After the traversal is completed, the complete needle handle profile is obtained. At this time, calculate the minimum circumscribed rectangle of the complete needle handle profile to complete the positioning.

[0189] S650. Based on the main body needle tube positioning result and the needle handle positioning result, perform defect detection on the needle tube assembly in the package to obtain a detection result.

[0190] The specific detection process is as follows:

[0191] S651. When there is no minimum circumscribed rectangle of the main body needle tube in the main body needle tube positioning result, it is determined that there is a needle tube missing defect;

[0192] Specifically, input Needles. If it is empty, return the needle tube missing defect and end the detection.

[0193] S652, when there is a minimum circumscribed rectangle of the main needle tube in the main needle tube positioning result, the center of the needle handle and the end point of the topmost straight line of the main needle tube form a first reference straight line, and the starting point of the topmost straight line of the main needle tube and the end point of the bottommost straight line of the main needle tube form a second reference straight line, calculate the angle between the first reference straight line and the second reference straight line, and perform bent needle detection based on the angle between the first reference straight line and the second reference straight line.

[0194] If Needles is not empty, form a straight line Line with the center of the needle handle and the end point of the top straight line in the main needle tube Needles 0 , the starting point of the top straight line in the main needle tube and the end point of the bottom straight line form a straight line Line 1 , calculate Line 0 and Line 1 If the angle is greater than a given threshold, it is determined to be a bent needle defect. This comparison can simultaneously detect the overall bending of the main needle tube and the bending in the middle of the main needle tube, greatly improving the detection rate of bent needle defects.

[0195] The machine vision-based defect detection method for the indwelling needle packaging needle tube assembly of the present invention performs positioning and defect detection of the indwelling needle needle tube assembly through machine vision and image processing methods, avoiding the problem of missed detection caused by manual detection fatigue. At the same time, the special imaging method can photograph the indwelling needle product through the cardboard in the packaging box, avoiding missed detection of bent needle damage caused by the packaging process; innovatively proposes strategies and methods for positioning the needle tube in the case of occlusion and interference between components, and through the reasonable selection of the reference straight line, high-precision bent needle detection is also achieved. The present application makes up for the missed detection of bent needle defects caused by the indwelling needle tube from the production to the packaging stage. Compared with manual detection, the missed detection rate can be greatly reduced, which improves the detection efficiency.

[0196] like Figure 9 As shown, the present application also provides a defect detection device for an indwelling needle packaging needle tube assembly based on machine vision, comprising:

[0197] An acquisition module, used to acquire an original image of the indwelling needle package, wherein the original image is a grayscale image;

[0198] a hemostat positioning module, configured to extract a preselected contour of the hemostat from the original image, extract a minimum circumscribed rectangle of the hemostat based on the preselected contour, and perform defect detection on the hemostat based on the preselected contour;

[0199] a middle needle tube positioning module, for positioning the middle needle tube from the original image based on the minimum circumscribed rectangle of the hemostat to obtain a middle needle tube positioning result; and using the middle needle tube positioning result to perform inclination correction on the minimum circumscribed rectangle of the hemostat;

[0200] The main syringe positioning module is used to position the main syringe and the syringe handle respectively based on the corrected minimum circumscribed rectangle of the hemostat, so as to obtain the main syringe positioning result and the syringe handle positioning result;

[0201] The defect detection module is used to detect the defects of the syringe assembly in the package based on the main syringe positioning result and the syringe handle positioning result, so as to obtain the detection result.

[0202] The defect detection device for the syringe assembly in the indwelling needle package based on machine vision of the present invention performs the positioning and defect detection of the indwelling needle syringe assembly through machine vision and image processing methods, avoiding the problem of missed detection caused by manual detection fatigue. At the same time, a special imaging method can take pictures of the indwelling needle product through the cardboard in the packaging box, avoiding the missed detection of the bent needle damage caused during the packaging process; innovatively proposes a strategy and method for syringe positioning in the case of occlusion and interference between components, and realizes high-precision bent needle detection by reasonably selecting the reference straight line. This application makes up for the missed detection of bent needle defects caused during the production to packaging stage of the indwelling needle syringe. Compared with manual detection, the missed detection rate can be greatly reduced, and the detection efficiency can be improved.

[0203] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0204] The memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory, so that the terminal executes any method in this embodiment.

[0205] For the computer-readable storage medium in this embodiment, those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to the computer program. The foregoing computer program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk or optical disc that can store program codes.

[0206] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store a computer program, the communication interface is used for communication, and the processor and the transceiver are used to run the computer program, so that the electronic terminal executes each step of the above method.

[0207] In this embodiment, the memory may include a random access memory (Random Access Memory, abbreviated as RAM), and may also include a non-volatile memory, such as at least one disk memory.

[0208] The above-mentioned processor may be a general-purpose processor, including a Central Processing Unit (CPU for short), a Network Processor (NP for short), etc.; it may also be a Digital Signal Processor (DSP for short), an Application Specific Integrated Circuit (ASIC for short), a Field-Programmable Gate Array (FPGA for short), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0209] In the above embodiments, although the present invention has been described in conjunction with specific embodiments of the present invention, many substitutions, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims.

[0210] The above embodiments are only illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Any person familiar with this technology can modify or change the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or changes completed by those with ordinary knowledge in the technical field without departing from the spirit and technical idea disclosed by the present invention should still be covered by the claims of the present invention.

Claims

1. A method for detecting defects in the packaging needle tube assembly of an indwelling needle based on machine vision, characterized in that: Includes steps: Acquire an original image of the indwelling needle package, wherein the original image is a grayscale image; extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline; Based on the minimum circumscribed rectangle of the hemostat, the middle needle tube is located from the original image to obtain a middle needle tube location result; and the middle needle tube location result is used to correct the inclination of the minimum circumscribed rectangle of the hemostat; Based on the minimum circumscribed rectangle of the modified hemostat, the main needle tube and the needle handle are positioned respectively to obtain the main needle tube positioning result and the needle handle positioning result; Based on the main needle tube positioning result and the needle handle positioning result, defect detection is performed on the needle tube assembly in the package to obtain a detection result.

2. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1 is characterized in that: Extracting a preselected contour of the hemostat from the original image comprises: Performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image; Performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat; Performing multiple opening operations on the mask image to obtain a mask image with the needle tube portion removed; Contour extraction is performed on the mask image with the needle tube portion removed, and a preselected contour is screened out using a preset area threshold.

3. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 2 is characterized in that: Extracting the minimum circumscribed rectangle of the hemostat based on the preselected contour, and performing defect detection on the hemostat based on the preselected contour, comprising: Fitting the minimum circumscribed rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle; Calculating the straight-line distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculating the longitudinal distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, wherein the center coordinates of the packaging box are preset; The preselected contour is screened based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first screening parameter to obtain a first target contour, wherein the first screening parameter includes a width screening parameter, a straight-line distance screening parameter, and a longitudinal distance screening parameter; When the first target contour does not exist, it is determined that the hemostat component in the indwelling needle package is missing; When the first target contour exists, the first target contour is drawn in the blank image to obtain a hemostat preselected binary image; performing a morphological closing operation on the hemostat pre-selected binary image to obtain a processed hemostat pre-selected binary image; Extracting the binary contour in the processed pre-selected binary image of the hemostat, calculating the distance between the binary contour and the center coordinate of the packaging box, and taking the binary contour with the smallest distance from the center coordinate of the packaging box as the contour of the hemostat; The minimum circumscribed rectangle of the outline of the hemostat is calculated to obtain the minimum circumscribed rectangle of the hemostat.

4. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1, characterized in that: The middle needle tube positioning is performed from the original image based on the minimum circumscribed rectangle of the hemostat to obtain the middle needle tube positioning result, including: Extracting the initial contour of the middle needle tube, and calculating the reference distance between the center of the initial contour of the middle needle tube and the center of the minimum circumscribed rectangle of the hemostat; Obtaining the center of the minimum circumscribed rectangle of the hemostat, and moving the center of the minimum circumscribed rectangle of the hemostat along the long side direction by the reference distance to obtain a middle needle tube reference position; A minimum circumscribed rectangle having a length equal to the real length of the middle needle tube and a width equal to the real width of the middle needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the middle needle tube; Extracting a middle needle tube image from the original image based on an initial rectangle of the middle needle tube; Extracting the contour of the middle needle tube image to obtain the contour in the middle needle tube image; and filtering the contour in the middle needle tube image based on a preset second filtering parameter to obtain a second target contour in the middle needle tube image; Fit the second target contour into a straight line, extract the slope k, intercept b, and maximum ordinate y of the straight line t , minimum ordinate y b ; and based on the slope k, intercept b, maximum ordinate y t , minimum ordinate y b Calculate the maximum horizontal coordinate x of the second target contour b and the minimum horizontal coordinate x t : and the maximum ordinate y based on the second target profile b , minimum ordinate y t , the maximum and minimum horizontal coordinates x t and the maximum horizontal coordinate x b Determine the minimum circumscribed rectangle of the second object contour; The strategic value of the minimum circumscribed contour of each second target contour is calculated, and the second target contour with the minimum strategic value is used as the contour of the middle needle tube.

5. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 4 is characterized in that: The mathematical expression of the policy value is: angle m +|d0-h m |-w m ; In the formula, angle m is the angle between the straight line ml and the connecting line hl, wherein the straight line ml is a straight line obtained by fitting the second target contour, and the connecting line hl is a line connecting the center of the hemostat and the vertex of the straight line ml; d0 is the actual length of the middle needle tube, h m is the length of the minimum circumscribed rectangle of the second target contour, w m is the width of the minimum circumscribed rectangle of the second target contour; is the vector expression symbol of the line ml, is the vector expression symbol of the connecting line hl; (x1, y1) and (x2, y2) are the starting point and end point coordinates of the middle needle tube respectively, and (x3, y3) is the center coordinate of the hemostat.

6. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1, characterized in that: Based on the minimum circumscribed rectangle of the corrected hemostat, the main needle tube is positioned from the original image to obtain the main needle tube positioning result, including: Obtaining the center of the minimum circumscribed rectangle of the modified hemostat, and moving the center of the minimum circumscribed rectangle of the modified hemostat by a first preset distance along the long side direction to obtain a main needle tube reference position; A minimum circumscribed rectangle having a length equal to the real length of the main needle tube and a width equal to the real width of the main needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the main needle tube; Extracting a main needle tube image from the original image based on an initial rectangle of the main needle tube; Extracting the contour of the main needle tube image to obtain the contour in the main needle tube image; and filtering the contour in the main needle tube image based on a preset third filtering parameter to obtain a third target contour in the main needle tube image; Fitting the third target contour into a straight line, and taking a straight line whose end is at a distance from the center of the minimum circumscribed rectangle of the modified hemostat that is not greater than a preset distance threshold as a candidate straight line; Determine the length of the candidate straight line and the angle with the horizontal coordinate. When the length of the candidate straight line is greater than a preset length threshold and the angle of the target straight line is greater than a preset angle threshold, take the candidate straight line as the target straight line, and extract the average gray value I of the contour area corresponding to the target straight line. a ; Based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than a preset length threshold, a point in the contour corresponding to the target straight line that meets the target condition is used as the point of the bottom of the main needle tube, and a bottom straight line is constructed based on the point of the bottom of the main needle tube, wherein the target condition includes that the difference between the ordinate and the end point coordinate of the corresponding fitting straight line is less than a preset difference l b ; The bottom straight line, the target straight line, the contour area corresponding to the target straight line, and the average gray value I of the contour area corresponding to the target straight line are calculated. a Save to the candidate list; The target straight lines in the candidate list are scored, and the minimum circumscribed rectangle of the main needle tube is determined based on the scores of the target straight lines.

7. The method for detecting defects of an indwelling needle tube assembly based on machine vision according to claim 6, characterized in that: The target lines in the candidate list are scored, and the minimum circumscribed rectangle of the main needle tube is determined based on the scores of the target lines, including: Score the target line in the candidate list, where the mathematical expression of the score is: score=W1l-d hn -a hn -W2I a Where W1 is the first weight, W2 is the second weight, l is the length of the target line, d hn is the distance from the corrected hemostat center to the target straight line, α hn is the angle between the target straight line and the longitudinal axis; The target straight line with the largest score is taken as the standard straight line; When only one target straight line is included in the candidate list, the minimum circumscribed rectangle of the main needle tube is obtained based on the minimum circumscribed rectangle of the outline of the standard straight line; When there are multiple target lines in the candidate list, the other target lines in the candidate list are screened based on a preset overlapping screening condition to obtain a target line that does not overlap with the standard line; the target lines that do not overlap with the standard line are screened based on a preset geometric screening condition to obtain a fitting line at the bottom of the main needle tube; The overlapping screening conditions include: the difference between the average grayscale and the average grayscale in the area of ​​the standard straight line does not exceed a preset grayscale threshold, and the vertical coordinate of the starting point is not less than the vertical coordinate of the end point of the standard straight line or the vertical coordinate of the end point is not less than the vertical coordinate of the starting point of the standard straight line; The geometric screening conditions include: the distance from the end point to the center of the corrected hemostat, the angle with the standard straight line, and the angle between the derived straight line and the standard straight line all fall into corresponding screening ranges, wherein the derived straight line is a straight line formed by the end point and the end point of the standard straight line; Based on the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube, the minimum circumscribed rectangle of the outline of the standard straight line and the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube are used as the minimum circumscribed rectangle of the main needle tube.

8. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1, characterized in that: The needle handle is positioned based on the minimum circumscribed rectangle of the corrected hemostat, and the needle handle positioning result is obtained, including: Acquire the center of the minimum circumscribed rectangle of the modified hemostat, and move the center of the minimum circumscribed rectangle of the modified hemostat by a second preset distance along the long side direction to obtain a needle handle reference position; A minimum circumscribed rectangle having a length equal to the actual length of the needle handle and a width equal to the actual width of the needle handle is set at the needle handle reference position to obtain an initial rectangle of the needle handle; extracting a needle handle image from the original image based on an initial rectangle of the needle handle; Extracting contours from the needle handle image, and taking the contour with the largest area as the standard needle handle contour; Calculate the minimum circumscribed rectangle Rect of the standard needle handle outline NeedleBase and the upper edge center point pt of the minimum circumscribed rectangle of the standard needle handle outline std ; and the minimum enclosing rectangle Rect NeedleBase The center and the upper edge center point pt std Connect the lines to get a new straight line std ; Extract the center point of the upper edge of the minimum circumscribed rectangle of other contours in the needle handle image, and compare the center point of the upper edge of the minimum circumscribed rectangle of other contours with the center point of the upper edge pt std Connect the lines to get a new straight line test ; Determine the new straight line Line std and the new straight line test The angle between the new straight line Line std and the new straight line test When the included angle of the other contours is less than a preset reference threshold, the other contours are connected to the standard needle handle contour to obtain an updated standard needle handle contour; The minimum circumscribed rectangle of the updated standard needle handle contour is calculated, and the minimum circumscribed rectangle of the updated standard needle handle contour is used as the minimum circumscribed rectangle of the needle handle.

9. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1, characterized in that: Based on the main needle tube positioning result and the needle handle positioning result, the needle tube assembly in the package is inspected for defects to obtain inspection results, including: When the minimum circumscribed rectangle of the main needle tube does not exist in the main needle tube positioning result, it is determined that there is a needle tube missing defect; When there is a minimum circumscribed rectangle of the main needle tube in the main needle tube positioning result, the center of the needle handle and the end point of the topmost straight line of the main needle tube form a first reference straight line, and the starting point of the topmost straight line of the main needle tube and the end point of the bottommost straight line of the main needle tube form a second reference straight line, the angle between the first reference straight line and the second reference straight line is calculated, and bent needle detection is performed based on the angle between the first reference straight line and the second reference straight line.

10. A defect detection device for indwelling needle packaging needle tube assembly based on machine vision, characterized in that: include: An acquisition module, used to acquire an original image of the indwelling needle package, wherein the original image is a grayscale image; a hemostat positioning module, configured to extract a preselected contour of the hemostat from the original image, extract a minimum circumscribed rectangle of the hemostat based on the preselected contour, and perform defect detection on the hemostat based on the preselected contour; a middle needle tube positioning module, for positioning the middle needle tube from the original image based on the minimum circumscribed rectangle of the hemostat to obtain a middle needle tube positioning result; and for performing inclination correction on the minimum circumscribed rectangle of the hemostat using the middle needle tube positioning result; The main needle tube positioning module is used to position the main needle tube and the needle handle based on the corrected minimum circumscribed rectangle of the hemostat, and obtain the main needle tube positioning result and the needle handle positioning result; The defect detection module is used to perform defect detection on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result to obtain a detection result.

Images