Method and device for detecting defects of a needle tube assembly of a retention needle package based on machine vision

Abstract

The present application relates to the field of machine vision, in particular to a method and device for detecting defects of a needle tube assembly of a retention needle package based on machine vision, which positions and detects defects of the needle tube assembly of the retention needle through a method of machine vision and image processing, avoids missed detection caused by fatigue of artificial detection, and at the same time, a special imaging mode can shoot the retention needle product through the paperboard in the packaging box, avoiding missed detection of the bent needle damage caused in the packaging process; innovatively, a strategy and method for positioning the needle tube under the condition that there is shielding and interference between the assemblies are proposed, and through reasonable selection of the reference straight line, high-precision bent needle detection is also achieved. The present application makes up for the missed detection of the bent needle defect of the needle tube of the retention needle from the production to the packaging stage. Compared with artificial detection, the missed detection rate can be greatly reduced, and the detection efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of machine vision, and in particular to a method and device for detecting defects in a needle tube assembly of an indwelling needle package based on machine vision. Background Art

[0002] Regarding the inspection solutions for indwelling needle products, the currently disclosed technologies only mention the automatic detection method of the indwelling needle bushing burr using a machine vision system, or the needle tip positioning method during use. No solutions for the inspection of the needle tube assembly after packaging have been found. The quality inspection after packaging currently still relies on manual visual inspection.

[0003] Manual quality inspection is prone to missed inspections and is inefficient. In addition, the syringes are inspected before assembly, but the multiple steps in the assembly and packaging process introduce new defects that are easily missed. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide a method and device for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides a method for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision, comprising the steps of:

[0007] Acquire an original image of the indwelling needle package, wherein the original image is a grayscale image;

[0008] extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline;

[0009] Positioning the central needle tube from the original image based on the minimum circumscribed rectangle of the hemostat to obtain a central needle tube positioning result; and correcting the inclination of the minimum circumscribed rectangle of the hemostat using the central needle tube positioning result;

[0010] Based on the minimum circumscribed rectangle of the modified hemostat, the main needle tube and the needle handle are positioned respectively to obtain the main needle tube positioning results and the needle handle positioning results;

[0011] Based on the main needle tube positioning result and the needle handle positioning result, the needle tube assembly in the package is inspected for defects to obtain an inspection result.

[0012] In one embodiment of the present application, extracting a preselected outline of the hemostat from the original image includes:

[0013] Performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image;

[0014] performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat;

[0015] performing multiple opening operations on the mask image to obtain a mask image with the needle tube portion removed;

[0016] Contour extraction is performed on the mask image with the needle tube portion removed, and a preselected contour is screened out using a preset area threshold.

[0017] In one embodiment of the present application, extracting a minimum circumscribed rectangle of the hemostat based on the preselected contour, and performing defect detection on the hemostat based on the preselected contour, includes:

[0018] Fitting the minimum circumscribed rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle;

[0019] Calculating the linear distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculating the longitudinal distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, wherein the center coordinates of the packaging box are preset;

[0020] Filtering the preselected contour based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first screening parameter to obtain a first target contour, wherein the first screening parameter includes a width screening parameter, a straight-line distance screening parameter, and a longitudinal distance screening parameter;

[0021] When the first target contour does not exist, it is determined that the hemostat component in the indwelling needle package is missing;

[0022] When the first target outline exists, the first target outline is drawn in the blank image to obtain a hemostat preselected binary image;

[0023] performing a morphological closing operation on the hemostat preselected binary image to obtain a processed hemostat preselected binary image;

[0024] extracting a binary contour from the processed pre-selected binary image of the hemostat, calculating a distance between the binary contour and the center coordinate of the packaging box, and taking the binary contour with the smallest distance from the center coordinate of the packaging box as the contour of the hemostat;

[0025] The minimum circumscribed rectangle of the outline of the hemostat is calculated to obtain the minimum circumscribed rectangle of the hemostat.

[0026] In one embodiment of the present application, the middle needle tube positioning is performed from the original image based on the minimum circumscribed rectangle of the hemostat to obtain the middle needle tube positioning result, including:

[0027] Extracting an initial outline of the middle needle tube, and calculating a reference distance between the center of the initial outline of the middle needle tube and the center of the minimum circumscribed rectangle of the hemostat;

[0028] Obtaining the center of the minimum circumscribed rectangle of the hemostat, and moving the center of the minimum circumscribed rectangle of the hemostat along the long side direction by the reference distance to obtain a middle needle tube reference position;

[0029] A minimum circumscribed rectangle having a length equal to the true length of the middle needle tube and a width equal to the true width of the middle needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the middle needle tube;

[0030] extracting a middle needle tube image from the original image based on an initial rectangle of the middle needle tube;

[0031] Performing contour extraction on the middle needle tube image to obtain a contour within the middle needle tube image; and filtering the contour within the middle needle tube image based on a preset second filtering parameter to obtain a second target contour within the middle needle tube image;

[0032] Fit the second target contour into a straight line, extract the slope k, intercept b, and maximum ordinate y of the straight line t , minimum ordinate y b ; and based on the slope k, intercept b, maximum ordinate y t , minimum ordinate y b Calculate the maximum horizontal coordinate x of the second target contour b and the minimum horizontal coordinate x t : and the maximum ordinate y based on the second target profile b , minimum ordinate y t , maximum and minimum horizontal coordinates x t and the maximum horizontal coordinate x b Determine the minimum circumscribed rectangle of the second object contour;

[0033] The strategic value of the minimum circumscribed contour of each second target contour is calculated, and the second target contour with the minimum strategic value is used as the contour of the middle needle tube.

[0034] In one embodiment of the present application, the mathematical expression of the policy value is: angle m +|d0-h m |-w m ;

[0035] Where,

[0036]

[0037] angle m is the angle between the straight line ml and the connecting line hl, wherein the straight line ml is the straight line obtained by fitting the second target contour, and the connecting line hl is the line connecting the center of the hemostat and the vertex of the straight line ml;

[0038] d0 is the actual length of the middle needle tube, h m is the length of the minimum circumscribed rectangle of the second target contour, w m is the width of the minimum bounding rectangle of the second target outline; is the vector expression symbol of the line ml, is the vector expression symbol of the connecting line hl; (x1, y1) and (x2, y2) are the starting point and end point coordinates of the middle needle tube respectively, and (x3, y3) is the center coordinate of the hemostat.

[0039] In one embodiment of the present application, the main needle tube is located from the original image based on the minimum circumscribed rectangle of the corrected hemostat, and the main needle tube positioning result is obtained, including:

[0040] Obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a first preset distance along the long side direction to obtain a main needle tube reference position;

[0041] A minimum circumscribed rectangle having a length equal to the actual length of the main needle tube and a width equal to the actual width of the main needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the main needle tube;

[0042] extracting a main needle tube image from the original image based on an initial rectangle of the main needle tube;

[0043] Performing contour extraction on the main needle tube image to obtain a contour within the main needle tube image; and filtering the contour within the main needle tube image based on a preset third filtering parameter to obtain a third target contour within the main needle tube image;

[0044] fitting the third target contour into a straight line, and taking a straight line whose end is at a distance from the center of the minimum circumscribed rectangle of the modified hemostat no greater than a preset distance threshold as a candidate straight line;

[0045] Determine the length of the candidate line and the angle with the horizontal coordinate. When the length of the candidate line is greater than a preset length threshold and the angle of the target line is greater than a preset angle threshold, take the candidate line as the target line and extract the average grayscale value I of the contour area corresponding to the target line. a ;

[0046] based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than a preset length threshold, taking a point in the contour corresponding to the target straight line that satisfies a target condition as a point at the bottom of the main needle tube, and constructing a bottom straight line based on the point at the bottom of the main needle tube, wherein the target condition includes that the difference between the longitudinal coordinate and the terminal point coordinate of the corresponding fitting straight line is less than a preset difference l b ;

[0047] storing the bottom straight line, the target straight line, the contour region corresponding to the target straight line, and the average gray value I a of the contour region corresponding to the target straight line into a candidate list;

[0048] scoring the target straight lines in the candidate list, and determining the minimum circumscribed rectangle of the main needle tube based on the scores of the target straight lines.

[0049] In an embodiment of the present application, scoring the target straight lines in the candidate list and determining the minimum circumscribed rectangle of the main needle tube based on the scores of the target straight lines includes:

[0050] scoring the target straight lines in the candidate list, wherein the mathematical expression of the score score is:

[0051] score = W1l - d hn - α hn - W2I a

[0052] wherein W1 is a first weight, W2 is a second weight, l is the length of the target straight line, d hn is the distance from the corrected hemostat center to the target straight line, α hn is the included angle between the target straight line and the longitudinal axis;

[0053] taking the target straight line with the largest score as a standard straight line;

[0054] when the candidate list contains only one target straight line, obtaining the minimum circumscribed rectangle of the main needle tube based on the minimum circumscribed rectangle of the contour of the standard straight line;

[0055] when there are multiple target straight lines in the candidate list, screening other target straight lines in the candidate list based on a preset overlap screening condition to obtain a target straight line that does not overlap with the standard straight line, and screening the target straight line that does not overlap with the standard straight line based on a preset geometric screening condition to obtain a fitting straight line at the bottom of the main needle tube;

[0056] The overlapping screening conditions include: the difference between the average grayscale and the average grayscale in the area of ​​the standard straight line does not exceed a preset grayscale threshold, and the vertical coordinate of the starting point is not less than the vertical coordinate of the end point of the standard straight line, or the vertical coordinate of the end point is not less than the vertical coordinate of the starting point of the standard straight line;

[0057] The geometric screening conditions include: the distance from the end point to the center of the corrected hemostat, the angle with the standard straight line, and the angle between the derived straight line and the standard straight line all falling within corresponding screening ranges, wherein the derived straight line is a straight line formed by the end point and the end point of the standard straight line;

[0058] Based on the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube, the minimum circumscribed rectangle of the outline of the standard straight line and the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube are used as the minimum circumscribed rectangle of the main needle tube.

[0059] In one embodiment of the present application, the needle handle is positioned based on the minimum circumscribed rectangle of the modified hemostat to obtain a needle handle positioning result, including:

[0060] Obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a second preset distance along the long side direction to obtain a needle handle reference position;

[0061] A minimum circumscribed rectangle having a length equal to the actual length of the needle handle and a width equal to the actual width of the needle handle is set at the needle handle reference position to obtain an initial rectangle of the needle handle;

[0062] extracting a needle handle image from the original image based on an initial rectangle of the needle handle;

[0063] Extracting contours from the needle handle image and taking the contour with the largest area as the standard needle handle contour;

[0064] Calculate the minimum circumscribed rectangle Rect of the standard needle handle outline NeedleBase and the upper edge center point pt of the minimum circumscribed rectangle of the standard needle handle outline std ; and the minimum enclosing rectangle Rect NeedleBase The center and the upper edge center point pt std Connect the lines to get a new straight line std ;

[0065] Extract the center point of the upper edge of the minimum circumscribed rectangle of other contours in the needle handle image, and compare the center point of the upper edge of the minimum circumscribed rectangle of other contours with the center point of the upper edge pt std Connect the lines to get a new straight line test ;

[0066] Determine the new straight line Linestd With the new straight line test The angle between the new straight line std With the new straight line test When the included angle of the other contours is less than a preset reference threshold, the other contours are connected to the standard needle handle contour to obtain an updated standard needle handle contour;

[0067] The minimum circumscribed rectangle of the updated standard needle handle outline is calculated, and the minimum circumscribed rectangle of the updated standard needle handle outline is used as the minimum circumscribed rectangle of the needle handle.

[0068] In one embodiment of the present application, a defect detection is performed on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result, and the detection results obtained include:

[0069] When the minimum circumscribed rectangle of the main needle tube does not exist in the main needle tube positioning result, it is determined that a needle tube missing defect exists;

[0070] When the minimum circumscribed rectangle of the main needle tube exists in the main needle tube positioning result, the center of the needle handle and the end point of the topmost straight line of the main needle tube form a first reference straight line, and the starting point of the topmost straight line of the main needle tube and the end point of the bottommost straight line of the main needle tube form a second reference straight line, calculate the angle between the first reference straight line and the second reference straight line, and perform bent needle detection based on the angle between the first reference straight line and the second reference straight line.

[0071] The present application also provides a device for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision, which is characterized by comprising:

[0072] an acquisition module, configured to acquire an original image of the packaging of the indwelling needle, wherein the original image is a grayscale image;

[0073] a hemostat positioning module, configured to extract a preselected outline of the hemostat from the original image, extract a minimum circumscribed rectangle of the hemostat based on the preselected outline, and perform defect detection on the hemostat based on the preselected outline;

[0074] a middle needle tube positioning module, configured to locate the middle needle tube from the original image based on the minimum circumscribed rectangle of the hemostat, obtaining a middle needle tube positioning result; and to perform tilt correction on the minimum circumscribed rectangle of the hemostat using the middle needle tube positioning result;

[0075] The main needle tube positioning module is used to position the main needle tube and the needle handle based on the corrected minimum circumscribed rectangle of the hemostat, and obtain the main needle tube positioning results and the needle handle positioning results;

[0076] The defect detection module is used to perform defect detection on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result to obtain a detection result.

[0077] The beneficial effects of the present invention are as follows: the method and device for defect detection of indwelling needle packaging needle tube assemblies based on machine vision of the present invention perform positioning and defect detection of indwelling needle needle tube assemblies through machine vision and image processing methods, thus avoiding the problem of missed detection caused by manual inspection fatigue. At the same time, the special imaging method can photograph the indwelling needle product through the cardboard in the packaging box, thus avoiding missed detection of bent needle damage caused by the packaging process; innovative strategies and methods for positioning needle tubes in situations where there are occlusions and interferences between components are proposed, and high-precision bent needle detection is also achieved through the reasonable selection of reference lines. This application makes up for the missed detection of bent needle defects caused by indwelling needle tubes from the production to the packaging stage. Compared with manual inspection, the missed detection rate can be greatly reduced, which is conducive to improving inspection efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0079] Figure 1 This is a schematic diagram of an indwelling needle sample in one embodiment of the present application;

[0080] Figure 2 This is an imaging effect diagram of an indwelling needle in one embodiment of the present application;

[0081] Figure 3 Schematic diagram of the structure of the various parts of the indwelling needle assembly in one embodiment of the present application;

[0082] Figure 4 This is a schematic diagram of a bent needle defect image in one embodiment of the present application;

[0083] Figure 5 Schematic diagram of the entire process of positioning and detecting a needle assembly in one embodiment of the present application;

[0084] Figure 6 This is a flow chart of a method for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision, shown in one embodiment of the present application;

[0085] Figure 7 This is a schematic diagram of the positioning of the middle needle tube and the hemostat in one embodiment of the present application;

[0086] Figure 8 This is a schematic diagram of the shielding effect of the protective cap on the needle tube in one embodiment of the present application;

[0087] Figure 9 This is a structural diagram of a machine vision-based defect detection device for an indwelling needle packaging needle tube assembly shown in one embodiment of the present application. DETAILED DESCRIPTION

[0088] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0089] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer can be changed arbitrarily, and the layer layout type may also be more complicated.

[0090] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention; however, it is apparent to one skilled in the art that the embodiments of the present invention may be practiced without these specific details.

[0091] Figure 1 This is a schematic diagram of an indwelling needle sample in an embodiment of the present application. The packaging of the indwelling needle tested in this application is as follows: Figure 1 shown.

[0092] Figure 2 This is an imaging effect diagram of an indwelling needle in an embodiment of the present application. This application is based on Figure 2 The original image shown is tested.

[0093] Figure 3 FIG. 1 is a structural diagram of the various parts of the indwelling needle assembly in one embodiment of the present application. Figure 3 As shown, the needle assembly of the indwelling needle product includes three parts: a hemostat, a needle tube, and a needle handle. The needle tube is covered with a sheath, and the inside of the needle handle is injected with needle handle glue.

[0094] Figure 4 This is a schematic diagram of a bent needle defect image in an embodiment of the present application, as shown in FIG. Figure 4 As shown, it is necessary to locate the various parts of the needle assembly through the plastic packaging shell to ultimately detect bent needle defects.

[0095] In order to avoid the reflection problem of the plastic packaging shell, backlight imaging is used. At the same time, since the packaging paper on the back of the indwelling needle package is opaque, in order to image the indwelling needle component more clearly, a more penetrating red light source is used to backlight the indwelling needle product. The resulting image is as follows Figure 2As shown in the figure, it can be seen that in the process of manually placing the indwelling needle into the packaging box, the posture of the indwelling needle varies greatly. Therefore, the occlusion rules of the needle tube assembly are likely to be different, and the component positioning algorithm needs to be adapted.

[0096] Figure 5 This is a schematic diagram of the entire process of positioning and detecting the needle assembly in one embodiment of the present application. The overall detection process is as follows: Figure 5 As shown in the figure, the needle assembly consists of three parts: the hemostat, the needle tube, and the needle handle. The needle handle is coated with handle glue. The needle assembly is the part with a lower grayscale value in the image. Since the ultimate goal of the inspection is to determine whether the needle tube is bent, the needle assembly must be accurately positioned first. Secondly, the comparison baseline of the needle tube must be determined. Since the entire needle tube is inserted into the hemostat and the needle handle is short, there is basically no possibility of bending. Therefore, the two can be used as the reference.

[0097] Figure 6 This is a specific implementation flow chart of a method for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision in an embodiment of the present application. Figure 6 As shown, the method for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision in this embodiment may include the following steps:

[0098] S610, acquiring an original image of the indwelling needle package, wherein the original image is a grayscale image;

[0099] S620, extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline;

[0100] In this embodiment, the hemostat needs to be positioned first, and the subsequent positioning detection is performed based on the hemostat. The specific process is as follows:

[0101] S621, performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image;

[0102] S622, performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat;

[0103] In steps S621-S622, due to the high interference in the background, two threshold segmentation methods are combined to obtain the mask image of the hemostat. The first binary image B1 is obtained by fixed threshold segmentation, and the second binary image B2 is obtained by adaptive threshold segmentation. An image AND operation is performed on B1 and B2 to obtain the mask image H_img of the hemostat.

[0104] S623, performing multiple opening operations on the mask image to obtain a mask image with the needle tube portion removed;

[0105] Since the grayscale values ​​of the needle and the hemostat are similar, it is necessary to remove the remaining needle parts on H_img and perform multiple opening operations on H_img. The kernel used here is set to {k1, k2, …}. The size of the kernel in the image column direction is much larger than the size in the image row direction. The updated hemostat mask image H_img is obtained.

[0106] S624, performing contour extraction on the mask image with the needle tube portion removed, and screening out preselected contours using a preset area threshold;

[0107] S625, fitting the minimum circumscribed rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle;

[0108] S626, calculating the straight-line distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculating the longitudinal distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, wherein the center coordinates of the packaging box are preset;

[0109] In steps S624-S626, the contour of H_img is extracted, and the pre-selected contour is filtered out by traversing the area threshold. The minimum bounding rectangle is fitted to it, so that the width w of the candidate rectangle and the center point coordinates (pt1_x, pt1_y) can be obtained. We need to finally filter out the hemostat based on its basically fixed position in the packaging box. Assume that the center point coordinates of the minimum bounding rectangle of the packaging box are (pt0_x, pt0_y), then the distance dist between the candidate rectangle and the center of the packaging box is H It can be calculated according to the following formula:

[0110] dist H =√((pt1 x -pt0 x )) 2 +(pt1 y -pt0 y )) 2

[0111] The vertical distance dH between the candidate rectangle and the center of the packaging box y =pt1 y -pt0 y .

[0112] S627, filtering the preselected contour based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first filtering parameter to obtain a first target contour, wherein the first filtering parameter includes a width filtering parameter, a straight-line distance filtering parameter, and a longitudinal distance filtering parameter;

[0113] By limiting w, dist H , dH y range, further obtain the filtered contours, and place these contours in the list cnt_list.

[0114] S628: When the first target contour does not exist, it is determined that the hemostat component in the indwelling needle package is missing; specifically, if cnt_list does not exist, a hemostat component missing defect is directly returned and the detection ends.

[0115] S629, when the first target outline exists, drawing the first target outline in the blank image to obtain a preselected binary image of the hemostat;

[0116] Specifically, if cnt_list exists, initialize an image with all values ​​0 that is consistent with the size of the grayscale image, and draw all the contours in cnt_lis on it to obtain the hemostat pre-selected binary image.

[0117] S6291, performing a morphological closing operation on the hemostat preselected binary image to obtain a processed hemostat preselected binary image;

[0118] S6292, extracting a binary contour from the processed pre-selected binary image of the hemostat, calculating a distance between the binary contour and the center coordinate of the packaging box, and taking the binary contour with the smallest distance from the center coordinate of the packaging box as the contour of the hemostat;

[0119] S6293: Calculate the minimum circumscribed rectangle of the outline of the hemostat to obtain the minimum circumscribed rectangle of the hemostat.

[0120] In steps S6292-S6293, the contours are sorted in ascending order according to the horizontal distance between the center of the contour and the center of the packaging box. The contour with the smallest distance is selected as the contour of the hemostat. At the same time, the minimum circumscribed rectangle h_rect of the hemostat is calculated to obtain the initial angle h_angle of the hemostat and the width w of the hemostat. h However, due to a series of morphological operations, the tilt angle of the hemostat has been distorted, and the tilt angle of the hemostat needs to be determined by using the angle of the middle needle tube connected to the hemostat.

[0121] S630, locating the middle needle tube in the original image based on the minimum circumscribed rectangle of the hemostat to obtain a middle needle tube positioning result; and using the middle needle tube positioning result to perform tilt correction on the minimum circumscribed rectangle of the hemostat;

[0122] The purpose of positioning the central needle is to correct the tilt angle of the minimum circumscribed rectangle of the hemostat. The specific process is as follows:

[0123] S631, extracting the initial outline of the middle needle tube, and calculating a reference distance between the center of the initial outline of the middle needle tube and the center of the minimum circumscribed rectangle of the hemostat;

[0124] Specifically, first adjust the angle α of the middle needle tube to mn Initialized to h_angle, first we need to get the initial outline of the middle needle. The distance from the center point of the minimum circumscribed rectangle of the initial outline to the center of the hemostat is dm. Let the actual length of the middle needle be d0.

[0125] S632, obtaining the center of the minimum circumscribed rectangle of the hemostat, and moving the center of the minimum circumscribed rectangle of the hemostat along the longitudinal direction by the reference distance to obtain a middle needle tube reference position;

[0126] Place the hemostat in the h Move the needle along the long side by a distance of dm. Using the hemostat as a reference, find the center of the needle tube.

[0127] S633, set a length equal to the actual length d0 of the middle needle tube and a width equal to the actual width w of the middle needle tube at the reference position of the middle needle tube. h The minimum circumscribed rectangle of the needle tube is obtained to obtain the initial rectangle of the middle needle tube;

[0128] S634, extracting a middle needle tube image from the original image based on the initial rectangle of the middle needle tube;

[0129] The specific process includes: drawing the initial rectangle of the middle needle tube in a blank image to obtain a middle needle tube mask image; and performing an AND operation on the middle needle tube mask image and the original image to obtain a middle needle tube image.

[0130] S635, performing contour extraction on the middle needle tube image to obtain a contour within the middle needle tube image; and filtering the contour within the middle needle tube image based on a preset second filtering parameter to obtain a second target contour within the middle needle tube image;

[0131] Specifically, the middle needle tube image is segmented by adaptive threshold, and opening and closing operations are performed respectively to extract the contour. The initial minimum circumscribed rectangle of the middle needle tube is set as mn_rect. In the process of traversing the contour, the contour of the middle needle tube is screened out by limiting the thresholds of area, length, and aspect ratio.

[0132] S636, fitting the second target contour into a straight line, extracting the slope k, intercept b, and maximum ordinate y of the straight line t , minimum ordinate y b ; and based on the slope k, intercept b, maximum ordinate y t, minimum ordinate y b Calculate the maximum horizontal coordinate x of the second target contour b and the minimum horizontal coordinate x t : and the maximum ordinate y based on the second target profile b , minimum ordinate y t , maximum and minimum horizontal coordinates x t and the maximum horizontal coordinate x b Determine the minimum circumscribed rectangle of the second object contour;

[0133] Specifically, the above contours are fitted into straight lines using the least squares method to obtain the slope k and intercept b of the straight lines, and the minimum and maximum y coordinates y in the contour ordinate set Y are found. t ,y b , corresponding to the highest and lowest positions of the contour, and use the following formula to calculate the horizontal coordinate x corresponding to the vertical coordinate of the contour b and x t :

[0134]

[0135] The minimum bounding rectangle and fitted straight line obtained from the filtered contour are stored in the lists MRect and MLine respectively. It is necessary to find the minimum bounding rectangle at the top of MRect that meets the needle tube characteristics. First, it is necessary to calculate the angle between the fitted straight line ml of the candidate middle needle tube in MLine and the straight line hl connecting the vertex from the center of the hemostat to ml. Considering that the rectangle with a smaller angle is closest to the middle needle tube, secondly, the length of the selected rectangle h m It should be as close as possible to the actual length d0 of the middle needle tube. Finally, the width of the selected rectangle w m It should be as large as possible to eliminate the possibility of misjudging the surrounding shadow lines as the central needle tube. In summary, the following strategy is adopted to determine the rectangle closest to the central needle tube:

[0136] S637, calculating the strategic value of the minimum circumscribed contour of each second target contour, and taking the second target contour with the minimum strategic value as the contour of the middle needle tube;

[0137] The mathematical expression of the policy value is: angle m +|d0-h m |-w m ; This application needs to find the minimum policy value, which is written as min{angle m +|d0-h m |-w m}.

[0138] The rectangle that satisfies the minimum value of the strategy calculated by the above formula is the minimum circumscribed rectangle of the central needle tube. mn, so the angle of the middle needle is rect mn The angle is set as α mn .

[0139] The linear equation of ml is set as The corresponding vectorized expression is The equation of the line hl is set to The corresponding vectorized expression is Where (x1, y1) and (x2, y2) are the starting and ending coordinates of the middle needle tube, respectively, and (x3, y3) is the center coordinate of the hemostat. The angle between the two vectors is calculated according to the following formula:

[0140]

[0141] The process connection between the middle needle tube and the hemostat determines that there will be no relative tilt between the two, so the hemostat rect h The angle is corrected to the angle α of the middle needle tube mn , update rect h .

[0142] Figure 7 This is a schematic diagram of the positioning of the middle needle tube and the hemostat in one embodiment of the present application. The positioning results of the middle needle tube and the hemostat are shown in FIG. Figure 7 shown.

[0143] S640, positioning the main needle tube and the needle handle based on the corrected minimum circumscribed rectangle of the hemostat, and obtaining main needle tube positioning results and needle handle positioning results;

[0144] This application uses the main needle tube positioning results and the needle handle positioning results to perform defect detection, and the defects mainly include main needle tube missing defects and bent needle defects.

[0145] Specifically, the main needle tube is located from the original image based on the minimum circumscribed rectangle of the corrected hemostat to obtain a positioning result of the main needle tube, including:

[0146] First, read the specifications of the indwelling needle from the configuration file. Because different specifications of needle tubes have different lengths, widths, and distances from the hemostat, let the distance from the main needle tube to the center of the hemostat be d nh The length and width of the main needle tube are h n and w n .

[0147] S6401, obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a first preset distance along the long side direction to obtain a main needle tube reference position;

[0148] S6402: setting a minimum circumscribed rectangle at the middle needle tube reference position, with a length equal to the actual length of the main needle tube and a width equal to the actual width of the main needle tube, to obtain an initial rectangle of the main needle tube;

[0149] S6403, extracting a main needle tube image from the original image based on the initial rectangle of the main needle tube;

[0150] S6404: performing contour extraction on the subject needle tube image to obtain a contour within the subject needle tube image; and filtering the contour within the subject needle tube image based on a preset third filtering parameter to obtain a third target contour within the subject needle tube image;

[0151] Steps S6401 to S6404 are similar to the above. First, the rect h Center moving distance d nh , the length and width are expanded to h n and w n , angle α mn , you can get the initialized minimum circumscribed rectangle of the main needle tube rect n , rect n Fill and draw on the blank image to get the mask image, perform image and operation on the mask image and the original grayscale image to get the initial main needle tube image, perform adaptive threshold segmentation and morphological opening operation on it to get a binary image, perform contour extraction on the binary image to get the third target contour. Then initialize the empty list Needles c It is used to store the components of candidate needle tubes and traverse each contour to screen them based on area, length, and aspect ratio.

[0152] S6405, fitting the third target contour into a straight line, and selecting a straight line whose end is at a distance from the center of the minimum circumscribed rectangle of the corrected hemostat that is not greater than a preset distance threshold as a candidate straight line;

[0153] The needle component contour that meets the requirements is fitted into a straight line needle, and the distance between the end of the straight line and the center of the hemostat is calculated. The straight lines with a distance not greater than a given threshold are screened out and used as candidate lines.

[0154] S6406, determining the length of the candidate line and the angle between the candidate line and the horizontal coordinate, and when the length of the candidate line is greater than a preset length threshold and the angle between the candidate line and the target line is greater than a preset angle threshold, taking the candidate line as the target line and extracting the average grayscale value I of the contour area corresponding to the target line. a ;

[0155] In this embodiment, the angle β between the candidate line and the horizontal axis is calculated, and the length l of the line is obtained by calculating the distance between the two points. If both β and l are greater than a given threshold, the target line is used to further calculate the average grayscale value I of the corresponding contour area. a .

[0156] S6407, based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than a preset length threshold, a point in the contour corresponding to the target straight line that meets the target condition is used as the point of the bottom of the main needle tube, and a bottom straight line is constructed based on the point of the bottom of the main needle tube, wherein the target condition includes that the difference between the vertical coordinate and the end point coordinate of the corresponding fitting straight line is less than a preset difference l b ;

[0157] Figure 8 Schematic diagram of the shielding effect of the protective cap on the needle tube in one embodiment of the present application. Figure 8 As shown in the figure, since the protective cap of the indwelling needle will block part of the needle tube, the needle tube may be divided into several discontinuous parts, or even the extracted needle tube may be deformed, which seriously interferes with the accuracy of subsequent detection. Therefore, it is necessary to locate the bottom area of ​​the needle tube separately. Assuming that the theoretical length of the bottom of the main needle tube is l0, if the minimum circumscribed rectangle length of the contour is greater than the given threshold, the y coordinate of the point at the bottom of the main needle tube in the contour must satisfy: the difference between the end point coordinate of the fitting line and the y coordinate is less than l b , put the points that meet the above conditions separately in an initialized new contour, and fit a straight line to it, then the straight line needle at the bottom of the needle body is obtained b .

[0158] S6408: The bottom straight line, the target straight line, the contour area corresponding to the target straight line, and the average gray value I of the contour area corresponding to the target straight line are calculated. a Save to candidate list;

[0159] In this embodiment, needle, contour, needle b , I a Store in a dictionary, loop through and store each dictionary in the list Needles c .

[0160] Next, the potential main body needle tube fitting straight line group screened out is sorted in descending order. The sorting rule is: the length of the fitting straight line l should be as long as possible, and the distance d from the center of the hemostat to the fitting straight line is hn , the angle α between the vertical axis of the hemostat and the fitting straight line hn , the average grayscale I of the main needle area a should be as small as possible, and a score is generated accordingly.

[0161] S6409: Score the target line in the candidate list. The mathematical expression of the score is:

[0162] score=w1l-d hn -α hn -w2I a

[0163] Where W1 is the first weight, W2 is the second weight, l is the length of the target line, d hn is the distance from the center of the corrected hemostat to the target straight line, α hn is the angle between the target straight line and the longitudinal axis;

[0164] In the specific embodiment of the present application, W1 can be 0.2, and W2 can be 0.5.

[0165] S64010, the target straight line with the largest score is used as the standard straight line;

[0166] Specifically, the fitting straight line with the largest score is set as the standard straight line needle std , store it in the initialized new list Needles.

[0167] S64011, when the candidate list contains only one target straight line, obtaining the minimum circumscribed rectangle of the main needle tube based on the minimum circumscribed rectangle of the outline of the standard straight line;

[0168] S64012: When there are multiple target lines in the candidate list, filter the other target lines in the candidate list based on a preset overlapping filter condition to obtain a target line that does not overlap with the standard line; filter the target lines that do not overlap with the standard line based on a preset geometric filter condition to obtain a fitting line for the bottom of the main needle tube;

[0169] The overlapping screening conditions include: the difference between the average grayscale and the average grayscale in the area of ​​the standard straight line does not exceed a preset grayscale threshold, and the vertical coordinate of the starting point is not less than the vertical coordinate of the end point of the standard straight line, or the vertical coordinate of the end point is not less than the vertical coordinate of the starting point of the standard straight line;

[0170] The geometric screening conditions include: the distance from the end point to the center of the corrected hemostat, the angle with the standard straight line, and the angle between the derived straight line and the standard straight line all falling within corresponding screening ranges, wherein the derived straight line is a straight line formed by the end point and the standard straight line;

[0171] S64013, based on the minimum circumscribed rectangle of the fitted straight line at the bottom of the main needle tube, the minimum circumscribed rectangle of the outline of the standard straight line and the minimum circumscribed rectangle of the fitted straight line at the bottom of the main needle tube are used as the minimum circumscribed rectangle of the main needle tube.

[0172] In steps S64012-S64013, if the candidate list Needles c If there is more than one component in , a second screening is required based on whether they are collinear.

[0173] The screening process is: traverse except needle std If the average grayscale is equal to the needle std The average grayscale difference does not exceed 30, and the starting point y coordinate of the component line is not less than needle std The y coordinate of the endpoint or the y coordinate of the endpoint of the component line is not greater than needle std If the starting point y coordinate is , it means that the component does not overlap with the standard straight line;

[0174] When there is no overlap, it is necessary to further calculate the distance d0 from the end point to the center of the hemostat and the distance between the end point and the needle. std The angle γ0 between the end point of the component line and the needle std Form a new straight line and calculate the distance between the new straight line and the needle std When d0, γ0, and γ1 all meet the given threshold, the component straight line is added to Needles. After the traversal is completed, the straight line components in Needles are sorted in ascending order according to the y coordinate of the starting point. At the same time, needle b The component line with the largest y coordinate of the end point is assigned, and the minimum circumscribed rectangle is fitted to all component contours that meet the conditions, and rect needle .

[0175] In one embodiment of the present application, the needle handle is positioned based on the minimum circumscribed rectangle of the modified hemostat to obtain a needle handle positioning result, including:

[0176] S6411, obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a second preset distance along the longitudinal direction to obtain a needle handle reference position;

[0177] S6412, setting a minimum circumscribed rectangle at the needle handle reference position, with a length equal to the actual length of the needle handle and a width equal to the actual width of the needle handle, to obtain an initial rectangle of the needle handle;

[0178] S6413, extracting a needle handle image from the original image based on the initial rectangle of the needle handle;

[0179] Steps S6411-S6413 are similar to the foregoing, and adaptive threshold segmentation is performed on the input gray-scale image. It is assumed that the distance from the center of the needle handle to the center of the lower edge of the tourniquet is d bh The center of the minimum circumscribed rectangle of the tourniquet is moved by d bh The length and width are each expanded to the actual length and width of the needle handle, a minimum circumscribed rectangle at the needle handle is obtained, and the minimum circumscribed rectangle is drawn on a blank image and subjected to image and operation with the gray-scale image to obtain an image of the needle handle region.

[0180] S6414, a contour is extracted from the needle handle image, and the largest contour in area is taken as a standard needle handle contour;

[0181] In an embodiment of the present application, due to the interference of the needle handle glue shadow, the obtained needle handle region can be discontinuous, and the needle handle image needs to be subjected to morphological opening operation and closing operation before the contour is extracted.

[0182] The contours that pass the area and length threshold screening are put into a list NeedleBases, and NeedleBases are arranged in descending order of area, and the first group is set as the standard needle handle contour NeedleBase std .

[0183] S6415, a minimum circumscribed rectangle Rect NeedleBase of the standard needle handle contour is calculated, and a center point pt std of the upper edge of the minimum circumscribed rectangle of the standard needle handle contour is calculated; and a line is drawn between the center of the minimum circumscribed rectangle Rect NeddleBase and the center point pt std of the upper edge to obtain a new straight line Line std ;

[0184] S6416, a center point of the upper edge of the minimum circumscribed rectangle of the other contour in the needle handle image is extracted, and a line is drawn between the center point of the upper edge of the minimum circumscribed rectangle of the other contour and the center point pt std of the upper edge to obtain a new straight line Line test ;

[0185] S6417, an included angle between the new straight line Line std and the new straight line Line test is determined, and when the included angle between the new straight line Line std and the new straight line Line test is less than a preset reference threshold, the other contour is connected to the standard needle handle contour to obtain an updated standard needle handle contour;

[0186] When the included angle is less than 10 degrees, it indicates that the other contours also follow the standard needle handle contour and can be inferred to belong to the standard needle handle contour. Therefore, the other contours that meet the conditions are linked to the standard needle handle contour. By traversing and judging the other contours one by one, the complete needle handle contour can be gradually extracted.

[0187] S6418: Calculate the minimum circumscribed rectangle of the updated standard needle handle outline, and use the minimum circumscribed rectangle of the updated standard needle handle outline as the minimum circumscribed rectangle of the needle handle.

[0188] After the traversal is completed, the complete needle handle outline is obtained. At this time, the minimum circumscribed rectangle of the complete needle handle outline is calculated to complete the positioning.

[0189] S650: Perform defect detection on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result to obtain a detection result.

[0190] The specific detection process is as follows:

[0191] S651, when the minimum circumscribed rectangle of the main needle tube does not exist in the main needle tube positioning result, it is determined that a needle tube missing defect exists;

[0192] Specifically, input Needles. If it is empty, a needle missing defect is returned and the detection ends.

[0193] S652, when the minimum circumscribed rectangle of the main needle tube exists in the main needle tube positioning result, the center of the needle handle and the end point of the topmost straight line of the main needle tube form a first reference straight line, and the starting point of the topmost straight line of the main needle tube and the end point of the bottommost straight line of the main needle tube form a second reference straight line, calculate the angle between the first reference straight line and the second reference straight line, and perform bent needle detection based on the angle between the first reference straight line and the second reference straight line.

[0194] If Needles is not empty, the center of the needle handle and the end point of the top straight line in the main needle tube Needles form a straight line Line0, and the starting point of the top straight line and the end point of the bottom straight line in the main needle tube form a straight line Line1. The angle between Line0 and Line1 is calculated. If the angle is greater than the given threshold, it is determined to be a bent needle defect. This comparison can simultaneously detect the overall bending of the main needle tube and the bending in the middle of the main needle tube, greatly improving the detection rate of bent needle defects.

[0195] The present invention's method for defect detection of indwelling needle packaging needle tube assemblies based on machine vision performs positioning and defect detection of indwelling needle needle tube assemblies through machine vision and image processing methods, avoiding the problem of missed detection caused by manual inspection fatigue. At the same time, a special imaging method can photograph the indwelling needle product through the cardboard in the packaging box, avoiding missed detection of bent needle damage caused by the packaging process; innovatively proposes strategies and methods for positioning needle tubes in situations where there are occlusions and interferences between components, and through the reasonable selection of reference lines, also achieves high-precision bent needle detection. This application makes up for the missed detection of bent needle defects caused by indwelling needle tubes from the production to the packaging stage. Compared with manual inspection, it can greatly reduce the missed detection rate, which is helpful to improve inspection efficiency.

[0196] like Figure 9 As shown, the present application also provides a device for detecting defects in an indwelling needle packaging needle tube assembly based on machine vision, comprising:

[0197] an acquisition module, configured to acquire an original image of the packaging of the indwelling needle, wherein the original image is a grayscale image;

[0198] a hemostat positioning module, configured to extract a preselected outline of the hemostat from the original image, extract a minimum circumscribed rectangle of the hemostat based on the preselected outline, and perform defect detection on the hemostat based on the preselected outline;

[0199] a middle needle tube positioning module, configured to locate the middle needle tube from the original image based on the minimum circumscribed rectangle of the hemostat, obtaining a middle needle tube positioning result; and to perform tilt correction on the minimum circumscribed rectangle of the hemostat using the middle needle tube positioning result;

[0200] The main needle tube positioning module is used to position the main needle tube and the needle handle based on the corrected minimum circumscribed rectangle of the hemostat, and obtain the main needle tube positioning results and the needle handle positioning results;

[0201] The defect detection module is used to perform defect detection on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result to obtain a detection result.

[0202] The device for defect detection of indwelling needle tube assemblies in packaging based on machine vision of the present invention performs positioning and defect detection of indwelling needle tube assemblies through machine vision and image processing methods, thus avoiding the problem of missed detection caused by fatigue of manual inspection. At the same time, the special imaging method can photograph the indwelling needle product through the cardboard in the packaging box, thus avoiding missed detection of damage to the bent needle caused by the packaging process. It innovatively proposes strategies and methods for positioning the needle tube in situations where there is occlusion and interference between components, and also achieves high-precision bent needle detection through the reasonable selection of reference lines. This application makes up for the missed detection of bent needle defects caused by indwelling needle tubes from the production to the packaging stage. Compared with manual inspection, it can greatly reduce the missed detection rate, which is conducive to improving inspection efficiency.

[0203] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0204] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0205] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0206] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0207] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0208] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0209] In the above embodiments, although the present invention has been described in conjunction with specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations that fall within the broad scope of the appended claims.

[0210] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A method for detecting defects in indwelling needle packaging tube assemblies based on machine vision, characterized in that: Including steps: Acquire an original image of the indwelling needle package, wherein the original image is a grayscale image; extracting a preselected outline of the hemostat from the original image, extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline; Positioning the central needle tube from the original image based on the minimum circumscribed rectangle of the hemostat to obtain a central needle tube positioning result; and correcting the inclination of the minimum circumscribed rectangle of the hemostat using the central needle tube positioning result; Based on the minimum circumscribed rectangle of the modified hemostat, the main needle tube and the needle handle are positioned respectively to obtain the main needle tube positioning results and the needle handle positioning results; Based on the main needle tube positioning result and the needle handle positioning result, the needle tube assembly in the package is inspected for defects to obtain an inspection result.

2. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 1, characterized in that: Extracting a preselected outline of the hemostat from the original image includes: Performing fixed threshold binarization segmentation on the original image to obtain a first binarized image; and performing adaptive threshold binarization segmentation on the original image to obtain a second binarized image; performing an AND operation on the first binarized image and the second binarized image to obtain a mask image of the hemostat; performing multiple opening operations on the mask image to obtain a mask image with the needle tube portion removed; Contour extraction is performed on the mask image with the needle tube portion removed, and a preselected contour is screened out using a preset area threshold.

3. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 2, characterized in that: Extracting a minimum circumscribed rectangle of the hemostat based on the preselected outline, and performing defect detection on the hemostat based on the preselected outline, comprising: Fitting the minimum circumscribed rectangle of the preselected contour to obtain a candidate rectangle and the width and center point coordinates of the candidate rectangle; Calculating the linear distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, and calculating the longitudinal distance between the center coordinates of the candidate rectangle and the center coordinates of the packaging box, wherein the center coordinates of the packaging box are preset; Filtering the preselected contour based on the width of the candidate rectangle, the straight-line distance, the longitudinal distance, and a preset first screening parameter to obtain a first target contour, wherein the first screening parameter includes a width screening parameter, a straight-line distance screening parameter, and a longitudinal distance screening parameter; When the first target contour does not exist, it is determined that the hemostat component in the indwelling needle package is missing; When the first target outline exists, the first target outline is drawn in the blank image to obtain a hemostat preselected binary image; performing a morphological closing operation on the hemostat preselected binary image to obtain a processed hemostat preselected binary image; extracting a binary contour from the processed pre-selected binary image of the hemostat, calculating a distance between the binary contour and the center coordinate of the packaging box, and taking the binary contour with the smallest distance from the center coordinate of the packaging box as the contour of the hemostat; The minimum circumscribed rectangle of the outline of the hemostat is calculated to obtain the minimum circumscribed rectangle of the hemostat.

4. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 1, characterized in that: The middle needle tube positioning is performed from the original image based on the minimum circumscribed rectangle of the hemostat to obtain a middle needle tube positioning result, including: Extracting an initial outline of the middle needle tube, and calculating a reference distance between the center of the initial outline of the middle needle tube and the center of the minimum circumscribed rectangle of the hemostat; Obtaining the center of the minimum circumscribed rectangle of the hemostat, and moving the center of the minimum circumscribed rectangle of the hemostat along the long side direction by the reference distance to obtain a middle needle tube reference position; A minimum circumscribed rectangle having a length equal to the true length of the middle needle tube and a width equal to the true width of the middle needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the middle needle tube; extracting a middle needle tube image from the original image based on an initial rectangle of the middle needle tube; Performing contour extraction on the middle needle tube image to obtain a contour within the middle needle tube image; and filtering the contour within the middle needle tube image based on a preset second filtering parameter to obtain a second target contour within the middle needle tube image; Fit the second target profile to a straight line and extract the slope of the straight line ,intercept , maximum vertical coordinate , minimum vertical coordinate ; and based on the slope ,intercept , maximum vertical coordinate , minimum vertical coordinate Calculate the maximum horizontal coordinate of the second target contour and the minimum horizontal coordinate : and the maximum ordinate based on the second target profile , minimum vertical coordinate , minimum horizontal coordinate and the maximum horizontal coordinate Determine the minimum circumscribed rectangle of the second object contour; The strategic value of the minimum circumscribed rectangle of each second target contour is calculated, and the second target contour with the minimum strategic value is used as the contour of the middle needle tube.

5. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 4, characterized in that: The mathematical expression of the policy value is: ; Where, is the angle between the straight line ml and the connecting line hl, wherein the straight line ml is a straight line obtained by fitting the second target contour, and the connecting line hl is a line connecting the center of the hemostat and the vertex of the straight line ml; is the actual length of the middle needle tube, is the length of the minimum circumscribed rectangle of the second target contour, is the width of the minimum bounding rectangle of the second target outline; is the vector expression symbol of the line ml, is the vector expression symbol of the line hl; ( , )、( , ) are the starting point and end point coordinates of the middle needle tube, ( , ) is the center coordinate of the hemostat.

6. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 4, characterized in that: The main needle tube is located from the original image based on the minimum circumscribed rectangle of the corrected hemostat to obtain a positioning result of the main needle tube, including: Obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a first preset distance along the long side direction to obtain a main needle tube reference position; A minimum circumscribed rectangle having a length equal to the actual length of the main needle tube and a width equal to the actual width of the main needle tube is set at the middle needle tube reference position to obtain an initial rectangle of the main needle tube; extracting a main needle tube image from the original image based on an initial rectangle of the main needle tube; Performing contour extraction on the main needle tube image to obtain a contour within the main needle tube image; and filtering the contour within the main needle tube image based on a preset third filtering parameter to obtain a third target contour within the main needle tube image; fitting the third target contour into a straight line, and taking a straight line whose end is at a distance from the center of the minimum circumscribed rectangle of the modified hemostat no greater than a preset distance threshold as a candidate straight line; Determine the length of the candidate line and the angle with the horizontal coordinate. When the length of the candidate line is greater than a preset length threshold and the angle of the candidate line is greater than a preset angle threshold, use the candidate line as the target line and extract the average grayscale value of the contour area corresponding to the target line. ; Based on the minimum circumscribed rectangle of the contour corresponding to the target straight line, when the length of the minimum circumscribed rectangle of the contour corresponding to the target straight line is greater than a preset length threshold, the point in the contour corresponding to the target straight line that meets the target condition is used as the point of the bottom of the main needle tube, and a bottom straight line is constructed based on the point of the bottom of the main needle tube, wherein the target condition includes that the difference between the vertical coordinate and the end point coordinate of the corresponding fitting straight line is less than a preset difference ; The bottom straight line, the target straight line, the contour area corresponding to the target straight line, and the average gray value of the contour area corresponding to the target straight line are calculated. Save to candidate list; The target lines in the candidate list are scored, and the minimum circumscribed rectangle of the main needle tube is determined based on the scores of the target lines.

7. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 6, characterized in that: Scoring the target lines in the candidate list, and determining the minimum circumscribed rectangle of the main needle tube based on the scores of the target lines, including: Score the target line in the candidate list, where the score The mathematical expression is: Where, is the first weight, is the second weight, is the length of the target line, is the distance from the center of the corrected hemostat to the target straight line, is the angle between the target straight line and the longitudinal axis; The target straight line with the largest score is taken as the standard straight line; When the candidate list contains only one target straight line, the minimum circumscribed rectangle of the main needle tube is obtained based on the minimum circumscribed rectangle of the outline of the standard straight line; When there are multiple target lines in the candidate list, the other target lines in the candidate list are screened based on a preset overlapping screening condition to obtain a target line that does not overlap with the standard line; the target lines that do not overlap with the standard line are screened based on a preset geometric screening condition to obtain a fitting line for the bottom of the main needle tube; The overlapping screening conditions include: the difference between the average grayscale and the average grayscale in the area of ​​the standard straight line does not exceed a preset grayscale threshold, and the vertical coordinate of the starting point is not less than the vertical coordinate of the end point of the standard straight line, or the vertical coordinate of the end point is not less than the vertical coordinate of the starting point of the standard straight line; The geometric screening conditions include: the distance from the end point to the center of the corrected hemostat, the angle with the standard straight line, and the angle between the derived straight line and the standard straight line all falling within corresponding screening ranges, wherein the derived straight line is a straight line formed by the end point and the end point of the standard straight line; Based on the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube, the minimum circumscribed rectangle of the outline of the standard straight line and the minimum circumscribed rectangle of the fitting straight line at the bottom of the main needle tube are used as the minimum circumscribed rectangle of the main needle tube.

8. The method for detecting defects of an indwelling needle tube assembly packaged with a machine vision according to claim 1, characterized in that: The needle handle is positioned based on the minimum circumscribed rectangle of the corrected hemostat, and the needle handle positioning result is obtained, including: Obtaining the center of the minimum circumscribed rectangle of the corrected hemostat, and moving the center of the minimum circumscribed rectangle of the corrected hemostat by a second preset distance along the long side direction to obtain a needle handle reference position; A minimum circumscribed rectangle having a length equal to the actual length of the needle handle and a width equal to the actual width of the needle handle is set at the needle handle reference position to obtain an initial rectangle of the needle handle; extracting a needle handle image from the original image based on an initial rectangle of the needle handle; Extracting contours from the needle handle image and taking the contour with the largest area as the standard needle handle contour; Calculate the minimum circumscribed rectangle of the standard needle handle outline and the upper edge center point of the minimum circumscribed rectangle of the standard needle handle outline ; and the minimum enclosing rectangle The center and the center point of the upper edge Connect the lines to get a new straight line ; Extract the center point of the upper edge of the minimum circumscribed rectangle of other contours in the needle handle image, and compare the center point of the upper edge of the minimum circumscribed rectangle of other contours with the center point of the upper edge. Connect the lines to get a new straight line ; Determine the new straight line With the new straight line The angle between the new straight line With the new straight line When the included angle of the other contours is less than a preset reference threshold, the other contours are connected to the standard needle handle contour to obtain an updated standard needle handle contour; The minimum circumscribed rectangle of the updated standard needle handle outline is calculated, and the minimum circumscribed rectangle of the updated standard needle handle outline is used as the minimum circumscribed rectangle of the needle handle.

9. The method for detecting defects in an indwelling needle packaged needle tube assembly based on machine vision according to claim 1, characterized in that: Based on the main needle tube positioning result and the needle handle positioning result, the needle tube assembly in the package is inspected for defects, and the inspection results are obtained, including: When the minimum circumscribed rectangle of the main needle tube does not exist in the main needle tube positioning result, it is determined that a needle tube missing defect exists; When the minimum circumscribed rectangle of the main needle tube exists in the main needle tube positioning result, the center of the needle handle and the end point of the topmost straight line of the main needle tube form a first reference straight line, and the starting point of the topmost straight line of the main needle tube and the end point of the bottommost straight line of the main needle tube form a second reference straight line, calculate the angle between the first reference straight line and the second reference straight line, and perform bent needle detection based on the angle between the first reference straight line and the second reference straight line.

10. A machine vision-based defect detection device for an indwelling needle packaging needle tube assembly, characterized in that: include: an acquisition module, configured to acquire an original image of the packaging of the indwelling needle, wherein the original image is a grayscale image; a hemostat positioning module, configured to extract a preselected outline of the hemostat from the original image, extract a minimum circumscribed rectangle of the hemostat based on the preselected outline, and perform defect detection on the hemostat based on the preselected outline; a middle needle tube positioning module, configured to locate the middle needle tube from the original image based on the minimum circumscribed rectangle of the hemostat, obtaining a middle needle tube positioning result; and to perform tilt correction on the minimum circumscribed rectangle of the hemostat using the middle needle tube positioning result; The main needle tube positioning module is used to position the main needle tube and the needle handle based on the corrected minimum circumscribed rectangle of the hemostat, and obtain the main needle tube positioning results and the needle handle positioning results; The defect detection module is used to perform defect detection on the needle tube assembly in the package based on the main needle tube positioning result and the needle handle positioning result to obtain a detection result.

Images