Infusion set accessory detection method, system, storage medium and device based on machine vision

Abstract

An inspection method, system, storage medium and device for infusion set accessories based on machine vision, including an infusion set, the infusion set includes four accessories: an end cap, a rotating luer connector, a needle cap, and a Robert clip. The inspection method for infusion set accessories based on machine vision designed by the present invention performs inspections on the entire infusion set, is used to detect whether the accessories on the infusion set are completely assembled, realizes accessory detection based on color features and differentiates accessories of different colors by adopting HSV color space technology, and then uses the template matching method to match accessories with similar colors that are difficult to distinguish through the HSV color space, quickly and accurately detecting the accessories and improving the inspection efficiency and inspection quality.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and in particular, to a detection method, system, storage medium and device for infusion set accessories based on machine vision. Background Art

[0002] There are a large number of catheters used in medical devices, such as infusion sets. The assembly of these devices requires the installation and fixation of catheters. The product quality of these medical devices is extremely important in any medical device or system. Therefore, the ability to automatically monitor multiple quality and technical parameters of infusion sets in the production line has always been the goal pursued by people. In the prior art, manual detection is obviously extremely inefficient and the quality is difficult to guarantee.

[0003] Chinese Patent CN102495072A announced on June 13, 2012 discloses an automatic detection method for defects of infusion catheters by a machine vision system, which uses the template matching method to detect infusion catheters and infusion bottles. However, when using the template matching method, if the measured part is blocked, the placement angle is different from the template, or the size of the measured part captured is different from the template, the template matching method is not applicable.

[0004] Chinese Patent CN104483321B authorized and announced on July 28, 2017 discloses an automatic detection system and method for injection syringes based on machine vision. By rotating the syringe and taking pictures of each angle of the infusion set, after preprocessing, the contour of the area with a specific brightness is extracted, and whether the defect is within the standard is judged by setting a threshold. However, this method has complex steps and a large amount of training.

[0005] Moreover, the existing detection methods mainly detect one accessory. The infusion set is a commonly used device in medical devices, mostly used for intravenous infusion. Its production standards are strict and the requirement for a sterile environment is high. The infusion set consists of a needle cap, a rotating luer connector, an end cap, a Robert clip, and a hose. During processing, the needle cap, rotating luer connector, end cap, and Robert clip are easily omitted during processing, and it is necessary to detect whether each accessory is installed. Currently, there is a lack of a detection method for the assembly quality of the entire infusion set on the market. Summary of the Invention

[0006] The problem solved by the present invention is how to provide a detection method for the assembly quality of the entire infusion set and improve the detection efficiency.

[0007] To solve the above problems, a detection method for infusion set accessories based on machine vision, the infusion set includes four accessories: an end cap, a rotating luer connector, a needle cap, and a Robert clip, and includes the following steps:

[0008] Step 1: Set a template corresponding to each accessory;

[0009] Step 2: Photograph the entire infusion set to be detected to obtain a clear original image;

[0010] Step 3: Preprocess the original image;

[0011] Step 4: Convert the preprocessed image to the HSV color space;

[0012] Step 5: Perform accessory detection based on color features according to the extracted hue H, saturation S, and value V, and label the distinguished accessories;

[0013] Step 6: Use the template matching method to perform accessory detection based on template matching for the remaining undetected accessories.

[0014] The beneficial effects of the present invention are as follows: The method for detecting accessories of an infusion set based on machine vision designed by the present invention detects the entire infusion set, and is used to detect whether the accessories on the infusion set are completely assembled. By using the HSV color space technology, accessory detection based on color features is realized to distinguish accessories of different colors, and then the template matching method is used to match accessories with similar colors that are difficult to distinguish through the HSV color space, so as to quickly and accurately detect the accessories, improving the detection efficiency and detection quality.

[0015] Preferably, the specific steps of setting the template for each accessory in Step 1 are as follows:

[0016] Step 101: Photograph each type of accessory under the same shooting conditions as a standard template image, and the standard template is enlarged by 1.05 times proportionally to generate a sub-template image for matching when the accessory is close to the lens;

[0017] Step 102: Photograph the background image under the same shooting conditions;

[0018] Step 103: Convert each standard template image obtained in Step 101 to the HSV color space to obtain the HSV color accessory parameters corresponding to each accessory;

[0019] Step 104: Convert the background image obtained in Step 102 to the HSV color space to obtain the HSV color background parameters.

[0020] Preferably, Step 3 specifically includes the following steps:

[0021] Step 301: Perform white balance operation on the original image to eliminate the influence of the light source on the color in the original image;

[0022] Step 302: Then use Gaussian filtering to denoise the image;

[0023] Step 303: Then enhance the color of the image by increasing the contrast.

[0024] Preferably, the conversion from the RGB color space to the HSV color space in step 4 is specifically carried out according to the following conversion formula:

[0025]

[0026]

[0027] V = max(R, G, B)

[0028] In the formula, the value range of the hue H is 0° - 360°; the value range of the saturation S is 0% - 100%. The larger the S value, the closer the color saturation is to the spectral color, and the higher the saturation; the value range of the lightness V is 0 - 1. When V = 0, it represents black; when V = 1 and S = 0, it represents white.

[0029] Preferably, step 5 specifically includes the following steps:

[0030] Step 501: Preset the HSV color accessory parameters corresponding to each accessory according to the color characteristics of the templates of each accessory, and preset the HSV color background parameters corresponding to the background image;

[0031] Step 502: Extract the component histograms of the hue H component, saturation S component, and lightness V component in the image after HSV conversion, and judge the magnitude of the color difference between the accessories in the image after HSV conversion. If the difference is large, go to step 503; if the difference is small, go to step 6;

[0032] 503: Perform accessory detection based on color characteristics.

[0033] 7. Preferably, step 501 specifically includes the following steps:

[0034] Step 5011: Judge whether there is a transparent accessory among all the accessories. If there is, go to step 5012; if not, go to step 5013;

[0035] Step 5012: Judge whether there is a white accessory among all the accessories. If there is, go to step 5014; if not, go to step 5015;

[0036] Step 5013: Select the HSV color accessory parameters and HSV color background parameters corresponding to all the accessories in the template;

[0037] Step 5014: Screen out the HSV color accessory parameters and HSV color background parameters corresponding to the remaining accessories except the white accessories and transparent accessories from the template;

[0038] Step 5015: Screen out the HSV color accessory parameters and HSV color background parameters corresponding to other accessories except the transparent color accessories from the template;

[0039] The specific steps for determining the color difference between accessories in the image after HSV conversion in Step 502 are as follows:

[0040] Step 5021: Determine whether the difference in the hue H component between accessories or between an accessory and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between accessories or between an accessory and the background is large; otherwise, proceed to Step 5022;

[0041] Step 5022: Determine whether the difference in the saturation S component between accessories or between an accessory and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between accessories or between an accessory and the background in the image after HSV conversion is large; otherwise, proceed to Step 5023;

[0042] Step 5023: Determine whether the difference in the value V component between accessories or between an accessory and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between accessories or between an accessory and the background in the image after HSV conversion is large; if not, it is determined that the color feature difference between accessories or between an accessory and the background in the image after HSV conversion is small;

[0043] Step 503 specifically includes the following steps:

[0044] Step 5031: Extract the H, S, and V components from the image after HSV color space conversion in Step 4 to obtain the H, S, and V components of all pixel points in the entire image;

[0045] Step 5032: Through the preset HSV color accessory parameters and HSV color background parameters in Step 1, find the pixel points that conform to the corresponding color features in the HSV component histogram of the entire image obtained in Step 5031. If the number of pixel points that meet the color features reaches the set threshold number, the accessory corresponding to the HSV color accessory parameters exists; if the number of pixel points does not meet the set threshold number, it is regarded as noise; if no pixel points corresponding to the color features are detected, it means that the corresponding accessory does not exist.

[0046] Preferably, Step 6 specifically includes the following steps:

[0047] Step 601: Perform template matching based on gray values, specifically including:

[0048] A1: Gray-scale process the preprocessed image:

[0049] Gary = 0.299R + 0.587G + 0.144B

[0050] A2. Perform matching using the NCC algorithm:

[0051] d) After the image grayscale processing, the image S has an area of M×N formed by the number of rows and columns of image pixels. The standard template image and the sub-template image are images T with an area of m×n formed by the number of rows and columns of image pixels;

[0052] e) Each image T is translated on the image S in turn. An arbitrarily selected covered area on the image S is used as a sub-image, denoted as s(i,j), and the coordinates of its upper left vertex in the image S are (i,j);

[0053] f) Each image T traverses and searches the image S from top to bottom and from left to right in turn, and calculates the grayscale correlation value r(x,y) at each sub-image position. The calculation formula for the grayscale correlation value r(x,y) is:

[0054]

[0055] In the formula, W1 and W2 are two matching windows of the image T and the sub-image s(i,j) respectively. The two matching windows are both images with the number of rows and columns of image pixels forming an area of m×n, and the value of r(x,y) is the matching degree between the template image T window and the sub-image window;

[0056] A3. Set the threshold α of the grayscale correlation value. If the grayscale correlation value r(x,y)≥α, the position of the sub-image s(i,j) is recorded as the matching position and the matching ends; if the grayscale correlation value r(x,y)<α, go to step 602;

[0057] Step 602. Perform matching using the template rotation method, specifically including:

[0058] B1. Rotate the image T around its center. The number of rotations is u, the initial rotation direction is clockwise, the initial rotation angle is θ, the maximum number of rotations is u′, the number of templates of the image T is f, the maximum number of templates is f′, and the grayscale correlation value is 0;

[0059] B2. Judge whether f is greater than f′. If so, go to step 603; otherwise, go to B3;

[0060] B3. Judge whether u is greater than u′. If so, go to B11; otherwise, go to B4;

[0061] B4. Rotate the image T by θ along the center of the image T in the rotation direction;

[0062] B5. Return to steps A1 and A2 in step 601 for template matching, and then go to B6;

[0063] B6. Determine whether the current grayscale correlation value is greater than the threshold α. If it is, the matching is successful and the matching ends. Otherwise, proceed to B7;

[0064] B7. Determine whether the current grayscale correlation value is greater than the grayscale correlation value of the previous rotation. If it is, proceed to B8. Otherwise, proceed to B9;

[0065] B8. Maintain the current rotation direction and rotation angle, and then proceed to B10;

[0066] B9. The rotation direction is opposite to the current rotation direction, and the rotation angle is half of the current rotation angle, that is Then proceed to B10;

[0067] B10. u = u + 1, and return to B3;

[0068] B11. f = f + 1, and return to B2;

[0069] Step 603. Adopt template matching based on local shape, specifically including:

[0070] C1. Combine the color feature auxiliary area to allocate parts:

[0071]

[0072] In the formula, F is the overall similarity, ω is the weight, Color is the color similarity,, Z is the shape similarity. When F reaches a certain threshold, the matching is successful;

[0073] C2. Set several local shape templates for the corresponding parts;

[0074] C3. First, perform image grayscale processing on the image preprocessed in Step 3 to reduce the amount of image processing. Then, automatically obtain the binarization threshold using the OSTU method, perform binarization processing on the image. Finally, use the Canny operator for contour extraction;

[0075] C4. Use several local shape templates as the standard templates, and magnify the standard template image by 1.05 times as the sub-template. Use the squared Euclidean distance between the standard template and the sub-image, and between the sub-template and the sub-image as the similarity metric, and adopt the template matching method based on Euclidean distance for matching:

[0076]

[0077] In the formula, w(x,y) is the template image function, K and L are the length and width of the template image, and f(x + i, y + i) is the sub-image function;

[0078] By setting up template matching algorithms based on grayscale values, template rotation matching, and template matching based on local shapes to detect accessories at various angles and different scales, and by combining a matching method that combines color and shape features in the template matching based on local shapes, it is possible to accurately detect occluded accessories.

[0079] An infusion set accessory detection system based on machine vision, comprising a preset module for presetting accessory and background templates, an image acquisition module for acquiring an original image, an image preprocessing module for image color enhancement, an image color space conversion module for HSV color space conversion, a first detection module for identifying accessories based on an image conversion module, and a second detection module based on template matching. The image acquisition module is connected to the image preprocessing module. The image conversion module and the second detection module are connected to the image preprocessing module to obtain the preprocessed image. The first detection module is connected to the preset module and the image conversion module.

[0080] Preferably, the second detection module includes a grayscale value template matching unit, a template rotation matching unit, and a local shape matching unit connected in sequence.

[0081] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed, it can implement the above-mentioned infusion set accessory detection method based on machine vision.

[0082] A computing device, comprising:

[0083] One or more processors, a memory, and one or more programs, where one or more programs are stored in the memory and configured to be executed by the one or more processors. The one or more programs include those for executing the above-mentioned infusion set accessory detection method based on machine vision. Description of the Drawings

[0084] Figure 1 A picture of the complete assembly of the infusion tube of the present invention;

[0085] Figure 2 The histogram of each component extracted from the white cap after conversion to the HSV color space;

[0086] Figure 3 The histogram of each component extracted from the transparent rotating luer adapter after conversion to the HSV color space;

[0087] Figure 4 The histogram of each component extracted from the light blue cap after conversion to the HSV color space;

[0088] Figure 5 The histogram of each component extracted from the dark blue Robert clip after conversion to the HSV color space;

[0089] Figure 6 It is a shape comparison diagram when the white protective cap and the transparent rotating Luer connector part are blocked;

[0090] Figure 7 It is the contour diagram of the white protective cap extracted by the present invention;

[0091] Figure 8 It is the contour diagram of the upper half part of the white protective cap extracted by the present invention;

[0092] Figure 9 It is the contour diagram of the left half part of the white protective cap extracted by the present invention;

[0093] Figure 10 It is the contour diagram of the right half part of the white protective cap extracted by the present invention;

[0094] Figure 11 It is the overall contour diagram of the transparent rotating Luer connector extracted by the present invention;

[0095] Figure 12 It is the contour diagram of the upper half part of the transparent rotating Luer connector extracted by the present invention;

[0096] Figure 13 It is the contour diagram of the left half part of the transparent rotating Luer connector extracted by the present invention;

[0097] Figure 14 It is the HSV component histogram of the whole picture extracted in Specific Example 1. Specific Embodiment

[0098] To make the above objects, features and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention will be given with reference to the accompanying drawings. Specific Example 1

[0100] A method for detecting infusion set accessories based on machine vision, including an infusion set. As Figure 1 shown, the infusion set in this specific embodiment includes four accessories, namely a light blue needle, a transparent rotating Luer connector, a white end cap, and a dark blue Robert clip, assembled on the infusion tube, and includes the following steps:

[0101] Step 1: Set templates corresponding to each accessory; specifically, it includes the following steps:

[0102] The specific steps for setting the template of each accessory in Step 1 include the following steps:

[0103] Step 101: Take pictures of each type of accessory under the same shooting conditions as standard template images, and the standard templates are enlarged by 1.05 times proportionally to generate sub-template images for matching when the accessories are close to the lens;

[0104] Step 102: Take a background image under the same shooting conditions;

[0105] Step 103: Perform HSV color space conversion on each standard template image obtained in Step 101 to obtain the HSV color accessory parameters corresponding to each accessory;

[0106] Step 104: Perform HSV color space conversion on the background image obtained in Step 102 to obtain the HSV color background parameters;

[0107] Step 2: Take a clear original image of the entire infusion set to be detected;

[0108] Step 3: Preprocess the original image; specifically, it includes the following steps:

[0109] Step 301: Perform white balance operation on the original image to exclude the influence of the light source on the color in the original image;

[0110] Step 302: Then use Gaussian filtering to denoise the image;

[0111] Step 303: Then enhance the color of the image by increasing the contrast;

[0112] Step 4: Perform HSV color space conversion on the preprocessed image; for the conversion from RGB color space to HSV color space, H represents hue, S represents saturation, and V represents value. The specific conversion formula is:

[0113]

[0114]

[0115] V = max(R, G, B)

[0116] In the formula, the value range of hue H is 0° - 360°; the value range of saturation S is 0% - 100%. The larger the S value, the closer the color saturation is to the spectral color and the higher the saturation; the value range of value V is 0 - 1. When V = 0, it represents black; when V = 1 and S = 0, it represents white;

[0117] Step 5: Perform accessory detection based on color features according to the extracted hue H, saturation S, and value V, and label the distinguished accessories. Figures 2 - 5 For each component histogram of the blue cap, transparent rotating Luer connector, white cap, and dark blue Robert clip converted to the HSV color space in this specific embodiment, this specific embodiment specifically includes the following steps:

[0118] Step 501: Preset the HSV color accessory parameters corresponding to each accessory according to the color characteristics of the templates of the accessories, and preset the HSV color background parameters corresponding to the background image. The specific steps are as follows:

[0119] Step 5011: Determine whether there is a transparent color accessory among all the accessories. If there is, go to Step 5012; if not, go to Step 5013.

[0120] Step 5012: Determine whether there is a white accessory among all the accessories. If there is, go to Step 5014; if not, go to Step 5015.

[0121] Step 5013: Select the HSV color accessory parameters and HSV color background parameters corresponding to all the accessories in the template.

[0122] Step 5014: Screen out the HSV color accessory parameters and HSV color background parameters corresponding to the remaining accessories in the template except for the white accessories and transparent color accessories.

[0123] Step 5015: Screen out the HSV color accessory parameters and HSV color background parameters corresponding to the other accessories in the template except for the transparent color accessories.

[0124] For example, for the dark blue Robert clip in this specific embodiment, the set HSV color accessory parameters of the dark blue Robert clip are: the H component is in the range of (0.6 - α, 0.6 + α); the S component is in the range of (0.91 - α, 0.99 + α); the V component is in the range of (0.5 - α, 0.6 + α).

[0125] Step 502: Extract the component histograms of the hue H component, saturation S component, and value V component in the image after HSV conversion, and determine the magnitude of the color difference between the accessories in the image after HSV conversion. If the difference is large, go to Step 503; if the difference is small, go to Step 6.

[0126] Among them, determining the magnitude of the color difference between the accessories in the image after HSV conversion specifically includes the following steps:

[0127] Step 5021: Determine whether the difference in the hue H component between accessories or between an accessory and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between accessories or between an accessory and the background is large; otherwise, go to Step 5022.

[0128] Step 5022: Determine whether the difference in the saturation S component between accessories or between an accessory and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between accessories or between an accessory and the background in the image after HSV conversion is large; otherwise, go to Step 5023.

[0129] Step 5023: Determine whether the difference in the V component of lightness between parts or between a part and the background is greater than or equal to 0.1. If so, it is determined that the color feature difference between parts or between a part and the background in the image after HSV conversion is large; if not, it is determined that the color feature difference between parts or between a part and the background in the image after HSV conversion is small.

[0130] 503. Perform part detection based on color features, specifically including:

[0131] Step 5031: Extract the H, S, and V components from the image after HSV color space conversion in step 4 to obtain the H, S, and V components of all pixel points in the entire image.

[0132] Step 5032: Through the preset HSV color part parameters and HSV color background parameters in step 1, find the pixel points that meet the corresponding color features in the HSV component histogram of the entire image as shown in Figure 12 . If the number of pixel points that meet the color features reaches the set threshold number, the part corresponding to the HSV color part parameter exists; if the number of pixel points does not meet the set threshold number, it is regarded as noise; if no pixel points corresponding to the color features can be detected, it means that the corresponding part does not exist.

[0133] In addition, in the process of performing part detection based on color features in this specific embodiment, if one or more parts have been detected to exist, the HSV color features corresponding to this part will no longer be used for comparison; if one or more parts are determined to not exist, the HSV color features corresponding to the corresponding part will no longer be used for comparison.

[0134] Step 6: Use the template matching method to perform part detection based on template matching for the remaining undetected parts. For example, for the white protective cap and the transparent rotating Luer connector described in this specific embodiment, use the template matching method to perform part detection, as shown in Figures 6 - 12 . Specifically, it includes the following steps:

[0135] Step 601: Perform template matching based on grayscale values, specifically including:

[0136] A1. Grayscale the preprocessed image:

[0137] Gary = 0.299R + 0.587G + 0.144B

[0138] A2. Use the NCC algorithm for matching:

[0139] a) After the image grayscale processing, the image becomes image S with an area of M×N formed by the number of rows and columns of image pixels. The standard template image and the sub-template image are image T with an area of m×n formed by the number of rows and columns of image pixels;

[0140] b) Each image T is translated on image S in turn. Arbitrarily select a covered area on image S as a sub-image, denoted as s(i,j), and the coordinates of its upper left vertex in image S are (i,j);

[0141] c) Each image T traverses and searches image S from top to bottom and from left to right in turn, and calculates the grayscale correlation value r(x,y) at each sub-image position. The calculation formula of the grayscale correlation value r(x,y) is:

[0142]

[0143] In the formula, W1 and W2 are two matching windows of image T and sub-image s(i,j) respectively. The two matching windows are both formed by the number of rows and columns of image pixels with an area of m×n, and the value of r(x,y) is the matching degree between the window of template image T and the sub-image window;

[0144] A3. Set the threshold α of the grayscale correlation value. If the grayscale correlation value r(x,y)≥α, then the position of the sub-image s(i,j) is recorded as the matching position and the matching ends; if the grayscale correlation value r(x,y)<α, then go to step 602;

[0145] Step 602. Adopt the template rotation method for matching, which specifically includes:

[0146] B1. Rotate image T around its center. The number of rotations is u, the initial rotation direction is the clockwise direction, the initial rotation angle is θ, the maximum number of rotations is u′, the number of templates of image T is f, the maximum number of templates is f′, and the grayscale correlation value is 0;

[0147] B2. Judge whether f is greater than f′. If so, go to step 603; otherwise, go to B3;

[0148] B3. Judge whether u is greater than u′. If so, go to B11; otherwise, go to B4;

[0149] B4. Rotate image T by θ along the center of image T in the rotation direction;

[0150] B5. Return to A1 and A2 in step 601 for template matching, and then go to B6;

[0151] B6. Judge whether the current grayscale correlation value is greater than the threshold α. If so, the matching is successful and the matching ends; otherwise, go to B7;

[0152] B7. Determine whether the current gray - level correlation value is greater than the gray - level correlation value of the previous rotation. If so, go to B8; otherwise, go to B9;

[0153] B8. Keep the current rotation direction and rotation angle, and then go to B10;

[0154] B9. The rotation direction is opposite to the current rotation direction, and the rotation angle is half of the current rotation angle, that is Then go to B10;

[0155] B10. u = u + 1, and return to B3;

[0156] B11. f = f + 1, and return to B2;

[0157] Step 603. Adopt template matching based on local shape, specifically including:

[0158] C1. Combine color feature auxiliary area distribution parts:

[0159]

[0160] In the formula, F is the overall similarity, ω is the weight, Color is the color similarity,, Z is the shape similarity. When F reaches a certain threshold, the matching is successful;

[0161] C2. Set several local shape templates for the corresponding parts;

[0162] C3. First, perform image grayscale processing on the image pre - processed in step 3 to reduce the amount of image processing. Then, use the OSTU method to automatically obtain the binarization threshold, perform binarization processing on the image. Finally, use the Canny operator for contour extraction;

[0163] C4. Take several local shape templates as standard templates, and magnify the standard template image by 1.05 times as a sub - template. Use the squared Euclidean distance between the standard template and the sub - image as well as the sub - template and the sub - image as the similarity measure, and adopt the template matching method based on Euclidean distance for matching:

[0164]

[0165] In the formula, w(x,y) is the template image function, K and L are the length and width of the template image, and f(x + i,y + i) is the sub - image function;

[0166] By setting template matching algorithms based on gray - level values, template rotation matching, and local - shape - based template matching to detect parts at various angles and different scales, and combining color and shape feature matching methods in local - shape - based template matching, parts that are occluded can be accurately detected. Specific Embodiment 2

[0168] An infusion set accessory detection system based on machine vision, comprising a preset module for presetting accessories and background templates, an image acquisition module for acquiring original images, an image preprocessing module for image color enhancement, an image color space conversion module for HSV color space conversion, a first detection module for identifying accessories based on the image conversion module, and a second detection module based on template matching. The image acquisition module is connected to the image preprocessing module. The image conversion module and the second detection module are connected to the image preprocessing module to obtain the preprocessed image. The first detection module is connected to the preset module and the image conversion module. The second detection module includes a grayscale value template matching unit, a template rotation matching unit, and a local shape matching unit connected in sequence. Specific Embodiment 3

[0170] A computer-readable storage medium, on which a computer program is stored. When the computer program is executed, it can implement the machine vision-based infusion set accessory detection method as described in Specific Embodiment 1. Specific Embodiment 4

[0172] A computing device, comprising:

[0173] One or more processors, a memory, and one or more programs, where one or more programs are stored in the memory and configured to be executed by the one or more processors. The one or more programs include those for executing the machine vision-based infusion set accessory detection method as described in Specific Embodiment 1.

[0174] Although the present disclosure is disclosed as above, the protection scope of the present disclosure is not limited thereto. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present disclosure, and these changes and modifications will all fall within the protection scope of the present invention.

Claims

1. A detection method for infusion set accessories based on machine vision. The infusion set includes four accessories: a end cap, a rotating Luer connector, a needle cap, and a Robert clip. It is characterized in that, It includes the following steps: Step 1: Set templates for each accessory; Step 2: Take pictures of the entire infusion set to be detected to obtain clear original images; Step 3: Preprocess the original images; Step 4: Convert the preprocessed images to the HSV color space; Step 5. Based on the extracted hue , saturation and lightness perform accessory detection based on color features and label the distinguished accessories; specifically, it includes the following steps: Step 501: Screen the HSV color accessory parameters corresponding to the accessories and the HSV color background parameters corresponding to the preset background image according to the color characteristics of the templates of each accessory. Specifically, it includes: Step 5011: Determine whether there are transparent accessories among all accessories. If there are, go to Step 5012; if not, go to Step 5013; Step 5012: Determine whether there are white accessories among all accessories. If there are, go to Step 5014; if not, go to Step 5015; Step 5013: Select the HSV color accessory parameters and HSV color background parameters corresponding to all accessories in the template; Step 5014: Screen the HSV color accessory parameters and HSV color background parameters corresponding to the remaining accessories except white and transparent accessories from the template; Step 5015: Screen the HSV color accessory parameters and HSV color background parameters corresponding to other accessories except transparent accessories from the template; Step 502: Extract the hue component, saturation component, and value component histograms of the components in the HSV-converted image, and determine the magnitude of the color differences between the various accessories in the HSV-converted image. If the differences are large, proceed to Step 503; if the differences are small, proceed to Step 6. Among them, the specific difference judgment regarding the color difference size includes the following steps: Step 5021: Determine the difference in hue components between parts or between a part and the background. If the difference is greater than or equal to 0.1, it is determined that the color feature difference between parts or between a part and the background is large; otherwise, proceed to Step 5022; Step 5022: Determine the saturation between parts and parts or between parts and the background Whether the difference in components is greater than or equal to 0.

1. If so, it is determined that there is a large difference in color characteristics between parts and parts or between parts and the background in the image after HSV conversion; otherwise, proceed to Step 5023; Step 5023, determine the lightness difference between parts and parts or between parts and the background Whether the component difference is greater than or equal to 0.

1. If so, it is determined that the color feature difference between parts and parts or between parts and the background in the image after HSV conversion is large; if not, it is determined that the color feature difference between parts and parts or between parts and the background in the image after HSV conversion is small; Step 503: Conduct accessory detection based on color characteristics. Specifically, it includes: Step 5031: Extract the component from the image that has undergone HSV color space conversion in Step 4, to obtain the components of all the pixels in the entire image; Step 5032: Through the preset HSV color accessory parameters and HSV color background parameters in Step 1, find the pixel points that meet the corresponding color characteristics in the HSV component histogram of the entire picture obtained in Step 5031. If the number of pixel points that meet the color characteristics reaches the set threshold number, the accessory corresponding to the HSV color accessory parameters exists; if the number of pixel points does not meet the set threshold number, it is regarded as noise; if the pixel points corresponding to the color characteristics cannot be detected, it means that the corresponding accessory does not exist; Step 6: Use the template matching method to conduct template-matching-based accessory detection on the remaining undetected accessories.

2. The method for detecting infusion set accessories based on machine vision according to claim 1, wherein, The specific steps for Step 1 to set the template for each accessory include the following steps: Step 101: Take pictures of each type of accessory under the same shooting conditions as the standard template images. The standard templates are enlarged by 1.05 times proportionally to generate sub-template images for matching when the accessories are close to the lens; Step 102: Take pictures of the background image under the same shooting conditions; Step 103: Convert each standard template image obtained in Step 101 to the HSV color space to obtain the HSV color accessory parameters corresponding to each accessory; Step 104: Convert the background image obtained in Step 102 to the HSV color space to obtain the HSV color background parameters.

3. The method for detecting infusion set accessories based on machine vision according to claim 1, wherein The specific steps for Step 3 include the following steps: Step 301: Perform white balance operation on the original images to eliminate the influence of the light source on the colors in the original images; Step 302: Then use Gaussian filtering to denoise the images; Step 303: Then enhance the colors of the images by increasing the contrast.

4. The method for detecting infusion set accessories based on machine vision according to claim 2, characterized in that, The specific steps for Step 6 include the following steps: Step 601: Conduct template matching based on gray values. Specifically, it includes: A1. Grayscale the preprocessed image: ； A2. Perform matching using the NCC algorithm: a) After the image grayscale processing, the image S is formed by the number of rows and columns of image pixels, and the area is The standard template image and the sub-template image are images T formed by the number of rows and columns of image pixels, and the area is of the image T; b) Each image T is translated on image S in turn, and an arbitrarily selected covered area on image S is used as a sub-image; c) Each image T traverses and searches image S from top to bottom and from left to right in turn, and calculates the gray correlation value at each sub-image position; A3. Set the threshold of the gray-scale correlation value , if the gray-scale correlation value is greater than the threshold , record the sub-graph position as the matching position and end the matching; if the gray-scale correlation value is less than the threshold , then go to step 602; Step 602. Perform matching using the template rotation method, specifically including: B1. Rotate the image T around its center, and the number of rotations is , the initial rotation direction is clockwise, and the initial rotation angle is , the maximum number of rotations is , the number of templates of the image T is , the maximum number of templates is , and the gray-scale correlation value is 0; B2. Judgment Is it greater than , if yes, go to step 603, otherwise, go to B3; B3. Judgment Is it greater than , if yes, go to B11, otherwise, go to B4; B4. Rotate the image T along the center of the image T in the rotation direction ; B5. Return to A1 and A2 in Step 601 for template matching, and then enter B6; B6. Determine whether the current gray-scale correlation value is greater than the threshold value , if so, the matching is successful and the matching ends; otherwise, proceed to B7; B7. Determine whether the current gray correlation value is greater than the gray correlation value of the previous rotation. If so, enter B8; otherwise, enter B9; B8. Keep the current rotation direction and rotation angle, and then enter B10; B9. The rotation direction is opposite to the current rotation direction, and the rotation angle is half of the current rotation angle, that is , and then proceed to B10; B10、 , return B3; B11、 , return B2; Step 603. Perform template matching based on local shape, specifically including: C1. Combine color feature auxiliary area allocation parts: ； Wherein, is the overall similarity, is the shape similarity, is the weight, is the color similarity; When reaches a certain threshold, the matching is successful; C2. Set several local shape templates for corresponding parts; C3. First, grayscale the image preprocessed in Step 3 to reduce the amount of image processing. Then, automatically obtain the binarization threshold using the OSTU method, perform binarization processing on the image. Finally, use the Canny operator for contour extraction; C4. Use several local shape templates as standard templates, and magnify the standard template image by 1.05 times as a secondary template. Use the squared Euclidean distance between the standard template and the sub-image, and between the secondary template and the sub-image as the similarity metric, and perform matching using the Euclidean distance template matching method: ； In the formula, is the template image function, is the length and width of the template image, is the sub-image function.

5. An infusion set accessory detection system based on machine vision, which is used to implement the method described in any one of claims 1 to 4, and is characterized in that, It includes a preset module for presetting parts and background templates, an image acquisition module for acquiring the original image, an image preprocessing module for image color enhancement, an image color space conversion module for HSV color space conversion, a first detection module for identifying parts based on the image color space conversion module, and a second detection module based on template matching. The image acquisition module is connected to the image preprocessing module. The image color space conversion module and the second detection module are connected to the image preprocessing module to obtain the preprocessed image. The first detection module is connected to the preset module and the image color space conversion module.

6. The infusion set accessory detection system based on machine vision according to claim 5, characterized in that, The second detection module includes a gray value template matching unit, a template rotation matching unit, and a local shape matching unit connected in sequence.

7. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed, it can implement the machine vision-based infusion set part detection method according to any one of claims 1 to 4.

8. A computing device, characterized in that, Including: One or more processors, a memory, and one or more programs, where one or more programs are stored in the memory and configured to be executed by the one or more processors. The one or more programs include those for executing the machine vision-based infusion set part detection method according to any one of claims 1 to 4.

Images