Atomizer component defect detection system based on machine vision and VIT-YOLO

Abstract

The invention relates to the technical field of machine vision, in particular to an atomizer component defect detection system based on machine vision and VIT-YOLO, and the atomizer component is positioned based on a pre-constructed positioning model. And after the positioning is completed, performing flaw detection and positioning on the single component image by using a pre-constructed flaw detection model, if flaws exist, extracting gray information and geometric information of flaw features, and finally performing analysis and verification by using the gray information and geometric information of the flaw features to complete flaw detection of the atomizer component. The defect detection system can effectively assist in positioning the previous process, can effectively help related enterprises to perform leak checking and vacancy filling of the previous process, can greatly avoid misleading caused by human eye sense during manual detection, further reduces the labor cost, and greatly improves the production efficiency of products.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to an atomizer component defect detection system based on machine vision and VIT-YOLO. Background Art

[0002] As a common medical device, nebulizers play a vital role in the treatment of respiratory diseases and are widely used in the medical field. Their working principle is to convert liquid medication into inhalable mist particles, allowing the medication to act more effectively in the patient's respiratory tract. During the nebulizer production process, the quality of the components directly affects the nebulizer's performance and safety, and in turn is related to the patient's treatment effect and health and safety. For example, if there are defects in the key components of the nebulizer, it may lead to uneven atomized particle size, affecting the inhalation effect of the drug and failing to achieve the intended treatment purpose. In severe cases, it may also cause medical accidents and harm to the patient. Therefore, strict quality inspection of nebulizer components is of great practical significance and is a key link in ensuring product quality and patient safety.

[0003] Traditional atomizer component inspection methods rely primarily on manual visual inspection, which has numerous limitations. Manual inspection is not only inefficient and difficult to meet the demands of large-scale production, but is also susceptible to subjective factors, resulting in poor accuracy and consistency in test results. Summary of the Invention

[0004] In view of this, an object of the present invention is to provide an atomizer component defect detection system based on machine vision and VIT-YOLO to solve the problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] The present invention provides an atomizer component defect detection system based on machine vision and VIT-YOLO, comprising the following steps:

[0007] A preprocessing module is used to obtain original images obtained by shooting the atomizer from multiple angles, and perform preprocessing and feature fusion on the original images to obtain preprocessed images and feature images, wherein the original images include front images, back images, and side images;

[0008] a component positioning module, configured to locate the characteristic image based on a pre-built positioning model to obtain preliminary positioning information of the target component of the atomizer; and to perform quantity and size verification on the preliminary positioning information to obtain positioning information when the quantity and size verification pass;

[0009] a foreign body detection module configured to crop the preprocessed image based on the positioning information to obtain a single object image; locate defects in the single object image based on a pre-built defect detection model to obtain a defect detection result; and, when a defect detection frame exists in the defect detection result, extract grayscale information and geometric information within the defect detection frame;

[0010] The feature analysis module is used to analyze the grayscale information and geometric information within the defect detection frame to obtain a defect analysis and judgment result.

[0011] In one embodiment of the present application, preprocessing and feature fusion are performed on the original image to obtain a preprocessed image and a feature image, including:

[0012] Performing format conversion on the original image to obtain a target format image;

[0013] Converting the target format image into a grayscale space to obtain a grayscale image;

[0014] Performing Gaussian filtering on the grayscale image to obtain a preprocessed image;

[0015] performing text elimination on the filtered image to obtain an intermediate image;

[0016] The intermediate image of the front image, the intermediate image of the back image, and the intermediate image of the side image are fused into a feature image with three-channel features.

[0017] In one embodiment of the present application, the feature image is positioned based on a pre-built positioning model to obtain preliminary positioning information of the target component of the atomizer, including:

[0018] Scaling the feature image to obtain a positioning image;

[0019] Input the positioning image into a pre-built positioning model for inference to obtain bounding box information, wherein the bounding box information includes the center coordinates of the bounding box, the width and height of the bounding box, the corresponding component classification label of the bounding box, and the confidence level;

[0020] The bounding box information is fused based on the non-maximum suppression algorithm to obtain the final detection box information;

[0021] When the final detection frame information is empty, it is determined that there is no positioning result; otherwise, the final detection frame information is used as preliminary positioning information of the target component of the atomizer.

[0022] In one embodiment of the present application, the quantity and size verification of the preliminary positioning information includes:

[0023] Comparing the number of classification labels in the preliminary positioning information with the set number, and when the number of classification labels is consistent with the set number and there are no duplicate classification labels, it is determined that the quantity detection has passed; when the number of classification labels is less than the set number, it is determined that there are a few defects; when the number of classification labels is greater than the set number, it is determined that there are multiple defects; when there are duplicate classification labels, it is determined that there are multiple defects;

[0024] The maximum and minimum values ​​of the width and height of the bounding box in the preliminary positioning information are respectively compared with the detected size range, and the confidence in the preliminary positioning information is compared with the confidence threshold; when the maximum and minimum values ​​of the width and height of the bounding box both fall into the detected size range and the confidence is greater than the confidence threshold, it is determined that the size verification has passed; when the maximum or minimum value of the length and width of the bounding box does not fall into the detected size range, it is determined that the size does not match; when the confidence is not greater than the confidence threshold, it is determined that the positioning has failed.

[0025] In one embodiment of the present application, the defect detection result includes the center coordinates of the rotated bounding box, the width and height information of the rotated bounding box, the rotation angle of the rotated bounding box, the defect type label, and the confidence level.

[0026] In one embodiment of the present application, extracting the grayscale information and geometric information within the defect detection frame includes:

[0027] Sets the rotation matrix based on the rotation angle ;

[0028] Define the initial coordinates of the four vertices of the rotation bounding box , ,in, are the initial coordinates of the four vertices of the rotation bounding box respectively;

[0029] The rotation matrix and the center coordinates of the rotation bounding box are used to rotate and offset the rotation bounding box to obtain the coordinates of the four vertices after rotation. , wherein the mathematical expression for rotating and offsetting the rotation bounding box is:

[0030]

[0031] Where, is the center coordinate of the rotation bounding box;

[0032] From the coordinates of the four vertices after rotation Confirmed Maximum value in direction 、 Minimum value of direction 、 Maximum value in direction as well as Minimum value of direction , get the lower left corner point and the upper right corner ;

[0033] Based on the lower left corner point and the upper right corner point constructing a regular rectangle, and extracting a feature area of ​​the preprocessed image based on the regular rectangle;

[0034] The grayscale information and geometric information of the feature area are extracted.

[0035] In one embodiment of the present application, the grayscale information and geometric information within the defect detection frame are analyzed to obtain a defect analysis result, including:

[0036] Load the defect attribute configuration information of the current detection area;

[0037] When the defect attribute configuration information of the current detection area is empty, determining that the current detection area has interference features; when the defect attribute configuration information of the current detection area is not empty, dividing the defect attribute configuration information of the current detection area into defect verification information and interference verification information;

[0038] The grayscale information and geometric information within the defect detection frame are verified based on the interference verification information. When the interference verification is passed, it is determined that interference features exist in the current detection area. When the interference verification is not passed, the grayscale information and geometric information within the defect detection frame are verified based on the defect verification information. When the defect verification is passed, it is determined that defect features exist in the current detection area.

[0039] In one embodiment of the present application, the interference verification information includes first reference value ranges of multiple grayscale parameters and first reference value ranges of multiple geometric parameters. Verifying the grayscale information and geometric information within the defect detection frame based on the interference verification information includes:

[0040] When the value of each grayscale parameter in the grayscale information falls into the first reference value range of the corresponding grayscale parameter, and the value of each geometric parameter in the grayscale information falls into the first reference value range of the corresponding geometric parameter, it is determined that the interference check has passed; otherwise, it is determined that the interference check has not passed.

[0041] In one embodiment of the present application, the defect verification information includes second reference value ranges of multiple grayscale parameters and second reference value ranges of multiple geometric parameters. Verifying the grayscale information and geometric information within the defect detection frame based on the defect verification information includes:

[0042] When the value of each grayscale parameter in the grayscale information falls into the second reference value range of the corresponding grayscale parameter, and the value of each geometric parameter in the grayscale information falls into the second reference value range of the corresponding geometric parameter, it is determined that the defect check has passed; otherwise, it is determined that the defect check has failed.

[0043] In one embodiment of the present application, it further includes:

[0044] a result analysis and generation module, configured to, when the component positioning module fails to pass the quantity or size verification, mark the preliminary positioning information in the original image, determine the atomizer as defective, and synchronize the defect corresponding to the preliminary positioning information into the final inspection result; and, when the component positioning module passes the quantity or size verification, extract the target defect with the highest priority from the defect detection results obtained by the foreign matter detection module, mark the target defect in the original image, determine the atomizer as defective, and synchronize the target defect into the final inspection result;

[0045] The dynamic adjustment and output module is used to output the original image with defects and the final detection results to the target object.

[0046] The beneficial effects of the present invention are as follows: the present invention is an atomizer component defect detection system based on machine vision and VIT-YOLO, which first locates the atomizer component based on a pre-built positioning model. After the positioning is completed, the pre-built defect detection model is used to perform defect detection and positioning on the single component image. If a defect exists, the grayscale information and geometric information of the defect feature are extracted, and finally the grayscale information and geometric information of the defect feature are used for analysis and verification to complete the defect detection of the atomizer component. The present application can effectively assist in the positioning of the front-end process. The use of the defect detection system in the present application can effectively help related companies to check for omissions and fill in the gaps in the front-end process, and can avoid the misleading caused by the human eye sense during manual inspection to a great extent, while further reducing labor costs and greatly improving product production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:

[0048] Figure 1 This is an application scenario diagram of the atomizer component defect detection system based on machine vision and VIT-YOLO in this application;

[0049] Figure 2 This is a structural diagram of an atomizer component defect detection system based on machine vision and VIT-YOLO shown in one embodiment of the present application;

[0050] Figure 3This is a flowchart of initialization configuration in an embodiment of the present application;

[0051] Figure 4 This is a pre-processing flow chart in one embodiment of the present application;

[0052] Figure 5 This is a schematic diagram of a component positioning process in an embodiment of the present application;

[0053] Figure 6 Schematic diagram of the foreign body detection process in one embodiment of the present application;

[0054] Figure 7 This is a schematic diagram of a feature analysis process in one embodiment of the present application;

[0055] Figure 8 This is a specific flow chart of feature analysis in one embodiment of the present application;

[0056] Figure 9 This is a schematic diagram of the result analysis process in one embodiment of the present application;

[0057] Figure 10 This is a schematic diagram of the specific analysis process in the embodiments of this application;

[0058] Figure 11 This is a topological diagram of the result output module in the embodiment of this application. DETAILED DESCRIPTION

[0059] The following describes the embodiments of the present invention through specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments. The details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other unless they conflict.

[0060] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. Therefore, the drawings only show the layers related to the present invention and are not drawn according to the number, shape and size of the layers in actual implementation. In actual implementation, the type, quantity and proportion of each layer can be changed arbitrarily, and the layer layout type may also be more complicated.

[0061] In the following description, numerous details are set forth to provide a more thorough explanation of the embodiments of the present invention; however, it is apparent to one skilled in the art that the embodiments of the present invention may be practiced without these specific details.

[0062] Figure 1This is the application scenario diagram of the atomizer component defect detection system based on machine vision and VIT-YOLO in this application, such as Figure 1 , use an industrial camera to take multi-angle group pictures of the atomizer product to obtain the corresponding grayscale image group; transmit the image to the industrial computer and forward it to the cloud server for defect detection; obtain the product inspection results and push the results to the industrial computer and subsequent data processing center.

[0063] Figure 2 FIG. 1 is a structural diagram of an atomizer component defect detection system based on machine vision and VIT-YOLO shown in one embodiment of the present application. Figure 2 As shown, the atomizer component defect detection system based on machine vision and VIT-YOLO in this embodiment includes:

[0064] A. Configuration module: Responsible for initializing the detection system, including reading configuration files, loading algorithms and system parameters Pars, loading multi-angle images captured by the camera (Process Image), and loading corresponding detection and classification models.

[0065] B. A preprocessing module, configured to obtain original images obtained by photographing the atomizer from multiple angles, and perform preprocessing and feature fusion on the original images to obtain preprocessed images and feature images, wherein the original images include front images, back images, and side images;

[0066] C. A component positioning module, configured to locate the feature image based on a pre-built positioning model to obtain preliminary positioning information of the target component of the atomizer; and to perform quantity and size verification on the preliminary positioning information to obtain positioning information when the quantity and size verification pass;

[0067] D. A foreign body detection module, configured to crop the pre-processed image based on the positioning information to obtain a single object image; locate defects in the single object image based on a pre-built defect detection model to obtain a defect detection result; and, when a defect detection frame exists in the defect detection result, extract grayscale information and geometric information within the defect detection frame;

[0068] E. a feature analysis module, configured to analyze the grayscale information and geometric information within the defect detection frame to obtain a defect analysis and determination result;

[0069] F. Feature analysis module: Use the system parameter Pars to analyze and determine each Feature feature information and encapsulate it into Defect.

[0070] G. Result analysis and generation module: Analyze and output the defects in E.

[0071] Figure 3 This is a flowchart of the initialization configuration in an embodiment of the present application, such as Figure 3 As shown, the initial call configuration process is as follows:

[0072] A1. Read the configuration file. The configuration table consists of three parts. The first part is the system preset configuration, which includes size parameters, system setting parameters, etc. The second part is Meta information, which includes the location information of the defect and detection parameters. The detection parameters include the color and geometric information of the feature, such as the minimum detection area. , the shortest detection size , detection density range , detect the aspect ratio range , detect the roundness interval , detection circularity interval , detect the gray value interval , detect the grayscale variance interval , detect the gradient interval ...etc.; its 3D model configuration parameters include the confidence threshold for each object detected by the model and the model path, etc.

[0073] A2. Load multi-angle images taken by the camera, including front image, back image and side image;

[0074] A3. Load the relevant parameters in A1, including the atomizer size parameters, such as the longest diameter range : Shortest diameter interval , the maximum tolerance threshold of the lens's major and minor axes , lens aspect ratio wait.

[0075] A3, image object initialization, according to A1 and A3 data, each image is packaged to form image data , which contains the image data and the image ID.

[0076] A5. Initialize the test results. Initialize the test results of each part, mainly including positioning test results. , foreign body detection results , comprehensive results of test objects wait.

[0077] Figure 4 This is a pre-processing flow chart in an embodiment of the present application, such as Figure 4 As shown, preprocessing is responsible for Secondary processing is performed in the image, including removing packaging text and image fusion to generate corresponding feature images. Etc. The specific process is as follows:

[0078] B1. Converting the format of the original image to obtain a target format image;

[0079] First load, decode, read the camera image data, and convert it from RAW to RGB format to obtain the target format image ;

[0080] B2. transforming the target format image into a grayscale space to obtain a grayscale image;

[0081] Then Transform to grayscale space to obtain grayscale image and deposit .

[0082] B3, performing Gaussian filtering on the grayscale image to obtain a preprocessed image;

[0083] For grayscale images Verify Gaussian filtering is performed to obtain , the purpose is to reduce the interference caused by environmental noise.

[0084] B4, performing text elimination on the filtered image to obtain an intermediate image;

[0085] In this application, surface text is extracted using binarization and other methods, and common machine vision techniques such as morphological operations and contour extraction are performed to obtain text bounding box information, and the text is then eliminated based on this information.

[0086] B5. Fusing the intermediate image of the front image, the intermediate image of the back image, and the intermediate image of the side image into a feature image having three-channel features.

[0087] The front, back and sides after processing Fused into three-channel feature images .

[0088] Figure 5 This is a schematic diagram of the component positioning process in an embodiment of the present application, such as Figure 5 As shown, by using the YOLO model loaded in A, Processing is performed, and the bounding box information obtained from the inference result is combined with the Pars in A to detect the object size, the presence of obvious defects, etc.

[0089] Furthermore, this article uses the YOLOV5 model to perform the object detection process of the atomizer, the details are as follows:

[0090] C1. Scaling the feature image to obtain a positioning image;

[0091] For the feature image extracted in the previous article , this application reconstructs it and recompresses the image to 512x512x3 to obtain the positioning image .

[0092] C2. Input the positioning image into the pre-built positioning model for reasoning to obtain the bounding box information , wherein the bounding box information includes the coordinates of the center of the bounding box, the width and height of the bounding box, the corresponding component classification label of the bounding box, and the confidence level;

[0093]

[0094] in, Represents the center coordinates of the bounding box, Indicates the width of the bounding box, Indicates the height of the bounding box, Indicates the corresponding component classification label of the bounding box, Indicates the confidence of the classification label.

[0095] C3, based on the non-maximum suppression algorithm, the bounding box information Perform fusion to obtain the final detection frame information ;

[0096] Non-Maximum Suppression (NMS) is a technique used to remove redundant detection boxes in object detection tasks. In object detection, the model may generate multiple overlapping bounding boxes for the same object. The NMS algorithm uses a specific strategy to select the most appropriate bounding box and remove any other overlapping bounding boxes.

[0097] C4. When the final detection frame information is empty, it is determined that there is no positioning result; otherwise, the final detection frame information is used as preliminary positioning information of the target component of the atomizer.

[0098] like Empty, set If the result is "No Product", the entire inspection process is exited and the process goes to step F. Otherwise, the quantity and size verification is continued, including:

[0099] C5, yes The detection box Info obtained in Determine the category and number of valid components. The specific determination method is as follows:

[0100] The prior knowledge is as follows:

[0101] The nebulizer components only include the mask, three-way pipe, mouthpiece, cup cover, and rope, and none of them can be missing. That is, the component number category cls_num is set to 5, so the set quantity is 5.

[0102] Comparing the number of classification labels in the preliminary positioning information with the set number, and when the number of classification labels is consistent with the set number and there are no duplicate classification labels, it is determined that the quantity detection has passed; when the number of classification labels is less than the set number, it is determined that there are a few defects; when the number of classification labels is greater than the set number, it is determined that there are multiple defects; when there are duplicate classification labels, it is determined that there are multiple defects;

[0103] From prior knowledge, There are only 5 categories as above and detection failure, a total of 6 items, among which "detection failure" means that the model loaded in A did not detect any of the above 5 components;

[0104] like All detected The quantity is not , it is considered that the positioning is abnormal. Among them, less than set up For "few pieces", if it is greater than set up is "multiple pieces" and exits to step F;

[0105] like Detected in If there are any multiple identical results in the , it means that there are duplicate categories. Set is "Multiple Items" and exits to step F.

[0106] C6. The maximum width and height of the bounding box in the preliminary positioning information and minimum value Comparing the initial positioning information with the detected size range respectively, and comparing the confidence level in the initial positioning information with the confidence threshold;

[0107] The maximum width and height of the bounding box and minimum value When both fall within the detected size range and the confidence level is greater than the confidence threshold, the size verification is determined to have passed; when the maximum or minimum length and width of the bounding box does not fall within the detected size range, the size is determined to be inconsistent; when the confidence level is not greater than the confidence threshold, the positioning is determined to have failed.

[0108] like and are all smaller than the minimum detection size configured in A, then Set to "Size does not match"; if is less than the minimum detection confidence level configured in A, then Set to "positioning failed", the judgment expression is as follows:

[0109]

[0110] C7, finally use the detection box information in C3 right Crop to get the relevant positioning image , and perform mask processing to generate object positioning image .

[0111] Figure 6 This is a schematic diagram of the foreign body detection process in an embodiment of the present application. Figure 6 As shown, if the positioning module in C passes, the object positioning image ROI Image obtained in C is used to crop the Process Image to obtain a single object image . Then for a single object image The self-developed VIT-YOLO model is used to locate and identify defects. If defects exist, the feature is stored and calculated for each defect. The feature contains geometric and color information, such as length, width, roundness, average grayscale, maximum grayscale, etc. The specific process is as follows:

[0112] D1. Initialize related storage space, such as model result array . And use the object positioning image generated in C Crop the image to get a single object image , and Convert the image to 768x768 size;

[0113] D2. Use the self-developed ViT-YOLO model to perform model inference on the Clip Image generated in D1 and obtain the inference result res, as follows:

[0114] The model described in this article is our self-developed ViT-YOLO. By combining the Transformer and our self-developed YoloV7SS model, it is a hybrid model that uses both convolution and self-attention mechanisms.

[0115] The model architecture is briefly described as follows: We chose the MHSA-Darknet backbone network, the BiFPN path aggregation neck network, and the anchor-based YOLOv7 detection head as the architecture of ViT-YOLO.

[0116] The YoloV7SS model is our optimized YOLOv7 model, which can realize the output of the rotating bounding box, that is, is a 7-dimensional vector, as follows:

[0117] above s contains the object rotation rectangle information , confidence information , and detection category information

[0118]

[0119] Where, is the center coordinate of the rotation bounding box, is the width of the rotation bounding box, is the height of the rotated bounding box, is the defect type label, is the confidence of the defect type label.

[0120] Additionally, the model architecture uses the Adam optimization algorithm, which combines the advantages of Adagrad and Adadelta. It not only adaptively adjusts the learning rate but also effectively estimates the first- and second-order moments of the parameters, enabling more accurate parameter updates. To prevent overfitting, strategies such as early stopping and regularization are employed.

[0121] During the training process, when the learning rate was set to 0.001, the batch size was set to 32, and the number of iterations was set to 300, the model achieved good performance on the test set, with the detection accuracy reaching more than 98%, and the recall rate and F1 value were also at a high level.

[0122] D3. Combined with the relevant parameters configured in A, the res generated in D2 is filtered to find the detection boxes that meet the basic size and confidence level, and stored in modelRes. This is similar to step C5 and will not be repeated here.

[0123] D4. If modelRes is empty in D3, it means that the model has not detected any foreign matter. Set ModelResult to "good product" and exit to F. Otherwise, go to D5.

[0124] D5. The model output in this application uses a rotated bounding box. This is because the YOLO network has a rotated box positioning function, which allows for more accurate positioning of oblique, slender objects. Therefore, before extracting the grayscale and geometric information within the defect detection frame, the rotated bounding box must be converted to a regular rectangle before feature extraction.

[0125] Each detection box information in modelRes is processed in a loop, and the corresponding rectangular image is intercepted as an example image for object grayscale information extraction. The rotated rectangle is converted to a rectangular image as follows:

[0126] D5-1. Convert the angle to radians and set the rotation matrix based on the radians ;

[0127] D5-2. Define the initial coordinates of the four vertices of the rotation bounding box , ,in, are the initial coordinates of the four vertices of the rotation bounding box respectively;

[0128] D5-3. Rotate and offset the rotation bounding box based on the rotation matrix and the center coordinates of the rotation bounding box to obtain the coordinates of the four vertices after rotation. , wherein the mathematical expression for rotating and offsetting the rotation bounding box is:

[0129]

[0130] Where, is the center coordinate of the rotation bounding box;

[0131] D5-4. From the coordinates of the four vertices after rotation Confirmed Maximum value in direction 、 Minimum value of direction 、 Maximum value in direction as well as Minimum value of direction , get the lower left corner point and the upper right corner ;

[0132] D5-5, based on the lower left corner point and the upper right corner point constructing a regular rectangle, and extracting a feature area of ​​the preprocessed image based on the regular rectangle;

[0133] In this application, after constructing a regular rectangle, the area in the preprocessed image is cropped based on the regular rectangle, and feature extraction is performed to crop the area. The mathematical expression is:

[0134]

[0135] D5-6. Extracting grayscale information and geometric information of the feature area.

[0136] Among them, the grayscale information includes the average grayscale value, grayscale value variance, gradient average, minimum grayscale value, maximum grayscale value and grayscale value at the center position, and the geometric information includes contour size, contour density, aspect ratio, roundness and circularity.

[0137] Figure 7 FIG. 1 is a flow chart of feature analysis in one embodiment of the present application. Figure 7 As shown, Pars is used to analyze and determine each feature information and package it into a Defect. The feature analysis module process is as follows: first, the current corresponding configuration is loaded from the meta information obtained from A, then each feature in the feature set in D is filtered, and finally the corresponding result is packaged into a Defect. The specific process is as follows:

[0138] E1. Load the defect attribute configuration information of the current inspection area;

[0139] The defect attribute configuration information loaded first contains the defect attributes, including the names of various defects and their corresponding geometry and color information.

[0140] E2, Feature Analysis: Filter and analyze features using the defect attribute configuration information obtained in E1, encapsulating the results into a defect (Defect). Defect is a higher-level encapsulation of Feature, consisting of both defect attributes and features. Attributes store the analyzed classification results of the feature, while Features contain the feature geometry and color information mentioned in A11. Figure 8 This is a specific flow chart of feature analysis in one embodiment of the present application, such as Figure 8 As shown in Figure 2, the feature analysis process includes:

[0141] E21. Assign the information loaded in E1 to a new variable config_data, and check whether the defect attribute configuration information config_data of the current detection area is empty. If the defect attribute configuration information config_data of the current detection area is empty, determine that the feature of the current detection area has an interference feature. If the defect attribute configuration information config_data of the current detection area is not empty, proceed to step E22.

[0142] E22, divide the defect attribute configuration information of the current detection area into two categories: defect verification information nosie and interference verification information defect, and store them in two storage spaces respectively, namely as well as .

[0143] E23. Verify the grayscale information and geometric information within the defect detection frame based on the interference verification information. If the interference verification passes, determine that interference features exist in the current detection area. Otherwise, proceed to step E24. In this application, filtering and classification are performed by configuring the analysis function analyze_function. If the conditions are met, Defect.attribute.name = noise, the result is packaged as Defect, and the detection ends; if not, proceed to E24.

[0144] E24. If the interference check fails, the grayscale information and geometric information within the defect detection frame are verified based on the defect check information. If the defect check passes, it is determined that defect features exist in the current detection area. This application performs filtering and classification by configuring the analysis function analyze_function. If the conditions are met, Defect.attribute.name = defect, the result is encapsulated as Defect, and the detection ends. If not, Defect.attribute.name = other, and the detection ends.

[0145] In this application, the analysis logic of the analysis function analyze_function is as follows, except that the corresponding defect attribute parameters are inconsistent when performing interference analysis and defect analysis respectively.

[0146]

[0147]

[0148]

[0149] In the above formula, Indicates the Grayscale information of the detection area, including the average grayscale value , gray value variance , gradient average , minimum grayscale value , maximum grayscale value and the grayscale value at the center . No. Grayscale information of the detection area, including outline size , contour density , aspect ratio , roundness and circularity .

[0150] in, Indicates that the contour is larger than the preset minimum detection size;

[0151] Indicates that the contour density is within the preset detection density range;

[0152] Indicates that the aspect ratio is within the preset detection aspect ratio range;

[0153] Indicates that the roundness is within the preset detection roundness range;

[0154] Indicates that the circularity is within the preset detection circularity range;

[0155] Indicates that the average grayscale value is greater than the preset detection grayscale value range;

[0156] Indicates that the gray value variance is within the preset detection variance range;

[0157] The average gradient value is within the preset detection gradient range;

[0158] Indicates that the minimum grayscale value is within the preset minimum grayscale range;

[0159] Indicates that the maximum grayscale value is less than the preset maximum grayscale range;

[0160] Indicates that the grayscale value of the center position is within the grayscale value interval of the center position;

[0161] It means that if all the above conditions are met, the verification is passed and the geometric information and grayscale information of the corresponding detection area are saved; otherwise, the verification fails and the geometric information and grayscale information of the corresponding detection area are discarded.

[0162] Figure 9 FIG. 1 is a flow chart of result analysis in one embodiment of the present application. Figure 9 As shown, all defects obtained in E are combined into the defect pool Defects. According to the priority configuration, the data in Defects is analyzed, and the defect information with the highest priority is assigned to the current image result imageResult. Figure 10 This is a schematic diagram of the specific analysis process in the embodiments of this application, such as Figure 10 The specific process is as follows:

[0163] F1. Initialize the result data in ObjectResult to "NG";

[0164] F2. Load the positioning result LocateResult of the component positioning module. If the component positioning module fails the quantity or size verification (i.e., the positioning result is not "pass"), mark the preliminary positioning information in the original image, determine the atomizer as defective, and synchronize the defect corresponding to the preliminary positioning information into the final inspection result.

[0165] Specifically, Assign the same defect code to synchronize the defect results, convert the relevant annotation data to the original image size, and then copy it to the data structure corresponding to ObjectResult.

[0166] F3. When the component positioning module passes the quantity or size verification, extract the target defect with the highest priority from the defect detection results obtained by the foreign object detection module, mark the target defect in the original image, determine the atomizer as a defective product, and synchronize the target defect into the final inspection result;

[0167] Specifically, the highest priority defect code is assigned to , and convert the relevant annotation data to the original image size, and then copy it to the data structure corresponding to ModelResult.

[0168] F4. Output the corresponding detection information and marking information according to the corresponding detection results, and end this module.

[0169] Figure 11 This is a topological diagram of the result output module in the embodiment of the present application, such as Figure 11 The present application also provides a dynamic adjustment and output module for outputting the original image marked with defects and the final detection results to the target object, specifically including:

[0170] G1. Combine the system parameters in A and package the output results. If you need to draw an image to mark the location of the defect, then according to the relevant position coordinate information of Defect, such as .

[0171] G2. Output the product detection result Res corresponding to the image obtained by the detection step F through the relevant API, and present the result to the corresponding front-end display interface and related data processing center, such as physical software, Web, mobile APP, etc. The topology diagram is as follows Figure 11 shown.

[0172] G3: If the displayed results do not match expectations, you can dynamically adjust the parameters through the relevant software and view the current test results in real time. Once the expected results are achieved, the parameters can be saved and applied to the test system. Return to G2 to continuously monitor the test results and system operating status, achieving closed-loop adjustment.

[0173] The present invention provides an atomizer component defect detection system based on machine vision and VIT-YOLO, which first locates the atomizer component based on a pre-built positioning model. After the positioning is completed, the pre-built defect detection model is used to perform defect detection and positioning on the single component image. If a defect exists, the grayscale information and geometric information of the defect feature are extracted. Finally, the grayscale information and geometric information of the defect feature are used for analysis and verification to complete the defect detection of the atomizer component. The present application can effectively assist in the positioning of the front-end process. The use of the defect detection system in the present application can effectively help related companies to check for omissions and fill in the gaps in the front-end process, and can greatly avoid the misleading caused by the human eye sense during manual inspection, while further reducing labor costs and greatly improving product production efficiency.

[0174] This embodiment also provides an electronic terminal, including: a processor and a memory;

[0175] The memory is used to store computer programs, and the processor is used to execute the computer programs stored in the memory, so that the terminal executes any one of the methods in this embodiment.

[0176] Regarding the computer-readable storage medium in this embodiment, those skilled in the art will appreciate that all or part of the steps in the aforementioned method embodiments can be implemented using hardware associated with the computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When executed, the program performs the steps in the aforementioned method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0177] The electronic terminal provided in this embodiment includes a processor, a memory, a transceiver and a communication interface. The memory and the communication interface are connected to the processor and the transceiver and complete communication with each other. The memory is used to store computer programs, the communication interface is used for communication, and the processor and the transceiver are used to run computer programs so that the electronic terminal executes the various steps of the above method.

[0178] In this embodiment, the memory may include a random access memory (RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage.

[0179] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0180] In the above embodiments, although the present invention has been described in conjunction with specific embodiments of the present invention, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art based on the foregoing description. The embodiments of the present invention are intended to cover all such replacements, modifications and variations that fall within the broad scope of the appended claims.

[0181] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the present invention. Anyone skilled in the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by one of ordinary skill in the art without departing from the spirit and technical principles disclosed herein are intended to be covered by the claims of the present invention.

Claims

1. A system for detecting defects in atomizer components based on machine vision and VIT-YOLO, characterized in that: include: A preprocessing module is used to obtain original images obtained by shooting the atomizer from multiple angles, and perform preprocessing and feature fusion on the original images to obtain preprocessed images and feature images, wherein the original images include front images, back images, and side images; a component positioning module, configured to locate the characteristic image based on a pre-built positioning model to obtain preliminary positioning information of the target component of the atomizer; and to perform quantity and size verification on the preliminary positioning information to obtain positioning information when the quantity and size verification pass; a foreign body detection module configured to crop the preprocessed image based on the positioning information to obtain a single object image; locate defects in the single object image based on a pre-built defect detection model to obtain a defect detection result; and, when a defect detection frame exists in the defect detection result, extract grayscale information and geometric information within the defect detection frame; The feature analysis module is used to analyze the grayscale information and geometric information within the defect detection frame to obtain a defect analysis and judgment result.

2. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, wherein The characteristics are, The original image is preprocessed and feature fused to obtain a preprocessed image and a feature image, including: Performing format conversion on the original image to obtain a target format image; Converting the target format image into a grayscale space to obtain a grayscale image; Performing Gaussian filtering on the grayscale image to obtain a preprocessed image; performing text elimination on the filtered image to obtain an intermediate image; The intermediate image of the front image, the intermediate image of the back image, and the intermediate image of the side image are fused into a feature image with three-channel features.

3. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, characterized in that: The characteristic image is positioned based on a pre-built positioning model to obtain preliminary positioning information of the target component of the atomizer, including: Scaling the feature image to obtain a positioning image; Input the positioning image into a pre-built positioning model for inference to obtain bounding box information, wherein the bounding box information includes the center coordinates of the bounding box, the width and height of the bounding box, the corresponding component classification label of the bounding box, and the confidence level; The bounding box information is fused based on the non-maximum suppression algorithm to obtain the final detection box information; When the final detection frame information is empty, it is determined that there is no positioning result; otherwise, the final detection frame information is used as preliminary positioning information of the target component of the atomizer.

4. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, characterized in that: Performing quantity and size verification on the preliminary positioning information, including: Comparing the number of classification labels in the preliminary positioning information with the set number, and when the number of classification labels is consistent with the set number and there are no duplicate classification labels, it is determined that the quantity detection has passed; when the number of classification labels is less than the set number, it is determined that there are a few defects; when the number of classification labels is greater than the set number, it is determined that there are multiple defects; when there are duplicate classification labels, it is determined that there are multiple defects; The maximum and minimum values ​​of the width and height of the bounding box in the preliminary positioning information are respectively compared with the detected size range, and the confidence in the preliminary positioning information is compared with the confidence threshold; when the maximum and minimum values ​​of the width and height of the bounding box both fall into the detected size range and the confidence is greater than the confidence threshold, it is determined that the size verification has passed; when the maximum or minimum value of the length and width of the bounding box does not fall into the detected size range, it is determined that the size does not match; when the confidence is not greater than the confidence threshold, it is determined that the positioning has failed.

5. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, characterized in that: The defect detection result includes the center coordinates of the rotated bounding box, the width and height information of the rotated bounding box, the rotation angle of the rotated bounding box, the defect type label, and the confidence level.

6. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 5, characterized in that: Extracting grayscale information and geometric information within the defect detection frame includes: Sets the rotation matrix based on the rotation angle ; Define the initial coordinates of the four vertices of the rotation bounding box , ,in, are the initial coordinates of the four vertices of the rotation bounding box respectively; The rotation matrix and the center coordinates of the rotation bounding box are used to rotate and offset the rotation bounding box to obtain the coordinates of the four vertices after rotation. , wherein the mathematical expression for rotating and offsetting the rotation bounding box is: Where, is the center coordinate of the rotation bounding box; From the coordinates of the four vertices after rotation Confirmed Maximum value in direction 、 Minimum value of direction 、 Maximum value in direction as well as Minimum value of direction , get the lower left corner point and the upper right corner ; Based on the lower left corner point and the upper right corner point constructing a regular rectangle, and extracting a feature area of ​​the preprocessed image based on the regular rectangle; The grayscale information and geometric information of the feature area are extracted.

7. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, characterized in that: Analyze the grayscale information and geometric information within the defect detection frame to obtain a defect analysis result, including: Load the defect attribute configuration information of the current detection area; When the defect attribute configuration information of the current detection area is empty, determining that the current detection area has interference features; when the defect attribute configuration information of the current detection area is not empty, dividing the defect attribute configuration information of the current detection area into defect verification information and interference verification information; The grayscale information and geometric information within the defect detection frame are verified based on the interference verification information. When the interference verification is passed, it is determined that interference features exist in the current detection area. When the interference verification is not passed, the grayscale information and geometric information within the defect detection frame are verified based on the defect verification information. When the defect verification is passed, it is determined that defect features exist in the current detection area.

8. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 7, characterized in that: The interference verification information includes first reference value ranges of multiple grayscale parameters and first reference value ranges of multiple geometric parameters, wherein the grayscale information and geometric information within the defect detection frame are verified based on the interference verification information, including: When the value of each grayscale parameter in the grayscale information falls into the first reference value range of the corresponding grayscale parameter, and the value of each geometric parameter in the grayscale information falls into the first reference value range of the corresponding geometric parameter, it is determined that the interference check has passed; otherwise, it is determined that the interference check has not passed.

9. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 7, characterized in that: The defect verification information includes second reference value ranges of multiple grayscale parameters and second reference value ranges of multiple geometric parameters, wherein verifying the grayscale information and geometric information within the defect detection frame based on the defect verification information includes: When the value of each grayscale parameter in the grayscale information falls into the second reference value range of the corresponding grayscale parameter, and the value of each geometric parameter in the grayscale information falls into the second reference value range of the corresponding geometric parameter, it is determined that the defect check has passed; otherwise, it is determined that the defect check has failed.

10. The atomizer component defect detection system based on machine vision and VIT-YOLO according to claim 1, characterized in that: Also includes: a result analysis and generation module, configured to, when the component positioning module fails the quantity or size verification, mark the preliminary positioning information in the original image, determine the atomizer as defective, and synchronize the defects corresponding to the preliminary positioning information into the final inspection result; When the component positioning module passes the quantity or size verification, extract the target defect with the highest priority from the defect detection results obtained by the foreign matter detection module, mark the target defect in the original image, determine the atomizer as a defective product, and synchronize the target defect into the final detection result; The dynamic adjustment and output module is used to output the original image with defects and the final detection results to the target object.

Images