Industrial part surface defect detection and counting method under small sample

Abstract

The invention discloses a method for detecting and counting surface defects of industrial parts under small samples. Firstly, an image unit is used for collecting an image in the falling process of the industrial part; secondly, performing sample labeling on the acquired image, and performing supplementary data set enhancement; secondly, establishing an industrial part defect detection and counting model, obtaining a VMP-YOLOX model through fusion of ViT and CNN and improvement of a target detection head in the defect detection process, and performing training by adopting an enhanced data set; and finally, deploying the trained VMP-YOLOX model in an industrial production field so as to complete the detection and counting of the surface defects of the industrial parts. The training data set is a small sample, so that the workload required for collecting a large number of data sets is reduced; only a small number of positive samples are labeled, so that the workload of labeling is reduced; in addition, the detection precision is improved by adopting a ViT and CNN hybrid model.

Description

Technical Field

[0001] The invention belongs to the field of target detection, and in particular relates to a method for detecting and counting surface defects of industrial parts under small samples. Background Art

[0002] At present, with the continuous and constant upgrading of consumption levels, the production of high-quality and high-reliability industrial parts has become an unstoppable development trend. Under such a background, the role of surface defect detection is crucial. In terms of reducing production costs, it can accurately identify defective products and avoid unnecessary waste of resources. In terms of improving product quality, it can timely discover and solve problems to ensure that the quality of products meets high standards; in terms of improving production efficiency, efficient detection processes can reduce production pauses and rework, making the entire production process smoother and more efficient.

[0003] At present, there are several main methods for detecting surface defects of industrial parts:

[0004] 1. By combining the effects of magnetic powder and magnetic field, using magnetic field sensors to collect data, and removing noise through image preprocessing steps, enhancing image quality, and helping to analyze and identify surface defects of parts. This detection method is relatively cumbersome to operate and is affected by noise.

[0005] 2. The depressions on the edge of the part surface can be effectively detected by using the convex hull detection and contour detection methods. This detection method is only suitable for detecting edge defects.

[0006] 3. Use video stream analysis to analyze the time-based sequence images of parts collected synchronously by multiple cameras in real time, form a two-dimensional relationship diagram with the positions of multiple cameras, and determine the location of defects. This detection method has high requirements for the number of images and lighting.

[0007] Existing methods for detecting surface defects of industrial parts require a large number of samples to train the model. The types of surface defects of industrial parts are diverse, and a large amount of manual labeling work is required. In addition, the existing detection methods have low accuracy, and the model training and recognition speeds are low, making it impossible to complete the defect counting. Summary of the invention

[0008] Aiming at the deficiencies of the prior art, the present invention proposes a method for detecting and counting surface defects of industrial parts under small samples.

[0009] In a first aspect, the present invention provides a method for detecting and counting surface defects of industrial parts under a small sample, comprising the following steps:

[0010] Step 1. Using an image unit to collect images of industrial parts in the falling process;

[0011] Step 2. Annotate the collected images and perform supplementary data set enhancement;

[0012] Step 3. Establish an industrial parts defect detection and counting model. In the process of defect detection, the VMP-YOLOX model is obtained by fusing ViT and CNN and improving the target detection head, and the enhanced dataset is used for training.

[0013] Step 4. Deploy the trained VMP-YOLOX model at the industrial production site to complete the detection and counting of surface defects of industrial parts.

[0014] In a second aspect, the present invention provides an electronic device comprising a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction implements the steps of the method described above when executed by the processor.

[0015] In a third aspect, the present invention provides a readable storage medium, on which a program or instruction is stored, and the program or instruction, when executed by a processor, implements the steps of the method described above.

[0016] The beneficial effects of the present invention are:

[0017] ① The training data set is a small sample, which reduces the workload required to collect a large data set;

[0018] ② It is difficult to exhaust all the types of surface defects of industrial parts. In this invention, only a small number of positive samples are annotated, which reduces the workload of annotation;

[0019] ③The detection accuracy is improved by using the ViT and CNN hybrid model. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is an overall flow chart of an embodiment of the present application;

[0021] Figure 2 This is a partial data set diagram of industrial parts;

[0022] Figure 3 It is the diagram of the positive feedback mechanism enhanced PFMA model;

[0023] Figure 4 This is the improved YOLOX model diagram;

[0024] Figure 5 This is a diagram of the ViT-moblievit-pooling model.

[0025] Figure 6 This is the Dyhead structure diagram. DETAILED DESCRIPTION

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0027] like Figure 1 As shown, a method for detecting and counting surface defects of industrial parts under a small sample size provided in an embodiment of the present application is specifically:

[0028] Step 1: Assemble the hardware equipment for industrial parts surface defect detection.

[0029] At the beginning of the hardware, industrial parts are unloaded through a PLC controller, and a linear array camera is installed under the unloading funnel. The linear array camera constitutes the image unit, which captures the process image of the free fall of the parts, so this image is used as the input image of the industrial parts defect detection and counting model. This application believes that if industrial parts are detected on a conveyor belt, there will be adhesion and overlap, but when they fall, they are free fall, and the above situation rarely exists, which can improve the detection accuracy. The partial data set diagram obtained is as follows: Figure 2 As shown, the industrial part in the figure is a gasket, but the present application can also be directed to other industrial parts, including but not limited to screws, small copper tubes, etc.

[0030] Step 2: Annotate the samples of the images collected in step 1. Due to the limited number of defect samples, a positive-feedback mechanism augmentation (PFMA) method is proposed to supplement the dataset enhancement.

[0031] In the process of making image samples, due to the small number of actual defect samples, it is easy to cause imbalance between positive and negative samples. A positive feedback mechanism to enhance the PFMA method (such as Figure 3 As shown), the positive feedback mechanism enhanced PFMA method is specifically:

[0032] The location of industrial parts is manually marked, and the coordinate position and the positive sample image are input into a model with pre-trained parameters as the resnet50 backbone network. Irregular noise is added to the middle feature layer of the model and in the scaled coordinate area to interfere with the positive sample features. The interference of the noise is then fed back to the original image to generate an image of industrial parts with defects, which serves as a supplementary enhancement of the sample data set.

[0033] Step 3: Establish an industrial parts defect detection and counting model. In the defect detection process, a new VMP-YOLOX model is designed by integrating ViT and CNN and improving the target detection head.

[0034] Based on the YOLOX model (such as Figure 4The binary classification target detection is performed based on the feature map 2, 3, and 4. The ViT-mobile-pooling module is inserted between the feature map and the feature pyramid to perform feature recalibration respectively, and then the features are fused and used as the input of the subsequent network. A dynamic detection head is introduced between the feature pyramid and the inspection head to significantly improve the representation ability of the target detection head. The network weights are obtained through pre-training.

[0035] MobileViT is a model for use on mobile devices, and the detection and counting of industrial parts in this application also need to be deployed on an industrial computer, which requires a lightweight model. Therefore, this embodiment is improved on the basis of the MobileViT model in combination with the characteristics of industrial parts data. MobileViT uses a multi-layer Transformer encoder to process the segmented sequence, but the application of multi-layer Transformers in actual detection will increase the number of model parameters and reduce the detection speed, and too many layers of Transformers will cause the extracted features to be distorted; the improved ViT-moblievit-pooling in this embodiment only uses one layer of Transformer. At the same time, the token mixer module in the Transformer module is not the more complex the extracted features, the better. In order to further reduce the parameters and computational complexity of the Transformer, the multi-head self-attention structure in the Transformer is replaced with a parameter-free spatial pooling operator as a token mixer module. This replacement hardly reduces the detection accuracy in the experiment.

[0036] Further, the ViT-mobilevit-pooling module of this embodiment is as follows Figure 5 As shown, the module is designed to effectively extract global representations from input feature maps.

[0037] The ViT-moblievit-pooling module starts with an input feature map F of dimensions H×W×C, where H is the height, W is the width, and C is the number of channels. Local features are first extracted through two parallel convolution operations: an n×n convolution (usually with a kernel size of 3×3 or larger), and a 1×1 convolution. These operations are used to capture local information in the input data. The local features are then passed through an “Unfold” operation, which reshapes the feature map into a sequential form in preparation for global processing by the Transformer module. In the “Unfold” step, the H×W-dimensional spatial feature map is converted into a sequence of N×P, where N is the sequence length and P is the flattened feature dimension of each patch. These sequence features are then passed to the Transformer module, which learns global feature dependencies through a self-attention mechanism. The output of the Transformer is an N×P sequence that contains a global representation of the input image. The "Fold" step after the Transformer module is the inverse operation of "Unfold", which reorganizes the output of the Transformer into a feature map of the original dimension for fusion with the local feature map. In the "Fusion" step, the global representation and the expanded local features are fused through a 1×1 convolution operation to expand the number of channels to strengthen the feature representation of the model. This fusion process usually includes addition and activation function operations to enhance nonlinear representation.

[0038] Furthermore, the Transformer module includes normalization, pooling, and channel MLP operations, which are designed to further extract and refine feature representation. The normalization step ensures the consistency and stability of the feature values, while the pooling step reduces the dimension while retaining key information. Channel MLP, as the last operation, extracts complex feature dependencies across channels through a multi-layer perceptron.

[0039] In the architecture of ViTMobile, the Pooling module plays a vital role. Its purpose is to convert local feature representations into representations with more global semantics. The original intention of the design of the entire Pooling module is to improve the model's ability to understand the global context by combining local and global information processing strategies without significantly increasing the computational burden.

[0040] In this embodiment, spatial pooling is used to replace the query (Q), key (K), and value (V) operations in self-attention. This method greatly reduces the amount of computation while maintaining the effectiveness of the model.

[0041] Self-attention is shown in formula (1):

[0042]

[0043] The basic unit of PoolFormer is PoolFormer Block, which mainly includes the following parts: Spatial Pooling Module, which uses spatial pooling (such as average pooling or maximum pooling) to capture local spatial features. Spatial pooling achieves feature compression and abstraction by reducing the spatial dimension of the feature map while retaining important spatial information. The pooled features are processed by a multi-layer perceptron (MLP). MLP consists of two fully connected layers with a nonlinear activation function (such as ReLU) in the middle. The MLP module is used to increase the nonlinear representation ability of the model. Residual Connection, PoolFormer Block uses residual connection to facilitate the training of deep networks. Residual connection can help gradients flow directly to the previous layer, alleviating the problem of gradient disappearance.

[0044] Perform spatial pooling operation on the input feature X:

[0045] X pool =Pool(X) (2)

[0046] Among them, pool represents the spatial pooling operation (average pooling), X pool is the pooled feature. The pooled feature is processed by a multi-layer perceptron (MLP):

[0047] X mlp =MLP(X pool ) (3)

[0048] Among them, MLP represents the multi-layer perceptron operation, which consists of two fully connected layers and a nonlinear activation function. mlp is the feature after MLP processing. Finally, the output of MLP is normalized through the layer (Norm) and added to the input feature X as a residual connection to obtain the final output of PoolFormer Block:

[0049] X out =X+Norm(X mlp ) (4)

[0050] Among them, Norm represents the layer normalization operation, X out It is the final output of PoolFormer Block.

[0051] Through this design, PoolFormer achieves effective feature extraction while maintaining the computational efficiency of the model, making it suitable for resource-constrained environments and application scenarios that require efficient computing.

[0052] Furthermore, the dynamic detection head (Dynamic Head): its structure mainly uses the attention mechanism to strengthen the following three perception capabilities of the model, thereby improving the performance of the detector. Figure 6 It is the structure of DyHead block. The role of scale-aware attention is to improve the scale perception ability of the detector, so that the receptive field can be dynamically adjusted according to the target size; spatial-aware attention mainly improves the spatial position perception ability of the detector, and the convolution kernel can be dynamically adjusted according to different inputs; task-aware attention changes the expression ability of different channels to improve the task perception ability of the detector.

[0053] Step 4. Deploy the trained VMP-YOLOX model at the industrial production site to complete the detection and counting of surface defects of industrial parts. For example: the automated inspection system on the industrial production line helps companies detect and count surface defects of parts in real time during the production process; or, in the production quality control link, use this application to inspect the quality of parts to ensure that each batch of parts produced meets the standards and reduce the subjectivity of manual inspection; or automatically detect surface defects of metals such as steel, aluminum alloys, and copper alloys to avoid product performance degradation due to defects, or in the production and quality inspection of automotive parts, especially in key areas such as autonomous driving vehicles, engine components, and body parts. Specifically:

[0054] When parts fall freely from the conveyor belt, there may be overlapping parts in the images captured by the linear scan camera, but the overlapping parts cannot be distinguished when the defects and number counting are performed by the VMP-YOLOX model. At the same time, there are repeated objects at the edges of the front and back frames of the images sent back by the linear scan camera. Single defect and number counting will not only misjudge overlapping objects, but also repeatedly count objects at the edge of the image, making the detection efficiency low.

[0055] In order to detect the problem of normal part overlap, this embodiment compares the areas of all rectangular boxes that pass the VMP-YOLOX model with the maximum area of ​​a single part manually input by the user. If the area of ​​the rectangular box is less than or equal to the maximum area of ​​a single part, it is judged that there is no overlap in the area of ​​the rectangular box; if the area of ​​the rectangular box is greater than the maximum area of ​​a single part, it is judged that there is overlap in the part. By comparing the area of ​​the rectangular box with the maximum area of ​​a single part, the number of parts with overlapping rectangular box areas is obtained, and the correct data is counted in the total number.

[0056] In view of the problem of detecting defects but counting the front and back frames, this embodiment realizes accurate counting of the front and back frames by judging the number of objects on the edge of the front and back frames. The frames of the video stream are divided into odd frames and even frames, and two parameters are set for the two frames, namely the number of top parts and the number of bottom parts. The rectangular box with a center point of 0-5 pixels from the top is taken as the top part, and the rectangular box with a center point of 0-5 pixels from the bottom is taken as the bottom part. Then the top parts and the bottom parts are overlapped and screened. If there is a rectangular box with an area greater than the maximum area of ​​a single part, it is judged that there is overlap. After splitting the overlapping parts, if the areas of the two parts still meet the conditions of the top or bottom parts, they are counted in the number of top or bottom parts, and if not, they are not counted. Compare the number of bottom parts and the number of top parts in adjacent odd and even frames. If they are equal, they are recorded once, and the upper part is recorded but not the lower part. If they are not equal, the number of multiple values ​​is taken as the final value. The reason for taking the value with a larger number as the final value is that when the gasket is cut, most of the gasket is on the even frame or odd frame, and the small part cannot be detected correctly. At this time, taking the value with a larger number as the final value will not be missed.

[0057] The embodiment of the present application further provides an electronic device including a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs include computer executable instructions for performing the following:

[0058] Step 1. Using an image unit to collect images of industrial parts in the falling process;

[0059] Step 2. Annotate the collected images and perform supplementary data set enhancement;

[0060] Step 3. Establish an industrial parts defect detection and counting model. In the process of defect detection, the VMP-YOLOX model is obtained by fusing ViT and CNN and improving the target detection head, and the enhanced dataset is used for training.

[0061] Step 4. Deploy the trained VMP-YOLOX model at the industrial production site to complete the detection and counting of surface defects of industrial parts.

[0062] The embodiment of the present application also provides a readable storage medium, which is loaded with one or more programs mentioned above and is used to execute the above computer instructions.

[0063] In summary, this application uses computer vision technology to identify and judge the surface defects of industrial parts. It can accurately and quickly identify defective parts in industrial parts, reduce misjudgments caused by lack of human experience, and effectively prevent or reduce the flow of defective products into the market, thereby reducing the occurrence of safety accidents. The industrial parts surface defect detection method based on the improved VGG model based on feature fusion and spatial channel attention mechanism can improve the recognition rate of industrial parts surface defect detection. In the case of small sample training, the detection accuracy of this application reached 95.2%, and the detection speed was 0.6s / sheet.

[0064] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting and counting surface defects of industrial parts under small sample conditions, characterized in that The method comprises the following steps: Step 1. Using an image unit to collect images of industrial parts in the falling process; Step 2. Annotate the collected images and perform supplementary data set enhancement; Step 3. Establish an industrial parts defect detection and counting model. In the process of defect detection, the VMP-YOLOX model is obtained by fusing ViT and CNN and improving the target detection head, and the enhanced dataset is used for training. Step 4. Deploy the trained VMP-YOLOX model at the industrial production site to complete the detection and counting of surface defects of industrial parts.

2. The method for detecting and counting surface defects of industrial parts under small sample conditions according to claim 1 is characterized in that: In step 2, a positive feedback mechanism is used to enhance the supplementary data set.

3. The method for detecting and counting surface defects of industrial parts under small sample conditions according to claim 2 is characterized in that: First, the coordinates of the industrial parts are manually marked, and the coordinates and positive sample images are input into a resnet50 backbone network with pre-trained parameters. Irregular noise is added to the middle feature layer of the model and at the scaled coordinate position to interfere with the positive sample features. The interference of the noise is then fed back to the original image to generate an image of industrial parts with defects, which serves as a supplementary enhancement of the sample data set.

4. The method for detecting and counting surface defects of industrial parts with a small sample size according to claim 1 is characterized in that: The industrial parts defect detection and counting model uses the YOLOX model; Inserting a ViT-mobile-pooling module between the feature map and the feature pyramid of the YOLOX model for feature recalibration; A dynamic detection head is introduced between the feature pyramid and the inspection head.

5. The method for detecting and counting surface defects of industrial parts with a small sample size according to claim 4 is characterized in that: The ViT-moblievit-pooling module is used to effectively extract global representation from the input feature map; the single-layer Transformer encoder in the ViT-moblievit-pooling module learns global feature dependencies through a self-attention mechanism.

6. The method for detecting and counting surface defects of industrial parts with a small sample size according to claim 5 is characterized in that: It also includes improving the multi-head self-attention structure in the Transformer encoder into a parameter-free spatial pooling operator as a token mixer module.

7. The method for detecting and counting surface defects of industrial parts with a small sample size according to claim 1 is characterized in that: To detect the problem of normal part overlap, the maximum area of ​​a single part manually entered by the user is used to compare the areas of all rectangular boxes that pass the VMP-YOLOX model: If the area of ​​the rectangular frame is less than or equal to the maximum area of ​​a single part, it is determined that there is no overlap in the area of ​​the rectangular frame; If the area of ​​the rectangular frame is larger than the maximum area of ​​a single part, it is judged that the parts are overlapped; By comparing the area of ​​the rectangular frame with the maximum area of ​​a single part, the number of parts that overlap with the rectangular frame area is obtained, and the correct data is counted in the total number.

8. The method for detecting and counting surface defects of industrial parts with a small sample size according to claim 1 is characterized in that: In order to solve the problem of detecting defects but counting the front and back frames, the accurate counting of the front and back frames can be achieved by judging the number of objects at the edges of the front and back frames. Specifically: The frames of the video stream are divided into odd frames and even frames, and two parameters are set for each type of frame, namely, the number of top parts and the number of bottom parts; The rectangular box whose center point is 0-5 pixels away from the top is regarded as the top part, and the rectangular box whose center point is 0-5 pixels away from the bottom is regarded as the bottom part; Perform overlapping screening on the top and bottom parts. If there is a rectangular box whose area is larger than the maximum area of ​​a single part, it is determined that there is overlap; After splitting the overlapping parts, if the areas of the two parts still meet the top or bottom part conditions, they will be counted in the number of top or bottom parts; if not, they will not be counted; Compare the number of bottom parts and the number of top parts of adjacent odd frames and even frames. If they are equal, record them once; if they are not equal, take the multiple value as the final value.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the method according to any one of claims 1 to 8.

10. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images