Workpiece identification method based on small sample training

By establishing a mutual enhancement structure and adding the MLLAttention module to the YOLOv8 network, the problem of poor generalization ability of deep learning models under small sample data is solved, and the accuracy and speed of artifact recognition are significantly improved, which is suitable for small sample artifact recognition tasks .

CN119091222APending Publication Date: 2024-12-06SHENYANG INST OF COMPUTING TECH CO LTD THE CHINESE ACAD OF SCI
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411261479.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-10
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the case of small samples, traditional deep learning models such as convolutional neural networks are difficult to learn enough features from finite samples, resulting in poor generalization ability and overfitting, especially in the case of uneven categories of data. The features of the YOLOv8 algorithm The extraction module cannot fully function, resulting in poor recognition.

Method used

By establishing a small sample data enhancement network with mutual enhancement structure, data expansion is used using data mining models and comparative learning models, and MLLAttention module is added to the YOLOv8 network to improve the network structure to deepen feature extraction capabilities and realize effective identification of small sample data and segmentation.

Benefits of technology

It significantly improves the accuracy and speed of workpiece recognition, and can display the position, category and recognition score of workpieces in real time on mobile devices, enhancing the accuracy of recognition and visual analysis capabilities.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091222A_ABST
    Figure CN119091222A_ABST
Patent Text Reader

Abstract

The invention relates to a workpiece identification method based on small sample training. The method mainly utilizes a high-pixel camera to capture the shape and size of a workpiece. Small sample data enhancement is carried out through a consistency-based mutual enhancement structure, and the speed and accuracy of the model for executing object detection and segmentation tasks on a mobile device are remarkably improved by introducing an efficient and lightweight Mamba-Like Line Attention module. Experimental results show that the original YOLOv8 is improved, and compared with the original YOLOv8 model, the model with the MLLA module is improved in the aspects of segmentation precision and recognition precision.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of computer vision, in particular to a workpiece recognition method based on small sample training. Background Art

[0002] In the production and assembly process, part model recognition is an important link. Traditional part recognition methods are easily affected by light and noise, and are not suitable for large-scale and multi-type recognition tasks. Deep learning target recognition methods can avoid the shortcomings of traditional methods, break through their performance bottlenecks, and improve the speed and accuracy of part recognition. However, assembly parts usually have the characteristics of small samples, which are difficult to learn using traditional convolutional neural networks.

[0003] Characteristics of small sample artifacts First, the number of samples is very limited, which may be only dozens or even less. In this case, it is difficult for traditional deep learning models (such as convolutional neural networks) to learn enough features from these small samples, resulting in poor generalization ability of the model and prone to overfitting. In addition, small sample data often has the problem of category imbalance. The number of samples in some categories is very small, which makes it difficult for the model to fully learn the characteristics of these categories during training, resulting in poor recognition effect. The core of the YOLOv8 algorithm lies in its efficient feature extraction and target detection mechanism, but in the case of small samples, its feature extraction module (such as convolution layer, pooling layer, etc.) may not be able to extract sufficiently representative features from limited samples. Specifically, YOLOv8's feature pyramid network (FPN) and multi-scale detection mechanism may not work fully on small sample data, resulting in poor detection effect. Summary of the invention

[0004] In order to better solve the above-mentioned deficiencies in the prior art, the present invention proposes a workpiece recognition method based on small sample training, by: 1) establishing a mutual reinforcement structure to enhance the data; 2) using the MLL Attention module to improve and train the YOLOv8 network structure, so as to achieve fast and effective recognition and segmentation of workpieces of unknown targets, and support subsequent industrial production links and industrial automation process.

[0005] The technical solution adopted by the present invention to achieve the above-mentioned purpose is: a workpiece recognition method based on small sample training, comprising the following steps:

[0006] S1, obtain the workpiece surface image, perform label processing, and create a label-image pair data set;

[0007] S2. Establish a small sample data enhancement network with a mutual enhancement structure, including a data mining model and a contrastive learning model. The data mining model is used to mine the intrinsic relationship between existing label-image pair data and generate image data and corresponding labels; the contrastive learning model is used to compare the image data generated by the data mining model with the original real image, so as to judge the quality of the generated image data, and call back the network structure parameters of the mining model to meet the quality and accuracy requirements, so as to obtain an ideal small sample data enhancement network for data expansion;

[0008] S3. Improve the YOLOv8 network structure, iteratively train the network with the label-image pair data set after data expansion, and back-propagate to optimize the network parameters to obtain the optimal improved YOLOv8 network structure, so that it can identify the target artifact in the positioning image and output the label of the artifact;

[0009] S4. Input the unknown target image to be detected into the optimal improved YOLOv8 network structure, and automatically output the category label and position regression box of the unknown artifact.

[0010] The workpiece surface image is taken by a high-pixel camera at an industrial site, and the workpiece surface image is taken at multiple angles with or without obstruction, and multiple types of workpiece images are taken from open source data sets.

[0011] The label processing is to manually mark the current workpiece with information: workpiece type, color, size and shape information, and mark the circumscribed rectangle of the workpiece outline.

[0012] The data mining model uses a convolutional neural network model to enhance the data.

[0013] The contrastive learning model uses a convolutional neural network model to perform quality error comparison calculation on the data.

[0014] The improved YOLOv8 network structure includes: adding an MLL Attention module after the C2f module at the end of each detection unit in the original detection head network, performing attention feature extraction again, deepening the network feature extraction capability, and using the output of the MLL Attention module as the detection and recognition result to achieve multi-scale resolution pixel-level recognition, segmentation and positioning.

[0015] The improved YOLOv8 network structure includes a backbone network and a detection head network;

[0016] The backbone network consists of a combination of a series of convolutional Conv and C2f modules, and the final pooling pyramid SPPF module, which outputs features of several different scales.

[0017] The detection head network includes several detection units, each of which splices the output features of the previous level detection unit and the features output by the backbone network and uses the C2f module and MLL Attention module to further extract deep features and then output them.

[0018] The step S3 iteratively training the network comprises:

[0019] Set network parameters, loss function, optimizer, training batches, and iteration rounds;

[0020] The data after data expansion is divided into a training set and a validation set. The network is trained using the training set data, and the model accuracy is verified using the validation set data for iterative training.

[0021] An artifact recognition system based on small sample training, including a control background and a front-end interface;

[0022] The control background is provided with the following modules, each of which cooperates with each other to realize the workpiece identification and positioning of the unknown target image; a data set module, which is used to obtain the workpiece surface image, perform label processing, and produce a label-image pair data set; a data enhancement module, which is used to establish a small sample data enhancement network with a mutually enhanced structure, including a data mining model and a contrast learning model, wherein the data mining model is used to mine the intrinsic connection between the existing label-image pair data, generate image data and corresponding labels; the contrast learning model is used to compare the image data generated by the data mining model with the original real image, so as to judge the quality of the generated image data, and call back the network structure parameters of the mining model to meet the quality and accuracy requirements, so as to obtain an ideal small sample data enhancement network for data expansion; a workpiece identification module, which is used to establish an improved YOLOv8 network structure, iteratively train the network with the label-image pair data set after data expansion, and back-propagate and optimize the network parameters to obtain the optimal improved YOLOv8 network structure, so that it can identify and locate the target workpiece in the image, and output the label of the workpiece;

[0023] The front-end interface uses human-computer interaction to collect images to be detected, set network parameters, loss function, optimizer, training batches and iteration rounds, and visualize the workpiece identification and positioning results of the control background calculation and processing, including: the location, category and recognition score of the workpiece, which is used to judge the status of the workpiece.

[0024] The present invention has the following beneficial effects and advantages:

[0025] 1. Visualization. The improved YOLOv8 algorithm can display the results of workpiece recognition in real time on the monitoring interface. Users can clearly see the location, category and recognition score of the workpiece, making it easy to quickly determine the status of the workpiece.

[0026] 2. Interactive analysis: Users can click on the workpiece icon on the screen to view detailed information about the workpiece, including its history, quality inspection results, etc.

[0027] 3. Accuracy. The speed and accuracy of object detection and segmentation tasks on mobile devices have been significantly improved through the improved YOLOv8 algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the overall framework;

[0029] Figure 2 To improve the YOLOv8 algorithm schematic;

[0030] Figure 3 Parsing diagram for making datasets for training and classification tasks. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation method of the present invention is described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.

[0033] A method for artifact recognition based on small sample training includes the following steps:

[0034] Step 1: Data collection of the workpiece: Use a high-pixel camera to collect images of the workpiece and capture image information.

[0035] Step 2: A small sample data enhancement algorithm based on a consistent mutual enhancement structure uses the existing small sample image data and the mutual enhancement structure to enhance the small sample data according to the existing internal connections to expand the data set;

[0036] Step 3: Based on the improved YOLOv8 algorithm, the color, type, and size information of the workpiece are used according to the data in step 2 to perform training prediction to improve recognition and segmentation accuracy;

[0037] Step 4: Input the unknown target image to be detected into the optimal improved YOLOv8 network structure, and automatically output the category label and position regression box of the unknown artifact.

[0038] The data of the workpiece image in step 1 is image data with a pixel of 4032x3024. The high-pixel camera is MV-CL084-96CM Hikvision's 8K Camera Link black-and-white linear array camera.

[0039] The basic process of the small sample method based on contrast consistency mutual reinforcement structure described in step 2 is as follows:

[0040] Step 21: In the initial stage, there are unlabeled new class instances in the base class dataset and the synthetic dataset. The mining model in the mutual enhancement structure finds the unlabeled new class instances in the dataset and assigns them pseudo labels.

[0041] The base dataset is a multi-angle shot taken by a high-pixel camera in an industrial field under different natural environmental factors such as occlusion and different lighting conditions. The synthetic dataset is an open source image dataset on the Internet.

[0042] Step 22: Filter the pseudo-labels and merge them with the real labels of the images to generate new supervision information for the images.

[0043] Step 23: The learning model is trained based on the new supervision information provided by the mining model. Contrastive learning is introduced during the training process to help the learning model resist the influence of noisy pseudo-labels to stabilize the training and help better classification.

[0044] Step 24: After a certain period of training, the learning model updates the parameters of the mining model in a momentum manner to pass the new knowledge to the mining model.

[0045] Through continuous iterations from step 21 to step 24, continuous evolution between the dual models is achieved, and an ideal state is achieved in the dual model structure where the mining model can provide new class labeling information more accurately and the learning model can continuously learn and transfer new knowledge.

[0046] The optimization of the original YOLOv8 algorithm described in step 3 includes the following steps:

[0047] 1) If Figure 2 As shown in the figure, the original YOLOv8 network structure is divided into a backbone network and a detection head network, which is subdivided into Conv convolution operation, SPPF pooling pyramid, Upsample upsampling operation, Concat splicing operation, C2F module, and MLL Attention module.

[0048] The backbone network consists of a series of convolutional Conv and C2f modules, and the final pooling pyramid SPPF module, which outputs features of three different scales (first feature, second feature, and third feature).

[0049] The detection head network includes four detection units (a first detection unit, a second detection unit, a third detection unit, and a fourth detection unit). The first detection unit performs feature extraction on the third feature and the second feature after splicing, and outputs a first depth feature; the second detection unit performs feature extraction on the first depth feature and the second feature after splicing, and outputs a first segmentation result; the third detection unit performs feature extraction on the first segmentation result and the first depth feature after splicing, and outputs a second segmentation result; the fourth detection unit performs feature extraction on the second segmentation result and the third feature of the backbone network after splicing, and outputs a third segmentation result.

[0050] 2) The present invention proposes to add an MLL Attention module after the C2f module at the end of the second detection unit, the third detection unit, and the fourth detection unit in the original detection head network, perform attention feature extraction again, deepen the network feature extraction capability, and output the detection results after the MLL Attention module.

[0051] The MLL Attention module is an existing network structure, and the steps of processing data by the MLL Attention module are as follows:

[0052] Step 1: Input data.

[0053] First, the input data passes through a multi-layer perceptron (MLP) and then undergoes a normalization operation.

[0054] Step 2: Process the MLLA Block.

[0055] For the data processed in step 1, linear operations are first performed, the SiLU activation function is applied, and then the convolutional layer is used to further process the result, and the SiLU activation function is applied again.

[0056] Step 3: Perform element-wise summation, Hadamard product, and SiLU activation on the data output from step 2 and the original MLP output.

[0057] The step S3 iteratively trains the network including: setting the network parameters to 150 epochs, 16 batches, and SGD gradients. If the training loss is NAN, the amp can be turned off, and the loss function is a classification loss function.

[0058] The data after data expansion is divided into a training set and a validation set. The network is trained using the training set data, and the model accuracy is verified using the validation set data for iterative training.

[0059] The present invention also provides a workpiece recognition system based on small sample training, including a control background and a front-end interface;

[0060] The control background is provided with the following modules, each of which cooperates with each other to realize the workpiece identification and positioning of unknown target images; a data set module is used to obtain the workpiece surface image, perform label processing, and produce a label-image pair data set; a data enhancement module is used to establish a small sample data enhancement network with a mutual enhancement structure, including a data mining model and a contrast learning model, and the data mining model is used to mine the intrinsic connection between the existing label-image pair data, generate image data and corresponding labels; the contrast learning model is used to compare the image data generated by the data mining model with the original real image, so as to judge the quality of the generated image data, and call back the network structure parameters of the mining model to meet the quality and accuracy requirements, and obtain an ideal small sample data enhancement network for data expansion; the workpiece identification module is used to establish an improved YOLOv8 network structure, iteratively train the network with the label-image pair data set after data expansion, and back-propagate to optimize the network parameters to obtain the optimal improved YOLOv8 network structure, so that it can identify and locate the target workpiece in the image and output the label of the workpiece;

[0061] The front-end interface uses human-computer interaction to collect images to be detected, set network parameters, loss function, optimizer, training batches and iteration rounds, and visualize the workpiece identification and positioning results of the control background calculation and processing, including: the location, category and recognition score of the workpiece, which is used to judge the status of the workpiece.

[0062] This method selects some images from the base class data and performs UDCut-Mix processing with some new class images to generate some new synthetic data, which together with the base class data set form a new data set for dual-model classification and recognition, as shown in the attached figure. Figure 3 shown.

[0063] For the experimental accuracy prediction of images, Table 1 shows that the traditional and improved YOLOv8 have been compared, and both accuracy and recall have increased significantly. However, MAP50-95 and MAP50 have not improved significantly, which may be related to the synthetic dataset. The synthetic dataset is not trained sufficiently, and there is a clear difference in the recognition of new classes and other basic classes.

[0064] Table 1 Traditional YOLOV8 effect diagram

[0065]

[0066] Table 2 Effect diagram of this method improving YOLOV8

[0067]

[0068] The advantages of the present invention are: 1) establishing a mutual enhancement structure to enhance data; 2) using the MLL Attention module to improve and train the YOLOv8 network structure to achieve positioning and recognition of unknown targets.

[0069] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should be regarded as within the scope of protection of the present invention.

Claims

1. An artifact recognition method based on small sample training, It is characterized in that The following steps are involved: S1, obtain the workpiece surface image, perform label processing, and create a label-image pair data set; S2. Establishing a small sample data enhancement network with a mutual enhancement structure, including a data mining model and a contrastive learning model, wherein the data mining model is used to mine the intrinsic connection between existing label-image pair data and generate image data and corresponding labels; The contrastive learning model is used to compare the image data generated by the data mining model with the original real image, so as to judge the quality of the generated image data, and call back the network structure parameters of the mining model to make it meet the quality and accuracy requirements, and obtain an ideal small sample data enhancement network for data expansion; S3. Improve the YOLOv8 network structure, iteratively train the network with the label-image pair data set after data expansion, and back-propagate to optimize the network parameters to obtain the optimal improved YOLOv8 network structure, so that it can identify the target artifact in the positioning image and output the label of the artifact; S4. Input the unknown target image to be detected into the optimal improved YOLOv8 network structure, and automatically output the category label and position regression box of the unknown artifact.

2. According to the method for artifact recognition based on small sample training according to claim 1, It is characterized in that The workpiece surface image is taken by a high-pixel camera at an industrial site, and the workpiece surface image is taken at multiple angles with or without obstruction, and multiple types of workpiece images are taken from open source data sets.

3. According to the workpiece recognition method based on small sample training as described in claim 1, It is characterized in that The label processing is to manually mark the current workpiece with information: workpiece type, color, size and shape information, and mark the circumscribed rectangle of the workpiece outline.

4. According to the method for identifying artifacts based on small sample training according to claim 1, It is characterized in that The data mining model uses a convolutional neural network model to enhance the data.

5. According to the method for artifact recognition based on small sample training as described in claim 1, It is characterized in that The contrastive learning model uses a convolutional neural network model to perform quality error comparison calculation on the data.

6. The method for identifying artifacts based on small sample training according to claim 1, It is characterized in that The improved YOLOv8 network structure includes: adding an MLL Attention module after the C2f module at the end of each detection unit in the original detection head network, performing attention feature extraction again, deepening the network feature extraction capability, and using the output of the MLL Attention module as the detection and recognition result to achieve multi-scale resolution pixel-level recognition, segmentation and positioning.

7. The method for identifying artifacts based on small sample training according to claim 6, It is characterized in that The improved YOLOv8 network structure includes a backbone network and a detection head network; The backbone network consists of a combination of a series of convolutional Conv and C2f modules, and the final pooling pyramid SPPF module, which outputs features of several different scales. The detection head network includes several detection units, each of which splices the output features of the previous level detection unit and the features output by the backbone network and uses the C2f module and MLL Attention module to further extract deep features and then output them.

8. The method for artifact recognition based on small sample training according to claim 6, It is characterized in that The step S3 iteratively training the network comprises: Set network parameters, loss function, optimizer, training batches, and iteration rounds; The data after data expansion is divided into a training set and a validation set. The network is trained using the training set data, and the model accuracy is verified using the validation set data for iterative training.

9. A workpiece recognition system based on small sample training according to any one of claims 1 to 8, It is characterized in that Including control background and front-end interface; The control background is provided with the following modules, each of which cooperates with each other to realize the workpiece recognition and positioning of the unknown target image; a data set module, which is used to obtain the workpiece surface image, perform label processing, and produce a label-image pair data set; a data enhancement module, which is used to establish a small sample data enhancement network with a mutual enhancement structure, including a data mining model and a contrastive learning model, and the data mining model is used to mine the intrinsic connection between the existing label-image pair data, generate image data and corresponding labels; The contrastive learning model is used to compare the image data generated by the data mining model with the original real image, so as to judge the quality of the generated image data, and call back the network structure parameters of the mining model to make it meet the quality and accuracy requirements, and obtain an ideal small sample data enhancement network for data expansion; The artifact recognition module is used to establish an improved YOLOv8 network structure, iteratively train the network with the label-image pair data set after data expansion, and back-propagate to optimize the network parameters to obtain the optimal improved YOLOv8 network structure, so that it can identify the target artifact in the positioning image and output the label of the artifact; The front-end interface uses human-computer interaction to collect images to be detected, set network parameters, loss function, optimizer, training batches and iteration rounds, and visualize the workpiece identification and positioning results of the control background calculation and processing, including: the location, category and recognition score of the workpiece, which is used to judge the status of the workpiece.