Random region mosaic data augmentation method and system for industrial defect detection

Abstract

A random region mosaic data augmentation method for industrial defect detection is proposed. First, a parameter list, Patch-List, is defined to control the size of the mosaic blocks. Different sized mosaic blocks are randomly selected for each data augmentation operation. Second, based on the selected mosaic block size, the image is divided into several grids, and the probability parameter P is used to... m Determine whether to perform mosaic blurring on each grid; by introducing M iou With M p Two threshold parameters are used to determine the mosaic coverage of the target region. If the mosaic coverage of a target region exceeds the threshold, its label information is automatically removed; the model adjusts the threshold in each training round using P. t The data augmentation is activated with a probability that continues until model training is complete. A random region mosaic data augmentation system is also provided. This invention can improve the classification and localization accuracy of detection models, while being simple to deploy and easy to use.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computer vision and image processing, and particularly relates to a random region mosaic data enhancement method and system for industrial defect detection. BACKGROUND

[0002] Industrial defect detection, as an important part of intelligent manufacturing, directly affects the product quality and the automation level of the production line. At present, the automatic defect detection system based on deep learning technology and machine vision has gradually become a research hotspot in the industry. However, due to the complexity and randomness of industrial defects and the scarcity of defect samples in the actual production environment, the number of available samples is small, and the diversity of defects is low.

[0003] Data enhancement technology, as an effective means, increases the diversity of the data set by simulating various changes of the actual data, thereby improving the generalization ability and detection accuracy of the model. However, the traditional data enhancement method is difficult to fully simulate the diverse characteristics of defects, thereby limiting the generalization ability of the detection model. The existing data enhancement strategies, such as basic geometric transformation and color disturbance, can increase the diversity of the data set, but often ignore the targeted enhancement of the defect region characteristics. In view of the special needs of industrial defect detection, it is particularly important to develop a data enhancement technology that can accurately simulate the changes of the defect region. SUMMARY

[0004] In order to overcome the problem that the existing data enhancement method has poor enhancement effect on industrial defects, and to improve the classification and positioning effect of the model in the defect detection task, the application provides a random region mosaic data enhancement method and system for industrial defect detection.

[0005] The technical scheme adopted by the application to solve the technical problems is:

[0006] A random region mosaic data enhancement method for industrial defect detection, comprising the following steps:

[0007] Step 1: reading the defect image in the training set and the label information of the corresponding class and position;

[0008] Step 2: setting a Patch-List parameter list, which stores a plurality of pre-defined values for controlling the width and height of the generated mosaic blocks

[0009] Step 3: in the training process of the model, the Patch-List parameter list set in step 2 is randomly selected, and the The image is divided into several grids, and after the division, each grid of the image is traversed, and a probability parameter P is set m to determine whether to perform mosaic blur processing on each grid, and the image after mosaic is obtained.

[0010] Step 4: Based on step 3, a threshold parameter M is set iou and M p When the size of the mosaic block used is greater than M iou and M p , the label information of the target is deleted.

[0011] Step 5: In the training process of the model, the data enhancement is started with a certain probability. After starting, the training is performed according to steps 2-4 above until the model converges completely.

[0012] Further, in step 2, the Patch-List parameter list specifically includes a list of multiple integer (INT) types, and the number of integers contained is by default between 3-6.

[0013] Further, in step 3, the method of randomly selecting the Patch-List parameter list is implemented using the random.choice(Patch-List) function. Secondly, the image is divided into several grids as shown in formula (1);

[0014]

[0015] Where N w and N H are the number of grids divided along the width direction and the number of grids divided along the height direction of the image, respectively, and H and W are the width and height of the image, respectively.

[0016] Further, in step 3, the mosaic blur processing is performed by first calculating the average color of the extracted region using the np.mean() method, and then filling the current grid cell with the average color in the current grid to achieve the mosaic effect.

[0017] In step 4, the analysis of the target region covered by the mosaic is specifically: when the size of the mosaic block used is greater than M and M pAt this time, the current divided grid cell is defined as a bounding box, and its coordinate form is [Xb1, Yb1, Xb2, Yb2]. In addition, the label coordinate format is converted into the same form as the grid cell bounding box [Xl1, Yl1, Xl2, Yl2], the IoA value between the grid cell and each label is calculated, and the total IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. When the accumulated IoA is less than or equal to M iou , the label is retained, while when the mosaic block size used is less than M p , all target labels in the figure are retained.

[0018] In step 5, the training process is started with certain conditions and probabilities, and the specific data enhancement is as follows: set the probability parameter P t , use the random.random() function to generate a random number in each round of model training, and compare it with P t . When the generated random number P t , the round of training starts the mosaic data enhancement, and the reverse does not use the mosaic data enhancement. The default value of P t is between 10-50%.

[0019] A random area mosaic data enhancement system for industrial defect detection, comprising:

[0020] A data reading module for loading and preprocessing the industrial images to be detected and the corresponding label information from the local storage system, while performing preprocessing operations including but not limited to image denoising, size adjustment (to ensure that all images have the same resolution before inputting into the model), normalization (scaling pixel values to a unified range such as 0 to 1), and possible image enhancement preliminary processing (such as contrast adjustment, brightness adjustment, etc.), to improve image quality and facilitate subsequent analysis;

[0021] A mosaic enhancement module for performing mosaic data enhancement processing on the image dataset. Simulate the image quality problems that may be encountered in the actual detection scene, such as occlusion, blur, etc., to increase the diversity and complexity of the dataset, thereby improving the robustness and generalization ability of the model for industrial defect detection;

[0022] A label post-processing module for removing the case where part of the target label is severely occluded (IoA is greater than M p ) by a coarse-grained mosaic block iou (greater than M iou ), to avoid the model learning useless feature information;

[0023] The model training module is used for inputting a test image set number acquired by the image acquisition module into an algorithm for training, and iteratively training the model in a forward propagation and backward propagation manner.

[0024] The model testing module is used for inputting a test image set number acquired by the image acquisition module into an algorithm for detection and information calculation precision testing, and is used for evaluating the performance of the trained model on unknown data, and ensuring that the model has reliable accuracy and generalization ability in actual application.

[0025] The technical concept of the present application is that, in view of the data bottleneck problem of supervised learning in the field of defect detection, considering that defect data is difficult to obtain, defect object labeling is difficult, and the effect of existing data enhancement methods on the defect detection task is poor, the present application proposes a general data enhancement method for industrial defect detection. Through a large number of analysis and verification on the public defect detection data set, the feasibility and reliability of the algorithm are verified.

[0026] The present application simulates the complex scenes such as occlusion and pollution that may be encountered in actual production by randomly selecting regions in the image and applying the mosaic effect (i.e. local pixel blur or occlusion), thereby effectively enhancing the complexity and diversity of the data set without introducing new defects. The method aims to improve the robustness and generalization ability of the deep learning model for defect detection, and provides strong support for the intelligent development of industrial defect detection.

[0027] The beneficial effects of the present application mainly include: the model can effectively improve the image quality problems that may be encountered in the actual detection process, such as occlusion, blur, etc., and at the same time forces the model to train through more abstract local semantic information, improves the recognition ability of the model for defects, increases the diversity and complexity of the data set, improves the generalization of the model and reduces the overfitting, and the scheme can well meet the accuracy and speed requirements of defect detection in industrial scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0028] Fig. 1 is a flowchart of the whole algorithm of the random region mosaic data enhancement method for industrial defect detection provided by the embodiment of the present application.

[0029] Fig. 2 is a flowchart of the visualization steps of the data enhancement method proposed by the present application and the corresponding generated images and post-processing of labels.

[0030] Fig. 3Fig. 1 is a schematic diagram of detection effects before and after using the data enhancement method proposed in the present application on the basis of a YOLOv9-S model, wherein (a) is a manual annotation result, (b) is a YOLOv9-S model detection result, and (c) is a YOLOv9-S model detection result after adding the proposed data enhancement method. DETAILED DESCRIPTION

[0031] The present application will be further described below with reference to the accompanying drawings.

[0032] Reference Figs. 1-3 A random region mosaic data enhancement method for industrial defect detection, comprising the following steps:

[0033] Step 1: reading a first picture in a training set and its corresponding annotation information, the annotation information including coordinate information of an object positioning frame and class information of the object;

[0034] Step 2: setting a parameter list Patch-List for setting the size of a mosaic block, and randomly selecting a value in the Patch-List list as the size of the mosaic block;

[0035] The Patch-List parameter list specifically is a list containing a plurality of integer (INT) types, and the number of the contained integers is by default between 3 and 6;

[0036] Step 3: according to the size of the value obtained in Step 2 , evenly dividing the image in the width and height directions into S grids, i.e., dividing the whole image into S×S grids; and setting a probability parameter P m and traversing all the grids divided in Step 3, each grid having a P m probability of executing mosaic processing, the mosaic processing mode being to take the average of all pixel values in the region and uniformly fill the region;

[0037] The method for randomly selecting the Patch-List parameter list adopts the random.choice(Patch-List) function, and the operation of evenly dividing the image into a plurality of grids is shown in formula (1);

[0038]

[0039] wherein N w and N H are the number of grids divided along the image width direction and the number of grids divided along the image height direction, respectively, and H and W are the width and height of the picture, respectively.

[0040] The mosaic blurring process described above first uses the np.mean() method to calculate the average color of the extracted region, and then fills the current grid cell with the average color within the current grid to achieve the mosaic effect.

[0041] Step 4: Set the threshold parameter M used for label post-processing algorithm judgment. p and M iou Calculate and determine the size of the mosaic blocks used in the image. Is it greater than the set threshold M? p Simultaneously, determine whether the IOA (Indicators of Area) of the target label's location and the mosaic area is greater than the set threshold M. iou If both conditions are met, the tag information for that target is removed. Otherwise, it is retained.

[0042] In step 4, the analysis of the target area covered by the mosaic specifically involves: when the size of the mosaic blocks used... Greater than M p At this time, the currently divided grid cell is defined as a bounding box with coordinates in the form [Xb1,Yb1,Xb2,Yb2]. Furthermore, the label coordinates are converted to the same form as the grid cell bounding box [Xl1,Yl1,Xl2,Yl2]. The IoA value between the grid cell and each label is calculated, and the overall IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. The accumulated IoA value is kept if it is less than or equal to M. iou The label, and the size of the mosaic blocks used by Dangdang. Less than M p At that time, all target labels in the graph are preserved.

[0043] Step 5: At the beginning of each training round, first use the random.random() function to generate random numbers and match them with the set probability value P. t When comparing, if the generated random number is greater than P t If the condition is met, then mosaic data augmentation will be enabled for that round of training; otherwise, mosaic data augmentation will not be enabled for that round of training. This process will be repeated until the training ends.

[0044] In step 5, the data augmentation is activated under certain conditions and with certain probabilities during the training process, specifically by setting the probability parameter P. t In each round of model training, the random.random() function is used to generate random numbers and P is applied. t When comparing the generated random number P, t If the condition is met, then mosaic data augmentation will be enabled for that round of training; otherwise, it will not be enabled. The set P... t The default value is between 10% and 50%.

[0045] A random region mosaic data augmentation method for industrial defect detection includes:

[0046] The data reading module is used to load and preprocess the industrial images to be detected and their corresponding label information from the local storage system. At the same time, it performs preprocessing operations, including but not limited to image denoising, resizing (to ensure that all images have the same resolution before being input into the model), normalization (scaling pixel values ​​to a uniform range, such as between 0 and 1), and possible preliminary image enhancement processing (such as contrast adjustment, brightness adjustment, etc.) to improve image quality and facilitate subsequent analysis.

[0047] The mosaic enhancement module is used to perform mosaic data enhancement processing on image datasets. It simulates image quality problems that may be encountered in real-world detection scenarios, such as occlusion and blurring, to increase the diversity and complexity of the dataset, thereby improving the model's robustness and generalization ability for industrial defect detection.

[0048] The label post-processing module is used to remove some target labels that are covered by coarse-grained mosaic blocks. Greater than M p Severe occlusion (IoA greater than M) iou In cases where the model learns useless feature information, this is to prevent the model from learning such information.

[0049] The model training module is used to input the set of test images obtained from the image acquisition module into the algorithm for training. The training adopts forward propagation and back propagation to iteratively train the model.

[0050] The model testing module is used to input the test image set obtained from the image acquisition module into the algorithm for detection and information calculation accuracy testing. It is used to evaluate the performance of the trained model on unknown data and ensure that the model has reliable accuracy and generalization ability in practical applications.

[0051] The embodiment adopts the public steel surface defect NEU-DET data set provided by Northeastern University, which contains 1800 labeled images in total, and has the following six types of defects: 'Crazing', 'Inclusion', 'Patches', 'Pitted_surface', 'Rolled-in_scale', 'Scratches'. In the embodiment, 80% of the images are used as the training set, 10% of the images are used as the verification set, and 10% of the images are used as the test set. First, the training set in the NEU-DET data set and the corresponding labeling information are read, and after steps 1 to 5, Precision, Recall, mAP50 are used as the main evaluation indexes of the detection result on the NEU-DET data set, and the deep neural network model trained by the traditional data enhancement method is compared. The results are shown in Table 1. As shown in Table 1, the average accuracy of the method is generally better than that of the network trained by the traditional data enhancement method. Table 1 is the comparison result of the deep neural network model trained by the traditional region discard method.

[0052]

[0053]

[0054] Table 1

[0055] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and is only for the purpose of description. The protection scope of the present application should not be regarded as being limited to the specific forms stated in the embodiments, and the protection scope of the present application also extends to equivalent technical means that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A random region mosaic data augmentation method for industrial defect detection, characterized in that, The method includes the following steps: Step 1: Read the defect images in the training set and their corresponding category and location label information; Step 2: Set the Patch-List parameter list. This list stores several predefined values ​​used to control the size of the generated mosaic blocks. ; Step 3: During model training, the model will be randomly selected from the Patch-List parameter list set in Step 2, and the selected parameters will be used to train the model. The image is divided into several grids on average. After the division, each image grid is traversed, and a probability parameter is set. To determine whether to perform mosaic blurring on each grid, the image with mosaic effect is obtained; Step 4: Set the threshold parameter based on Step 3. and The analysis also examines the target area covered by the mosaic, and determines the IoA of the target area covered by the mosaic blocks when it is greater than a certain value. And the size of the mosaic tiles used Greater than When this happens, the tag information for that target will be deleted; Step 5: During the model training process, enable the data augmentation with a set probability. After enabling it, train according to steps 2-4 above until the model fully converges.

2. The random region mosaic data augmentation method for industrial defect detection as described in claim 1, characterized in that, In step 2, the Patch-List parameter list is specifically a list containing multiple integers (INT), with the number of integers being between 3 and 6 by default.

3. The random region mosaic data augmentation method for industrial defect detection as described in claim 1 or 2, characterized in that, In step 3, the method of randomly selecting the Patch-List parameter list is implemented by the random.choice(Patch-List) function. Then, the operation of dividing the image into several grids on an average basis is shown in formula (1). , (1)； in, and These represent the number of grids divided along the image width direction and the number of grids divided along the image height direction, respectively. and These represent the width and height of the image, respectively.

4. The random region mosaic data augmentation method for industrial defect detection as described in claim 1 or 2, characterized in that, In step 3, the mosaic blurring process first uses the np.mean() method to calculate the average color of the extracted region, and then fills the current grid cell with the average color within the current grid to achieve the mosaic effect.

5. The random region mosaic data augmentation method for industrial defect detection as described in claim 1 or 2, characterized in that, In step 4, the analysis of the target area covered by the mosaic specifically involves: when the size of the mosaic blocks used... Greater than At this time, the currently divided grid cell is defined as a bounding box with coordinates [Xb1, Yb1, Xb2, Yb2]. Furthermore, the label coordinates are converted to the same format as the grid cell bounding box [Xl1, Yl1, Xl2, Yl2]. The IoA value between the grid cell and each label is calculated, and the overall IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. Values ​​with a cumulative IoA less than or equal to the target are retained. The label, and when the size of the mosaic blocks used. Less than At that time, all target labels in the graph are preserved.

6. The random region mosaic data augmentation method for industrial defect detection as described in claim 1 or 2, characterized in that, In step 5, the data augmentation is activated under certain conditions and probabilities during the training process, specifically by setting probability parameters. In each round of model training, the random.random() function is used to generate random numbers and match them with the model's output. When the generated random number is greater than 1, the comparison is made. If the specified conditions are met, then mosaic data augmentation will be enabled for that round of training; otherwise, it will not be enabled. The value is between 10% and 50%.

7. A system implementing the random region mosaic data augmentation method for industrial defect detection as described in claim 1, characterized in that, The system includes: The data reading module is used to load and preprocess the industrial images to be detected and their corresponding tag information from the local storage system. At the same time, it performs preprocessing operations, including image denoising, size adjustment, normalization and preliminary image enhancement, in order to improve image quality and facilitate subsequent analysis. The mosaic enhancement module is used to perform mosaic data enhancement processing on image datasets to simulate image quality problems encountered in actual detection scenarios, thereby increasing the diversity and complexity of the dataset and improving the robustness and generalization ability of the model for industrial defect detection. The tag post-processing module is used to remove target tags that are severely obscured by coarse-grained mosaic blocks. The criteria for judging mosaic blocks are... Greater than The criterion for severe occlusion is that the IoA is greater than 1. This avoids the model learning useless feature information; The model training module is used to input the set of test images obtained from the image acquisition module into the algorithm for training. The training adopts forward propagation and back propagation to iteratively train the model. The model testing module is used to input the test image set obtained from the image acquisition module into the algorithm for detection and information calculation accuracy testing. It is used to evaluate the performance of the trained model on unknown data and ensure that the model has reliable accuracy and generalization ability in practical applications.

Images