Random region mosaic data enhancement method and system for industrial defect detection

Abstract

A random area mosaic data enhancement method for industrial defect detection comprises the following steps: firstly, defining a parameter list Patch-List for controlling the size of mosaic blocks, and randomly selecting mosaic blocks with different sizes during each data enhancement; secondly, dividing the image into a plurality of grids according to the size of a selected mosaic block, and determining whether to execute mosaic blurring processing on each grid or not through a probability parameter Pm; and judging the mosaic coverage degree of the target area by introducing two threshold parameters of Miou and Mp. If the mosaic coverage of a certain target area exceeds a threshold value, automatically removing the label information of the target area; and starting the data enhancement by the model at the probability of Pt in each round of training process until the model training is finished. The invention further provides a random region mosaic data enhancement system. The method can improve the classification precision and positioning precision of the detection model, and is simple in deployment and easy to use.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer vision and image processing, and in particular relates to a random area mosaic data enhancement method and system for industrial defect detection. Background Art

[0002] As an important part of intelligent manufacturing, the accuracy and efficiency of industrial defect detection directly affect product quality and the automation level of production lines. At present, automatic defect detection systems based on deep learning technology and machine vision have gradually become a research hotspot in the industry. However, due to the complexity and randomness of industrial defects and the scarcity of defect samples in actual production environments, the number of available samples is small and the diversity of defects is low.

[0003] Data augmentation technology is an effective means to increase the diversity of data sets by simulating various changes in actual data, thereby improving the generalization ability of the model and the detection accuracy. However, traditional data augmentation methods are currently unable to fully simulate the diverse characteristics of defects, thus limiting the generalization ability of the detection model. Existing data augmentation strategies, such as basic geometric transformations and color perturbations, can increase the diversity of data sets, but often ignore the targeted enhancement of defect area characteristics. In view of the special needs of industrial defect detection, it is particularly important to develop a data augmentation technology that can accurately simulate changes in defect areas. Summary of the invention

[0004] In order to overcome the problem that existing data enhancement methods have poor enhancement effects on industrial defects, and to improve the classification and positioning effects of the model on defect detection tasks, the present invention provides a random area mosaic data enhancement method and system for industrial defect detection.

[0005] The technical solution adopted by the present invention to solve its technical problem is:

[0006] A random area mosaic data enhancement method for industrial defect detection includes the following steps:

[0007] Step 1: Read the defect images in the training set and their corresponding category and location label information;

[0008] Step 2: Set the Patch-List parameter list, which stores several pre-defined values ​​to control the width and height of the generated mosaic blocks

[0009] Step 3: During the model training process, the Patch-List parameter list set in step 2 will be randomly selected, and the selected The image is evenly divided into several grids. After the division is completed, each image grid is traversed and the probability parameter P is set. m To determine whether to perform mosaic blur processing on each grid and obtain the mosaic image;

[0010] Step 4: Set the threshold parameter M based on step 3 iou With M p , and analyze the target area to be detected covered by the mosaic. When the coverage of the target area to be detected by the mosaic block and the size of the mosaic block used are greater than M iou With M p , delete the label information of the target;

[0011] Step 5: Enable data augmentation with a certain probability during model training. After enabling, perform training according to steps 2 to 4 above until the model fully converges.

[0012] Furthermore, in step 2, the Patch-List parameter list is specifically a list containing multiple integer (INT) types, and the number of integers contained is between 3 and 6 by default.

[0013] Furthermore, in step 3, the method of randomly selecting the Patch-List parameter list is implemented by using the random.choice(Patch-List) function, and then the image is evenly divided into a number of grids as shown in formula (1);

[0014]

[0015] Among them, N w With N H They are the number of grids divided along the width direction of the image and the number of grids divided along the height direction of the image, and H and W are the width and height of the image respectively.

[0016] Furthermore, in step 3, the mosaic blur processing is performed by first using the np.mean() method to calculate the average color of the extracted area, and then filling the current grid cell with the average color in the current grid to achieve a mosaic effect.

[0017] In step 4, the analysis of the target area to be detected covered by the mosaic is specifically as follows: when the size of the mosaic block used is Greater than M pWhen the current grid unit is divided, it is defined as a bounding box with coordinates in the form of [Xb1, Yb1, Xb2, Yb2]. In addition, the label coordinate format is converted to the same form as the grid unit bounding box [Xl1, Yl1, Xl2, Yl2]. The IoA value between the grid unit and each label is calculated, and the overall IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. The accumulated IoA is less than or equal to M. iou The label, and the mosaic block size used by Dangdang Less than M p When , all target labels in the graph are retained.

[0018] In step 5, the data enhancement is enabled under certain conditions and probabilities during the training process, specifically: setting the probability parameter P t In each round of training of the model, the random.random() function is used to generate random numbers and t When the generated random number P is compared t , then mosaic data enhancement is enabled for this round of training, otherwise mosaic data enhancement is not enabled for this round of training. t The value defaults to between 10-50%.

[0019] A random area mosaic data enhancement system for industrial defect detection, comprising:

[0020] The data reading module is used to load and preprocess the industrial images to be inspected and the corresponding label information from the local storage system, and perform preprocessing operations, including but not limited to image denoising, resizing (to ensure that all images have the same resolution before being input into the model), normalization (scaling pixel values ​​to a uniform range, such as between 0 and 1), and possible preliminary image enhancement processing (such as contrast adjustment, brightness adjustment, etc.) to improve image quality and facilitate subsequent analysis;

[0021] The mosaic enhancement module is used to perform mosaic data enhancement processing on the image dataset. It simulates image quality problems that may be encountered in actual inspection scenarios, such as occlusion and blur, to increase the diversity and complexity of the dataset, thereby improving the robustness and generalization ability of the model for industrial defect detection;

[0022] The label post-processing module is used to remove some target labels that are blocked by coarse-grained mosaic blocks ( Greater than M p ) Severe occlusion (IoA greater than M iou ) to prevent the model from learning useless feature information;

[0023] The model training module is used to input the test image sets obtained by the image acquisition module into the algorithm for training. The training adopts forward propagation and back propagation to iteratively train the model.

[0024] The model testing module is used to input the test image sets obtained from the image acquisition module into the algorithm for detection and information calculation accuracy testing. It is used to evaluate the performance of the trained model on unknown data and ensure that the model has reliable accuracy and generalization ability in practical applications.

[0025] The technical concept of the present invention is: to solve the data bottleneck problem of supervised learning in the field of defect detection, considering that defect data is difficult to obtain, defect object labeling is difficult, and the existing data enhancement methods have poor effects on defect detection tasks and the improvement is not obvious, the present invention proposes a general data enhancement method for industrial defect detection. Through a large number of analysis and verification on public defect detection data sets, the feasibility and reliability of the algorithm have been confirmed.

[0026] The present invention simulates complex scenes such as occlusion and pollution that may be encountered in actual production by randomly selecting areas in the image and applying a mosaic effect (i.e., blurring or occluding local pixels), thereby effectively enhancing the complexity and diversity of the data set without introducing new defects. This method aims to improve the robustness and generalization ability of deep learning models for defect detection, and provide strong support for the intelligent development of industrial defect detection.

[0027] The beneficial effects of the present invention are mainly manifested in: it can effectively improve the image quality problems that the model may encounter in the actual detection process, such as occlusion, blur, etc., while forcing the model to be trained through more abstract local semantic information, improving the model's defect recognition ability, increasing the diversity and complexity of the data set, improving the generalization of the model and reducing overfitting. The adoption of this solution can well meet the accuracy and speed requirements of defect detection in industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a flowchart of an overall algorithm of a random area mosaic data enhancement method for industrial defect detection provided by an embodiment of the present invention.

[0029] Figure 2 It is a flow chart of the visualization steps of the data enhancement method proposed in the present invention and the corresponding image generation and label post-processing.

[0030] Figure 3It is a schematic diagram of the detection effect before and after using the data enhancement method proposed in the present invention on the basic YOLOv9-S model, wherein (a) is the manual annotation result, (b) is the detection result of the YOLOv9-S model, and (c) is the detection result of the YOLOv9-S model after adding the proposed data enhancement method. DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings.

[0032] Reference Figure 1 to Figure 3 , a random area mosaic data enhancement method for industrial defect detection, comprising the following steps:

[0033] Step 1: Read the first image in the training set and its corresponding annotation information, where the annotation information includes the coordinate information of the positioning frame of the object and the category information of the object;

[0034] Step 2: Set the parameter list Patch-List for setting the mosaic block size and randomly select a value in the Patch-List list. as the size of the mosaic piece;

[0035] The Patch-List parameter list is specifically a list of multiple integer (INT) types, and the number of integers included is between 3 and 6 by default;

[0036] Step 3: Based on the result obtained in step 2 The value divides the image into S grids in width and height directions, that is, the entire image is divided into S×S grids; and sets the probability parameter P m And traverse all the grids obtained in step 3, each grid has P m The probability of performing mosaic processing is that all pixel values ​​in the area are averaged and then uniformly filled in the area;

[0037] The method for randomly selecting the Patch-List parameter list is implemented by using the random.choice(Patch-List) function, and then the image is evenly divided into a number of grids as shown in formula (1);

[0038]

[0039] Among them, N w With N H are the number of grids divided along the width direction of the image and the number of grids divided along the height direction of the image, respectively. H and W are the width and height of the image, respectively.

[0040] The mosaic blur processing is performed by first using the np.mean() method to calculate the average color of the extracted area, and then filling the current grid cell with the average color in the current grid to achieve a mosaic effect;

[0041] Step 4: Set the threshold parameter M for label post-processing algorithm judgment p and M iou , calculate and determine the mosaic block size used in the image Is it greater than the set threshold M? p At the same time, determine whether the IOA of the target label area and the mosaic area is greater than the set threshold M iou ? If both conditions are met, the label information of the target is removed. Otherwise, it is retained.

[0042] In step 4, the analysis of the target area to be detected covered by the mosaic is specifically as follows: when the size of the mosaic block used is Greater than M p When the current grid unit is divided, it is defined as a bounding box with coordinates in the form of [Xb1, Yb1, Xb2, Yb2]. In addition, the label coordinate format is converted to the same form as the grid unit bounding box [Xl1, Yl1, Xl2, Yl2]. The IoA value between the grid unit and each label is calculated, and the overall IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. The accumulated IoA is less than or equal to M. iou The label, and the mosaic block size used by Dangdang Less than M p When , all target labels in the graph are retained.

[0043] Step 5: At the beginning of each round of training, first use the random.random() function to generate random numbers and compare them with the set probability value P t Compare, when the generated random number is greater than P t , then mosaic data enhancement is enabled in this round of training, otherwise mosaic data enhancement is not enabled in this round of training, and the above process is repeated until the end of training.

[0044] In step 5, the data enhancement is enabled under certain conditions and probabilities during the training process, specifically: setting the probability parameter P t In each round of training of the model, the random.random() function is used to generate random numbers and t When the generated random number P is compared t , then mosaic data enhancement is enabled for this round of training, otherwise mosaic data enhancement is not enabled for this round of training. t The value defaults to between 10-50%.

[0045] A random area mosaic data enhancement for industrial defect detection, including:

[0046] The data reading module is used to load and preprocess the industrial images to be inspected and the corresponding label information from the local storage system, and perform preprocessing operations, including but not limited to image denoising, resizing (to ensure that all images have the same resolution before being input into the model), normalization (scaling pixel values ​​to a uniform range, such as between 0 and 1), and possible preliminary image enhancement processing (such as contrast adjustment, brightness adjustment, etc.) to improve image quality and facilitate subsequent analysis;

[0047] The mosaic enhancement module is used to perform mosaic data enhancement processing on the image dataset. It simulates image quality problems that may be encountered in actual inspection scenarios, such as occlusion and blur, to increase the diversity and complexity of the dataset, thereby improving the robustness and generalization ability of the model for industrial defect detection;

[0048] The label post-processing module is used to remove some target labels that are blocked by coarse-grained mosaic blocks ( Greater than M p ) Severe occlusion (IoA greater than M iou ) to prevent the model from learning useless feature information;

[0049] The model training module is used to input the test image sets obtained by the image acquisition module into the algorithm for training. The training adopts forward propagation and back propagation to iteratively train the model.

[0050] The model testing module is used to input the test image sets obtained from the image acquisition module into the algorithm for detection and information calculation accuracy testing. It is used to evaluate the performance of the trained model on unknown data and ensure that the model has reliable accuracy and generalization ability in practical applications.

[0051] This embodiment uses the public steel surface defect NEU-DET dataset provided by Northeastern University. The dataset contains a total of 1,800 annotated images, with the following six types of defects: 'Crazing', 'Inclusion', 'Patches', 'Pitted_surface', 'Rolled-in_scale', 'Scratches'. In this embodiment, 80% of the images are used as training sets, 10% of the images are used as validation sets, and 10% of the images are used as test sets. First, the training set in the NEU-DET dataset and its corresponding annotation information are read. After steps 1 to 5, Precision, Recall, and mAP50 are used as the main evaluation indicators of the detection results on the NEU-DET dataset, and compared with the deep neural network model trained by the traditional data enhancement method. The results are shown in Table 1. It can be seen from Table 1 that the average accuracy of the method of the present invention is generally better than that of the network trained by the traditional data enhancement method. Table 1 shows the comparison results of the deep neural network model trained with the traditional area rejection method.

[0052]

[0053]

[0054] Table 1

[0055] The contents described in the embodiments of this specification are merely enumerations of implementation forms of the inventive concept and are for illustrative purposes only. The protection scope of the present invention should not be considered to be limited to the specific forms described in this embodiment, and the protection scope of the present invention also extends to equivalent technical means that can be thought of by ordinary technicians in this field based on the inventive concept.

Claims

1. A random area mosaic data enhancement method for industrial defect detection, characterized in that: The method comprises the following steps: Step 1: Read the defect images in the training set and their corresponding category and location label information; Step 2: Set the Patch-List parameter list, which stores several pre-defined values ​​to control the width and height of the generated mosaic blocks Step 3: During the model training process, the Patch-List parameter list set in step 2 will be randomly selected, and the selected The image is evenly divided into several grids. After the division is completed, each image grid is traversed and the probability parameter P is set. m To determine whether to perform mosaic blur processing on each grid and obtain the mosaic image; Step 4: Set the threshold parameter M based on step 3 iou With M p , and analyze the target area to be detected covered by the mosaic. When the coverage of the target area to be detected by the mosaic block and the size of the mosaic block used are greater than M iou With M p , delete the label information of the target; Step 5: During the model training process, enable data augmentation with a set probability, and then train according to steps 2 to 4 above until the model fully converges.

2. The random area mosaic data enhancement method for industrial defect detection according to claim 1, characterized in that: In step 2, the Patch-List parameter list is specifically a list including multiple integer (INT) types, and the number of integers included is between 3 and 6 by default.

3. The random area mosaic data enhancement method for industrial defect detection according to claim 1 or 2, characterized in that: In step 3, the method for randomly selecting the Patch-List parameter list is implemented by using the random.choice(Patch-List) function, and then the image is evenly divided into a number of grids as shown in formula (1); Among them, N w With N H They are the number of grids divided along the width direction of the image and the number of grids divided along the height direction of the image, and H and W are the width and height of the image respectively.

4. The random area mosaic data enhancement method for industrial defect detection according to claim 1 or 2, characterized in that: In step 3, the mosaic blur processing is performed by first using the np.mean() method to calculate the average color of the extracted area, and then filling the current grid cell with the average color in the current grid to achieve a mosaic effect.

5. The random area mosaic data enhancement method for industrial defect detection according to claim 1 or 2, characterized in that: In step 4, the analysis of the target area to be detected covered by the mosaic is specifically as follows: when the size of the mosaic block used is Greater than M p When the current grid unit is divided, it is defined as a bounding box with coordinates in the form of [Xb1, Yb1, Xb2, Yb2]. In addition, the label coordinate format is converted to the same form as the grid unit bounding box [Xl1, Yl1, Xl2, Yl2]. The IoA value between the grid unit and each label is calculated, and the overall IoA value belonging to the target is obtained by accumulating the IoA value of each label itself. The accumulated IoA is less than or equal to M. iou The label, and the mosaic block size used by Dangdang Less than M p When , all target labels in the graph are retained.

6. The random area mosaic data enhancement method for industrial defect detection according to claim 1 or 2, characterized in that: In step 5, the data enhancement is enabled under certain conditions and probabilities during the training process. Specifically, the probability parameter P is set. t In each round of model training, the random.random() function is used to generate random numbers and t When the generated random number P is compared t , then mosaic data enhancement is enabled for this round of training. In any case, mosaic data enhancement is not enabled for this round of training. The set P t The value is between 10-50%.

7. A system for implementing the random area mosaic data enhancement method for industrial defect detection as claimed in claim 1, characterized in that: The system comprises: A data reading module is used to load and preprocess the industrial images to be detected and the corresponding label information from the local storage system, and perform preprocessing operations, including but not limited to image denoising, resizing, normalization, and possible image enhancement preliminary processing to improve image quality and facilitate subsequent analysis; The mosaic enhancement module is used to perform mosaic data enhancement processing on the image dataset to simulate the image quality problems that may be encountered in actual inspection scenarios, so as to increase the diversity and complexity of the dataset, thereby improving the robustness and generalization ability of the model for industrial defect detection; The label post-processing module is used to remove the situation where some target labels are severely blocked by coarse-grained mosaic blocks. The judgment condition of the mosaic block is Greater than M p , the judgment condition for severe occlusion is that IoA is greater than M iou , to prevent the model from learning useless feature information; The model training module is used to input the test image sets obtained by the image acquisition module into the algorithm for training. The training adopts forward propagation and back propagation to iteratively train the model. The model testing module is used to input the test image sets obtained from the image acquisition module into the algorithm for detection and information calculation accuracy testing. It is used to evaluate the performance of the trained model on unknown data and ensure that the model has reliable accuracy and generalization ability in practical applications.

Images