Pixel-level data enhancement method based on small target detection in complex scene

Abstract

The invention provides a pixel-level data enhancement method based on small target detection in a complex scene. The pixel-level data enhancement method comprises the following steps: S1, preparing data; s2, randomly copying and pasting the instance target; s3, updating the enhanced data label and the mask; and S4, determining whether the data volume is sufficient. According to the pixel-level data enhancement method based on small target detection in the complex scene provided by the invention, the data set is expanded by simply copying and pasting the instance target on the target detection data, so that the limitation of spending a large amount of manual annotation in the past is broken through; meanwhile, compared with an existing enhancement means, the data enhancement effect is more suitable for solving the problem of small target detection in a complex scene, one-pixel-level data enhancement is achieved, the enhanced data is relatively attached to original target detection data, mosaic style pixel blocks cannot appear near pasted target pixels, and the detection accuracy is improved. The problem of confusion with a low-pixel detected target is solved, and invalid operation of data enhancement due to complex data background environment is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to a pixel-level data enhancement method based on small target detection in complex scenes. Background Art

[0002] With the rapid development of deep learning, deep learning has an absolute advantage in target detection tasks. However, due to the different appearances, shapes and postures of various detected objects, coupled with interference from factors such as illumination and occlusion during imaging, target detection is still challenging. If you want to achieve good target detection results, you need to rely on a large number of accurate and well-labeled data sets to be passed into the deep learning network for layer-by-layer learning.

[0003] However, how to obtain a large number of annotated data sets is a thorny issue. Currently, a lot of manpower is usually spent on manual data annotation, and then the annotated data sets are selected according to a certain proportion to perform data enhancement strategies, such as geometric transformation, color change, random cropping, random occlusion and other operations, so as to obtain multiple data levels. In addition, existing data enhancement, such as geometric transformation, color change, random cropping, random occlusion, random mapping and other operations, will improve the generalization and accuracy of the general target detection model. However, for image data sets with complex background environments and low-pixel targets, the data enhancement effect is not obvious, which can easily cause the target detection model to mislearn and increase the misrecognition rate. Although the data set size has been expanded, it cannot effectively improve the performance of the target detection model.

[0004] Therefore, it is necessary to provide a pixel-level data enhancement method based on small target detection in complex scenes to solve the above technical problems. Summary of the invention

[0005] The present invention provides a pixel-level data enhancement method based on small target detection in complex scenes, which solves the problem that a large amount of manpower is required and the image data set has complex background environment, and when the detected target is low in pixels, the data enhancement effect is not obvious and the performance of the target detection model cannot be effectively improved.

[0006] In order to solve the above technical problems, the present invention provides a pixel-level data enhancement method based on small target detection in complex scenes, comprising the following steps:

[0007] S1. Prepare data. Prepare the data set for enhancement. The prepared data includes: the data set of the category to be enhanced (i.e. the detected target) instance format data and the corresponding instance label file, the target detection yolo format data set and the corresponding yolo label file;

[0008] S2, randomly copy and paste the instance target, randomly select an instance format data containing the enhanced category, and a YOLO format data of the pasted target detection. Extract the instance target from the selected instance format data, find the target pixel block in the corresponding image according to the instance label, and use edge analysis to find the smallest enclosing rectangle of the target instance pixel block;

[0009] S3, update the enhanced data label and mask, and update the pixel mask of the original pasted target detection data according to the pixel mask of the pasted instance target. Perform random scale jitter with a corresponding threshold of α on the YOLO format label obtained in the above step 2, and add the jittered label to the txt label document of the original pasted target detection data to implement the small target data pasting and corresponding label update operation;

[0010] S4: Check whether the data volume is sufficient. If it is sufficient, perform data enhancement. If not, execute S2 to S3 in sequence until a satisfactory data volume is reached. Only then complete the data enhancement operation and pass the expanded data set into the deep learning model for subsequent model training.

[0011] Preferably, the operation process of edge analysis is as follows: Step 1: copy the original data image in the instance format, convert the read instance target image into a grayscale image and save it separately; Step 2: perform Gaussian noise reduction filtering on the converted grayscale image to remove useless noise pixels on the image, and perform Gaussian filtering on the pixel h(i,j) on the image to obtain H(i,j); Step 3: find the edge, that is, the position where the grayscale intensity changes the most. In the image, the gradient is usually used to represent the degree and direction of the change in the grayscale value. This patent selects the Sobel operator to calculate the gradient and direction. The Sobel operator G in the horizontal direction is x, used to detect the edge in the y-axis direction; Step 4: Use non-maximum suppression to eliminate edge false detection, exclude non-edge pixels, and retain only candidate edges. Traverse all pixel points on the gradient matrix and retain pixel values ​​with maximum values ​​in the edge direction. The purpose is to make the blurred boundary clear. That is, the maximum value of the gradient intensity on each pixel is retained, and other values ​​are deleted. For each pixel point, the following operation is performed: its gradient direction is approximated to the closest one among 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees or 315 degrees. At the same time, compare the gradient intensity of the pixel point and its gradient direction, positive and negative direction pixels. If the pixel point has the largest gradient intensity, it is retained, otherwise it is deleted, that is, the pixel point is assigned a value of 0; Step 5: Use the hysteresis threshold, that is, the high threshold and the low threshold, to determine which are the true edges of the target pixel block. Here, an upper threshold and a lower threshold are set. If a pixel in the image is greater than the upper threshold, it is considered to be a boundary, and if it is less than the lower threshold, it is considered not to be a boundary. If it is between the two, it is considered to be a candidate edge pixel. For the candidate edge pixel, the next step of analysis and processing is required. When the candidate edge pixel is adjacent to the 8 neighboring pixels on its edge, it can be determined to be an edge and retained, otherwise it is regarded as a non-edge and discarded, thereby completing the edge detection operation of the instance target pixel block; Step 6: Based on the edge of the extracted target instance pixel block, the rotating caliper algorithm is used to enumerate the edges of the polygon, try to make the bounding rectangle of the pixel block, compare the area of ​​the bounding rectangle, and select the bounding rectangle with the smallest area as the minimum bounding rectangle of the target instance pixel block.

[0012] Preferably, the Gaussian filter function calculation formula is:

[0013]

[0014] σ is the standard deviation, k is the size of the Gaussian filter kernel. This patent selects a 5×5 Gaussian filter, σ=1.4, and the approximate value of the Gaussian filter K is calculated to be:

[0015]

[0016] Use the specified Gaussian filter K to traverse each pixel in the image to obtain the smoothed denoised pixel value.

[0017] Preferably, the operator G x The mathematical expression is:

[0018]

[0019] Used to detect edges in the y-axis direction, the Sobel operator G in the vertical direction y , used to detect the edge in the x-axis direction. yThe mathematical expression of the operator is:

[0020] Using operator G x , G y Calculate the gradient G and direction θ of the image pixel change. The expressions of gradient G and direction θ are:

[0021]

[0023] Preferably, the minimum circumscribed rectangular frame is subjected to a coordinate conversion operation, the coordinates are normalized according to the yolo format, and a new txt file is created. Based on the coordinate information after the minimum circumscribed rectangular frame is converted, the label of the pasted instance target is converted into a yolo format label operation, and the converted label is written into the newly created txt file. Here, the instance data converted into the yolo format is obtained, and the size of its yolo format label box is set to H*W. The random scale jitter threshold is set to α. Because the enhanced data set is used for small target detection, a scaling operation is performed on the pasted instance target. The α value range is set to (0.2, 0.8). After random scale jitter, the size of the pasted instance yolo format label box is updated to H*W. new *W new , which is calculated as:

[0024] H new =H*α

[0025] W new =W*α

[0026] However, it should be noted that if the instance target is directly pasted onto the detection data of the pasted target, the detected target in the original image will be covered. Here, it is necessary to implement a sliding window strategy based on the label file of the pasted target detection data to find the range where there is no target to paste and perform a random pasting operation to avoid the problem of target occlusion and coverage.

[0027] Compared with the related art, the pixel-level data enhancement method based on small target detection in complex scenes provided by the present invention has the following beneficial effects:

[0028] The present invention provides a pixel-level data enhancement method based on small target detection in complex scenes. The method expands the data set by simply copying and pasting instance targets in target detection data, breaking through the limitation of previously spending a lot of manual annotation. At the same time, the effect of data enhancement is more suitable for solving the problem of small target detection in complex scenes than the existing enhancement means, achieving pixel-level data enhancement. The enhanced data is more consistent with the original target detection data, and no mosaic-style pixel blocks appear near the pasted target pixels, which solves the problem of confusion with low-pixel detected targets and avoids invalid operations of data enhancement due to complex data background environment, which is helpful to subsequently improve the detection performance of deep models. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flowchart of a preferred embodiment of a pixel-level data enhancement method based on small target detection in complex scenes provided by the present invention;

[0030] Figure 2 for Figure 1 Schematic diagram of the effect of pixel-level data enhancement for a single image data shown; DETAILED DESCRIPTION

[0031] The present invention will be further described below in conjunction with the accompanying drawings and implementation modes.

[0032] Please refer to Figure 1 and Figure 2 ,in, Figure 1 A flowchart of a first embodiment of a pixel-level data enhancement method based on small target detection in complex scenes provided by the present invention; Figure 2 for Figure 1 A pixel-level data enhancement method based on small target detection in complex scenes includes the following steps:

[0033] S1. Prepare data. Prepare the data set for enhancement. The prepared data includes: the data set of the category to be enhanced (i.e. the detected target) instance format data and the corresponding instance label file, the target detection yolo format data set and the corresponding yolo label file;

[0034] S2, randomly copy and paste the instance target, randomly select an instance format data containing the enhanced category, and a YOLO format data of the pasted target detection. Extract the instance target from the selected instance format data, find the target pixel block in the corresponding image according to the instance label, and use edge analysis to find the smallest enclosing rectangle of the target instance pixel block;

[0035] S3, update the enhanced data label and mask, and update the pixel mask of the original pasted target detection data according to the pixel mask of the pasted instance target. Perform random scale jitter with a corresponding threshold of α on the YOLO format label obtained in the above step 2, and add the jittered label to the txt label document of the original pasted target detection data to implement the small target data pasting and corresponding label update operation;

[0036] S4: Check whether the data volume is sufficient. If it is sufficient, perform data enhancement. If not, execute S2 to S3 in sequence until a satisfactory data volume is reached. Only then complete the data enhancement operation and pass the expanded data set into the deep learning model for subsequent model training.

[0037] The operation process of edge analysis is as follows: Step 1: copy the original data image in the instance format, convert the read instance target image into a grayscale image and save it separately; Step 2: perform Gaussian noise reduction filtering on the converted grayscale image to remove useless noise pixels on the image, and perform Gaussian filtering on the pixel h(i,j) on the image to obtain H(i,j); Step 3: find the edge, that is, the position where the grayscale intensity changes the most. In the image, the gradient is usually used to represent the degree and direction of the change in the grayscale value. This patent selects the Sobel operator to calculate the gradient and direction. The Sobel operator G in the horizontal direction x, used to detect the edge in the y-axis direction; Step 4: Use non-maximum suppression to eliminate edge false detection, exclude non-edge pixels, and retain only candidate edges. Traverse all pixel points on the gradient matrix and retain pixel values ​​with maximum values ​​in the edge direction. The purpose is to make the blurred boundary clear. That is, the maximum value of the gradient intensity on each pixel is retained, and other values ​​are deleted. For each pixel point, the following operation is performed: its gradient direction is approximated to the closest one among 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees or 315 degrees. At the same time, compare the gradient intensity of the pixel point and its gradient direction, positive and negative direction pixels. If the pixel point has the largest gradient intensity, it is retained, otherwise it is deleted, that is, the pixel point is assigned a value of 0; Step 5: Use the hysteresis threshold, that is, the high threshold and the low threshold, to determine which are the true edges of the target pixel block. Here, an upper threshold and a lower threshold are set. If a pixel in the image is greater than the upper threshold, it is considered to be a boundary, and if it is less than the lower threshold, it is considered not to be a boundary. If it is between the two, it is considered to be a candidate edge pixel. For the candidate edge pixel, the next step of analysis and processing is required. When the candidate edge pixel is adjacent to the 8 neighboring pixels on its edge, it can be determined to be an edge and retained, otherwise it is regarded as a non-edge and discarded, thereby completing the edge detection operation of the instance target pixel block; Step 6: Based on the edge of the extracted target instance pixel block, the rotating caliper algorithm is used to enumerate the edges of the polygon, try to make the bounding rectangle of the pixel block, compare the area of ​​the bounding rectangle, and select the bounding rectangle with the smallest area as the minimum bounding rectangle of the target instance pixel block.

[0038] The Gaussian filter function calculation formula is:

[0039]

[0040] σ is the standard deviation, k is the size of the Gaussian filter kernel. This patent selects a 5×5 Gaussian filter, σ=1.4, and the approximate value of the Gaussian filter K is calculated to be:

[0041]

[0042] Use the specified Gaussian filter K to traverse each pixel in the image to obtain the smoothed denoised pixel value.

[0043] The operator G x The mathematical expression is:

[0044]

[0045] Used to detect edges in the y-axis direction, the Sobel operator G in the vertical direction y , used to detect the edge in the x-axis direction. yThe mathematical expression of the operator is:

[0046] Using operator G x , G y Calculate the gradient G and direction θ of the image pixel change. The expressions of gradient G and direction θ are:

[0047]

[0048] The coordinate conversion operation is performed on the minimum bounding rectangle, the coordinates are normalized according to the yolo format, and a new txt file is created. According to the coordinate information after the minimum bounding rectangle is converted, the label of the pasted instance target is converted into a yolo format label operation, and the converted label is written into the newly created txt file. Here, the instance data converted into the yolo format is obtained, and the size of its yolo format label box is set to H*W. The random scale jitter threshold is set to α. Because the enhanced data set is used for small target detection, a scaling operation is performed on the pasted instance target. The α value range is set to (0.2, 0.8). After random scale jitter, the size of the pasted instance yolo format label box is updated to H. new *W new , which is calculated as:

[0049] H new =H*α

[0050] W new =W*α

[0051] However, it should be noted that if the instance target is directly pasted onto the detection data of the pasted target, the detected target in the original image will be covered. Here, it is necessary to implement a sliding window strategy based on the label file of the pasted target detection data to find the range where there is no target to paste and perform a random pasting operation to avoid the problem of target occlusion and coverage.

[0052] The sliding window strategy here is a way to find the pasting range during data enhancement pasting. First, we need to define a fixed-size window. The size of this window should be roughly the same as the size of the target we expect to detect. Here the window size is set to (H new +5,W new +5) is the size of the YOLO format annotation box of the pasted instance after random scale jitter. The extra 5 is designed here to ensure that the pasted instance target maintains a certain distance from the original target of the data to prevent confusion with the original label target pixel and avoid problems in subsequent model training and learning. When the sliding window strategy is initially executed, a coordinate point p(x p ,y p ). Then take the point p(xp ,y p ) is the upper left corner of the window, and is placed on the YOLO format data of the pasted target detection to determine whether the image in the anchor window contains the target pixel block. If it does not contain the target pixel block, the paste operation is performed, otherwise the window is moved to the right or to the left.

[0053] Move down a certain step length β, which can be half of the window width or height, or a smaller value. Here β is:

[0054] Then, the same judgment operation is performed on the image at the new window position. Repeat the steps of the above sliding window strategy until a window is slid to and the image in the anchor window does not contain the target pixel block, determine the pastable range, and perform the instance target pasting operation according to the pasting range.

[0055] Compared with the related art, the pixel-level data enhancement method based on small target detection in complex scenes provided by the present invention has the following beneficial effects:

[0056] By simply copying and pasting instance targets in the target detection data to expand the data set, the limitation of previous manual labeling is broken through. At the same time, the effect of data enhancement is more suitable for solving small target detection problems in complex scenarios than existing enhancement methods, achieving a pixel-level data enhancement. The enhanced data is more consistent with the original target detection data, and no mosaic-style pixel blocks will appear near the pasted target pixels, which solves the problem of confusion with low-pixel detected targets and avoids invalid data enhancement operations due to complex data background environment, which will help to improve the detection performance of deep models in the future.

[0057] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A pixel-level data enhancement method for small target detection in complex scenes, characterized in that: The following steps are involved: S1. Prepare data. Prepare the data set for enhancement. The prepared data includes: the data set of the category to be enhanced (i.e. the detected target) instance format data and the corresponding instance label file, the target detection yolo format data set and the corresponding yolo label file; S2, randomly copy and paste the instance target, randomly select an instance format data containing the enhanced category, and a YOLO format data of the pasted target detection. Extract the instance target from the selected instance format data, find the target pixel block in the corresponding image according to the instance label, and use edge analysis to find the smallest enclosing rectangle of the target instance pixel block; S3, update the enhanced data label and mask, and update the pixel mask of the original pasted target detection data according to the pixel mask of the pasted instance target. Perform random scale jitter with a corresponding threshold of α on the YOLO format label obtained in the above step 2, and add the jittered label to the txt label document of the original pasted target detection data to implement the small target data pasting and corresponding label update operation; S4: Check whether the data volume is sufficient. If it is sufficient, perform data enhancement. If not, execute S2 to S3 in sequence until a satisfactory data volume is reached. Only then complete the data enhancement operation and pass the expanded data set into the deep learning model for subsequent model training.

2. According to claim 1, a pixel-level data enhancement method based on small target detection in complex scenes is characterized in that: The operation process of edge analysis is as follows: first, copy the original data image in the instance format, convert the read instance target image into a grayscale image and save it separately; Step 2: Perform Gaussian noise reduction filtering on the converted grayscale image to remove useless noise pixels on the image. For the pixel h(i,j) on the image, perform Gaussian filtering to obtain H(i,j); Step 3: Find the edge, that is, the location where the grayscale intensity changes the most. In an image, the gradient is usually used to represent the degree and direction of the grayscale value change. This patent uses the Sobel operator to calculate the gradient and direction. The Sobel operator G in the horizontal direction x , used to detect the edge in the y-axis direction; Step 4: Use non-maximum suppression to eliminate false edge detection, exclude non-edge pixels, and retain only candidate edges. Traverse all pixel points on the gradient matrix and retain pixel values ​​with maximum values ​​in the edge direction. The purpose is to make the blurred boundary clear. That is, the maximum value of the gradient intensity on each pixel is retained, and other values ​​are deleted. For each pixel point, perform the following operation: approximate its gradient direction to the closest one among 0 degrees, 45 degrees, 90 degrees, 135 degrees, 180 degrees, 225 degrees, 270 degrees or 315 degrees. At the same time, compare the gradient intensity of the pixel point with its gradient direction and the positive and negative direction pixels. If the pixel point has the largest gradient intensity, it will be retained, otherwise it will be deleted, that is, the pixel point is assigned a value of 0; Step 5: Use the hysteresis threshold, that is, the high threshold and the low threshold, to determine which are the true edges of the target pixel block. Here, an upper threshold and a lower threshold are set. If the pixel point in the image is greater than the upper threshold, it is considered to be a boundary, and if it is less than the lower threshold, it is considered not to be a boundary. The pixels between the two are considered to be candidate edge pixels. For the candidate edge pixels, the next step of analysis and processing is required. When the candidate edge pixel point is adjacent to the 8 neighboring pixels on its edge, it can be determined to be an edge and retained, otherwise it is considered to be a non-edge and discarded, thereby completing the edge detection operation of the instance target pixel block; Step 6: Based on the edge of the extracted target instance pixel block, the rotating caliper algorithm is used to enumerate the edges of the polygon, try to make the bounding rectangle of the pixel block, compare the area of ​​the bounding rectangle, and select the bounding rectangle with the smallest area as the minimum bounding rectangle of the target instance pixel block.

3. According to claim 1, a pixel-level data enhancement method based on small target detection in complex scenes is characterized in that: The Gaussian filter function calculation formula is: σ is the standard deviation, k is the size of the Gaussian filter kernel. This patent selects a 5×5 Gaussian filter, σ=1.4, and the approximate value of the Gaussian filter K is calculated to be: Use the specified Gaussian filter K to traverse each pixel in the image to obtain the smoothed denoised pixel value.

4. The pixel-level data enhancement method based on small target detection in complex scenes according to claim 1, characterized in that: The operator G x The mathematical expression is: Used to detect edges in the y-axis direction, the Sobel operator G in the vertical direction y , used to detect the edge in the x-axis direction. y The mathematical expression of the operator is: Using operator G x , G y Calculate the gradient G and direction θ of the image pixel change. The expressions of gradient G and direction θ are: 。 5. The pixel-level data enhancement method based on small target detection in complex scenes according to claim 1, characterized in that: The coordinate conversion operation is performed on the minimum bounding rectangle, the coordinates are normalized according to the yolo format, and a new txt file is created. According to the coordinate information after the minimum bounding rectangle is converted, the label of the pasted instance target is converted into a yolo format label operation, and the converted label is written into the newly created txt file. Here, the instance data converted into the yolo format is obtained, and the size of its yolo format label box is set to H*W. The random scale jitter threshold is set to α. Because the enhanced data set is used for small target detection, a scaling operation is performed on the pasted instance target. The α value range is set to (0.2, 0.8). After random scale jitter, the size of the pasted instance yolo format label box is updated to H. new *W new , which is calculated as: H new =H*a W new =W*a However, it should be noted that if the instance target is directly pasted onto the detection data of the pasted target, the detected target in the original image will be covered. Here, it is necessary to implement a sliding window strategy based on the label file of the pasted target detection data to find the range where there is no target to paste and perform a random pasting operation to avoid the problem of target occlusion and coverage.

Images