A data augmentation method for small-scale targets

Abstract

The present invention provides a data augmentation method for small-scale targets. By using the method of copy and paste and adopting trilinear interpolation to improve the quality of inserted images, it solves the problem of sample imbalance of small-scale targets in neural networks, and realizes the function of improving the model performance of small-scale target detection and semantic segmentation based on deep learning technology. Without changing the basic structure of the image (such as texture, object, context semantic environment, etc.), the present invention effectively increases the proportion of small-scale targets in the training dataset, enhances the weight of the small-scale dataset in the training of deep neural networks, and alleviates the problem of sample imbalance. The present invention is used for the establishment and enhancement of datasets in practical scenarios such as the detection of lesions in clinical medicine, the discovery and tracking of suspicious targets in the air, etc., improves the overall performance of small-scale target detection and segmentation, helps the subsequent model training to achieve better results, and enhances the robustness of the model.

Description

Technical Field

[0001] The present invention belongs to the technical field of deep learning, and in particular relates to a data enhancement method for small-scale targets. Background Art

[0002] Object detection is an important research direction in the field of computer vision and is also the basis of other complex visual tasks. In recent years, with the rapid development of deep learning (network structures such as Alexnet, ENet, UNet, FastSCNN, etc.) and the use of some data enhancement methods, object detection has achieved great success in both accuracy and speed.

[0003] However, compared with regular-sized objects, small object detection has long been a difficult point in object detection. Problems with small-scale object detection include fewer available features, a small proportion in existing datasets, and unbalanced samples. This has led to the fact that on some public detection datasets such as MS COCO, the performance of small-scale object detection is usually less than half of that of large objects. Currently, popular data augmentation methods are to rotate, scale, color change, flip, etc. images. However, these data augmentation methods are based on the perspective of the entire image, rather than the target level, resulting in poor performance in the accuracy of small-scale object separation.

[0004] Traditional data enhancement methods, such as rotation, scaling, and flipping, do not increase the number of small-scale samples. The defects of traditional data enhancement methods make them unable to solve the problem that the number of small-scale samples is too small in the entire sample. In order to segment small-scale targets more accurately, an algorithm is needed to solve the problem of small-scale targets accounting for a small proportion in the data set and unbalanced samples. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a data enhancement method for small-scale targets, so as to improve the overall performance of detection and segmentation of small-scale targets.

[0006] The technical solution adopted by the present invention to solve the above technical problems is: a data enhancement method for small-scale targets, comprising the following steps:

[0007] S1: define and collect small-scale targets, convert the image format of small-scale targets and obtain small-scale target images with higher resolution through scaling;

[0008] S2: Select the mapping area of ​​the small-scale target to obtain a synthetic image that conforms to prior common sense;

[0009] S3: replicate and enhance small-scale objects to generate a small-scale object enhanced dataset;

[0010] S4: Evaluate the performance of the small-scale target enhancement dataset.

[0011] According to the above scheme, in step S1, the specific steps are as follows:

[0012] S11: Define the small scale as the ratio of the pixels of each target in the dataset to the total pixels, and determine and collect small-scale targets according to the definition.

[0013] S12: Select the small-scale target as the object to be copied according to the preset requirements; convert the collected small-scale target into a standard image format.

[0014] S13: Perform scaling operations on the collected small-scale targets to increase the robustness of the small-scale targets; at the same time, use the method of trilinear interpolation to generate the scaled small-scale targets, obtain high-resolution small-scale target images, ensure the clarity of the small-scale targets, and improve the image quality of the small-scale targets.

[0015] Furthermore, in step S12, the semantic information of the selected small-scale target is completely preserved.

[0016] According to the above scheme, in step S2, the specific steps are as follows:

[0017] S21: Based on the small-scale targets selected in step S12, determine the mapping area based on the context scene.

[0018] S22: Place the small-scale target in the mapping area, and select the placement position so that the small-scale target does not block other small-scale targets or key semantic information areas, and the small-scale target conforms to the mapping area, thereby obtaining a synthetic image that conforms to the prior common knowledge.

[0019] According to the above scheme, in step S3, the specific steps are as follows:

[0020] S31: Copy the small-scale target to the mapping area, and replace the background of the original image of the small-scale target with the background of the mapping area.

[0021] S32: Perform data augmentation on the small-scale target copied to the mapping area, including translation, rotation, scaling, adding Gaussian white noise, and blurring.

[0022] S33: Select the processed pictures; loop through steps S1 to S3 to generate a small-scale target enhancement dataset.

[0023] According to the above scheme, in step S4, the specific steps are as follows:

[0024] The evaluation metrics include counting the proportion of small-scale targets in the small-scale target enhancement dataset, and the detection performance and segmentation performance of small-scale targets in different depth convolutional object detection or semantic segmentation models.

[0025] According to the above solution, it further includes a method for generating a context-aware mapping area.

[0026] A computer storage medium stores a computer program executable by a computer processor, and the computer program executes a data augmentation method for small-scale targets.

[0027] The beneficial effects of the present invention are as follows:

[0028] 1. A data augmentation method for small-scale targets of the present invention uses the method of copy and paste and adopts the method of trilinear interpolation to improve the quality of inserted images, solves the problem of unbalanced samples of small-scale targets in neural networks, and realizes the function of improving the model performance of small-scale target detection and semantic segmentation based on deep learning technology.

[0029] 2. Without changing the basic structure of the image (such as texture, object, context semantic environment, etc.), the present invention effectively increases the proportion of small-scale targets in the training dataset, improves the weight of the small-scale dataset in the training of deep neural networks, and alleviates the problem of unbalanced samples.

[0030] 3. The present invention is used for the establishment and enhancement of datasets in practical scenarios such as the detection of lesions in clinical medicine, the discovery and tracking of suspicious targets in the air, etc., improves the overall performance of small-scale target detection and segmentation, helps the subsequent model training to achieve better results, and enhances the robustness of the model. Description of the Drawings

[0031] Figure 1 is a flowchart of an embodiment of the present invention.

[0032] Figure 2 is a schematic diagram of the instance operation of an embodiment of the present invention. Detailed Embodiments

[0033] The present invention will be further described in detail below with reference to the drawings and specific embodiments.

[0034] See Figure 1 , a data augmentation method for small-scale targets of an embodiment of the present invention includes the following steps:

[0035] S1: Collect and read small-scale targets;

[0036] S11: Determine small-scale targets according to the definition of small scale, that is, the ratio of the pixels of each target in the dataset to the total pixels, and collect small-scale targets;

[0037] S12: Since there may be multiple small-scale targets in an image, (see Figure 2For a) of this, select a suitable small-scale target as the object to be replicated according to requirements; convert the collected small-scale target into a standard image format;

[0038] S13: Perform scaling operations on the collected small-scale target to increase its robustness; at the same time, use trilinear interpolation to generate the scaled small-scale target and obtain a higher-resolution small-scale target image to ensure the clarity of the small-scale target (see Figure 2 For b) of this, improve the image quality of the small-scale target.

[0039] Preferably, the semantic information of the selected small-scale target should be preserved completely.

[0040] S2: Select the mapping area;

[0041] S21: Based on the small-scale target selected in S11, determine the background area according to the context scene;

[0042] S22: Place the small-scale target in the background and select a reasonable position to make the small-scale target independent of surrounding objects, not block other targets, and be able to blend into the background with the small-scale target.

[0043] Since the image after small-scale copy-paste synthesis needs to meet certain prior conditions, that is, the mapping area and the small-scale target should conform to certain prior knowledge, such as a person cannot float in the sky, a street lamp cannot grow on a wall, near objects are small and far objects are large, etc. Therefore, design an algorithm to select a suitable paste area for the small-scale target. Such a reasonable area should meet two conditions: First, it does not block other small-scale targets or key semantic information areas; Second, the small-scale target should conform to the background of the mapping area.

[0044] S3: Copy and transform the small-scale target;

[0045] S31: Based on the small-scale target and the image mapping area determined by S13 and S21 respectively, copy the small-scale target to the mapping area;

[0046] S32: Perform data augmentation on the small-scale target copied to the mapping area, including translation, rotation, scaling, adding Gaussian white noise, blurring, etc. of the small-scale target.

[0047] S33: Select the processed pictures to generate a dataset for data augmentation of the small-scale target.

[0048] Select another image (see Figure 2c), according to the appropriate region selected by S2, paste the small-scale target into a newly selected image. At this time, the pasted small-scale target still carries the background of the original image. Then, replace the background of the original image with the background of the new image to obtain the final image (see Figure 2 d) of Figure 2 . Finally, the small-scale target enhanced dataset is generated by looping S1, S2, and S3.

[0049] S4: Evaluate the small-scale target enhanced dataset.

[0050] Perform performance evaluation on the obtained small-scale enhanced dataset. The evaluation metrics include counting the proportion of small-scale objects in the enhanced small-scale dataset, and the detection performance and segmentation performance of small-scale objects in different-depth convolutional object detection or semantic segmentation models.

[0051] The data augmentation technology for small-scale targets includes, but is not limited to, algorithms for generating small-scale targets and enhancement, and also includes methods for generating context-aware mapping regions.

[0052] The above embodiments are only used to illustrate the design concept and features of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, all equivalent changes or modifications made according to the principles and design concepts disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A data augmentation method for small-scale targets, characterized in that: It includes the following steps: S1: Define and collect small-scale targets, convert the picture format of small-scale targets, and obtain higher-resolution small-scale target pictures through scaling; the specific steps are: S11: Define the small scale as the proportion of the pixels of each target in the dataset to the total pixels, and determine and collect small-scale targets according to the definition; S12: Select small-scale targets as the objects to be copied according to preset requirements; convert the collected small-scale targets into standard picture formats; S13: Perform scaling operations on the collected small-scale targets to increase the robustness of small-scale targets; at the same time, use the method of trilinear interpolation to generate scaled small-scale targets, obtain higher-resolution small-scale target pictures, ensure the clarity of small-scale targets, and improve the picture quality of small-scale targets; S2: Select the mapping area of small-scale targets to obtain a synthetic image that conforms to the well-known common sense of prior knowledge; the specific steps are: S21: Based on the small-scale targets selected in step S12, determine the mapping area based on the context scene; S22: Place the small-scale targets in the mapping area, and select the placement position so that the small-scale targets do not block other small-scale targets or key semantic information areas, and the small-scale targets conform to the mapping area, so as to obtain a synthetic image that conforms to the well-known common sense of prior knowledge; S3: Copy and enhance small-scale targets to generate a small-scale target enhanced dataset; The specific steps are: S31: Copy the small-scale targets to the mapping area, and replace the background of the original image of the small-scale targets with the background of the mapping area; S32: Perform data augmentation on the small-scale targets copied to the mapping area, including translation, rotation, scaling, adding Gaussian white noise, and blurring; S33, select the pictures processed above; loop steps S1 to S3 to generate a small-scale target enhanced dataset; S4: Evaluate the performance of the small-scale target enhanced dataset.

2. A data augmentation method for small-scale targets according to claim 1, characterized in that: In step S12, the semantic information of the selected small-scale targets is completely preserved.

3. A data augmentation method for small-scale targets according to claim 1, characterized in that: In step S4, the specific steps are: The evaluation indicators include counting the proportion of small-scale targets in the small-scale target enhanced dataset, the detection performance and segmentation performance of small-scale targets in different depth convolutional object detection or semantic segmentation models.

4. A data augmentation method for small-scale targets according to claim 1, characterized in that: It also includes a method for generating a context-aware mapping area.

5. A computer storage medium, characterized in that: It stores a computer program that can be executed by a computer processor, and this computer program executes a data augmentation method for small-scale targets according to any one of claims 1 to 4.

Images