An image data augmentation method

Abstract

The application belongs to the technical field of image data processing, and particularly relates to an image data augmentation method, which comprises the following steps: firstly, collecting images under a standard scene, and labeling according to preset scene labels to generate a training data set and obtain target data; then, training and constructing a generative adversarial network according to the training data set; finally, inputting the target data, a preset scene label value range and a number of scene labels corresponding to images in the training data set into the generative adversarial network, and inputting a preset sampling range and a preset sampling number into the generative adversarial network to output augmented image data; the generative adversarial network comprises a generator, a discriminator, a scene mapping network and a scene encoder. The application can solve the problem that the image augmentation method for the collected training data in the prior art does not consider the imbalance of scenes, resulting in a decrease in the precision of a deep learning model in some scenes.

Description

An image data augmentation method Technical Field

[0001] This invention belongs to the field of image data processing technology, and in particular relates to an image data augmentation method. Background Technology

[0002] With the development of artificial intelligence, deep learning models have become a hot research topic. Deep learning models are widely used in fields such as speech recognition and image recognition. Especially in the field of image recognition, deep learning models consist of multi-level learning and processing layers that extract abstract features. They use deep neural networks and propagation algorithms to learn, process, and analyze image features in order to identify targets and objects of various patterns.

[0003] To achieve better generalization capabilities, deep learning networks require a large amount of high-quality and diverse test data. However, collecting large amounts of test data, especially data from different scenarios, often presents significant challenges. On one hand, data is insufficient and acquisition costs are high; on the other hand, data quality is poor. Traditional data processing methods include rotation, flipping, cropping, and integration into the training framework. However, these methods still cannot yield a large amount of high-quality data. To address this, existing image data augmentation methods assume that image noise follows a normal distribution. They utilize three levels of feature images—feature maps, prior constraints, and constraint feature vectors—and perform channel concatenation. The prior constraints use normally distributed random noise. However, in reality, noise distribution is often a fusion of various noise types, such as Gaussian and Poisson noise. Therefore, the dataset is limited to the labeled dataset, resulting in imbalanced data across different scenarios. Summary of the Invention

[0004] The technical problem solved by this invention is to provide an image data augmentation method to address the issue that existing image augmentation methods for collected training data do not consider scene imbalances, leading to reduced accuracy of deep learning models in some scenes.

[0005] The basic solution provided by this invention: an image data augmentation method, comprising:

[0006] S1: Collect images in a standard scene, label them according to the preset scene labels, generate training dataset one, and obtain target data for object detection;

[0007] S2: Based on training dataset 1, train and establish a generative adversarial network for multi-scene fusion parameter fusion; the generative adversarial network includes a generator, a discriminator, a scene mapping network, and a scene encoder;

[0008] S3: Input the target data, the preset range of scene label values, and the number of scene labels corresponding to the image in the target data into the generative adversarial network, and input the preset sampling range and preset number of samples into the generative adversarial network to output the augmented image data.

[0009] Furthermore, the preset scene labels in S1 include color temperature parameters, brightness parameters, and motion blur parameters.

[0010] Furthermore, S2 includes:

[0011] S2-1: After taking the logarithm of the color temperature parameter and brightness parameter, and sampling them together with the motion blur parameter according to the preset sampling range and preset sampling number, a mask vector is formed, and then input into the mapping network along with the image to map out the style code.

[0012] S2-2: Extract style codes from the mapping network using a scene encoder.

[0013] Furthermore, S3 includes:

[0014] S3-1: Generate the output image by using the style code and the image in the target data;

[0015] S3-2: A discriminator is used to calculate the loss value of the generator through the loss function. The corresponding network parameters are adjusted according to the loss value until a converged generative adversarial network is obtained.

[0016] Furthermore, the loss function in S3-2 includes adversarial loss and style diversity loss, and the adversarial loss is calculated as follows:

[0017]

[0018] in, x represents the image, and m represents the mask vector;

[0019] The formula for calculating the style diversity loss is as follows:

[0020]

[0021] in,

[0022] Furthermore, the image data size in the target data is consistent with the input size of the generator.

[0023] The principle and advantages of this invention are as follows: When acquiring image data, this invention labels the acquired images according to image labels under standard scenes. At the same time, it sets up a generative adversarial network and improves the input parameters of the generative adversarial network, including the sampling labels, the sampling range, and the number of samples. Through the generation and labeling of style codes, the generated image data can firstly limit the range of standard scenes, and secondly, it is not limited to a small number of pre-acquired scenes. Thus, the generated images are more consistent with the images actually acquired by the imaging system, which can better improve the performance of deep learning models in real scenes, especially the performance of scenes not included in the training dataset. Attached Figure Description

[0024] Figure 1 is a flowchart of an embodiment of the present invention. Detailed Implementation

[0025] The following detailed description illustrates the specific implementation method:

[0026] The successful application of deep learning networks requires a large-scale dataset. Image augmentation technology expands the size of the training dataset by making a series of random changes to the training images to generate similar but different training samples. In addition, image augmentation also reduces the network model's dependence on certain attributes by randomly changing the training samples, thereby improving the model's generalization ability. For this reason, existing image augmentation methods assume that the image noise follows a normal distribution and process it through a generator, which leads to the problem of imbalanced scene in the generated dataset.

[0027] Therefore, this application proposes an image data augmentation method, the implementation of which is basically as shown in Figure 1: including:

[0028] S1: Collect images in standard scenes, label them according to preset scene labels, generate a training dataset, and obtain target data for object detection;

[0029] In S1, image data with different color temperatures, brightness, and motion blur under standard scenes are collected and labeled to generate a training dataset. Color temperature and brightness are measured by instruments, and motion blur is obtained based on the degree of blur of moving objects in the image data, specifically taking floating-point values ​​in the range [0, 1]. The collection under standard scenes is for training a generative adversarial network, which is the basis of the augmentation method in this case. These three labels are the main attributes that affect the image, but are not limited to these three labels, there are other attributes such as contrast.

[0030] The target data used for object detection is the data that is actually augmented. For example, when recognizing car license plates, the license plate image that needs to be augmented is the target data.

[0031] S2: Based on training dataset 1, train and establish a generative adversarial network for multi-scene fusion parameter fusion; the generative adversarial network includes a generator, a discriminator, a scene mapping network, and a scene encoder;

[0032] The preset range of scene labels is as follows: the color temperature is T, the brightness is L, and the motion blur is B in the standard scene. Therefore, the range of color temperature is [T1, T2], the range of brightness is [L1, L2], and the range of motion blur is [B1, B2].

[0033] The specific number of scene labels corresponding to the image data in the training dataset is as follows: for each image data in the training dataset, the number of generated corresponding color temperatures is J, the number of generated corresponding brightness is K, and the number of generated corresponding motion blurs is L.

[0034] The specific data mentioned above is input into a Generative Adversarial Network (GAN), which includes a generator, a discriminator, a scene mapping network, and a scene encoder. The scene mapping network is responsible for generating style codes. Specifically, it takes the logarithm of the color temperature and brightness parameters, and samples them together with the motion blur according to a preset sampling range and a preset number of samples to form a mask vector. The scene mapping network also maps the mask vector and the image together to form a style code. In this embodiment, the style code represents the image style under a standard scene. For example, the image style is set according to a person's gender. In this case, the style represents makeup, body features, and hairstyle. By using the generated style code, only a small amount of image data needs to be collected. Through the transfer processing of the GAN, more scene images and uniform scene images can be obtained.

[0035] The mask vector can be represented as m = [lgT, lgL, B]. In practice, the scene mapping network consists of a multilayer perceptron.

[0036] Therefore, S2 also includes:

[0037] S2-1: After taking the logarithm of the color temperature parameter and brightness parameter, and sampling them together with the motion blur parameter according to the preset sampling range and preset sampling number, a mask vector is formed, and then input into the mapping network along with the image to map out the style code.

[0038] S2-2: Extract style codes from the mapping network using a scene encoder.

[0039] S3: Input the target data, the preset range of scene label values, and the number of scene labels corresponding to the image in the target data into the generative adversarial network, and input the preset sampling range and preset number of samples into the generative adversarial network to output the augmented image data.

[0040] The target data of user target detection, the preset range of scene label values, and the number of scene labels corresponding to the images in the target data are input into the trained generative adversarial network. Then, the sampled data is input into the generative adversarial network according to the sampling range and preset number of samples set by the user, and the augmented image data is output.

[0041] In this embodiment, the image data size in the target data is consistent with the input size of the generator.

[0042] S3 includes:

[0043] S3-1: Generate the output image by using the style code and the image in the target data;

[0044] S3-2: A discriminator is used to calculate the loss value of the generator through the loss function. The corresponding network parameters are adjusted according to the loss value until a converged generative adversarial network is obtained.

[0045] The loss functions include adversarial loss and style diversity loss. The adversarial loss is calculated as follows:

[0046]

[0047] in, x represents the image, and m represents the mask vector;

[0048] The formula for calculating style diversity loss is as follows:

[0049]

[0050] in,

[0051] The generated style code is extracted by the scene editor and injected into the generator through instance normalization. The generator receives the style code and the image, and generates an image corresponding to the style code. The instance normalization method is specifically represented by the following formula:

[0052]

[0053] Where s represents the style code and x represents the image.

[0054] After the generator generates an image with the corresponding style code, a discriminator is used to calculate the generator's loss value using a loss function. The corresponding network parameters are then adjusted based on the loss value until a converged generative adversarial network is obtained. In other embodiments of this example, other loss functions may be used, and this application does not impose any restrictions.

[0055] The technical solution of this application collects a small number of images in a standard scene. Based on the set range of parameters such as color temperature, brightness and motion blur of the scene and the number of samples, a generative adversarial network is used to perform scene transfer and generate uniform images of the scene, which can better improve the performance of deep learning networks in real-world scenes.

[0056] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An image data augmentation method, characterized in that: include: S1: Collect images under standard scenes, label them according to preset scene labels, generate training dataset one, and obtain target data for target detection; the preset scene labels in S1 include color temperature parameters, brightness parameters, and motion blur parameters; S2: Based on training dataset one, train and establish a generative adversarial network for multi-scene fusion parameter fusion; the generative adversarial network includes a generator, a discriminator, a scene mapping network, and a scene encoder; The scene mapping network consists of a multilayer perceptron and is used to map mask vectors to images as style codes. S2 includes: S2-1: Taking the logarithm of the color temperature parameter and brightness parameter, and sampling them together with the motion blur parameter according to a preset sampling range and a preset sampling number to form a mask vector, and inputting it along with the image into the scene mapping network to map out the style code; S2-2: Extracting the style code from the scene mapping network through the scene encoder; S3: Inputting the target data, the preset scene label value range, and the number of scene labels corresponding to the image in the target data into the generative adversarial network, and inputting the preset sampling range and preset sampling number into the generative adversarial network, and outputting the augmented image data; S3 includes: S3-1: Generating the output image by passing the style code and the image in the target data through the generator; S3-2: Using a discriminator, calculating the loss value of the generator through a loss function, and adjusting the corresponding network parameters according to the loss value until a converged generative adversarial network is obtained.

2. The image data augmentation method according to claim 1, characterized in that: The loss function in S3-2 includes adversarial loss and style diversity loss. The adversarial loss is calculated as follows: in, Where x represents the image and m represents the mask vector; the formula for calculating the style diversity loss is as follows: in, ， 。 3. The image data augmentation method according to claim 2, characterized in that: The image data size in the target data is the same as the input size of the generator.

Images