A differentiable stochastic image data augmentation method and system

Abstract

The present application belongs to the field of computer vision and artificial intelligence, and particularly relates to a differentiable random image data augmentation method and system, aiming at solving the problem that the prior art does not consider the difference between the features of the data sets, so that the augmented data set is less helpful for the artificial intelligence method to more accurately achieve the target computer vision task. The present application comprises: defining an augmentation search space and operator differentiable parameters of the image data to be augmented; constructing a spatial search neural network model and training; performing relaxation sampling of the augmented operator obtained by training and normal distribution sampling of the transformation amplitude of the augmented operator, and normalizing the sampling results; and augmenting the image data to be augmented based on the final augmented operator and the transformation amplitude obtained by normalization. The present application learns the data augmentation strategy according to the characteristics of the data set itself, realizes fast and customized data augmentation, and thus helps the artificial intelligence method to more accurately achieve the target computer vision task.

Claims

1. A method for augmenting differentiable random image data, characterized in that, The expansion method includes: Step S100: Define the expansion search space of the image data to be expanded, and define the differentiable parameters of the operator corresponding to the expansion search space; Step S200: Construct a spatial search neural network model, and based on the image data to be expanded, perform forward training of the operator's differentiable parameters with the aim of minimizing the model loss, to obtain the normal distribution followed by the sampling weights and transformation amplitude of the expansion operator; Step S300: Perform relaxation sampling of the expansion operator based on the sampling weight of the expansion operator, perform normal distribution sampling based on the normal distribution followed by the transformation amplitude of the expansion operator, and normalize the sampling results to obtain the final expansion operator and transformation amplitude. Step S400: Expand the image data to be expanded based on the final expansion operator and the transformation amplitude; Step S100 defines the expansion search space for the image data to be expanded, including: Step S110: Define a set of operators for the image data to be expanded, the set of operators including spatial transformation operators and color transformation operators; Step S120: Define the transformation amplitude of each operator in the operator set and normalize the transformation amplitude. The spatial transformation operators include translation operators, rotation operators and affine operators. The color transformation operators include contrast stretching operators, grayscale operators and inversion operators. Step S130: The operator sampling weight set corresponding to the operator set and the normalized transformation amplitude of each operator constitute the expanded search space of the image data to be expanded; Step S100 defines the operator differentiable parameters, including: Step S140: Assign each operator in the operator set a learnable sampling weight, and achieve differentiability processing of the operator sampling weight through relaxation sampling to obtain differentiable operator sampling weights; Step S150: Make the transformation amplitude of each operator follow a learnable normal distribution, and realize the differentiability processing of the normal distribution of the operator transformation amplitude through reparameterization to obtain the normal distribution of the transformation amplitude of the differentiable operator. Step S160: The sampling weights of the differentiable operator and the normal distribution of the transformation amplitude of the differentiable operator constitute the operator differentiable parameters corresponding to the search space. The forward training of the operator's differentiable parameters in step S200 includes: Step S210: Construct a spatial search neural network model, define the number of rounds to be searched on the dataset, and randomly and evenly divide the image dataset to be expanded into two parts, one as the training set and the other as the validation set. Step S220: Fix the normal distribution of operator sampling weights and transformation amplitude, and update the weights of the spatial search neural network model used for the search using the training set; Step S230: Fix the weights of the spatial search neural network model used for the search, and update the normal distribution that the sampling weights and transformation amplitudes of the augmentation operator follow using the step validation set; Step S240, repeat steps S220 and S230 until the predetermined number of search rounds is reached; Steps S220 to S240 together constitute the second-level optimization.

2. The method for augmenting differentiable random image data according to claim 1, characterized in that, Step S120 defines the transformation magnitude of each operator in the operator set, including: Define the effective range of the transformation amplitude for each operator, including the translation distance for the translation operator, the rotation angle for the rotation operator, and the stretching ratio for the contrast stretching operator.

3. The method for augmenting differentiable random image data according to claim 1, characterized in that, Step S140 involves differentiating the operator sampling weights through relaxation sampling, including: ； in, For the set of operators Any operator in Differentiable sampling weights, For the set of operators Any operator Sampling weights, For the set of operators Any operator Sampling weights, , for The corresponding random coefficients, , for The corresponding random coefficients, Represents a uniform distribution. The temperature coefficient is set in advance.

4. The method for augmenting differentiable random image data according to claim 1, characterized in that, Step S150 involves differentiating the operator transformation amplitude from a normal distribution through reparameterization, including: ； ； in, For the operator's differentiable transformation amplitude, For the operator transformation amplitude, Indicates a normal distribution. The mean of the operator transformation amplitude. The standard deviation of the operator transformation amplitude. represent It conforms to a standard normal distribution.

5. The method for augmenting differentiable random image data according to claim 1, characterized in that, The forward training of the operator's differentiable parameters in step S200 is as follows: ； ； ； in, For image data to be expanded, The transformed image data to be augmented. For operators The amplitude of the differentiable transform, For the set of operators Any operator in, Representatives transmit during the forward process The gradient is not propagated during the reverse process. The gradient transfer function, The transformed output image, Represents single-heat operation. This is the result of single-heat operation. Operator As a result, For the set of operators The vector of relaxed sampling results for all operators.

6. The method for augmenting differentiable random image data according to claim 1, characterized in that, Step S300 includes: Step S310: Perform relaxation sampling of the augmented operator based on the sampling weights of the augmented operator, and perform one-hot processing on the result to obtain the augmented operator. ; Step S320: Perform normal distribution sampling based on the normal distribution followed by the transformation amplitude of the extended operator to obtain the sampled transformation amplitude. ; Step S330: Normalize the sampling results to obtain the final augmentation operator and transformation amplitude. .

7. The method for augmenting differentiable random image data according to claim 6, characterized in that, Step S330 normalizes the sampling results, including: ； in, It is based on the sampling results of the normal distribution followed by the operator transformation amplitude. Yes Normalized sampling results.

8. A differentiable random image data augmentation system, used to perform the differentiable random image data augmentation method according to any one of claims 1-7, characterized in that, The expansion system includes: The image preprocessing module is configured to load the image data to be expanded from the storage medium and perform normalization preprocessing. The operator definition and storage module is configured to define the expanded search space of the image data to be expanded, define the operator differentiable parameters corresponding to the expanded search space, and store the candidate transformation operator, operator sampling weight, operator transformation amplitude range, and the normal distribution parameters followed by the operator transformation amplitude. The spatial search neural network module is configured to build and store the spatial search neural network structure and parameters used for searching transformation operators. The sampling module is configured to perform all random sampling during operator space search and image augmentation, including normal distribution sampling, uniform distribution sampling, and relaxation sampling. The operator search and update module is configured to implement the differentiable processing of the learnable parameters of the operator, and to search and update the normal distribution of the sampling weights and transformation amplitudes of the candidate operators based on second-order optimization, so as to obtain the normal distribution of the sampling weights and transformation amplitudes of the final expanded operator. The image augmentation module is configured to control the sampling module by applying the content stored in the operator definition and storage module, sampling to obtain a set of augmentation operators and transformation amplitudes, and using these augmentation operators and transformation amplitudes to augment the normalized image data to be augmented in the image preprocessing module. The image export module is used to display the images before and after expansion, as well as to export the expanded images.

Images