An adaptive automatic data augmentation method, system, medium and terminal

Abstract

The application discloses a kind of self-adapting automatic data enhancement method, system, medium and terminal, belong to computer vision field.This method selects the initial search boundary of a group of data enhancement operators of predefined parameter;By applying disturbance to parameter, and observing the change of model loss value, approximate simulation is carried out gradient of model loss relative to parameter change, and then parameter is adaptively updated;In the training process, the maximum model loss is selected for training, and the data enhancement operator parameter with the lowest contribution is frozen, and the process is iterated until the current model training is completed;Based on golden section search algorithm, according to the final performance of model, update search boundary until the global performance optimal model is obtained, for image processing.Through repeating this cycle, the application aims to improve the performance of deep neural network model without specific task professional knowledge, and reduce the computing overhead, applicable to image classification, target detection and semantic segmentation etc.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to an adaptive automatic data augmentation method, system, medium, and terminal. Background Technology

[0002] Data augmentation (DA) plays a crucial role in improving the generalization ability and accuracy of deep neural networks (DNNs) in visual tasks such as image classification, object detection, and semantic segmentation. Previously manually designed data augmentation methods have been gradually replaced by automated data augmentation methods due to their high cost, reliance on human expertise, and poor versatility.

[0003] Existing automatic data augmentation methods mainly fall into two categories. One is a search-based approach to obtain optimal data augmentation parameters (e.g., randomly sampled data augmentation operators and the augmentation magnitude of those operators), while the other adaptively optimizes the data augmentation parameters during model training. Search-based methods typically employ general search algorithms (e.g., grid search) to search for the optimal data augmentation parameters with respect to the loss value within a search space with specified boundaries. This approach incurs high search costs when training large models and datasets. In contrast, methods that optimize data augmentation parameters online during training randomly sample parameters and optimize them using a specified update strategy. These methods continuously optimize the data augmentation parameters based on the loss or gradient until training is complete.

[0004] Existing automated data augmentation methods rely on large search spaces or continuous parameter optimization to achieve the desired final performance, and they do indeed achieve state-of-the-art results on a variety of vision tasks. However, the computational cost of training these models is high, especially for today's large models. Furthermore, most current automated data augmentation algorithms are only suitable for image classification tasks, and perform poorly for other computer vision tasks such as object detection and semantic segmentation. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide an adaptive automatic data augmentation method, system, medium and terminal to address the shortcomings of the prior art, and to solve the problems that current search-based automatic data augmentation algorithms have large search spaces, high computational overhead for training, and do not need to keep all data augmentation operator parameters updated when optimizing data augmentation strategies online during model training.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an adaptive automatic data augmentation method, comprising the following steps:

[0008] S1. For a set of data augmentation operators with predefined parameters, select the initial search boundary N from the hyperparameter search boundary;

[0009] S2. Add perturbations to the parameters of the data augmentation operator. Based on the change in the model's loss value before and after adding the perturbations, approximate the gradient of the model loss with respect to the change in the data augmentation operator parameters. Adaptively update the data augmentation operator parameters based on the gradient to obtain the model loss L. i ;

[0010] S3. Select the model loss with the largest perturbation after S2 for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0011] S4. Repeat steps S2-S3 until the training of the current model is complete. Based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model. ′ ;

[0012] S5. Repeat steps S1-S4 until the model with the best global performance is obtained;

[0013] S6. Obtain the globally optimal model and process the image to be processed to obtain the image processing result.

[0014] The method of approximating the gradient of the model loss relative to the changes in the parameters of the data augmentation operator based on the change in the model loss value before and after adding the perturbation is as follows:

[0015] Perturbations are added to the parameters of the data augmentation operators to obtain data with parameters. The data augmentation operator is then used to obtain the gradient of the model loss relative to the change in the parameters of the data augmentation operator before and after adding a perturbation to the data augmentation operator:

[0016] G i =(L i -L original ) / Δp i

[0017] Among them, G i For the gradient, L i The loss value of the model, Δp i For the perturbation, M i Data augmentation operator parameters, To add perturbation to the data augmentation operator parameters, L original This represents the original loss value of the model.

[0018] The original loss value of the model is as follows:

[0019] For training images (x, y) of batch_size, after processing by a set of data augmentation operators with predefined parameters, the original loss value of the model can be expressed as:

[0020] L original =L(F(o) n (…o2(o1(x)))),y)

[0021] Where L() represents the loss function selected for model training, x is the training dataset images, y is the image label, and o i F is the data augmentation operator; F is the target model that needs to be trained.

[0022] The adaptive update of the data augmentation operator parameters based on the gradient is achieved using the following formula:

[0023]

[0024] in, To update the parameters of the data augmentation operator, M i For the data augmentation operator parameters, α is the parameter update step size, sign() is used to return the sign of the input data, and G... i For gradient.

[0025] The freezing of the data augmentation operator parameters that contribute the least to model training is specifically as follows:

[0026] During model training, the number of times each operation achieves the maximum model loss is recorded. After training for a preset number of epochs, the contribution of different data augmentation operators to the model loss after parameter updates is statistically analyzed and measured, and the parameters of the data augmentation operators with the lowest contribution to model training are frozen.

[0027] The hyperparameter search boundary based on the golden section search algorithm is updated to N according to the final performance of the obtained model. ′ Specifically:

[0028] Given the hyperparameter search boundary, the search interval is divided into two intervals using the golden ratio. A new search interval is determined within the smaller interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.

[0029] Repeat steps S1-S4 to continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.

[0030] Secondly, the present invention provides a system for an adaptive automatic data augmentation method, comprising the following modules:

[0031] The hyperparameter search module is used to select an initial search boundary N from the hyperparameter search boundary. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained after each training iteration. ′ This continues until the model with the best global performance is obtained;

[0032] The adaptive update module for data augmentation operator parameters is used to add perturbations to the data augmentation operator parameters. Based on the change in the model's loss value before and after adding the perturbations, it approximates the gradient of the model loss relative to the change in the data augmentation operator parameters, and adaptively updates the data augmentation operator parameters based on the gradient to obtain the model loss.

[0033] The data augmentation operator parameter freezing strategy module is used to select the model loss with the largest value after adding perturbations for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0034] The model application module is used to obtain the globally optimal model to process the image to be processed and obtain the image processing result.

[0035] Thirdly, the present invention provides a computer-readable storage medium for storing one or more programs, the one or more programs including instructions that, when executed by a computing device, cause the computing device to perform the steps of the adaptive automatic data augmentation method described above.

[0036] Fourthly, the present invention provides a terminal device, comprising:

[0037] One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for performing the adaptive automatic data augmentation method described above.

[0038] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0039] An adaptive automatic data augmentation method is proposed, which, compared with existing state-of-the-art automatic data augmentation algorithms, improves the final performance of DNN models, reduces the computational overhead required for DNN model training, and enhances the versatility of automatic data augmentation algorithms beyond image classification tasks.

[0040] Furthermore, considering that the final performance of the DNN model exhibits a single-peak characteristic of first increasing and then decreasing as the hyperparameter search boundary N increases, this invention selects a more efficient one-dimensional parameter search algorithm, namely the golden section search algorithm, which effectively reduces the number of iterations required to obtain the optimal performance model and reduces the computational overhead required for model training.

[0041] Furthermore, considering the fact that most data augmentation operator parameters are not differentiable, this invention approximates the gradient of the model loss with respect to the data augmentation operator parameters before and after the perturbation by adding perturbations to the data augmentation operator parameters during model training. Based on this gradient, the data augmentation operator parameters are updated respectively, realizing adaptive updating of data augmentation operator parameters (augmentation magnitude) during model training. This reduces the search dimension of the hyperparameter search boundary from two dimensions to one dimension, further reducing the computational overhead required for model training.

[0042] Furthermore, during model training, this invention measures the contribution of the data augmentation operator to the model training after updating the statistical parameters every certain number of epochs, and determines the freezing strategy of the data augmentation operator parameters based on the contribution, thereby reducing the computational overhead required for updating the data augmentation operator parameters during model training.

[0043] It is understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below:

[0045] Figure 1 This is a flowchart of the efficient automatic data augmentation algorithm proposed in this invention;

[0046] Figure 2 This diagram illustrates how the model classification accuracy and computational cost required to train the model on the CIFAR-10 dataset change as the hyperparameter N increases.

[0047] Figure 3 A schematic diagram showing how the model classification accuracy and computational cost required to complete model training on the CIFAR-100 dataset change as the hyperparameter N increases;

[0048] Figure 4 A schematic diagram showing how the classification accuracy of the model on the SVHN dataset and the computational cost required to complete model training change as the hyperparameter N increases;

[0049] Figure 5 The execution flow of the parameter freezing strategy for data augmentation operators;

[0050] Figure 6 A visual comparison illustration for an object detection task;

[0051] Figure 7 A visual comparison diagram for semantic segmentation tasks;

[0052] Figure 8This is a block diagram of a terminal device provided according to an embodiment of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0055] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0056] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" relationship.

[0057] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0058] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0059] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0060] The present invention will be further described below with reference to the accompanying drawings:

[0061] like Figure 1 As shown, the present invention provides an adaptive automatic data augmentation method, comprising the following steps:

[0062] S1. For a set of data augmentation operators with predefined parameters, select the initial search boundary N from the hyperparameter search boundary;

[0063] S2. Add perturbations to the parameters of the data augmentation operator. Based on the change in the model's loss value before and after adding the perturbations, approximate the gradient of the model loss with respect to the change in the data augmentation operator parameters. Adaptively update the data augmentation operator parameters based on the gradient to obtain the model loss L. i ;

[0064] S3. Select the model loss with the largest perturbation after S2 for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0065] S4. Repeat steps S2-S3 until the training of the current model is complete. Based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model. ′ ;

[0066] S5. Repeat steps S1-S4 until the model with the best global performance is obtained;

[0067] S6. Obtain the globally optimal model and process the image to be processed to obtain the image processing result.

[0068] This invention provides users with an efficient automatic data augmentation algorithm for computer vision tasks such as image classification, object detection, and semantic segmentation. It can simultaneously improve the final performance of DNN models, reduce the computational overhead required for DNN model training, and enhance the versatility of automatic data augmentation algorithms beyond image classification tasks.

[0069] The method of approximating the gradient of the model loss relative to the changes in the parameters of the data augmentation operator based on the change in the model loss value before and after adding the perturbation is as follows:

[0070] For training images (x, y) of batch_size, after processing by a set of data augmentation operators with predefined parameters, the original loss value of the model can be expressed as:

[0071]

[0072] Where L() represents the loss function selected for model training, x is the training dataset images, y is the image label, and o i For data augmentation operators; M is

[0073] Perturbations are added to the parameters of the data augmentation operators to obtain data with parameters. The data augmentation operator is then used to obtain the gradient of the model loss relative to the change in the parameters of the data augmentation operator before and after adding a perturbation to the data augmentation operator:

[0074] G i =(L i -L original ) / Δp i

[0075] Among them, G i For the gradient, L i For, Δp i For the perturbation, M i Data augmentation operator parameters, To add perturbation to the data augmentation operator parameters, L original This represents the original loss value of the model.

[0076] The adaptive update of the data augmentation operator parameters based on the gradient is achieved using the following formula:

[0077]

[0078] in, To update the parameters of the data augmentation operator, M i For the data augmentation operator parameters, α is the parameter update step size, sign() is used to return the sign of the input data, and G... i For gradient,

[0079] like Figure 5 As shown, freezing the data augmentation operator parameters that contribute the least to model training specifically refers to:

[0080] During model training, the number of times each operation achieves the maximum model loss is recorded. After training for a preset number of epochs, the contribution of different data augmentation operators to the model loss after parameter updates is statistically analyzed and measured, and the parameters of the data augmentation operators with the lowest contribution to model training are frozen.

[0081] Because the adaptive parameter update strategy obtains the corresponding model loss after updating the magnitude parameter of each digital-to-analog conversion operation, and uses the maximum loss to optimize the target model, different operations will produce different enhancement effects on the same image data, and their impact on the model loss will also be different.

[0082] The hyperparameter search boundary based on the golden section search algorithm is updated to N according to the final performance of the obtained model. ′ Specifically:

[0083] Given the hyperparameter search boundary, the search interval is divided into two intervals using the golden ratio. A new search interval is determined within the smaller interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.

[0084] Repeat steps S1-S4 to continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.

[0085] In one embodiment of the present invention, a system for an adaptive automatic data augmentation method is provided, specifically including a hyperparameter search module, an adaptive update module for data augmentation operator parameters, and a data augmentation operator parameter freezing strategy module;

[0086] The hyperparameter search module is used to select an initial search boundary N from the hyperparameter search boundary. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained after each training iteration. ′ This continues until the model with the best global performance is obtained;

[0087] The adaptive update module for data augmentation operator parameters is used to add perturbations to the data augmentation operator parameters. Based on the change in the model's loss value before and after adding the perturbations, it approximates the gradient of the model loss relative to the change in the data augmentation operator parameters, and adaptively updates the data augmentation operator parameters based on the gradient to obtain the model loss.

[0088] The data augmentation operator parameter freezing strategy module is used to input augmented images into the model, select the model with the largest model loss for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0089] The model application module is used to obtain the globally optimal model to process the image to be processed and obtain the image processing result.

[0090] In another embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (memory). This computer-readable storage medium is a memory device in a terminal device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and extended storage media supported by the terminal device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device.

[0091] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the above-described calculation method; one or more instructions in the computer-readable storage medium may be loaded and executed by a processor to perform the following steps:

[0092] S1. For a set of data augmentation operators with predefined parameters, select the initial search boundary N from the hyperparameter search boundary;

[0093] S2. Add perturbations to the parameters of the data augmentation operator. Based on the change in the model's loss value before and after adding the perturbations, approximate the gradient of the model loss with respect to the change in the data augmentation operator parameters. Adaptively update the data augmentation operator parameters based on the gradient to obtain the model loss L. i ;

[0094] S3. Select the model loss with the largest perturbation after S2 for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0095] S4. Repeat steps S2-S3 until the training of the current model is complete. Based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model. ′ ;

[0096] S5. Repeat steps S1-S4 until the model with the best global performance is obtained;

[0097] S6. Obtain the globally optimal model and process the image to be processed to obtain the image processing result.

[0098] In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, the computer program including program instructions, and the processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a computing method, including:

[0099] S1. For a set of data augmentation operators with predefined parameters, select the initial search boundary N from the hyperparameter search boundary;

[0100] S2. Add perturbations to the parameters of the data augmentation operator. Based on the change in the model's loss value before and after adding the perturbations, approximate the gradient of the model loss with respect to the change in the data augmentation operator parameters. Adaptively update the data augmentation operator parameters based on the gradient to obtain the model loss L. i ;

[0101] S3. Select the model loss with the largest perturbation after S2 for model training, and freeze the data augmentation operator parameters that contribute the least to model training.

[0102] S4. Repeat steps S2-S3 until the training of the current model is complete. Based on the golden section search algorithm, update the hyperparameter search boundary according to the final performance of the model. ′ ;

[0103] S5. Repeat steps S1-S4 until the model with the best global performance is obtained;

[0104] S6. Obtain the globally optimal model and process the image to be processed to obtain the image processing result.

[0105] Please see Figure 8The terminal device 600 is an electronic device, which takes the form of a general-purpose computing device. The components of the electronic device may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0106] The storage unit stores program code, which can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the above-described method section of this specification according to various exemplary embodiments of the present invention.

[0107] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0108] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0109] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0110] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0111] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0112] Example

[0113] This invention compares its automatic data augmentation algorithm with other automatic data augmentation algorithms in an experimental environment with an Intel(R) Xeon(R) Platinum 8160T CPU@2.10GHz, 64GB of memory, an NVIDIA Tesla V100 GPU, an Ubuntu 18.04 operating system, PyTorch version 1.12.1, CUDA version 11.6, and Python version 3.8.

[0114] Table 1 compares the performance of the automatic data augmentation algorithm EADA of this invention with other state-of-the-art automatic data augmentation algorithms on the CIFAR-10 and CIFAR-100 datasets.

[0115]

[0116]

[0117] Please see Figure 2 This indicates that for the image classification task on the CIFAR-10 dataset, the final classification accuracy of the DNN model exhibits a unimodal curve property of first increasing and then decreasing as N increases.

[0118] Please see Figure 3 This indicates that for the image classification task on the CIFAR-100 dataset, the final classification accuracy of the DNN model exhibits a unimodal curve property of first increasing and then decreasing as N increases.

[0119] Please see Figure 4 This indicates that for the image classification task of the SVHN dataset, the final classification accuracy of the DNN model exhibits a unimodal curve property of first increasing and then decreasing as N increases.

[0120] Table 1 shows that on the classic image classification datasets CIFAR-10 and CIFAR-100, compared to other state-of-the-art (SOTA) automatic data augmentation algorithms such as AutoAugment (AA), Fast AutoAugment (Fast AA), Faster AutoAugment (Faster AA), RandAugment (RA), UniformAugment (UA), TrivialAugment (TA), Adversarial AutoAugment (Adv.AA), DivAug, Differentiable RandAugment (DRA), and UADA (baseline refers to training the model using only RandomCrop and RandomFlip), the automatic data augmentation algorithm EADA of this invention outperforms most top-ranked SOTA methods and can reduce the computational cost required for model training while maintaining similar final classification accuracy.

[0121] Table 2 compares the performance of the EADA automatic data augmentation algorithm of this invention with other state-of-the-art (SOTA) automatic data augmentation algorithms on the ImageNet dataset.

[0122]

[0123] Table 3 compares the performance of the EADA automatic data augmentation algorithm of this invention with other state-of-the-art (SOTA) automatic data augmentation algorithms on the tiny-ImageNet dataset.

[0124]

[0125] Tables 2 and 3 show the performance comparison of EADA with other automatic data augmentation algorithms using ResNet-18 and ResNet-50 on the large-scale datasets ImageNet and tiny-ImageNet. The baseline refers to training the model using only RandomResizedCrop and RandomHorizontalFlip. The experimental data shows that EADA consistently achieves better results than UADA.

[0126] Table 4 compares the computational costs of different automatic data augmentation algorithms on the CIFAR-10 dataset compared to RA.

[0127]

[0128] Table 4 shows the computational cost comparison of different automatic data augmentation algorithms compared to RA on the CIFAR-10 dataset. Although EADA can achieve similar performance with UADA, DivAug and Adv.AA, EADA has a much lower training cost.

[0129] Table 5 compares the performance of the automatic data augmentation algorithm EADA of this invention on the target detection task.

[0130]

[0131] Table 6 compares the performance of the automatic data augmentation algorithm EADA presented in this invention on the semantic segmentation task.

[0132]

[0133] Tables 5 and 6 present comparative experiments on object detection and semantic segmentation tasks using different network structures and datasets (visual comparisons are shown below). Figure 6 and Figure 7 As shown in the figure, this demonstrates the effectiveness of EADA and its versatility in other vision tasks. Experimental data shows that the automatic data augmentation algorithm of this invention improves the final performance of the model.

[0134] This invention provides users with an adaptive automatic data augmentation method for computer vision tasks such as image classification, object detection, and semantic segmentation. This method improves the final performance of deep neural network models and reduces the computational overhead required to complete model training without requiring specialized domain knowledge for specific tasks.

[0135] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0136] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0137] Those skilled in the art will recognize that the units and computational steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0138] In the embodiments provided by this invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0139] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0140] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0141] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] The above description is merely illustrative of the technical concept of this invention and should not be construed as limiting the scope of protection of this invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.

[0146] The above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art can still make modifications or equivalent substitutions to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention are within the protection scope of the claims of the present invention pending approval.

Claims

1. An adaptive automatic data augmentation method, characterized in that, Includes the following steps: S1. For a set of data augmentation operators with predefined parameters, select the initial search boundary N from the hyperparameter search boundary; S2. Add perturbations to the parameters of the data augmentation operator. Based on the change in the model's loss value before and after adding the perturbations, approximate the gradient of the model loss with respect to the change in the data augmentation operator parameters. Adaptively update the data augmentation operator parameters based on the gradient to obtain the model loss L. i ; S3. Select the model loss with the largest perturbation after S2 for model training, and freeze the data augmentation operator parameters that contribute the least to model training. S4. Repeat steps S2-S3 until the training of the current model is complete. Based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model. ′ ; S5. Repeat steps S1-S4 until the model with the best global performance is obtained; S6. Obtain the globally optimal model and process the image to be processed to obtain the image processing result.

2. The adaptive automatic data augmentation method according to claim 1, characterized in that, The method of approximating the gradient of the model loss relative to the changes in the parameters of the data augmentation operator based on the change in the model loss value before and after adding the perturbation is as follows: Perturbations are added to the parameters of the data augmentation operators to obtain data with parameters. The data augmentation operator is then used to obtain the gradient of the model loss relative to the change in the parameters of the data augmentation operator before and after adding a perturbation to the data augmentation operator: G i =(L i -L original ) / Δp i Among them, G i For the gradient, L i The loss value of the model, Δp i For the perturbation, M i Data augmentation operator parameters, To add perturbation to the data augmentation operator parameters, L original This represents the original loss value of the model.

3. The adaptive automatic data augmentation method according to claim 2, characterized in that, The original loss value of the model is as follows: For training images (x, y) of batch_size, after processing by a set of data augmentation operators with predefined parameters, the original loss value of the model can be expressed as: L origunal =L(F(o n (…o2(o1(x)))),y) Where L() represents the loss function selected for model training, x is the training dataset images, y is the image label, and o i F is the data augmentation operator; F is the target model that needs to be trained.

4. The adaptive automatic data augmentation method according to claim 1, characterized in that, The adaptive update of the data augmentation operator parameters based on the gradient is achieved using the following formula: in, To update the parameters of the data augmentation operator, M i For the data augmentation operator parameters, α is the parameter update step size, sign() is used to return the sign of the input data, and G... i For gradient.

5. The adaptive automatic data augmentation method according to claim 1, characterized in that, The freezing of the data augmentation operator parameters that contribute the least to model training is specifically as follows: During model training, the number of times each operation achieves the maximum model loss is recorded. After training for a preset number of epochs, the contribution of different data augmentation operators to the model loss after parameter updates is statistically analyzed and measured, and the parameters of the data augmentation operators with the lowest contribution to model training are frozen.

6. The adaptive automatic data augmentation method according to claim 1, characterized in that, The hyperparameter search boundary based on the golden section search algorithm is updated to N according to the final performance of the obtained model. ′ Specifically: Given the hyperparameter search boundary, the search interval is divided into two intervals using the golden ratio. A new search interval is determined within the smaller interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.

7. The adaptive automatic data augmentation method according to claim 1, characterized in that, Repeat steps S1-S4 to continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.

8. A system for an adaptive automatic data augmentation method, characterized in that, Includes the following modules: The hyperparameter search module is used to select an initial search boundary N from the hyperparameter search boundary. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained after each training iteration. ′ This continues until the model with the best global performance is obtained; The adaptive update module for data augmentation operator parameters is used to add perturbations to the data augmentation operator parameters. Based on the change in the model's loss value before and after adding the perturbations, it approximates the gradient of the model loss relative to the change in the data augmentation operator parameters, and adaptively updates the data augmentation operator parameters based on the gradient to obtain the model loss. The data augmentation operator parameter freezing strategy module is used to select the model loss with the largest value after adding perturbations for model training, and freeze the data augmentation operator parameters that contribute the least to model training. The model application module is used to obtain the globally optimal model to process the image to be processed and obtain the image processing result.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform the steps of the adaptive automatic data augmentation method of any one of claims 1 to 7.

10. A terminal device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including steps for performing the adaptive automatic data augmentation method of any one of claims 1 to 7.

Images