Self-adaptive automatic data enhancement method and system, medium and terminal
Through the adaptive automatic data enhancement method, the data enhancement parameters are optimized using perturbation and golden segmentation search algorithms, and the problems of high computational overhead and poor task universality in the existing technology are solved, and efficient DNN model training and better visual task performance are achieved. .
Patent Information
- Application Number
- CN202411202577.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-29
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-29
AI Technical Summary
Existing automatic data augmentation methods are computationally expensive in model training, and most of them are only suitable for image classification tasks, which are not satisfactory for other computer vision tasks, especially in large models and large search spaces, making it difficult to effectively optimize data augmentation. parameter.
Adaptive automatic data augmentation method is adopted, by selecting the initial search boundary in the hyperparameter search boundary, adding perturbation to the data augmentation operator parameters, approximate the gradient of the simulation model loss relative to the data augmentation operator parameters, and adapting according to the gradient Update, freeze the data enhancement operator parameters with the lowest contribution degree, and use the golden segmentation search algorithm to update the hyperparameter search boundary until the model with the best global performance is obtained.
It reduces the computational overhead of DNN model training, improves the universality of automatic data augmentation algorithms except for image classification tasks, reduces the number of iterations and calculation overhead, and achieves more efficient model training and better visual task performance.
Smart Images

Figure CN119090745A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of computer vision, and in particular relates to an adaptive automatic data enhancement method, system, medium and terminal. Background Art
[0002] Data augmentation (DA) plays an important role in improving the generalization ability and result accuracy of deep neural networks (DNNs) in visual tasks such as image classification, object detection, and semantic segmentation. Previous manually designed data augmentation methods have been gradually replaced by automatic data augmentation methods due to their high cost, dependence on human expertise, and poor generalizability.
[0003] Existing automatic data augmentation methods are mainly divided into two categories. One is based on a search approach to obtain the optimal data augmentation parameters (e.g., randomly sampled data augmentation operators and the augmentation amplitude of the operator), while the other class adaptively optimizes data augmentation parameters during model training. Search-based methods usually use a general search algorithm (e.g., grid search) to search for the best data augmentation parameters with respect to the loss value in a search space with specified boundaries. Such methods have expensive search costs when training large models and datasets. For methods that optimize data augmentation parameters online during training, they randomly sample parameters and optimize them through specified update strategies. These methods continuously optimize data augmentation parameters based on loss or gradient until the end of training.
[0004] Existing automatic data enhancement methods rely on large search spaces or continuously optimize parameters to obtain the desired final performance, and they do achieve state-of-the-art results on various visual tasks, but the computational overhead required to complete model training is high, especially for today's large models. In addition, most of the current automatic data enhancement algorithms are only applicable to image classification tasks, but are not satisfactory for other computer vision tasks such as object detection and semantic segmentation. Summary of the invention
[0005] The technical problem to be solved by the present invention is to provide an adaptive automatic data enhancement method, system, medium and terminal in view of the deficiencies in the above-mentioned prior art, which is used to solve the problems that the current search-based automatic data enhancement algorithm has a large search space, the computational overhead required to complete training is high, and it is not necessary to keep all data enhancement operator parameters updated when optimizing the data enhancement strategy online during model training.
[0006] In order to achieve the above object, the present invention provides the following technical solutions:
[0007] In a first aspect, the present invention provides an adaptive automatic data enhancement method, comprising the following steps:
[0008] S1. For a set of data augmentation operators with predefined parameters, select an initial search boundary N from the hyperparameter search boundary;
[0009] S2. Add disturbances to the parameters of the data enhancement operator. According to the change in the loss value of the model before and after the disturbance is added, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, and the parameters of the data enhancement operator are adaptively updated according to the gradient to obtain the model loss L. i ;
[0010] S3, select the maximum model loss obtained after adding disturbance in S2 for model training, and freeze the parameters of the data enhancement operator with the lowest contribution to model training;
[0011] S4. Repeat steps S2-S3 until the training of the current model is completed. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model. ′ ;
[0012] S5. Repeat steps S1-S4 until a model with optimal global performance is obtained;
[0013] S6. Obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0014] According to the change in the loss value of the model before and after adding the disturbance, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, specifically:
[0015] Add disturbances to the parameters of the data enhancement operator respectively to obtain The data enhancement operator is used to obtain the gradient of the model loss relative to the change of the data enhancement operator parameters before and after adding disturbance to the data enhancement operator:
[0016] G i =(L i -L original ) / Δp i
[0017] Among them, G i is the gradient, L i is the loss value of the model, Δp i is the disturbance, M i Data augmentation operator parameters, is the parameter of the data enhancement operator for adding perturbations, L original is the original loss value of the model.
[0018] The original loss value of the model is:
[0019] For batch_size training images (x, y), after being processed by a set of data enhancement operators with predefined parameters, the original loss value of the model can be expressed as:
[0020] L original =L(F(o n (…o 2 (o 1 (x)))),y)
[0021] Among them, L() represents the loss function selected for model training, x is the training dataset image, y is the image label, and o i is the data enhancement operator; F is the target model to be trained.
[0022] The data enhancement operator parameters are adaptively updated according to the gradient, and the formula used is as follows:
[0023]
[0024] in, is the updated data enhancement operator parameter, M i is the data enhancement operator parameter, α is the update step size of the parameter, sign() is used to return the sign of the input data, G i is the gradient.
[0025] The freezing of the data enhancement operator parameters that contribute the least to model training is specifically:
[0026] During the model training process, the number of times each operation obtains the maximum model loss is recorded. After a preset number of epochs of training, the contribution of different data enhancement operators to the model loss after the parameters are updated is statistically and metrically calculated, and the parameters of the data enhancement operator with the lowest contribution to model training are frozen.
[0027] Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained. ′ , specifically:
[0028] The given hyperparameter search boundary is divided into two intervals using the golden ratio, and a new search interval is determined within the small interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.
[0029] Repeat steps S1-S4, and continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.
[0030] In a second aspect, the present invention provides a system for an adaptive automatic data enhancement method, comprising the following modules:
[0031] The hyperparameter search module is used to select the initial search boundary N from the hyperparameter search boundary, and based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model obtained after each training. ′ , until the model with the best global performance is obtained;
[0032] The adaptive update module of the data enhancement operator parameters is used to add disturbances to the data enhancement operator parameters, and approximate the gradient of the model loss relative to the change of the data enhancement operator parameters according to the change of the loss value of the model before and after the addition of the disturbance, and adaptively update the data enhancement operator parameters according to the gradient to obtain the model loss;
[0033] The data enhancement operator parameter freezing strategy module is used to select the maximum model loss after adding disturbances for model training, and freeze the data enhancement operator parameters with the lowest contribution to model training;
[0034] The model application module is used to obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0035] In a third aspect, the present invention provides a computer-readable storage medium storing one or more programs, wherein the one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the steps of the above-mentioned adaptive automatic data enhancement method.
[0036] In a fourth aspect, the present invention provides a terminal device, comprising:
[0037] One or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the above-mentioned adaptive automatic data enhancement method.
[0038] Compared with the prior art, the present invention has the following beneficial effects:
[0039] An adaptive automatic data enhancement method improves the final performance of the DNN model, reduces the computational overhead required for DNN model training, and improves the versatility of the automatic data enhancement algorithm in tasks other than image classification, compared to the existing state of the art automatic data enhancement algorithm.
[0040] Furthermore, in view of the fact that the final performance of the DNN model presents a unimodal characteristic of first rising and then falling with the increase of the hyperparameter search boundary N, the present invention selects a more efficient one-dimensional parameter search algorithm, namely the golden section search algorithm, which effectively reduces the number of iterations required to obtain the optimal performance model and reduces the computational overhead required for model training.
[0041] Furthermore, in view of the non-differentiable nature of most data enhancement operator parameters, the present invention, during the model training process, approximately simulates the gradient of the model loss relative to the data enhancement operator parameters before and after adding the disturbance by adding disturbances to the data enhancement operator parameters respectively, and updates the data enhancement operator parameters based on the gradient, thereby realizing adaptive updating of the data enhancement operator parameters (enhancement amplitude) during model training, reducing the search dimension of the hyperparameter search boundary from two dimensions to one dimension, and further reducing the computational overhead required for model training.
[0042] Furthermore, during the model training process, the present invention measures the contribution of the data enhancement operator to the model training after the statistical parameters are updated every a certain number of epochs, and determines the freezing strategy of the data enhancement operator parameters based on the contribution, thereby reducing the computational overhead required for updating the data enhancement operator parameters during the model training process.
[0043] It can be understood that the beneficial effects of the second to fourth aspects mentioned above can be found in the relevant description of the first aspect mentioned above, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following briefly introduces the drawings required for use in the embodiment:
[0045] Figure 1 This is a flow chart of the efficient automatic data enhancement algorithm proposed by the present invention;
[0046] Figure 2 A schematic diagram showing how the model classification accuracy and the computational overhead required to complete model training change on the CIFAR-10 dataset as the hyperparameter N increases;
[0047] Figure 3 This is a diagram showing how the model classification accuracy and the computational overhead required to complete model training on the CIFAR-100 dataset change as the hyperparameter N increases;
[0048] Figure 4 This is a schematic diagram showing how the model classification accuracy and the computational overhead required to complete model training change on the SVHN dataset as the hyperparameter N increases;
[0049] Figure 5 The execution process of the parameter freezing strategy for the data enhancement operator;
[0050] Figure 6 Schematic diagram of visual comparison for target detection tasks;
[0051] Figure 7 A visual comparison diagram for the semantic segmentation task;
[0052] Figure 8The present invention is a block diagram of a terminal device provided according to an embodiment. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] In the description of the present invention, it should be understood that the terms “include” and “comprises” indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.
[0055] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.
[0056] It should be further understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects are in an "or" relationship.
[0057] It should be understood that, although the terms first, second, third, etc. may be used to describe preset ranges, etc. in the embodiments of the present invention, these preset ranges should not be limited to these terms. These terms are only used to distinguish preset ranges from each other. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.
[0058] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.
[0059] Various structural schematic diagrams of the embodiments disclosed in the present invention are shown in the accompanying drawings. These figures are not drawn to scale, and some details are magnified and some details may be omitted for the purpose of clear expression. The shapes of various regions and layers shown in the figures and the relative sizes and positional relationships therebetween are only exemplary, and may deviate in practice due to manufacturing tolerances or technical limitations, and those skilled in the art may additionally design regions / layers with different shapes, sizes, and relative positions according to actual needs.
[0060] The present invention will be further described below in conjunction with the accompanying drawings:
[0061] like Figure 1 As shown, the present invention provides an adaptive automatic data enhancement method, comprising the following steps:
[0062] S1. For a set of data augmentation operators with predefined parameters, select an initial search boundary N from the hyperparameter search boundary;
[0063] S2. Add disturbances to the parameters of the data enhancement operator. According to the change in the loss value of the model before and after the disturbance is added, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, and the parameters of the data enhancement operator are adaptively updated according to the gradient to obtain the model loss L. i ;
[0064] S3, select the maximum model loss obtained after adding disturbance in S2 for model training, and freeze the parameters of the data enhancement operator with the lowest contribution to model training;
[0065] S4. Repeat steps S2-S3 until the training of the current model is completed. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model. ′ ;
[0066] S5. Repeat steps S1-S4 until a model with optimal global performance is obtained;
[0067] S6. Obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0068] The present invention provides users with an efficient automatic data enhancement algorithm for computer vision tasks such as image classification, target detection and semantic segmentation, which can simultaneously improve the final performance of the DNN model, reduce the computational overhead required for DNN model training, and improve the versatility of the automatic data enhancement algorithm in tasks other than image classification.
[0069] According to the change in the loss value of the model before and after adding the disturbance, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, specifically:
[0070] For batch_size training images (x, y), after being processed by a set of data enhancement operators with predefined parameters, the original loss value of the model can be expressed as:
[0071]
[0072] Among them, L() represents the loss function selected for model training, x is the training dataset image, y is the image label, and o i is the data enhancement operator; M is
[0073] Add disturbances to the parameters of the data enhancement operator respectively to obtain The data enhancement operator is used to obtain the gradient of the model loss relative to the change of the data enhancement operator parameters before and after adding disturbance to the data enhancement operator:
[0074] G i =(L i -L original ) / Δp i
[0075] Among them, G i is the gradient, L i =Δp i is the disturbance, M i Data augmentation operator parameters, is the parameter of the data enhancement operator for adding perturbations, L original is the original loss value of the model.
[0076] The data enhancement operator parameters are adaptively updated according to the gradient, and the formula used is as follows:
[0077]
[0078] in, is the updated data enhancement operator parameter, M i is the data enhancement operator parameter, α is the update step size of the parameter, sign() is used to return the sign of the input data, G i is the gradient,
[0079] like Figure 5 As shown, the freezing of the data enhancement operator parameters that contribute the least to model training is specifically:
[0080] During the model training process, the number of times each operation obtains the maximum model loss is recorded. After a preset number of epochs of training, the contribution of different data enhancement operators to the model loss after the parameters are updated is statistically and metrically calculated, and the parameters of the data enhancement operator with the lowest contribution to model training are frozen.
[0081] Since the adaptive parameter update strategy will obtain the corresponding model loss after updating the amplitude parameters of each digital-to-analog conversion operation and use the maximum loss to optimize the target model, different operations will produce different enhancement effects on the same image data and have different effects on the model loss.
[0082] Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained. ′ , specifically:
[0083] The given hyperparameter search boundary is divided into two intervals using the golden ratio, and a new search interval is determined within the small interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.
[0084] Repeat steps S1-S4, and continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.
[0085] In one embodiment of the present invention, a system for an adaptive automatic data enhancement method is provided, which specifically includes a hyperparameter search module, an adaptive update module for data enhancement operator parameters, and a data enhancement operator parameter freezing strategy module;
[0086] The hyperparameter search module is used to select the initial search boundary N from the hyperparameter search boundary, and based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model obtained after each training. ′ , until the model with the best global performance is obtained;
[0087] The adaptive update module of the data enhancement operator parameters is used to add disturbances to the data enhancement operator parameters, and approximate the gradient of the model loss relative to the change of the data enhancement operator parameters according to the change of the loss value of the model before and after the addition of the disturbance, and adaptively update the data enhancement operator parameters according to the gradient to obtain the model loss;
[0088] The data enhancement operator parameter freezing strategy module is used to input the enhanced images into the model respectively, select the largest model loss for model training, and freeze the data enhancement operator parameters with the lowest contribution to model training;
[0089] The model application module is used to obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0090] In another embodiment of the present invention, the present invention further provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a terminal device for storing programs and data. It can be understood that the computer-readable storage medium here can include both the built-in storage medium in the terminal device and the extended storage medium supported by the terminal device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and these instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory, or a non-volatile memory (Non-Volatile Memory), such as at least one disk memory.
[0091] One or more instructions stored in a computer-readable storage medium may be loaded and executed by a processor to implement the corresponding steps of the above-mentioned calculation method; one or more instructions in the computer-readable storage medium are loaded and executed by a processor as follows:
[0092] S1. For a set of data augmentation operators with predefined parameters, select an initial search boundary N from the hyperparameter search boundary;
[0093] S2. Add disturbances to the parameters of the data enhancement operator. According to the change in the loss value of the model before and after the disturbance is added, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, and the parameters of the data enhancement operator are adaptively updated according to the gradient to obtain the model loss L. i ;
[0094] S3, select the maximum model loss obtained after adding disturbance in S2 for model training, and freeze the parameters of the data enhancement operator with the lowest contribution to model training;
[0095] S4. Repeat steps S2-S3 until the training of the current model is completed. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model. ′ ;
[0096] S5. Repeat steps S1-S4 until a model with optimal global performance is obtained;
[0097] S6. Obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0098] In another embodiment of the present invention, a terminal device is provided, the terminal device includes a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions to implement the corresponding method flow or corresponding functions; the processor described in the embodiment of the present invention can be used for the operation of the computing method, including:
[0099] S1. For a set of data augmentation operators with predefined parameters, select an initial search boundary N from the hyperparameter search boundary;
[0100] S2. Add disturbances to the parameters of the data enhancement operator. According to the change in the loss value of the model before and after the disturbance is added, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, and the parameters of the data enhancement operator are adaptively updated according to the gradient to obtain the model loss L. i ;
[0101] S3, select the maximum model loss obtained after adding disturbance in S2 for model training, and freeze the parameters of the data enhancement operator with the lowest contribution to model training;
[0102] S4. Repeat steps S2-S3 until the training of the current model is completed. Based on the golden section search algorithm, the hyperparameter search boundary is updated to ′ ;
[0103] S5. Repeat steps S1-S4 until a model with optimal global performance is obtained;
[0104] S6. Obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
[0105] See also Figure 8The terminal device 600 is an electronic device, and the electronic device is in the form of a general computing device. The components of the electronic device may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.
[0106] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps according to various exemplary embodiments of the present invention described in the above method part of this specification.
[0107] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .
[0108] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.
[0109] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.
[0110] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.
[0111] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can usually be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.
[0112] Example
[0113] The automatic data enhancement algorithm of the present invention is compared with other automatic data enhancement algorithms in an experimental environment where the CPU is Intel(R) Xeon(R) Platinum 8160T CPU@2.10GHz, the memory is 64GB, the GPU is NVIDIA Tesla V100 GPU, the operating system is Ubuntu 18.04, the Pytorch version is 1.12.1, the CUDA version is 11.6, and the Python version is 3.8.
[0114] Table 1 Performance comparison of the automatic data enhancement algorithm EADA of the present invention and other SOTA automatic data enhancement algorithms on CIFAR-10 and CIFAR-100 datasets;
[0115]
[0116]
[0117] See also Figure 2 , indicating that for the image classification task of the CIFAR-10 dataset, the final classification accuracy of the DNN model shows the nature of a unimodal curve that first increases and then decreases as N increases.
[0118] See also Figure 3 , indicating that for the image classification task of the CIFAR-100 dataset, the final classification accuracy of the DNN model shows the nature of a unimodal curve that first increases and then decreases as N increases.
[0119] See also Figure 4 , indicating that for the image classification task of the SVHN dataset, the final classification accuracy of the DNN model shows the nature of a unimodal curve that first increases and then decreases as N increases.
[0120] The results in Table 1 show that on the classic image classification datasets CIFAR-10 and CIFAR-100, compared with other SOTA automatic data enhancement algorithms, such as AutoAugment (AA), Fast AutoAugment (Fast AA), FasterAutoAugment (Faster AA), RandAugment (RA), UniformAugment (UA), TrivialAugment (TA), Adversarial AutoAugment (Adv.AA), DivAug, Differentiable RandAugment (DRA) and UADA, the baseline refers to training the model using only RandomCrop and RandomFlip. The automatic data enhancement algorithm EADA of the present invention is superior to most of the top-ranked SOTA methods, and can reduce the computational cost required for model training under the premise that the final classification accuracy of the model is similar.
[0121] Table 2 Performance comparison of the automatic data enhancement algorithm EADA of the present invention and other SOTA automatic data enhancement algorithms on the ImageNet dataset
[0122]
[0123] Table 3 Performance comparison of the automatic data enhancement algorithm EADA of the present invention and other SOTA automatic data enhancement algorithms on the tiny-ImageNet dataset
[0124]
[0125] Tables 2 and 3 show the performance comparison of EADA and other automatic data augmentation algorithms using ResNet-18 and ResNet-50 on large-scale datasets ImageNet and tiny-ImageNet. The baseline refers to training the model using only RandomResizedCrop and RandomHorizontalFlip. According to the experimental data, EADA has consistently achieved better results than UADA.
[0126] Table 4 Comparison of the computational overhead of different automatic data augmentation algorithms compared to RA on the CIFAR-10 dataset
[0127]
[0128] Table 4 shows the computational overhead comparison of different automatic data enhancement algorithms compared to RA on the CIFAR-10 dataset. Although EADA can achieve similar performance to UADA, DivAug, and Adv.AA, the training cost of EADA is much lower.
[0129] Table 5 Performance comparison of the automatic data enhancement algorithm EADA of the present invention on the target detection task
[0130]
[0131] Table 6 Performance comparison of the automatic data enhancement algorithm EADA of the present invention on the semantic segmentation task
[0132]
[0133] Tables 5 and 6 show the comparative experiments on target detection and semantic segmentation tasks using different network structures and datasets (visual comparison as shown in Figure 6 and Figure 7 The experimental data show that the automatic data enhancement algorithm of the present invention improves the final performance of the model.
[0134] The present invention provides users with an adaptive automatic data enhancement method for computer vision tasks such as image classification, object detection and semantic segmentation, thereby improving the final performance of the deep neural network model and reducing the computational overhead required to complete model training without the need for professional domain knowledge for specific tasks.
[0135] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0136] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0137] Those of ordinary skill in the art will appreciate that the units and computing method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0138] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0139] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0140] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0141] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0142] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0143] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0144] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.
[0145] The above content is only for explaining the technical idea of the present invention, and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution according to the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.
[0146] The above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art can still modify or make equivalent substitutions to the specific implementations of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention are within the scope of protection of the claims of the present invention to be approved.
Claims
1. An adaptive automatic data enhancement method, It is characterized in that The following steps are involved: S1. For a set of data augmentation operators with predefined parameters, select an initial search boundary N from the hyperparameter search boundary; S2. Add disturbances to the parameters of the data enhancement operator. According to the change in the loss value of the model before and after the disturbance is added, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, and the parameters of the data enhancement operator are adaptively updated according to the gradient to obtain the model loss L. i ; S3, select the maximum model loss obtained after adding disturbance in S2 for model training, and freeze the parameters of the data enhancement operator with the lowest contribution to model training; S4. Repeat steps S2-S3 until the training of the current model is completed. Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model. ′ ; S5. Repeat steps S1-S4 until a model with optimal global performance is obtained; S6. Obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
2. The method of adaptive automatic data enhancement according to claim 1, It is characterized in that According to the change in the loss value of the model before and after adding the disturbance, the gradient of the model loss relative to the change in the parameters of the data enhancement operator is approximately simulated, specifically: Add disturbances to the parameters of the data enhancement operator respectively to obtain The data enhancement operator is used to obtain the gradient of the model loss relative to the change of the data enhancement operator parameters before and after adding disturbance to the data enhancement operator: G i =(L i -L original ) / Δp i Among them, G i is the gradient, L i is the loss value of the model, Δp i is the disturbance, M i Data augmentation operator parameters, is the parameter of the data enhancement operator for adding perturbations, L original is the original loss value of the model.
3. The method of adaptive automatic data enhancement according to claim 2, It is characterized in that The original loss value of the model is: For batch_size training images (x, y), after being processed by a set of data enhancement operators with predefined parameters, the original loss value of the model can be expressed as: L origunal =L(F(o n (…o 2 (o 1 (x)))),y) Among them, L() represents the loss function selected for model training, x is the training dataset image, y is the image label, and o i is the data enhancement operator; F is the target model to be trained.
4. The method of adaptive automatic data enhancement according to claim 1, It is characterized in that The data enhancement operator parameters are adaptively updated according to the gradient, and the formula used is as follows: in, is the updated data enhancement operator parameter, M i is the data enhancement operator parameter, α is the update step size of the parameter, sign() is used to return the sign of the input data, G i is the gradient.
5. The method of adaptive automatic data enhancement according to claim 1, It is characterized in that The freezing of the data enhancement operator parameters that contribute the least to model training is specifically: During the model training process, the number of times each operation obtains the maximum model loss is recorded. After a preset number of epochs of training, the contribution of different data enhancement operators to the model loss after the parameters are updated is statistically and metrically calculated, and the parameters of the data enhancement operator with the lowest contribution to model training are frozen.
6. The method of adaptive automatic data enhancement according to claim 1, It is characterized in that Based on the golden section search algorithm, the hyperparameter search boundary is updated to N according to the final performance of the model obtained. ′ , specifically: The given hyperparameter search boundary is divided into two intervals using the golden ratio, and a new search interval is determined within the small interval based on the objective function of the model's final performance, and the hyperparameter search boundary is updated.
7. The method of adaptive automatic data enhancement according to claim 1, It is characterized in that Repeat steps S1-S4, and continuously narrow the search interval during the iteration of the golden section search algorithm until the model with the best global performance is obtained.
8. A system for adaptive automatic data enhancement method, It is characterized in that Includes the following modules: The hyperparameter search module is used to select the initial search boundary N from the hyperparameter search boundary, and based on the golden section search algorithm, update the hyperparameter search boundary to N according to the final performance of the model obtained after each training. ′ , until the model with the best global performance is obtained; The adaptive update module of the data enhancement operator parameters is used to add disturbances to the data enhancement operator parameters, and approximate the gradient of the model loss relative to the change of the data enhancement operator parameters according to the change of the loss value of the model before and after the addition of the disturbance, and adaptively update the data enhancement operator parameters according to the gradient to obtain the model loss; The data enhancement operator parameter freezing strategy module is used to select the maximum model loss after adding disturbances for model training, and freeze the data enhancement operator parameters with the lowest contribution to model training; The model application module is used to obtain the model with the best global performance to process the image to be processed and obtain the image processing result.
9. A computer-readable storage medium storing one or more programs, It is characterized in that The one or more programs include instructions, which, when executed by a computing device, cause the computing device to perform the steps of the adaptive automatic data enhancement method according to any one of claims 1 to 7.
10. A terminal device, It is characterized in that include: One or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include steps for executing the adaptive automatic data enhancement method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Training method and device of disturbance generation model
CN114170479A
Automatic confrontation training method and device
CN114491448A
Model training and image processing method and device, equipment and storage medium
CN116310626A
Image classification method based on convolutional neural network and related device
CN117237713A
Semi-supervised semantic segmentation method and device of image, equipment and storage medium
CN118506005A
