Data enhancement processing and model training method and device, and electronic equipment

Abstract

The application provides a data enhancement processing and model training method and device, electronic equipment and a storage medium, comprising: determining at least one enhancement processing category of original data according to the data type of the original data; and performing data enhancement processing on the original data according to at least one enhancement processing parameter of the at least one enhancement processing category and a randomly generated noise parameter to obtain enhanced data of the original data. The application can ensure the rationality and universality of the data enhancement processing result by introducing a random noise parameter, and help improve the model training effect.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data augmentation processing and model training method, apparatus, electronic device and storage medium. Background Technology

[0002] Data augmentation is a technique in deep learning that generates new data (or augmented data) from the original data for model training. By synthesizing new data based on existing datasets, the model can fully extract key knowledge contained in a small amount of existing data during training, thereby improving the training effect of the model.

[0003] Traditional data augmentation methods perform augmentation on the original data using a predetermined sequence of operations to generate new data. However, due to the deterministic nature of the operation sequence, the new data obtained through this augmentation process largely maintains the correlation with the original data. While using this type of new data for model training can broaden the model's exposure and enable it to learn effectively, it cannot provide targeted correction for model defects.

[0004] To address the shortcomings of traditional data augmentation methods in resolving model deficiencies, adversarial training has been proposed. The process of manipulating original samples to generate adversarial examples can be considered a form of adversarial data augmentation. The goal of adversarial training is to prevent the model's output from being affected by reasonable perturbations added to the input, thereby improving the accuracy of the model's recognition results. However, adversarial augmentations generated based on excessive perturbations can become meaningless during model training and may even have a counterproductive effect. Therefore, appropriate constraints need to be designed to ensure that the generated adversarial examples do not deviate excessively from the distribution of the original data. However, current constraints for this purpose are often based on assumptions made by human prior knowledge, which cannot guarantee the reasonableness and breadth of the data perturbations.

[0005] Therefore, a data augmentation processing scheme that can guarantee the rationality and widespread applicability of data perturbation is needed. Summary of the Invention

[0006] In view of this, embodiments of this application provide a data augmentation processing and model training method, apparatus, electronic device and storage medium, so as to solve the problem of poor rationality and wide applicability of data perturbation in the prior art.

[0007] According to a first aspect of the embodiments of this application, a data augmentation processing method is provided, comprising: determining at least one augmentation processing category of the original data according to the data type of the original data; performing data augmentation processing on the original data according to at least one augmentation processing parameter of the at least one augmentation processing category and a randomly generated noise parameter to obtain augmented data of the original data.

[0008] According to a second aspect of the embodiments of this application, a model training method is provided, comprising: acquiring a target sample and adversarial samples of the target sample; and training a target model using the target sample and the adversarial samples; wherein the adversarial samples of the target sample are obtained using the data augmentation processing method described in the first aspect.

[0009] According to a third aspect of the embodiments of this application, a data augmentation processing apparatus is provided, comprising: an augmentation analysis unit, configured to determine at least one augmentation processing category of the original data based on the data type of the original data; and an augmentation processing unit, configured to perform data augmentation processing on the original data based on at least one augmentation processing parameter of the at least one augmentation processing category and randomly generated noise parameters, to obtain augmented data of the original data.

[0010] According to a fourth aspect of the embodiments of this application, a model training apparatus is provided, comprising: an acquisition unit for acquiring a target sample and adversarial samples of the target sample; and a training unit for training a target model using the target sample and the adversarial samples; wherein the adversarial samples of the target sample are obtained using the data augmentation processing method described in the first aspect, or using the data augmentation processing apparatus described in the third aspect.

[0011] According to a fifth aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to perform an operation corresponding to the data augmentation processing method described in the first aspect, or to perform an operation corresponding to the model training method described in the second aspect.

[0012] According to a sixth aspect of the embodiments of this application, a computer storage medium is provided, on which a computer program is stored, which, when executed by a processor, can implement the data augmentation processing method as described in the first aspect or the model training method as described in the second aspect.

[0013] By employing the technical solutions of the above embodiments, the data augmentation category to be performed on the original data can be automatically determined according to the data type of the original data. By introducing random noise, fixed data augmentation results can be avoided, and the rationality and breadth of data perturbation can be ensured, thereby improving the training effect of the model. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a flowchart illustrating the data augmentation processing method as an exemplary embodiment of this application.

[0016] Figure 2 This is a flowchart illustrating a data augmentation processing method as another exemplary embodiment of this application.

[0017] Figure 3 This is a schematic diagram illustrating the application of a training scheme for a data augmentation processing model, which is an exemplary embodiment of this application.

[0018] Figure 4 This is a flowchart illustrating a data augmentation processing method as another exemplary embodiment of this application.

[0019] Figure 5 This is a flowchart illustrating the model training method as an exemplary embodiment of this application.

[0020] Figure 6 This is a structural block diagram of a data enhancement processing apparatus that is an exemplary embodiment of this application.

[0021] Figure 7 This is a structural block diagram of a model training apparatus as an exemplary embodiment of this application.

[0022] Figure 8 This is a structural block diagram of an electronic device that is an exemplary embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present application. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative effort are within the scope of protection of the present application.

[0024] The data augmentation and model training methods, apparatuses, electronic devices, and storage media provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0025] The various embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] refer to Figure 1 The flowchart illustrates a data augmentation processing method according to an exemplary embodiment of this application, which mainly includes the following processing steps:

[0027] Step S102: Determine at least one enhancement processing category for the original data based on the data type of the original data.

[0028] Optionally, the data type of the raw data may include images, sounds, etc.

[0029] Specifically, the type of enhancement processing required for the original data can be determined based on the data type itself. For example, if the original data is identified as an image, the enhancement processing categories for image data can be determined to include color transformation processing, spatial transformation processing, etc.

[0030] Step S104: Perform data augmentation on the original data according to at least one augmentation parameter of at least one augmentation processing category and randomly generated noise parameters to obtain augmented data of the original data.

[0031] In this embodiment, the enhancement processing parameters corresponding to the enhancement processing category can be obtained according to the pre-defined correspondence between enhancement processing categories and enhancement processing parameters. Then, data enhancement processing is performed on the original data according to the randomly generated noise parameters and enhancement processing parameters to obtain enhanced data of the original data.

[0032] Alternatively, the noise parameters can be represented as a parameter vector.

[0033] Optionally, steps S102 and S104 can be performed using a data augmentation model.

[0034] For example, the data augmentation processing operation performed using the data augmentation model can be represented by the following formula 1:

[0035] x'=Λ g (x, z; θ) (Formula 1)

[0036] Where x' represents augmented data, x represents original data, z represents randomly generated noise parameters, θ represents model parameters of the data augmentation model, Λ represents the augmentation category performed on the original data, and g represents the augmentation parameters of augmentation category Λ.

[0037] In summary, the data augmentation method of this embodiment automatically determines the augmentation category to be performed on the original data by identifying the data type of the original data, and avoids generating fixed data augmentation results by introducing randomly generated noise parameters, thus fully ensuring the rationality and breadth of data perturbation.

[0038] Figure 2 A flowchart of a data augmentation processing method according to another exemplary embodiment of this application is shown. This embodiment illustrates a method for performing the above-described data augmentation processing method. Figure 1 The training scheme of the data augmentation processing model in the illustrated embodiment will be combined with the following. Figure 3 The technical solution of this embodiment is described in detail.

[0039] As shown in the figure, this embodiment mainly includes the following steps:

[0040] Step S202: Use the data augmentation model to be trained to perform data augmentation on the original sample to obtain the augmented sample of the original sample.

[0041] Optionally, the data augmentation model to be trained can perform data augmentation based on the original samples and randomly generated noise parameters to obtain augmented samples of the original samples.

[0042] Specifically, the data augmentation model to be trained can determine the augmentation category of the original sample based on the data type of the original sample, and perform data augmentation on the original sample according to the augmentation parameters of the augmentation category and the randomly generated noise parameters to obtain augmented samples of the original data.

[0043] In this embodiment, multiple original samples can be input into the data augmentation model. The data augmentation processing model can be combined with randomly generated noise parameters to perform data augmentation processing on the multiple original samples, thereby obtaining multiple augmented samples of the multiple original samples (in this application, they can also be referred to as adversarial samples of the original samples).

[0044] Alternatively, a data augmentation model can be used to determine the augmentation label of the augmented sample based on the original label of the original sample and the data augmentation process performed on the original sample.

[0045] Step S204: Perform similarity prediction and difference prediction on the original sample and the enhanced sample to obtain the model loss value.

[0046] Optionally, at least one similarity prediction model can be used to perform similarity prediction on the original sample and the enhanced sample.

[0047] Specifically, the original sample and the augmented sample can be used as training sample pairs and input into each similarity prediction model (e.g., Figure 3Similarity prediction is performed in the similarity prediction models 1 to N shown, and the similarity prediction results of each similarity prediction model are obtained.

[0048] In this embodiment, the similarity prediction model may include, but is not limited to, a 3D object detection model and a semantic segmentation model.

[0049] Optionally, a difference prediction model can be used to perform difference prediction on the original sample and the augmented sample.

[0050] Specifically, the original sample and the augmented sample can be used as training sample pairs and input into the difference prediction model (e.g., Figure 3 Perform difference prediction in the difference prediction model shown to obtain the difference prediction results between the original sample and the enhanced sample.

[0051] In this embodiment, the difference prediction model may include a two-dimensional target detection model.

[0052] Optionally, the similarity loss value of the data augmentation processing model to be trained can be obtained based on the similarity prediction result output by the similarity prediction model, and the difference loss value of the data augmentation processing model to be trained can be obtained based on the difference prediction result output by the difference prediction model.

[0053] In this embodiment, the similarity prediction model is a prediction model that has finished training, and the difference prediction model can be a prediction model that has finished training or is yet to be trained.

[0054] Step S206: Update the data augmentation processing model to be trained based on the model loss value.

[0055] In this embodiment, the model parameters of the data augmentation processing model to be trained can be iteratively updated based on the model loss value to optimize the model performance of the data augmentation processing model to be trained.

[0056] Step S208: Determine whether the data augmentation processing model to be trained meets the preset model training termination condition. If yes, proceed to step S210; otherwise, proceed to step S202.

[0057] In this embodiment, when the model parameters of the data augmentation processing model to be trained are updated after iteratively (θ... i ) and the parameter values ​​(θ) before performing the iterative update i-1 When the difference between the two values ​​is less than the preset difference threshold, the judgment result is obtained that the data augmentation processing model to be trained meets the preset model training termination condition.

[0058] In this embodiment, when it is determined that the data augmentation processing model to be trained does not meet the preset model training termination condition, it means that the model training has not ended, and step S202 is repeated.

[0059] Step S210: Obtain the data augmentation processing model.

[0060] In this embodiment, when it is determined that the data augmentation processing model to be trained meets the preset model training termination condition, it means that the model training has ended, and the data augmentation processing model that has finished training is obtained.

[0061] In summary, the data augmentation method provided in this embodiment improves the training effect of the data augmentation model by performing similarity prediction and difference prediction on the original samples and the augmented model generated by the data augmentation model, respectively, and combining the evaluation results of similarity and difference to perform model training.

[0062] Figure 4 This is a flowchart of a data augmentation processing method in another exemplary embodiment of this application. This embodiment is the one described above. Figure 2 Another embodiment of the illustrated example, as shown in the figure, mainly includes the following steps:

[0063] Step S402: Perform similarity prediction on the original sample and the enhanced sample to obtain the similarity prediction result between the original sample and the enhanced sample, and obtain the similarity loss value based on the similarity prediction result.

[0064] Optionally, the similarity prediction results include a similarity prediction vector.

[0065] Optionally, at least one similarity prediction model can be used to perform similarity prediction on the original sample and the enhanced sample to obtain at least one similarity prediction vector of at least one similarity prediction model.

[0066] In this embodiment, the similarity prediction model should be a prediction model that has finished training; that is, the model parameters of the similarity prediction model should be fixed.

[0067] Optionally, a similarity prediction model can be used to perform similarity prediction on the original sample and the enhanced sample based on at least one given prediction dimension to obtain a similarity prediction vector.

[0068] For example, when the prediction dimensions include object location and object category, a similarity prediction model can be used to perform location prediction and category prediction on the target object in the original sample to obtain the object location prediction vector and object category prediction vector of the original sample; and the similarity prediction model can be used to perform location prediction and category prediction on the target object in the enhanced sample to obtain the object location prediction vector and object category prediction vector of the enhanced sample; and a similarity prediction vector can be obtained based on the object location prediction vector and object category prediction vector of the original sample and the object location prediction vector and object category prediction vector of the enhanced sample.

[0069] Optionally, when there are multiple prediction dimensions, the prediction vectors corresponding to each prediction dimension can be concatenated. For example, the object location prediction vector and the object category prediction vector of the original sample can be concatenated to obtain the concatenated vector of the original sample, and the object location prediction vector and the object category prediction vector of the enhanced sample can be concatenated to obtain the concatenated vector of the enhanced sample. Based on the concatenated vector of the original sample and the concatenated vector of the enhanced sample, the similarity prediction vector can be obtained.

[0070] Optionally, the distance between at least one similarity prediction vector of at least one similarity prediction model and the all-1 vector can be calculated to obtain the similarity loss value.

[0071] Optionally, when using multiple similarity prediction models to perform similarity prediction on the original sample and the enhanced sample, the similarity prediction vectors corresponding to each similarity prediction model can be concatenated to obtain a comprehensive similarity vector of all similarity prediction vectors. (Also known as the combined similarity vector of all similarity prediction models) ), and calculate the comprehensive similarity vector. and all-1 vectors (I c The distance between them is used to obtain the similarity loss value. Right now, in, Represents the similarity loss value. I represents the comprehensive similarity vector. c Represents a vector of all 1s in dimension c. Represents the comprehensive similarity vector and all-1 vectors (I c The distance between them.

[0072] Step S404: Determine whether the similarity prediction result meets the preset similarity conditions. If not, proceed to step S406; if so, proceed to step S408.

[0073] Optionally, based on the original label of the original sample and the enhanced label of the enhanced sample, a judgment result can be obtained on whether the similarity prediction result between the original data and the enhanced data meets or does not meet the preset similarity conditions.

[0074] Specifically, based on the original labels of the original samples and the enhanced labels of the enhanced samples, a similarity reference result between the original samples and the enhanced samples can be obtained. Based on the similarity prediction result between the original data and the enhanced data and the similarity reference result, if the similarity prediction result matches the similarity reference result, a judgment result is obtained that the similarity prediction result meets the preset similarity conditions; otherwise, if the similarity prediction result does not match the similarity reference result, a judgment result is obtained that the similarity prediction result does not meet the preset similarity conditions.

[0075] In this embodiment, when the similarity prediction model gives similar prediction results for the original sample and the enhanced sample, a judgment result can be obtained that the similarity prediction results between the original data and the enhanced data meet the preset similarity conditions; when the similarity prediction model gives different prediction results for the original sample and the enhanced sample, a judgment result can be obtained that the similarity prediction results between the original data and the enhanced data do not meet the preset similarity conditions.

[0076] Step S406: Update the model parameters of the data augmentation processing model to be trained according to the similarity loss value, and continue to execute step S202.

[0077] Specifically, when the similarity prediction results between the original sample and the enhanced sample do not meet the preset similarity conditions, the similarity loss value can be used as a basis. Update the model parameters of the data augmentation processing model to be trained.

[0078] Step S408: Perform difference prediction on the original sample and the enhanced sample to obtain the difference prediction result between the original sample and the enhanced sample, and then proceed to step S412.

[0079] Optionally, the difference prediction results between the original sample and the enhanced sample include the distance value between the original sample and the enhanced sample.

[0080] In this embodiment, a difference prediction model can be used to perform distance calculation based on the sample features of the original sample and the sample features of the enhanced sample to obtain the distance value (D) between the original sample and the enhanced sample.

[0081] Optionally, the distance value (D) between the original sample and the augmented sample may include a distance function and may be implemented in a specific form such as multidimensional (e.g., c-dimensional) binary cross-entropy.

[0082] Step S410: Based on the difference prediction results between the original sample and the augmented sample, obtain the difference loss value. Based on the difference loss value and the similarity loss value, update the model parameters of the data augmentation processing model to be trained, and continue to execute step S202.

[0083] In this embodiment, the difference loss value can be obtained based on the distance between the original sample and the enhanced sample.

[0084] In this embodiment, the model parameters of the data augmentation processing model to be trained can be updated using the following formula 2, based on the difference loss value and the similarity loss value:

[0085]

[0086] Where, θ * This represents the updated model parameters of the data augmentation model to be trained. express Expected value Represents all similarity prediction models (e.g.) Figure 3 The comprehensive similarity vector of similarity prediction models 1 to N. Indicates based on and all-1 vectors (I c The similarity loss value determined by the distance between Λ and f, where f represents the difference prediction model. f This represents the data augmentation processing performed by the data augmentation model, Λ f (x; θ) represents the augmented sample of the original sample, f(x) represents the original sample, and D represents f(x) and Λ. f The distance value between (x; θ) Indicates in The difference loss value determined based on the D value under the preset similarity condition.

[0087] Step S412: Determine whether the data augmentation processing model to be trained meets the preset model training termination condition. If yes, proceed to step S414; otherwise, proceed to step S202.

[0088] In this embodiment, the model parameters of the data augmentation processing model to be trained can be updated after iterative processing, with the parameter values ​​(θ) updated accordingly. i ) and the parameter values ​​(θ) before performing the iterative update i-1 When the difference between the two values ​​is less than the preset difference threshold, the judgment result is obtained that the data augmentation processing model to be trained meets the preset model training termination condition.

[0089] In this embodiment, when it is determined that the data augmentation processing model to be trained does not meet the preset model training termination condition, it means that the model training has not ended, and step S202 is repeated.

[0090] Step S414: Determine whether the test results of the data augmentation processing model after training are satisfied with the preset test conditions. If satisfied, proceed to step S416; otherwise, proceed to step S202.

[0091] Specifically, test samples can be extracted from the sample set, and the data augmentation processing model that has finished training can be used to perform data augmentation processing on the test samples to obtain augmented samples of the test samples. If the similarity prediction model outputs similar prediction results for the test samples and augmented samples, and the difference prediction model outputs dissimilar prediction results for the test samples and augmented samples, it can be determined that the test results of the data augmentation processing model that has finished training meet the preset test conditions.

[0092] Step S416: Obtain the data augmentation processing model.

[0093] In summary, the data augmentation processing scheme of this embodiment predicts the similarity between the original sample and the augmented sample, and determines whether the similarity prediction result meets the preset similarity conditions. When it is determined that the similarity prediction result does not meet the preset similarity conditions, the model parameters of the data augmentation processing model are optimized and updated based on the similarity loss value determined by the similarity prediction result. Alternatively, if it is determined that the similarity prediction result meets the preset similarity conditions, the difference between the original sample and the augmented sample is further predicted to obtain the difference loss value determined by the difference prediction result. The model parameters of the data augmentation processing model are then optimized and updated based on the similarity loss value and the difference loss value. In this way, the rationality and applicability of the data augmentation processing results of the data augmentation processing model can be improved.

[0094] Specifically, this embodiment introduces a multi-head output supervision method during the training process of the data augmentation model. By setting up multiple similarity prediction models, the similarity prediction result is determined based on all the prediction outputs of all similarity prediction models (e.g., by jointly scoring all the prediction outputs of all similarity prediction models). Two types of objective optimization are generated by determining whether the similarity prediction result meets preset similarity conditions. The first type of objective optimization minimizes the scoring inconsistency loss (similarity loss value) when the similarity prediction result does not meet the preset similarity conditions. The first objective is to ensure that the data augmentation model, which updates based on the similarity loss value, can produce reasonable data augmentation results each time. The second objective is to further calculate the difference loss value based on the difference prediction results between the original sample and the augmented sample when the similarity prediction results meet the preset similarity conditions, so as to maximize the difference influence of the reasonable augmented sample (adversarial sample) on the output of the difference prediction model, so that the reasonable augmented sample and the original sample can produce drastically different prediction results in the difference prediction model.

[0095] Furthermore, by concatenating the prediction vectors of all similarity prediction models, this embodiment allows the generated similarity prediction results to have a wider range of values, ensuring that the augmented samples generated by the data augmentation model have a certain degree of generalization.

[0096] Furthermore, since the similarity prediction model involved in this embodiment is a prediction model with fixed parameters, and since the similarity prediction model is used to provide the rationality of the augmentation results, there is an assumption: the given similarity prediction model has a better understanding of some knowledge in the target task than the difference prediction model. This allows the difference prediction model to have optimization space by referring to the strengths of each similarity prediction model. This assumption can be achieved by obtaining better or more similarity prediction models, thereby increasing the probability of covering knowledge that the difference prediction model has not learned, and thus obtaining more augmented data that conforms to the distribution definition of the original data.

[0097] Figure 5 The processing flow of the model training method according to an exemplary embodiment of this application is illustrated. As shown in the figure, this embodiment mainly includes the following steps:

[0098] Step S502: Obtain the target sample and its adversarial sample.

[0099] In this embodiment, the adversarial examples of the target sample can be obtained using the data augmentation processing method described in any of the above embodiments.

[0100] Step S504: Train the target model using the target samples and adversarial samples.

[0101] Optionally, the target model may include Figure 3 The difference prediction model shown.

[0102] Optionally, the above methods can be reused after the target model has finished training. Figure 2 or Figure 4 The data augmentation method described in the embodiments trains and optimizes the model parameters of the data augmentation model to achieve the desired result for both the data augmentation model and the target model (e.g., Figure 3 Alternating training between the difference prediction models shown.

[0103] In summary, the model training scheme of this embodiment optimizes the target model based on the data augmentation model after training, thereby improving the training effect of the target model. Furthermore, through the data augmentation model and the target model (e.g., Figure 3 The alternating training scheme between the difference prediction models shown can generate more targeted data augmentation strategies for new target models, thus forming a closed loop.

[0104] Figure 6As shown in the figure, the data enhancement processing apparatus 600 of this embodiment mainly includes:

[0105] The enhancement analysis unit 602 is used to determine at least one enhancement processing category of the original data based on the data type of the original data.

[0106] The enhancement processing unit 604 performs data enhancement processing on the original data according to at least one enhancement processing parameter of the at least one enhancement processing category and a randomly generated noise parameter, to obtain enhanced data of the original data.

[0107] Optionally, the data augmentation processing apparatus 600 includes a data augmentation model built based on the augmentation analysis unit 602 and the augmentation processing unit 604.

[0108] Optionally, the data augmentation processing apparatus 600 further includes a model training unit (not shown) for training a data augmentation model, which includes: performing data augmentation processing on the original sample using the data augmentation processing model to be trained to obtain an augmented sample of the original sample; performing similarity prediction and difference prediction on the original sample and the augmented sample to obtain a model loss value; updating the data augmentation processing model to be trained based on the model loss value; and repeatedly performing the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained until the data augmentation processing model to be trained meets a preset model training termination condition; and obtaining the data augmentation processing model.

[0109] Optionally, the model training unit is further configured to: perform similarity prediction on the original sample and the enhanced sample to obtain a similarity prediction result between the original sample and the enhanced sample, and obtain a similarity loss value based on the similarity prediction result; when the similarity prediction result meets the preset similarity condition, perform difference prediction on the original sample and the enhanced sample to obtain a difference prediction result between the original sample and the enhanced sample; when the similarity prediction result does not meet the preset similarity condition, update the model parameters of the data augmentation processing model to be trained based on the similarity loss value, and execute the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained; obtain a difference loss value based on the difference prediction result, update the model parameters of the data augmentation processing model to be trained based on the difference loss value and the similarity loss value, and execute the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained, until the data augmentation processing model to be trained meets the preset model training termination condition.

[0110] Optionally, the model training unit is further configured to: obtain a similarity reference result between the original sample and the enhanced sample based on the original label of the original sample and the enhanced label of the enhanced sample; and, based on the similarity prediction result and the similarity reference result, if the similarity prediction result matches the similarity reference result, obtain a judgment result that the similarity prediction result satisfies the preset similarity condition, and if the similarity prediction result does not match the similarity reference result, obtain a judgment result that the similarity prediction result does not satisfy the preset similarity condition.

[0111] Optionally, the similarity prediction result includes a similarity prediction vector, and the model training unit is further configured to: use at least one similarity prediction model to perform similarity prediction on the original sample and the enhanced sample according to at least one given prediction dimension, to obtain at least one similarity prediction vector of the at least one similarity prediction model; calculate the distance between the at least one similarity prediction vector and the all-1 vector to obtain the similarity loss value.

[0112] Figure 7 As shown in the figure, the model training apparatus 700 of this embodiment mainly includes:

[0113] The acquisition unit 702 is used to acquire a target sample and an adversarial sample of the target sample, wherein the adversarial sample of the target sample is obtained by using the data augmentation processing method described in any embodiment or by using the data augmentation processing device described in any embodiment.

[0114] Training unit 704 is used to train the target model using the target sample and the adversarial sample.

[0115] Another embodiment of the present invention provides an electronic device, including: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other through the communication bus.

[0116] Figure 8 A structural block diagram of an electronic device as an exemplary embodiment of the present invention, such as... Figure 8 As shown, the electronic device 800 of this embodiment may include a processor 802, a communication interface 804, and a memory 806.

[0117] The processor 802, communication interface 804, and memory 806 can communicate with each other via communication bus 808.

[0118] The communication interface 804 is used to communicate with other electronic devices such as terminal devices or servers.

[0119] The processor 802 is used to execute the computer program 810, specifically to execute the relevant steps in the above-described method embodiments, that is, to execute the steps in the data augmentation processing method or model training method as described in the above-described embodiments.

[0120] Specifically, computer program 810 may include program code that includes computer operation instructions.

[0121] Processor 802 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0122] Memory 806 is used to store computer program 810. Memory 806 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0123] Another embodiment of the present invention provides a computer storage medium storing a computer program thereon, which, when executed by a processor, can implement the data augmentation processing method or model training method described in the above embodiments.

[0124] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of the present invention can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0125] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored on a remote recording medium or a non-transitory machine-readable medium and subsequently stored on a local recording medium, downloaded via a network. Thus, the methods described herein can be stored as software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the data augmentation processing method or model training method described herein is implemented. Furthermore, when a general-purpose computer accesses code used to implement the data augmentation processing method or model training method shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the data augmentation processing method or model training method shown herein.

[0126] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present invention.

[0127] It should be noted that although specific embodiments of this application have been described in detail with reference to the accompanying drawings, this should not be construed as limiting the scope of protection of this application. Various modifications and variations that can be made by those skilled in the art without inventive effort within the scope described in the claims still fall within the scope of protection of this application.

[0128] The examples of the embodiments in this application are intended to concisely illustrate the technical features of the embodiments in this application, so that those skilled in the art can intuitively understand the technical features of the embodiments in this application, and are not intended to be improper limitations on the embodiments in this application.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A data augmentation method, the method being executed by a data augmentation model, comprising: Based on the data type of the original data, determine at least one enhancement processing category for the original data; Based on at least one enhancement processing parameter of the at least one enhancement processing category and randomly generated noise parameters, perform data augmentation processing on the original data to obtain augmented data of the original data; The data augmentation model is trained in the following way: The original sample is augmented using the data augmentation model to be trained, thereby obtaining the augmented sample of the original sample; Similarity prediction and difference prediction are performed on the original sample and the augmented sample to obtain the model loss value. The data augmentation processing model to be trained is updated based on the model loss value. The step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is repeated until the data augmentation processing model to be trained meets the preset model training termination condition. Obtain the data augmentation processing model; Furthermore, the steps of performing similarity prediction and difference prediction on the original sample and the augmented sample to obtain a model loss value, updating the data augmentation processing model to be trained based on the model loss value, and repeatedly performing data augmentation processing on the original sample using the data augmentation processing model to be trained until the data augmentation processing model to be trained meets the preset model training termination condition include: A similarity prediction is performed on the original sample and the enhanced sample to obtain the similarity prediction result between the original sample and the enhanced sample, and a similarity loss value is obtained based on the similarity prediction result; When the similarity prediction result meets the preset similarity condition, the difference prediction is performed on the original sample and the enhanced sample to obtain the difference prediction result between the original sample and the enhanced sample. When the similarity prediction result does not meet the preset similarity condition, the model parameters of the data augmentation processing model to be trained are updated according to the similarity loss value, and the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is executed. Based on the difference prediction result, the difference loss value is obtained. Based on the difference loss value and the similarity loss value, the model parameters of the data augmentation processing model to be trained are updated, and the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is executed until the data augmentation processing model to be trained meets the preset model training termination condition.

2. The method according to claim 1, wherein, The similarity prediction result is determined to be satisfied or not satisfied with the preset similarity condition by means of the following methods: Based on the original labels of the original samples and the enhanced labels of the enhanced samples, a similarity reference result between the original samples and the enhanced samples is obtained; Based on the similarity prediction result and the similarity reference result, if the similarity prediction result matches the similarity reference result, a judgment result is obtained that the similarity prediction result satisfies the preset similarity condition; if the similarity prediction result does not match the similarity reference result, a judgment result is obtained that the similarity prediction result does not satisfy the preset similarity condition.

3. The method according to claim 1, wherein, The similarity prediction result includes a similarity prediction vector; The step of performing similarity prediction on the original sample and the enhanced sample to obtain a similarity prediction result between the original sample and the enhanced sample, and obtaining a similarity loss value based on the similarity prediction result, includes: Using at least one similarity prediction model, and based on at least one given prediction dimension, perform similarity prediction on the original sample and the enhanced sample to obtain at least one similarity prediction vector of the at least one similarity prediction model; Calculate the distance between the at least one similarity prediction vector and the all-1 vector to obtain the similarity loss value.

4. A model training method, comprising: Obtain the target sample and its adversarial examples; The target model is trained using the target sample and the adversarial sample; The adversarial sample of the target sample is obtained using the data augmentation processing method as described in any one of claims 1 to 3.

5. A data augmentation processing apparatus, comprising a data augmentation processing model constructed based on an augmentation analysis unit and an augmentation processing unit. The enhancement analysis unit is used to determine at least one enhancement processing category of the original data based on the data type of the original data. The enhancement processing unit performs data enhancement processing on the original data according to at least one enhancement processing parameter of the at least one enhancement processing category and a randomly generated noise parameter to obtain enhanced data of the original data; in, The data augmentation model was trained in the following way: The original sample is augmented using the data augmentation model to be trained, thereby obtaining the augmented sample of the original sample; Similarity prediction and difference prediction are performed on the original sample and the augmented sample to obtain the model loss value. The data augmentation processing model to be trained is updated based on the model loss value. The step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is repeated until the data augmentation processing model to be trained meets the preset model training termination condition. Obtain the data augmentation processing model; Furthermore, the steps of performing similarity prediction and difference prediction on the original sample and the augmented sample to obtain a model loss value, updating the data augmentation processing model to be trained based on the model loss value, and repeatedly performing data augmentation processing on the original sample using the data augmentation processing model to be trained until the data augmentation processing model to be trained meets the preset model training termination condition include: A similarity prediction is performed on the original sample and the enhanced sample to obtain the similarity prediction result between the original sample and the enhanced sample, and a similarity loss value is obtained based on the similarity prediction result; When the similarity prediction result meets the preset similarity condition, the difference prediction is performed on the original sample and the enhanced sample to obtain the difference prediction result between the original sample and the enhanced sample. When the similarity prediction result does not meet the preset similarity condition, the model parameters of the data augmentation processing model to be trained are updated according to the similarity loss value, and the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is executed. Based on the difference prediction result, the difference loss value is obtained. Based on the difference loss value and the similarity loss value, the model parameters of the data augmentation processing model to be trained are updated, and the step of performing data augmentation processing on the original sample using the data augmentation processing model to be trained is executed until the data augmentation processing model to be trained meets the preset model training termination condition.

6. A model training device, comprising: An acquisition unit is used to acquire a target sample and adversarial examples of the target sample; The training unit is used to train the target model using the target sample and the adversarial sample; The adversarial sample of the target sample is obtained by the data augmentation processing method as described in any one of claims 1 to 3, or by the data augmentation processing apparatus as described in claim 5.

7. An electronic device, characterized in that, include: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to perform the operation corresponding to the data augmentation processing method as described in any one of claims 1 to 3, or to perform the operation corresponding to the model training method as described in claim 4.

8. A computer storage medium, characterized in that, It stores a computer program that, when executed by a processor, can implement the data augmentation processing method as described in any one of claims 1 to 3, or the model training method as described in claim 4.

Images