Data enhancement processing method and device, equipment and storage medium

Abstract

The invention discloses a data enhancement processing method and device, equipment and a storage medium. The method comprises the steps that weighted sampling is conducted on elements in a database according to a first parameter, a first enhancement strategy is obtained, and the first parameter represents the probability of extracting the elements; performing enhancement processing on first training data based on the first enhancement strategy to obtain second training data; training a first model by using the second training data to obtain a second parameter, the second parameter representing the difficulty level of the second training data for model training; and updating the first parameter and / or updating elements in a database according to the second parameter, obtaining a second enhancement strategy based on the updated first parameter and / or the updated database, and performing data enhancement processing according to the second enhancement strategy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a data enhancement processing method, device, equipment and storage medium. Background Art

[0002] Data augmentation refers to a method of generating more training samples by performing various transformations or processing on training data during model training. The purpose is to increase the diversity of data and improve the generalization ability of the model, thereby reducing model overfitting.

[0003] At present, traditional data enhancement methods rely too much on human design, that is, the design of relevant data enhancement parameters, as well as the selection of data enhancement strategies and data enhancement ranges are usually based on human experience. For example, relevant parameters are set to fixed values, or the design randomly takes values ​​within a preset range. However, this design method makes the enhancement strategy fixed, lacks diversity, and lacks comparison and verification, making it impossible to evaluate whether the design is reasonable. If the design is too simple, it is easy to make the model fit too early and lose the meaning of data enhancement; if the design is too difficult, it is easy to make the model over-enhance and waste training samples.

[0004] In addition, most of the existing automatic data enhancement methods are based on the AutoAugment solution, which uses a search algorithm (such as reinforcement learning) to perform brute force search to find the best enhancement strategy from the search space so that the model can achieve the highest accuracy on the validation set, thereby automatically discovering the optimal data enhancement strategy. This type of method relies on the validation set and requires training multiple sub-models, which has high requirements for computing resources. Summary of the invention

[0005] In order to solve the existing technical problems, the embodiments of the present invention provide a data enhancement processing method, device, equipment and storage medium.

[0006] To achieve the above object, the technical solution of the embodiment of the present invention is implemented as follows:

[0007] In a first aspect, an embodiment of the present invention provides a data enhancement processing method, the method comprising: performing weighted sampling on elements in a database according to a first parameter to obtain a first enhancement strategy, wherein the first parameter represents a probability of extracting an element;

[0008] Performing enhancement processing on the first training data based on the first enhancement strategy to obtain second training data;

[0009] Using the second training data to train the first model, obtaining a second parameter, where the second parameter indicates the difficulty of the second training data for model training;

[0010] The first parameter is updated and / or the elements in the database are updated according to the second parameter, and a second enhancement strategy is obtained based on the updated first parameter and / or the updated database to perform data enhancement processing according to the second enhancement strategy.

[0011] In the above scheme, before performing weighted sampling on the elements in the database according to the first parameter to obtain the first enhancement strategy, the method further includes: initializing the database; the database includes: a first database including multiple enhancement operations, a second database including the enhancement amplitude of each enhancement operation, and a third database including multiple training data; the enhancement operation includes an operation range and an operation value;

[0012] Among them, the first database also includes a first update parameter corresponding to each enhancement operation; the second database also includes a second update parameter corresponding to each enhancement amplitude; the third database also includes a third update parameter corresponding to each training data; the first update parameter, the second update parameter and the third update parameter include: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0013] In the above scheme, the database also includes one or more fourth databases, the fourth database includes at least one operation object, and a fourth update parameter corresponding to each operation object, the fourth update parameter includes: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0014] In the above scheme, weighted sampling of elements in the database according to the first parameter to obtain the first enhancement strategy includes: obtaining enhancement operations from the first database based on the first parameter in the first update parameter; the number of the enhancement operations satisfies that is greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; when the number of enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining the enhancement amplitude corresponding to each enhancement operation from the second database; and generating the first enhancement strategy based on the enhancement operations and the corresponding enhancement amplitudes.

[0015] In the above scheme, weighted sampling of elements in the database according to the first parameter to obtain the first enhancement strategy includes: obtaining enhancement operations from the first database based on the first parameter in the first update parameter; the number of the enhancement operations satisfies that is greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; when the number of enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining the enhancement amplitude corresponding to each enhancement operation from the second database; based on the first parameter in the fourth update parameter, obtaining one or more operation objects from the fourth database; and generating the first enhancement strategy according to the enhancement operations, the corresponding enhancement amplitudes and the one or more operation objects.

[0016] In the above scheme, the enhancement processing of the first training data based on the first enhancement strategy includes: obtaining the first training data from the third database based on the first parameter in the third update parameter; and enhancing the first training data based on the first enhancement strategy.

[0017] In the above scheme, updating the first parameter and / or updating the elements in the database according to the second parameter includes: updating the sampling parameter according to the second parameter and the difficulty threshold; updating the first parameter and / or updating the elements in the database according to the updated sampling parameter and the sampling parameter threshold.

[0018] In the above scheme, the difficulty threshold includes a first threshold and a second threshold, and the first threshold is greater than the second threshold; updating the sampling parameter according to the second parameter and the difficulty threshold includes: when the second parameter is greater than the first threshold, updating the sampling parameter based on the sum of the sampling parameter and the first value; when the second parameter is less than the second threshold, updating the sampling parameter based on the difference between the sampling parameter and the first value.

[0019] In the above scheme, the sampling parameter threshold includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; updating the first parameter and / or updating the elements in the database according to the updated sampling parameters and the sampling parameter threshold includes: when the updated sampling parameter is greater than the third threshold or less than the fourth threshold, deleting the elements corresponding to the updated sampling parameters in the database; when the updated sampling parameter is less than or equal to the third threshold and greater than or equal to the fourth threshold, updating the first parameter according to the updated sampling parameter.

[0020] In the above scheme, updating the first parameter according to the updated sampling parameter includes: when the updated sampling parameter is greater than the sampling parameter before updating, increasing the first parameter; when the updated sampling parameter is less than the sampling parameter before updating, decreasing the first parameter.

[0021] In a second aspect, an embodiment of the present invention further provides a data enhancement processing device, the device comprising: a first processing unit, a second processing unit, a third processing unit and an updating unit; wherein,

[0022] The first processing unit is used to perform weighted sampling on elements in the database according to a first parameter to obtain a first enhancement strategy, wherein the first parameter represents a probability of extracting an element;

[0023] The second processing unit is used to perform enhancement processing on the first training data based on the first enhancement strategy to obtain second training data;

[0024] The third processing unit is used to train the first model using the second training data to obtain a second parameter, where the second parameter indicates the difficulty of the second training data for model training;

[0025] The updating unit is used to update the first parameter and / or update the elements in the database according to the second parameter, and obtain a second enhancement strategy based on the updated first parameter and / or the updated database to perform data enhancement processing according to the second enhancement strategy.

[0026] In a third aspect, an embodiment of the present invention further provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method described in the embodiment of the present invention when executing the program.

[0027] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the embodiment of the present invention.

[0028] In a seventh aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which implements the method described in the embodiment of the present invention when executed by a processor.

[0029] The embodiments of the present invention provide a data enhancement processing method, device, equipment and storage medium. On the one hand, a first enhancement strategy is obtained by weighted sampling elements in a database according to a first parameter, so that the number of enhancement operations is not limited, thereby improving the diversity of the enhancement strategy. On the other hand, a first model is trained to obtain second parameters by obtaining second training data after enhancement processing based on the first enhancement strategy on first training data, and the first parameters and / or elements in the database are updated according to the second parameters. The enhancement strategy is re-acquired using the updated first parameters and / or the updated database to perform the next round of data enhancement processing, so that the data enhancement processing method of the present invention does not rely on a validation set and does not need to train multiple sub-models. Dynamic adjustment of the enhancement difficulty can be achieved in only one model training process, thereby improving the model training efficiency while avoiding excessive and invalid enhancement. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a schematic diagram of the relationship between model training and accuracy;

[0031] Figure 2 This is a schematic diagram of the changes in the difficulty of training samples as the model is trained;

[0032] Figure 3 A schematic diagram showing the statistical frequency of high difficulty of noise audio used for audio noise enhancement;

[0033] Figure 4 This is a schematic diagram of the structural framework of the automatic data enhancement AutoAugment method;

[0034] Figure 5 Schematic diagram of a data enhancement processing method according to an embodiment of the present invention;

[0035] Figure 6 The process of obtaining the first enhancement strategy according to the embodiment of the present invention is shown as follows Figure 1 ;

[0036] Figure 7 The process of obtaining the first enhancement strategy according to the embodiment of the present invention is shown as follows Figure 2 ;

[0037] Figure 8 A schematic diagram of the composition of the first enhancement strategy and training data elements according to an embodiment of the present invention;

[0038] Fig. 9 Schematic diagram of the process of updating the first parameter and / or updating the element in the database in the embodiment of the present invention Figure 1 ;

[0039] Fig.10 Schematic diagram of a process of updating a first parameter and / or updating an element in a database according to an embodiment of the present invention Figure 2 ;

[0040] Fig.11 is a data enhancement processing flow chart of an embodiment of the present invention;

[0041] Fig.12 A schematic diagram of the structure of a data enhancement processing device provided by an embodiment of the present invention;

[0042] Fig.13 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0044] It should be understood that the terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0045] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable in appropriate circumstances, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0046] Before describing the data enhancement method according to the embodiment of the present invention in detail, the related technology is briefly described first.

[0047] Data augmentation refers to the method of generating more training samples by performing various transformations and processing on training samples during model training. For example, rotation, flipping, cropping, scaling, color change, etc. in images, or volume disturbance, speed change, noise addition, reverberation addition, random slicing, etc. in audio. The goal of data augmentation is to increase the diversity of data and improve the generalization ability of the model, thereby reducing model overfitting.

[0048] Figure 1 This is a schematic diagram of the relationship between model training and accuracy; Figure 1 As shown in the figure, taking the voiceprint recognition task as an example, the model is iterated on the voiceprint training set and test set with data enhancement, as well as the voiceprint training set and test set without data enhancement, to obtain the accuracy corresponding to different training rounds. Figure 1 It can reflect the impact of data enhancement on model training and performance.

[0049] in, Figure 1 The horizontal axis is the training rounds (epochs) of the model, and the vertical axis is the accuracy (Acc) (%). The dotted line a shows that the training accuracy of the model training set without data enhancement has reached 100% overfitting around the 60th generation, while the solid line c shows that the training accuracy of the model training set with data enhancement can only reach 70% at most. Although the training accuracy of the model training set with data enhancement is not as high as that of the model training set without data enhancement, the test accuracy of the corresponding model test set after data enhancement (such as Figure 1 The dashed line d) is higher than the test accuracy of the model test set without data enhancement (such as Figure 1 Dashed line b), which shows the effectiveness of data augmentation in improving the generalization ability of the model.

[0050] Although data enhancement can effectively prevent model overfitting, in actual application scenarios, the design of different data enhancement parameters, as well as the selection of different data enhancement strategies and amplitudes, will have a significant impact on the effect of data enhancement. Traditional data enhancement methods usually fix the enhancement parameters and amplitudes based on human experience, or set them to a preset range. Due to the lack of comparison and verification, it is impossible to determine whether the design of parameters, strategies, and amplitudes is reasonable. If the design is too simple, after the model is trained to a certain extent, the model has fully fitted this enhancement method, which can easily make this simple data enhancement method lose its meaning; if the design is too difficult, over-enhancement will occur, which can easily make the originally expected features in the training samples become noise samples due to over-enhancement, thereby resulting in a waste of training samples.

[0051] For example, Figure 1 As shown in the figure, although data augmentation can improve the generalization ability of the model, the design method of data augmentation makes the problem of data augmentation more obvious, that is, after the 80th generation, the accuracy of the solid line c basically no longer improves no matter how many times the training is iterated. At this time, the accuracy of the model on the training set is only about 70%, which is obviously in a state of underfitting.

[0052] Will Figure 1The enhanced training samples corresponding to the solid line c can be roughly divided into three types according to the degree of difficulty, namely low-difficulty samples, medium-difficulty samples, and high-difficulty samples. By further analyzing the data of the iterative process of the training samples, we can get the following: Figure 2 The diagram shows how the difficulty of training samples changes with model training.

[0053] Among them, first, it is set that when the model starts training (i.e., the 0th generation), all training samples after data enhancement are high-difficulty samples for the model. As the model training proceeds, if the model can easily perform voiceprint recognition on the training samples after data enhancement, and the recognition discrimination is large, it indicates that this type of sample has been fully fitted by the model, and this type of sample is a low-difficulty sample; if the model has difficulty in performing voiceprint recognition on the training samples after data enhancement, and the recognition discrimination is small, it indicates that this type of sample has not been fully fitted by the model, and this type of sample is a medium-difficulty sample; if the model cannot make correct voiceprint recognition on the training samples after data enhancement, and the recognition is not discriminative, it indicates that this type of sample has not been fitted by the model, and this type of sample is a high-difficulty sample.

[0054] like Figure 2 As shown in Figure 2, as the model training progresses, the number of low-difficulty training samples increases, while the number of medium- and high-difficulty training samples decreases. However, under the existing data enhancement mode, the number of medium- and high-difficulty training samples increases within a certain training round (i.e. Figure 2 After the 80th generation in the training data set, the model will no longer decrease, and subsequent training will not have any effect on improving the model performance. Even if the model is still underfitting on the training data, the model will not be able to gain benefits by continuing training.

[0055] Take the example of using noisy audio to enhance the audio training samples. Figure 3 A statistical diagram of the high difficulty of noise audio used for audio noise enhancement; Figure 3 As shown in the figure, the number of times the audio samples become high-difficulty samples after the audio training samples are subjected to noise enhancement operations using overly difficult noise audio and overly simple noise audio is counted, further reflecting the overly difficult and overly simple data enhancement situations.

[0056] Among them, from the first column of spectrograms with a high number of high difficulty after noise addition, it can be seen that because the noise audio used for the noise addition operation is too difficult, the audio samples after noise addition have always been difficult samples and are difficult to be fitted by the model. This data enhancement has no positive effect on model training. From the third column of spectrograms with a low number of high difficulty, it can be seen that because the noise audio used for the noise addition operation is too simple, the audio samples may still be difficult samples in the early stage of training, but this data enhancement loses its meaning in the middle and late stages of training. It is necessary to continuously reduce such overly complex and overly simple data enhancements during the training process to improve the efficiency of model training.

[0057] From the above analysis, we can see that the use of traditional data enhancement methods leads to Figures 1 to 3 The reasons for the phenomena shown are that, on the one hand, the artificially set fixed data enhancement strategy is relatively simple, so that the model has been fully fitted in the middle and late stages, and continuing to use the same enhancement strategy will no longer be beneficial. A large proportion of such training samples dilutes the proportion of training samples that can bring model improvement, resulting in slow or stagnant model performance improvement; on the other hand, due to the unreasonable design of the data enhancement method, the training samples after data enhancement will cover up or lose important feature information of the training samples, that is, noise data is generated. This noise data is difficult to be fitted by the model under an enhancement strategy with strong randomness (such as online data enhancement, randomly enhancing data once each round), that is, this training sample is still a medium-to-high difficulty training sample no matter how many rounds of training are performed.

[0058] In addition, the core idea of ​​AutoAugment is to use a search algorithm to perform brute force search to find the best enhancement strategy from the search space so that the model can achieve the highest progress on the validation set, thereby automatically discovering the optimal data enhancement strategy.

[0059] Figure 4 It is a schematic diagram of the structural framework of the relevant data enhancement method; Figure 4As shown, AutoAugment extracts an enhancement strategy from the search space through a search algorithm (for example, reinforcement learning, using a recurrent neural network (RNN) controller). The strategy includes the enhancement operation to be used, the probability and amplitude of using the operation in each round, and uses the enhancement strategy to train a subnetwork with a fixed structure to obtain the corresponding submodel. The submodel is tested on the validation set to converge to the accuracy Accuracy (Acc), and the search algorithm (i.e., RNN controller) is updated according to Acc to adjust the next search. Through multiple iterations, the optimal enhancement strategy can be searched from the search space, that is, the strategies of over-enhancement and invalid enhancement are automatically filtered out. Some related automatic data enhancement methods are improvements based on AutoAugment, such as Augmentationwise Sharing for AutoAugment (AWS), Population-based Augmentation (PBA), RandAugment, Fast AutoAugment (FastAA), UniformAugment (UA), TrivialAugment (TA), etc., which will not be elaborated here.

[0060] However, the AutoAugment data enhancement method still has many problems for different model training requirements. First, each enhancement policy in the AutoAugment search space contains 5 sub-policies, each sub-policy contains 2 serial enhancement operations, and each enhancement operation contains 2 hyperparameters: the probability of operation application and the magnitude of the operation. Taking 16 enhancement operations as an example, each operation value is discretized into 10 values, each operation probability is discretized into 11 values, and each sub-policy has two operations, that is, there are (16*10*11) 2 There are possibilities, that is, the search space of 5 sub-strategies is approximately (16*10*11) 10 =2.9*10 32 There are many possibilities. Searching for the optimal strategy in such a huge search space will consume a lot of time and resources.

[0061] Secondly, since each enhancement strategy (including 5 sub-strategies) requires training a sub-network with a fixed structure separately, and each sub-network needs to be tested for accuracy in the validation set, it is not feasible to train multiple sub-networks for large models where training a single network requires a large computational cost.

[0062] Third, AutoAugment limits the data augmentation strategy, and each sub-strategy only contains two augmentation operations. However, in actual application scenarios, this limited augmentation operation strategy does not have sufficient diversity.

[0063] In addition, since the enhancement strategy for training each sub-network in AutoAugment will remain fixed after being selected, even if the sub-strategy combination for the next enhancement strategy can be dynamically adjusted based on the performance of multiple sub-networks, the enhancement strategy is fixed during the training of each sub-network, and the difficulty of the data enhancement object cannot be dynamically adjusted, so the problem of improving model training efficiency cannot be solved.

[0064] Fourthly, the performance improvement of AutoAugment depends on the feedback from the validation set. If the validation set is not representative, it will easily have a significant impact on the effectiveness of the final strategy.

[0065] For the subsequent automatic data enhancement methods based on AutoAugment, in order to solve the problem of too large a search space, the search space is simplified by reducing the number of enhancement operation amplitudes, such as RandAugment (RA) and UA. Although this method will simplify the search space to a certain extent, it will relatively reduce the diversity of data enhancement, resulting in insufficient diversity of data enhancement. In order to improve the efficiency of model training, the search algorithm is improved, but since the improved algorithm still needs to train multiple sub-networks and search for enhancement strategies through the accuracy on the validation set, it still cannot solve the above-mentioned problems. In order to solve the problem of too large a search space and too much reliance on the validation set, the parametric search method is abandoned, such as UA and TA. Although it can effectively solve some problems, it will lead to the inability to filter invalid enhancements and excessive enhancements.

[0066] Based on this, the present invention proposes a data enhancement method, which aims to avoid high computational cost, high time consumption, over-reliance on the validation set, and automatically perform data enhancement without limiting the number of enhancement operations, thereby dynamically adjusting the difficulty of data enhancement and automatically filtering invalid enhancement and excessive enhancement, thereby improving model training efficiency.

[0067] Figure 5 FIG. 1 is a flow chart of a data enhancement processing method according to an embodiment of the present invention; Figure 5 As shown, the method includes:

[0068] Step 101: performing weighted sampling on elements in a database according to a first parameter to obtain a first enhancement strategy, wherein the first parameter represents the probability of extracting an element;

[0069] Step 102: performing enhancement processing on the first training data based on the first enhancement strategy to obtain second training data;

[0070] Step 103: training the first model using the second training data to obtain a second parameter, where the second parameter indicates the difficulty of the second training data for model training;

[0071] Step 104: Update the first parameter and / or update the elements in the database according to the second parameter;

[0072] Step 105: Obtain a second enhancement strategy based on the updated first parameter and / or the updated database, so as to perform data enhancement processing according to the second enhancement strategy.

[0073] In this embodiment, the database includes a plurality of elements for constituting an enhancement strategy, and the plurality of elements include at least enhancement operations, operation values, etc. used for data enhancement processing. In some optional embodiments, the elements included in the database may be different for different needs. For example, if the training data is image data, the elements in the database are elements related to data enhancement processing for image data, such as image rotation processing, sharpness adjustment processing, cropping processing, etc., and the corresponding operation values ​​refer to operation values ​​for enhancement operations. For another example, if the training data is audio data, the elements in the database are elements related to data enhancement processing for audio data, such as noise addition processing for audio data, etc. In this embodiment, for different elements, there may be corresponding first parameters (indicating the probability of extracting the element), that is, weighted sampling is performed according to the first parameters corresponding to each element to obtain the first enhancement strategy.

[0074] In this embodiment, the first training data is training data for training the first model, and the first training data is training data required for one round of training extracted from the training data set. The first model can be any neural network model, and the multiple training data included in the training data set can be at least one of image data, audio data and text data. Among them, the data in the training data set can adopt any data set currently used for machine learning, or can also be generated by image data and audio data collected by any image acquisition device, voice acquisition device, etc. For text data, it can be obtained by converting audio data. In this embodiment, the method of obtaining the training data set is not limited.

[0075] In this embodiment, weighted sampling is performed on the elements in the database according to the first parameter used to represent the probability of extracting each element in the database, and a first enhancement strategy for enhancing the first training data is obtained according to the extracted elements. The database is used to store multiple elements that can form multiple enhancement strategies, and each element corresponds to the first parameter representing the extraction probability. The contents contained in the database and the first parameter will be described in detail in the embodiments of the present invention later.

[0076] In some embodiments, before performing weighted sampling on elements in a database according to a first parameter to obtain a first enhancement strategy, the method further includes: initializing the database; the database includes: a first database including multiple enhancement operations, a second database including an enhancement amplitude of each enhancement operation, and a third database including multiple training data; the enhancement operation includes an operation range and an operation value; wherein the first database also includes a first update parameter corresponding to each enhancement operation; the second database also includes a second update parameter corresponding to each enhancement amplitude; the third database also includes a third update parameter corresponding to each training data; the first update parameter, the second update parameter and the third update parameter include: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0077] In this embodiment, before acquiring the first enhancement strategy according to the first parameter, the database needs to be initialized, and corresponding multiple parameters are set for the elements in the database through initialization.

[0078] Specifically, the database may include a first database, a second database and a third database, wherein the first database includes multiple enhancement operations and a first update parameter corresponding to each enhancement operation; the second database includes one or more enhancement amplitudes corresponding to each enhancement operation in the first database, and a second update parameter corresponding to each enhancement amplitude; the third database includes multiple training data and a third update parameter corresponding to each training data.

[0079] In this embodiment, the first update parameter, the second update parameter and the third update parameter are the parameters used to update or be updated after each round of model training is completed, and include at least the corresponding first parameter, sampling parameter, sampling parameter threshold and difficulty threshold, and may also include the sampling probability update temperature coefficient.

[0080] In this embodiment, the initialized first database includes multiple enhancement operations, each of which includes a value range of the enhancement amplitude corresponding to the operation, one or more enhancement amplitude values ​​that can be selected from the enhancement amplitude range, and a first update parameter related to the operation.

[0081] Among them, the enhancement operation is a single data enhancement method that can be used for different types of training samples to meet different enhancement requirements. For example, for image data, enhancement operations such as rotation, flipping, cropping, scaling, and color change can be used; for audio data, enhancement operations such as noise addition, speed perturbation, and adding reverberation can be used. The enhancement amplitude value represents the specific amplitude value in the enhancement amplitude range corresponding to the data enhancement operation. For example, for the rotation operation of image data, an enhancement amplitude value of 10 means that the image is rotated 10 degrees clockwise; for the speed perturbation of audio data, an enhancement amplitude value of 1.1 means that the audio is changed to 1.1 times the speed.

[0082] It should be noted that, in the embodiment of the present invention, there is no specific restriction on the value of the enhancement amplitude in the enhancement operation range, and the value may be taken arithmetic progression, that is, uniformly within the enhancement operation range at certain intervals, or non-uniformly within the enhancement operation range.

[0083] In this embodiment, the first update parameters related to each enhancement operation in the initialized first database include: the first parameter corresponding to each enhancement operation (i.e., the probability of extracting the enhancement operation), sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient.

[0084] Among them, the first parameter corresponding to each enhancement operation represents the probability of extracting the enhancement operation when weighted sampling is performed in the first database; the sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient corresponding to each enhancement operation are used to update the first parameter (the specific update process will be described in detail later). The present invention does not impose specific restrictions on the design of all elements in the initialized first database, which can be manually given based on experience or randomly set. Usually, the effect of manually given based on experience is better.

[0085] As an example, taking the training data of the first model as audio data, the initialized first database can be expressed as follows:

[0086] {AddNoise(Ranges=[6,10],Values=[6,8,10],

[0087] UP=(P=0.6,R=0,τ=0.01,

[0088] T D =[0.4,0.6],T R =[-∞,∞])),

[0089] SpeedPerturb(Ranges=[0.9,1.1],Values=[0.9,1.0,1.1],

[0090] UP=(P=0.5,R=0,τ=0.01,

[0091] T D =[0.4,0.6],T R =[-∞,∞])),

[0092] AddReverb(Ranges=[...],Values=[...],

[0093] UP=(...))

[0094] ......}

[0095] Among them, the first database is a database containing one or more enhancement operations designed for different enhancement requirements. AddNoise, SpeedPerturb and AddReverb respectively represent the data enhancement operations of adding noise, speed perturbation and adding reverberation that can be performed on audio training samples; Ranges represents the value range of the enhancement amplitude corresponding to each enhancement operation. For example, the enhancement amplitude range [6,10] in the noise addition operation means that the signal-to-noise ratio (SNR) of the noise addition ranges from 6 decibels to 10 decibels; Values ​​represents the enhancement amplitude value that can be selected in the enhancement amplitude range corresponding to each enhancement operation. For example, the enhancement amplitude value in the noise addition operation is [6,8,10], which means that the optional amplitude values ​​in the current noise addition operation are 6 decibels, 8 decibels and 10 decibels; UP represents the first update parameter related to each enhancement operation, which may specifically include the first parameter P, sampling parameter R and difficulty threshold T corresponding to each operation given manually. D , sampling parameter threshold T R And the operation sampling probability updates the temperature coefficient τ, the difficulty threshold T D and sampling parameter threshold T R Includes maximum and minimum values ​​respectively, for example, T D =[0.4,0.6] means setting the maximum value of the noise adding difficulty to 0.6 and the minimum value to 0.4. R =[-∞,∞] means setting the maximum and minimum values ​​of the sampling parameters of the noise addition operation to infinity.

[0096] It should be noted that, with respect to the setting of the value range of the enhancement amplitude corresponding to the enhancement operation, since the present invention can automatically adjust the amplitude extraction probability of the enhancement amplitude corresponding to each enhancement operation with each round of model training to reduce the probability of an unreasonable enhancement amplitude being extracted, the dependence on the enhancement amplitude range of the enhancement operation is relatively low. During initialization, only a rough enhancement amplitude range can be set. For example, when the model needs to be applied in a high-noise scene, the upper limit of the enhancement amplitude range corresponding to the noise addition operation can be set higher, and the lower limit of the corresponding signal-to-noise ratio can be set lower.

[0097] In this embodiment, the initialized second database includes one or more enhancement amplitudes corresponding to each enhancement operation in the first database, and second update parameters related to each enhancement amplitude. The second update parameters include the first parameter corresponding to each enhancement amplitude, sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient.

[0098] Among them, the first parameter corresponding to each enhancement amplitude represents the probability of extracting the enhancement amplitude when weighted sampling is performed in the second database; the sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient corresponding to each enhancement amplitude are used to update the first parameter (the specific update process will be described in detail later). The present invention does not impose specific restrictions on the design of all elements in the initialized second database, which can be manually given based on experience or randomly set. Usually, the effect of manually given based on experience is better.

[0099] As an example, based on the above-mentioned initialized first database, the initialized second database can be expressed as follows:

[0100] {AddNoise(M=(Values=6,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),

[0101] AddNoise(M=(Values=8,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),

[0102] AddNoise(M=(Values=10,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),

[0103] SpeedPerturb(M=(Values=0.9,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),

[0104] SpeedPerturb(M=(Values=1.0,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),SpeedPerturb(M=(Values=1.1,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30]))),AddReverb(M=(Values=...,UP=(...))),......}

[0105] The second database is a database corresponding to the first database and including one or more enhancement amplitudes. For example, in combination with the content included in the first database, when it is determined that the enhancement amplitude M can be 6 decibels, 8 decibels, and 10 decibels according to the enhancement amplitude range of the noise addition operation, the second update parameter UP related to the enhancement amplitude corresponding to each noise addition amplitude value may specifically include the first parameter P, the sampling parameter R, the difficulty threshold T, and the sampling parameter R corresponding to each enhancement amplitude given artificially. D , sampling parameter threshold T R And the sampling probability updates the temperature coefficient τ, the difficulty threshold T D and sampling parameter threshold T R Includes maximum and minimum values ​​respectively, for example, T D =[0.4,0.6] means setting the maximum value of the noise amplitude difficulty to 0.6 and the minimum value to 0.4. R =[-30,30] means setting the maximum value of the sampling parameter of the noise amplitude to 30 and the minimum value to -30.

[0106] It should be noted that, since each enhancement operation in the first enhancement strategy corresponds to only one enhancement amplitude, when initializing the second database, no matter which setting method of the first parameter is adopted, the sum of the first parameters of all enhancement amplitudes corresponding to one enhancement operation should be 1. For example, the above example adopts the method of evenly distributing the first parameter, so that the first parameter P corresponding to each enhancement amplitude in the noise addition operation is 0.333.

[0107] In this embodiment, the initialized third database includes a plurality of original unenhanced training data for training the first model, and a third update parameter associated with each training data. The second update parameter includes the first parameter corresponding to each training data, a sampling parameter, a sampling parameter threshold, a difficulty threshold, and a sampling probability update temperature coefficient.

[0108] Among them, the third parameter corresponding to each training data represents the probability of extracting the training data when weighted sampling is performed in the third database; the sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient corresponding to each training data are used to update the first parameter (the specific update process will be described in detail later). The present invention does not impose specific restrictions on the design of all elements in the initialized third database, which can be manually given based on experience or randomly set.

[0109] As an example, taking the training data of the first model as audio data, the initialized third database can be expressed as follows:

[0110]

[0111] The third database is a database including the original unenhanced training data for training the first model, wherein each training data corresponds to a third update parameter UP related to the training data, which may specifically include the first parameter P, sampling parameter R, difficulty threshold T corresponding to each training data given artificially. D , sampling parameter threshold T R And the sampling probability updates the temperature coefficient τ, the difficulty threshold T D and sampling parameter threshold T R Includes maximum and minimum values ​​respectively, for example, T D =[0.4,0.6] means setting the maximum value of the difficulty of the training data to 0.6 and the minimum value to 0.4. R =[-30,30] means setting the maximum value of the sampling parameter of the training data to 30 and the minimum value to -30.

[0112] It should be noted that, since each enhancement operation in the first enhancement strategy corresponds to only one training data, when initializing the third database, no matter which setting method of the first parameter is adopted, the sum of the first parameters corresponding to all training data should be 1. For example, the above example adopts the method of evenly distributing the first parameters, so that the first parameter P corresponding to each training data is N1 is the number of original data samples.

[0113] In some embodiments, the database also includes one or more fourth databases, the fourth database includes at least one operation object, and a fourth update parameter corresponding to each operation object, the fourth update parameter includes: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0114] In this embodiment, when additional operation objects are needed to assist in completing the enhancement operation on the training data, the database may also include one or more fourth databases. The fourth database is an optional database, which includes multiple operation objects and fourth update parameters corresponding to each operation object. The fourth update parameters include the first parameters, sampling parameters, sampling parameter thresholds and difficulty thresholds corresponding to each operation object, and may also include sampling probability update temperature coefficients.

[0115] For example, when a noise adding operation needs to be performed on audio data, additional noise audio is needed as an operation object to assist in completing the noise adding operation on the audio data.

[0116] It should be noted that the fourth parameter corresponding to each operation object represents the probability of extracting the operation object when weighted sampling is performed in the fourth database; the sampling parameter, sampling parameter threshold, difficulty threshold and sampling probability update temperature coefficient corresponding to each operation object are used to update the first parameter (the specific update process will be described in detail later). The present invention does not impose specific restrictions on the design of all elements in the initialized fourth database, which can be manually given based on experience or randomly set.

[0117] As an example, based on the above-mentioned initialized first database, when data enhancement is performed on the first training data based on the noise addition operation, the initialized fourth database can be expressed as follows:

[0118]

[0119] The fourth database is a database including one or more operation objects required for the noise addition operation. The noise addition operation on the first training data is implemented by extracting operation objects corresponding to the number of samples of the first training data. Each operation object corresponds to a fourth update parameter UP related to the operation object, which may specifically include the first parameter P, sampling parameter R, and difficulty threshold T corresponding to each operation object given manually. D Sampling parameter threshold T R And the sampling probability updates the temperature coefficient τ, the difficulty threshold T D and sampling parameter threshold T R Includes maximum and minimum values ​​respectively, for example, T D =[0.4,0.6] means setting the maximum value of the difficulty of the operation object to 0.6 and the minimum value to 0.4.R =[-30,30] means setting the maximum value of the sampling parameter of the operation object to 30 and the minimum value to -30.

[0120] It should be noted that if the first enhancement strategy includes an operation object, each enhancement operation in the first enhancement strategy can only correspond to one operation object. Therefore, when initializing the fourth database, no matter which setting method of the first parameter is used, the sum of the first parameters corresponding to all operation objects should be 1. For example, the above example adopts the method of evenly distributing the first parameters, so that the first parameter P corresponding to each operation object is N2 is the number of operation objects.

[0121] In some embodiments, Figure 6 The process of obtaining the first enhancement strategy according to the embodiment of the present invention is shown as follows Figure 1 ;like Figure 6 As shown, the weighted sampling of elements in the database according to the first parameter to obtain the first enhancement strategy includes:

[0122] Step 201: based on the first parameter in the first update parameter, obtaining enhancement operations from the first database; the number of the enhancement operations satisfies being greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database;

[0123] Step 202: when the number of the enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining the enhancement amplitude corresponding to each enhancement operation from the second database;

[0124] Step 203: Generate a first enhancement strategy according to the enhancement operation and the corresponding enhancement amplitude.

[0125] Here, after initializing the first database, the second database and the third database in the database, based on the first parameter in the first update parameter (that is, the sampling probability corresponding to each enhancement operation in the first database), the enhancement operation is extracted, and then based on the extracted enhancement operation, based on the first parameter in the second update parameter (that is, the sampling probability of each enhancement amplitude corresponding to each enhancement operation in the second database), an enhancement amplitude corresponding to each enhancement operation is extracted, and the first enhancement strategy is obtained according to the extracted enhancement operation and the enhancement amplitude corresponding to each extracted enhancement operation.

[0126] In this embodiment, by extracting enhancement operations in the first database based on the first parameter in the first update parameter, an enhancement operation set arrangement with one or more enhancement operations between 0 and the total number of enhancement operations can be obtained, and each enhancement operation arrangement corresponds to any training data in the first training data, that is, the number of enhancement operation arrangements is equal to the number of training data in the first training data.

[0127] For example, based on the initialized first database in the above example, it can be determined that the range of the number of optional enhancement operations is [0, 3]. Within this range, an enhancement operation arrangement L extracted based on the first parameter in the first update parameter can be expressed as:

[0128] L=(AddNoise(Ranges=[6,10],Values=[6,8,10],

[0129] UP=(P=0.6,R=0,τ=0.01,T D =[0.4,0.6],T R =[-∞,∞])),

[0130] SpeedPerturb(Ranges=[0.9,1.1],Values=[0.9,1.0,1.1],

[0131] UP=(P=0.5,R=0,τ=0.01,T D =[0.4,0.6],T R =[-∞,∞])))

[0132] The enhancement operation arrangement L includes two enhancement operations, namely, a noise addition operation and a speed change operation. The order of the enhancement operations in L is the order in which the enhancement operations are performed on the training data corresponding to L, that is, the noise addition operation is performed first, and then the speed change is performed. The embodiment of the present invention does not specifically limit the order of the enhancement operations in the enhancement operation arrangement, and the order of the enhancement operations can be determined according to the first parameter in the first update parameter.

[0133] It should be noted that the number of enhanced operations in the enhanced operation arrangement can be any value between [0 and the total number of operations], and the number of operations can be 0, that is, no enhanced operation is performed on the training data. In addition, each number of operations corresponds to a probability. The embodiment of the present invention does not specifically limit the probability of each number of operations in the range of the number of enhanced operations. The probability of each number of operations can be randomly set, or the probability of each number of operations can be set to be the same. Different probabilities can also be assigned to each number of operations as needed, thereby limiting the number of operations.

[0134] For example, without limiting the number of operations, the calculation method of the probability of extracting the enhanced operation arrangement L according to the first parameter in the first update parameter can be set to the following formula:

[0135]

[0136] Among them, P irepresents the first probability of the i-th enhancement operation in the enhancement operation arrangement L in the first database; P j represents the first probability of the jth enhancement operation in the first database except the enhancement operations in the enhancement operation arrangement L; |L| represents the number of enhancement operations in the enhancement operation arrangement L. The probability of extracting the enhancement operation arrangement L can be obtained by multiplying the product of the first parameters of all enhancement operations in L by the product of the probability (1-first parameter) that all enhancement operations except L in the first database are not extracted, and then dividing by the factorial of the number of operations of L.

[0137] In this embodiment, when the number of enhancement operations is greater than 0, after obtaining an enhancement operation arrangement including one or more enhancement operations based on the first parameter in the first update parameter, an enhancement amplitude corresponding to each enhancement operation in the arrangement is extracted from the second database based on the first parameter in the second update parameter to obtain the enhancement amplitude corresponding to the enhancement operation arrangement, and a first enhancement strategy is formed according to the enhancement operation arrangement and the corresponding enhancement amplitude.

[0138] For example, when the enhancement operation arrangement L includes a speed change operation, the first enhancement strategy S composed of the enhancement operation arrangement L and the corresponding enhancement amplitude can be expressed as follows:

[0139] S=(SpeedPerturb(Ranges=[0.9,1.1],Values=[0.9,1.0,1.1],

[0140] M=(Values=1.1,UP=(P=0.5,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30])),

[0141] UP=(P=0.5,R=0,τ=0.01,T D =[0.4,0.6],T R =[-∞,∞])))

[0142] Among them, based on the first parameter corresponding to the speed change operation in the second update parameter, that is, the probability of the enhancement amplitude corresponding to the speed change operation in the second database, the enhancement amplitude of the speed change operation is extracted as 1.1 times the speed. Therefore, the specific data enhancement processing strategy of the first enhancement strategy S is to change the audio speed of the training data to 1.1 times the speed.

[0143] In some embodiments, Figure 7 The process of obtaining the first enhancement strategy according to the embodiment of the present invention is shown as follows Figure 2 , performing weighted sampling on elements in the database according to the first parameter to obtain a first enhancement strategy includes:

[0144] Step 301: based on the first parameter in the first update parameter, obtaining enhancement operations from the first database; the number of the enhancement operations satisfies being greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database;

[0145] Step 302: when the number of the enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining the enhancement amplitude corresponding to each enhancement operation from the second database;

[0146] Step 303: based on the first parameter in the fourth update parameter, obtaining one or more operation objects from the fourth database;

[0147] Step 304: Generate a first enhancement strategy according to the enhancement operation, the corresponding enhancement amplitude and the one or more operation objects.

[0148] Here, in the case where there is a fourth database, after initializing the first database, the second database, the third database and the fourth database in the database, based on the first parameter in the first update parameter (i.e., the sampling probability corresponding to each enhancement operation in the first database), the enhancement operation is extracted, and then based on the extracted enhancement operation, based on the first parameter in the second update parameter (i.e., the sampling probability of each enhancement amplitude corresponding to each enhancement operation in the second database), an enhancement amplitude corresponding to each enhancement operation is extracted, and then based on the first parameter in the fourth update parameter (i.e., the sampling probability of each operation object in the fourth database), an operation object corresponding to the enhancement operation of the operation object is extracted, and the first enhancement strategy is obtained according to the extracted enhancement operation, the enhancement amplitude corresponding to each extracted enhancement operation, and the operation object corresponding to the extracted enhancement operation.

[0149] The description of step 301 and step 302 in the embodiment of the present invention is the same as the description of step 201 and step 202 described above, and will not be repeated here.

[0150] In this embodiment, when the number of enhancement operations is greater than 0, after obtaining the enhancement operation arrangement and the corresponding enhancement amplitude, the operation object corresponding to the enhancement operation is extracted in the fourth database based on the sampling probability of each operation object, and the first enhancement strategy is formed according to the enhancement operation arrangement, the enhancement amplitude corresponding to the enhancement operation arrangement and the operation object corresponding to the enhancement operation.

[0151] For example, based on the example of the enhancement operation arrangement L, the first enhancement strategy S composed of the enhancement operation arrangement and the first enhancement amplitude can be expressed as follows:

[0152] S=(AddNoise(Ranges=[6,10],Values=[6,8,10],

[0153] M=(Value=8,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30])),

[0154] Objects=noise2,UP=(P=0.6,R=0,τ=0.01,T D =[0.4,0.6],T R =[-∞,∞])),

[0155] SpeedPerturb(Ranges=[0.9,1.1],Values=[0.9,1.0,1.1],

[0156] M=(Values=1.1,UP=(P=0.5,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30])),

[0157] UP=(P=0.5,R=0,τ=0.01,T D =[0.4,0.6],T R =[-∞,∞])))

[0158] Among them, the first enhancement strategy S includes two enhancement operations: a noise addition operation and a speed change operation. Based on the first parameter corresponding to the acceleration operation in the second update parameter, and the first parameter corresponding to the speed change operation, that is, the probability of the enhancement amplitude corresponding to the acceleration operation in the second database, and the probability of the enhancement amplitude corresponding to the speed change operation, the enhancement amplitude of the noise addition operation is extracted to be 8 decibels, and the enhancement amplitude of the speed change operation is 1.1 times the speed. Since the 8-decibel noise addition operation on the training data requires the assistance of the operation object, it is necessary to extract an operation object corresponding to the noise addition operation from the fourth database based on the first parameter in the fourth update parameter. For example, the operation object Objects extracted from the fourth database according to the first parameter in the fourth update parameter is noise2.

[0159] Based on this, the specific data enhancement processing strategy of the first enhancement strategy S can be obtained as follows: first use the noise frequency noise2 to perform an 8 dB noise operation on the training data, and then change the audio speed of the noisy training data to 1.1 times the speed.

[0160] As an example, Figure 8Schematic diagram of the first enhancement strategy and element composition of training data in an embodiment of the present invention; Figure 8 As shown, the first enhancement strategy S includes a plurality of first enhancement operations O extracted according to the first parameter in the first update parameter. i (i=1, 2......n), taking the enhanced operation O1 including the object to be operated as an example, the enhanced operation O1 includes the enhanced operation range corresponding to the operation, the enhanced operation amplitude value, a first enhanced amplitude extracted according to the first parameter in the second update parameter, a first operation object extracted according to the first parameter in the fourth update parameter, and the first update parameter; the first enhanced amplitude includes the second update parameter corresponding to the enhanced amplitude; the first operation object includes the fourth update parameter corresponding to the operation object; in addition, the first training data corresponding to the first enhanced strategy S includes n training data; each training data includes the third update parameter corresponding to the training data; the first update parameter, the second update parameter, the third update parameter and the fourth update parameter include their respective corresponding first parameters, sampling parameters, sampling parameter thresholds, difficulty thresholds and sampling probability update temperature coefficients.

[0161] In this embodiment, after extracting various elements from the database according to the first parameter to form the first enhancement strategy, the first training data is enhanced based on the first enhancement strategy to obtain second training data, that is, the training data after data enhancement.

[0162] It should be noted that the present invention does not impose any specific restrictions on the first training data to be enhanced by each first enhancement strategy, that is, each first enhancement strategy can be randomly paired with any training data in the extracted first training data without repetition to perform data enhancement processing.

[0163] In some embodiments, the enhancing the first training data based on the first enhancement strategy includes: acquiring the first training data from the third database based on the first parameter in the third update parameter; and enhancing the first training data based on the first enhancement strategy.

[0164] In this embodiment, the first training data is extracted based on the first parameter in the third update parameter, that is, the sampling probability of each training data in the third database, and the number of training data in the first training data is the same as the number of generated first enhancement strategies, and the extracted first training data is enhanced using the generated first enhancement strategy.

[0165] It should be noted that the embodiment of the present invention does not impose any specific restrictions on the order of obtaining the first enhancement strategy and extracting the first training data, that is, multiple first enhancement strategies can be generated first, and then the first training data can be extracted according to the number of the first enhancement strategies; or first training data including a certain number of training data can be extracted first, and then the corresponding multiple first enhancement strategies can be generated according to the number of training data. The number of training data can be artificially given in advance or randomly preset.

[0166] In this embodiment, after the first training data is enhanced based on the first enhancement strategy to obtain the second training data after data enhancement, the first model is trained using the second training data to obtain a second parameter indicating the difficulty of the second training data for model training.

[0167] Specifically, the enhanced second training data is input into the first model for training, and the training result is output. The second parameter is calculated according to the training result and the label corresponding to the second training data as a parameter for evaluating the difficulty level of the second training data.

[0168] It should be noted that the first model is the model that needs to be trained corresponding to the first training data, and the embodiment of the present invention does not limit the selection of the first model. In addition, the embodiment of the present invention does not specifically limit the second parameter selected for evaluating the difficulty of the second training data. The second parameter can be calculated by calculating the loss function, or by calculating the distance between the training data and its classification center or the cosine similarity. The specific calculation method can adopt the existing calculation method, which will not be described in detail here.

[0169] In this embodiment, after the first model is trained using the second training data to obtain a second parameter indicating the difficulty of the second training data for model training, the first parameter is updated and / or the elements in the database are updated according to the second parameter.

[0170] In some embodiments, Fig. 9 Schematic diagram of the process of updating the first parameter and / or updating the element in the database in the embodiment of the present invention Figure 1 ;like Fig. 9 As shown, updating the first parameter and / or updating the element in the database according to the second parameter includes:

[0171] Step 401: Update the sampling parameter according to the second parameter and the difficulty threshold.

[0172] In this embodiment, after the first model is trained according to the second training data to obtain the second parameters, the sampling parameters in the database are updated according to the second parameters and the difficulty threshold of each element in the first enhancement strategy.

[0173] It should be noted that, due to the difference in databases, there are multiple difficulty thresholds in the first enhancement strategy, namely, the difficulty thresholds in the first update parameter, the second update parameter, the third update parameter, and the fourth update parameter. It is necessary to update the sampling parameters in the database with the second parameter in turn and the difficulty threshold of each element in the first enhancement strategy, namely, update the sampling parameters of the first update parameter, the second update parameter, the third update parameter, and the fourth update parameter in the first enhancement strategy in turn. The embodiment of the present invention does not specifically limit the update order of the sampling parameters in different databases.

[0174] In some embodiments, the difficulty threshold includes a first threshold and a second threshold, and the first threshold is greater than the second threshold; updating the sampling parameter according to the second parameter and the difficulty threshold includes: when the second parameter is greater than the first threshold, updating the sampling parameter based on the sum of the sampling parameter and the first value; when the second parameter is less than the second threshold, updating the sampling parameter based on the difference between the sampling parameter and the first value.

[0175] Here, the difficulty thresholds in different databases all include a first threshold and a second threshold that is smaller than the first threshold, that is, the difficulty threshold represents a difficulty range, including the first threshold as the maximum value and the second threshold as the minimum value. For example, in the example of the first database after initialization, the difficulty threshold T in the first update parameter D =[0.4,0.6], where 0.6 is the first threshold and 0.4 is the second threshold.

[0176] In this embodiment, the first threshold and the second threshold of the difficulty threshold of each element in the first enhancement strategy are compared with the second parameter in turn, that is, the second parameter is compared with the first threshold and the second threshold of the difficulty threshold of each element in the first enhancement strategy in turn, and the sampling parameters in the database are updated according to the comparison results.

[0177] Specifically, when the second parameter is greater than the first threshold of the difficulty threshold of the element in the first enhancement strategy, the sampling parameter is updated based on the sum of the corresponding sampling parameter and the first value; when the second parameter is less than the second threshold of the difficulty threshold of the element in the first enhancement strategy, the sampling parameter is updated based on the difference between the corresponding sampling parameter and the first value. The embodiment of the present invention does not limit the setting of the first value, which can usually be preset to 1.

[0178] For example, based on the example of the initialized first database, the noise adding operation is selected in the first enhancement strategy, that is, the difficulty threshold of the first update parameter in the first database corresponding to the enhancement operation is T D=[0.4,0.6], sampling parameter R=0, and the first value is preset to 1, when the calculated second parameter is 0.72, since 0.72 is greater than the first threshold 0.6 of the difficulty threshold, the sampling parameter 0 is added to the first value 1, that is, the updated sampling parameter is 1;

[0179] When the second parameter is calculated to be 0.24, since 0.24 is less than the second threshold value 0.4 of the difficulty threshold, the sampling parameter 0 is subtracted from the first value 1, that is, the updated sampling parameter obtained is -1;

[0180] When the calculated second parameter is 0.53, since 0.53 is greater than the second threshold 0.4 of the difficulty threshold and less than the first threshold 0.6 of the difficulty threshold, that is, the second parameter is within the range of the difficulty threshold, the sampling parameter will not be updated, that is, the sampling parameter will remain unchanged.

[0181] Step 402: Update the first parameter and / or update the elements in the database according to the updated sampling parameter and sampling parameter threshold.

[0182] In this embodiment, after updating the sampling parameters according to the second parameters and the difficulty threshold, the first parameters of each element in the first enhancement strategy are updated and / or the corresponding elements in the database are updated according to all updated sampling parameters of each element in the first enhancement strategy and the corresponding sampling parameter threshold.

[0183] It should be noted that due to the differences in databases, there are multiple sampling parameter thresholds in the first enhancement strategy, namely, the sampling parameter thresholds in the first update parameter, the second update parameter, the third update parameter, and the fourth update parameter. According to the sampling parameters of each element in the updated first enhancement strategy and the corresponding sampling parameter thresholds, only the first parameter of the element and / or the corresponding element in the database can be updated, and the update order is consistent with the order of updating the sampling parameters, that is, after updating the corresponding sampling parameters according to the second parameter and the difficulty threshold of all elements in the first enhancement strategy, the first parameter and / or the corresponding element in the database are updated in sequence according to the updated sampling parameters and the corresponding sampling parameter thresholds.

[0184] In some embodiments, the sampling parameter threshold includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; updating the first parameter and / or updating the elements in the database according to the updated sampling parameters and the sampling parameter threshold includes: when the updated sampling parameter is greater than the third threshold or less than the fourth threshold, deleting the elements corresponding to the updated sampling parameters in the database; when the updated sampling parameter is less than or equal to the third threshold and greater than or equal to the fourth threshold, updating the first parameter according to the updated sampling parameter.

[0185] Here, the sampling parameter threshold of each element in the first enhancement strategy includes the third threshold and the fourth threshold that is less than the third threshold, that is, the sampling parameter threshold represents the sampling parameter range, including the third threshold as the maximum value and the fourth threshold as the minimum value. For example, in the above-mentioned example of the second database after initialization, the sampling parameter threshold T in the second update parameter R =[-30,30], where 30 is the third threshold and -30 is the fourth threshold.

[0186] In this embodiment, the third threshold and the fourth threshold in the sampling parameter threshold of the current element in the first enhancement strategy are compared with the corresponding updated sampling parameters, and the first parameter of the element in the first enhancement strategy and / or the corresponding element in the database are updated according to the comparison result.

[0187] Specifically, when the updated sampling parameter is greater than the third threshold of the sampling parameter threshold of the element in the first enhancement strategy, or is less than the fourth threshold of the sampling parameter threshold of the element in the first enhancement strategy, the element corresponding to the updated sampling parameter in the corresponding database is deleted, that is, the corresponding enhancement operation or enhancement amplitude or operation object or training data is deleted; when the updated sampling parameter is less than or equal to the third threshold of the sampling parameter threshold of the element in the first enhancement strategy, and is greater than or equal to the fourth threshold of the element in the first enhancement strategy, the first parameter corresponding to the element is updated according to the updated sampling parameter.

[0188] It should be noted that, since the third threshold of the sampling parameter threshold of the enhancement operation can usually be set to positive infinity and the fourth threshold can usually be set to negative infinity, the enhancement operation is usually not deleted, that is, only the corresponding enhancement amplitude or operation object is deleted.

[0189] For example, based on the example of the second database after the initialization, the enhancement amplitude of the speed change operation selected in the first enhancement strategy is 1.1 times the speed, that is, the sampling parameter threshold of the second update parameter in the second database corresponding to the enhancement amplitude is T R =[-30,30], when the updated sampling parameter R' corresponding to the element in the first enhancement strategy is 32, since 32 is greater than the third threshold 30 of the sampling parameter threshold, the element corresponding to the updated sampling parameter in the database will be deleted, that is, the enhancement amplitude corresponding to the updated sampling parameter will be deleted.

[0190] SpeedPerturb(M=(Values=1.1,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30])));

[0191] When the updated sampling parameter R' corresponding to the element in the first enhancement strategy is -31, since -31 is less than the fourth threshold -30 of the sampling parameter threshold, the element corresponding to the updated sampling parameter in the database is deleted, that is, the enhancement amplitude corresponding to the updated sampling parameter in the second database is deleted: SpeedPerturb(M=(Values=1.1,UP=(P=0.333,R=0,τ=0.1,T D =[0.4,0.6],T R =[-30,30])));

[0192] When the updated sampling parameter R' in the database is 20, since 20 is greater than the fourth threshold value -30 of the sampling parameter threshold and less than the third threshold value 30 of the sampling parameter threshold, that is, the updated sampling parameter is within the range of the sampling parameter threshold, the first parameter in the corresponding database will be further updated according to the updated sampling parameter.

[0193] In this embodiment, when the updated sampling parameter is greater than the third threshold of the corresponding sampling parameter threshold, it means that in the current first enhancement strategy, the element corresponding to the updated sampling parameter causes the difficulty of the enhanced second training data to be too large, that is, it causes over-enhancement, and the training data is enhanced into a noise sample; when the updated sampling parameter is less than the fourth threshold of the corresponding sampling parameter threshold, it means that in the current first enhancement strategy, the element corresponding to the updated sampling parameter causes the difficulty of the enhanced second training data to be too small, that is, it causes invalid enhancement, and therefore it is necessary to delete the element corresponding to the sampling parameter, thereby updating the database and then updating the enhancement strategy to prevent over-enhancement or invalid enhancement.

[0194] It should be noted that, in an embodiment of the present invention, after the second parameter (difficulty) is calculated for all the second training data in a round of training, the sampling parameters of all database elements corresponding to the first enhancement strategy are updated according to the second parameter. Only after all the updates are completed, the final sampling parameters will update the corresponding first parameters / or database elements with their respective corresponding sampling parameter thresholds.

[0195] It can be understood that by counting the difficulty level of the same round, the enhancement operations, amplitudes, objects or training data itself that always lead to high difficulty of the second training data are deleted to prevent excessive enhancement or noise samples; the enhancement operations, amplitudes, objects or training data itself that always lead to low difficulty of samples are deleted to prevent invalid enhancement, thereby improving the efficiency of model training.

[0196] As an example, Fig.10 Schematic diagram of a process of updating a first parameter and / or updating an element in a database according to an embodiment of the present invention Figure 2 ;like Fig.10 As shown, the specific process is as follows:

[0197] First, the second parameter D according to the second training data is compared with the first threshold and the second threshold of the difficulty threshold of each element in the first enhancement strategy in turn. When D is greater than the first threshold, the sampling parameter is added to the preset first value 1 to obtain the updated sampling parameter; when D is less than the second threshold, the sampling parameter is subtracted from the preset first value 1 to obtain all updated sampling parameters;

[0198] Secondly, based on all updated sampling parameters, the updated sampling parameters of the element in the first enhancement strategy are compared with the third threshold and the fourth threshold of the corresponding sampling parameter threshold respectively. When R' is greater than the third threshold, the elements corresponding to the updated sampling parameters in the corresponding database are deleted; when R' is less than the fourth threshold, the elements corresponding to the updated sampling parameters in the corresponding database are deleted; when R' is less than or equal to the third threshold and greater than or equal to the fourth threshold, the first parameters in the corresponding database are updated according to the updated sampling parameters.

[0199] In this embodiment, after updating the first parameter and / or updating the elements in the database according to the second parameter, the enhancement strategy is reacquired based on the updated first parameter and / or the updated database, and the next round of data enhancement processing is performed according to the newly acquired enhancement strategy.

[0200] In some embodiments, updating the first parameter according to the updated sampling parameter includes: when the updated sampling parameter is greater than the sampling parameter before updating, increasing the first parameter; when the updated sampling parameter is less than the sampling parameter before updating, decreasing the first parameter.

[0201] In this embodiment, when the updated sampling parameter is within the corresponding sampling parameter threshold range, the first parameter in the corresponding database is updated according to the updated sampling parameter and the sampling parameter before the update, that is, the weighted sampling probability of the enhancement operation or enhancement amplitude or operation object or training data is updated. Specifically, when the updated sampling parameter is greater than the sampling parameter, the first parameter is increased; when the updated sampling parameter is less than the sampling parameter, the first parameter is decreased.

[0202] It should be noted that when the extracted enhancement operation is not unique, and the number of enhancement operations in the enhancement strategy is not limited, for example, the enhancement operations in the first database are independent of each other, that is, the first parameters corresponding to the enhancement operations in the first database do not affect each other. However, since one enhancement strategy corresponds to only one training data and one operation object, and different enhancement operations in one enhancement strategy correspond to only one enhancement amplitude, different from the first database, the enhancement amplitudes in the second database, the training data in the third database, and the operation objects in the fourth database are mutually exclusive, that is, the sum of all the first parameters of the elements in the second database, the third database, and the fourth database should all be 1.

[0203] As an example, the calculation method for updating the first parameter in the first database may be designed as the following formula:

[0204]

[0205] Among them, P i ' and P i represents the first parameter of the i-th enhancement operation in the first database before and after the update; R i ' and R i represent the sampling parameters of the i-th enhancement operation in the first database before and after the update respectively; τ is the sampling probability update temperature coefficient of the i-th enhancement operation in the first database.

[0206] It can be understood that when the updated sampling parameter is greater than the sampling parameter, the first parameter can be increased by the calculation method of formula (2), that is, when the enhancement operation causes the difficulty of the second training data to be higher, the probability of the enhancement operation being extracted again is increased; when the updated sampling parameter is less than the sampling parameter, the first parameter can be reduced by the calculation method of formula (2), that is, when the enhancement operation causes the difficulty of the second training data to be lower, the probability of the enhancement operation being extracted again is reduced.

[0207] As another example, the calculation method for updating the first parameter in the second database, the third database, and the fourth database may be designed as the following formula:

[0208]

[0209] Among them, P i ' and P i represents the first parameter of the i-th element (i.e., enhancement amplitude or training data or operation object) in the second database, the third database or the fourth database after and before updating, respectively; R i ' and R iRespectively represent the sampling parameters of the i-th element (i.e., enhancement amplitude, training data, or operation object) in the second database, the third database, or the fourth database after and before the update; T is the sampling probability update temperature coefficient of the i-th element (i.e., enhancement amplitude, training data, or operation object) in the second database, the third database, or the fourth database. Compared with formula (2), since the enhancement amplitudes in the second database, the training data in the third database, and the operation objects in the fourth database are mutually exclusive, it is necessary to compare the first parameter of the i-th pseudo-update in the database with the sum of the first parameters of all pseudo-updates in the database, so that the sum of all the first parameters of the elements in the second database, the third database, and the fourth database should all be 1.

[0210] It can be understood that when the updated sampling parameter is greater than the sampling parameter, the first parameter can be increased by the calculation method of formula (3), that is, when the enhancement amplitude or the operation object or the training data causes the difficulty of the second training data to be higher, the probability of the enhancement amplitude or the operation object or the training data being extracted again is increased; when the updated sampling parameter is less than the sampling parameter, the first parameter can be reduced by the calculation method of formula (3), that is, when the enhancement amplitude or the operation object or the training data causes the difficulty of the second training data to be lower, the probability of the enhancement amplitude or the operation object or the training data being extracted again is reduced.

[0211] In this embodiment, after the first parameter or the element in the database is updated, the acquisition of the enhancement strategy is re-executed based on the updated first parameter and / or the updated database to obtain the second enhancement strategy, so as to perform data enhancement on the re-extracted training data according to the second enhancement strategy, and then start the next round of model training based on the enhanced training data, calculate the second parameter based on the training result, continue to update the first parameter and / or the element in the database based on the second parameter, and then obtain a new enhancement strategy again, and repeat this cycle.

[0212] It should be noted that, since the data enhancement of the training data using the enhancement strategy is performed with each round of model training, the end condition of the data enhancement is the stop condition of the model training. The embodiment of the present invention does not impose any specific restrictions on the stop condition of the model training. The stop condition of the model training can be designed to be whether the maximum number of iterations is reached, that is, when the round of model training reaches the maximum number of iterations, the model training is stopped and the data enhancement of the training data is ended.

[0213] As an example, Fig.11 is a data enhancement processing flow chart of an embodiment of the present invention; Fig.11 As shown, the specific data enhancement processing method is as follows:

[0214] Step 501: extracting one or more enhancement operations from a first database according to a first parameter in a first update parameter;

[0215] Step 502: extracting enhancement amplitudes corresponding to the one or more enhancement operations from the second database according to the first parameter in the second update parameter;

[0216] Step 503: when the extracted enhancement operation requires an operation object, extract the operation object corresponding to the enhancement operation from the fourth database according to the first parameter in the fourth update parameter;

[0217] Step 504: obtaining a first enhancement strategy according to the first one or more enhancement operations, the enhancement amplitudes corresponding to the one or more enhancement operations, and the operation objects corresponding to the enhancement operations, respectively extracted from the first database, the second database, and the fourth database;

[0218] Step 505: extracting training data corresponding to the one or more enhancement operations from the third database according to the first parameter in the third update parameter;

[0219] Step 506: performing data enhancement on the extracted first training data based on the first enhancement strategy to obtain enhanced second training data;

[0220] Step 507: performing a round of model training on the first model according to the second training data, and calculating a second parameter representing the difficulty of the second training data according to the training result and the second training data;

[0221] Step 508: When the enhancement end condition is not met, that is, the maximum number of iterations of model training is not reached, the corresponding sampling parameters are updated according to the second parameter and the difficulty threshold of each element in the first enhancement strategy;

[0222] Step 509: Determine whether to delete the element in the database corresponding to the updated sampling parameter according to the updated sampling parameter and the corresponding sampling parameter threshold; if not, update the corresponding first parameter.

[0223] All databases are updated according to the updated first parameter and / or the elements in the database, and the acquisition of the enhancement strategy, the acquisition of the training data, the data enhancement of the training data, and the acquisition of the second parameter are re-executed until the enhancement end condition is met.

[0224] In this embodiment, a first enhancement strategy is obtained by weighted sampling elements in a database according to a first parameter, and the first model is trained to obtain second parameters based on second training data obtained after enhancement processing of the first training data by the first enhancement strategy. The first parameters and / or the elements in the database are updated according to the second parameters, and the enhancement strategy is updated using the updated first parameters and / or the updated database, so that the number of enhancement operations is not limited, thereby improving the diversity of enhancement strategies. The data enhancement processing method of the present invention does not rely on a validation set and does not require training of multiple sub-models. Dynamic adjustment of the enhancement difficulty can be achieved only in the process of sequential model training, thereby improving the model training efficiency while avoiding excessive enhancement and invalid enhancement.

[0225] Based on the above embodiments, an embodiment of the present invention further provides a data enhancement processing device, which is applied to a computer device. Fig.12 A schematic diagram of the structure of a data enhancement processing device provided in an embodiment of the present invention; Fig.12 As shown, the device includes: a first processing unit 61, a second processing unit 62, a third processing unit 63 and an updating unit 64; wherein,

[0226] The first processing unit 61 is used to perform weighted sampling on the elements in the database according to a first parameter to obtain a first enhancement strategy, where the first parameter represents the probability of extracting the element;

[0227] The second processing unit 62 is used to perform enhancement processing on the first training data based on the first enhancement strategy to obtain second training data;

[0228] The third processing unit 63 is used to train the first model using the second training data to obtain a second parameter, where the second parameter indicates the difficulty of the second training data for model training;

[0229] The updating unit 64 is used to update the first parameter and / or update the elements in the database according to the second parameter, obtain a second enhancement strategy based on the updated first parameter and / or the updated database, and perform data enhancement processing according to the second enhancement strategy.

[0230] In an optional embodiment of the present invention, the device further includes a preprocessing unit for initializing the database; the database includes: a first database including multiple enhancement operations, a second database including the enhancement amplitude of each enhancement operation, and a third database including multiple training data; the enhancement operation includes an operation range and an operation value;

[0231] Among them, the first database also includes a first update parameter corresponding to each enhancement operation; the second database also includes a second update parameter corresponding to each enhancement amplitude; the third database also includes a third update parameter corresponding to each training data; the first update parameter, the second update parameter and the third update parameter include: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0232] In an optional embodiment of the present invention, the database also includes one or more fourth databases, the fourth database includes at least one operation object, and a fourth update parameter corresponding to each operation object, the fourth update parameter includes: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

[0233] In an optional embodiment of the present invention, the first processing unit 61 is used to obtain an enhancement operation from the first database based on the first parameter in the first update parameter; the number of the enhancement operations satisfies that is greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; when the number of enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtain the enhancement amplitude corresponding to each enhancement operation from the second database; generate a first enhancement strategy according to the enhancement operations and the corresponding enhancement amplitudes.

[0234] In an optional embodiment of the present invention, the first processing unit 61 is used to obtain enhancement operations from the first database based on the first parameter in the first update parameter; the number of the enhancement operations satisfies that is greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; when the number of enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtain the enhancement amplitude corresponding to each enhancement operation from the second database; based on the first parameter in the fourth update parameter, obtain one or more operation objects from the fourth database; generate a first enhancement strategy according to the enhancement operations, the corresponding enhancement amplitudes and the one or more operation objects.

[0235] In an optional embodiment of the present invention, the second processing unit 62 is used to obtain first training data from the third database based on the first parameter in the third update parameter; and perform enhancement processing on the first training data based on the first enhancement strategy.

[0236] In an optional embodiment of the present invention, the updating unit 64 is used to update the sampling parameter according to the second parameter and the difficulty threshold; update the first parameter and / or update the elements in the database according to the updated sampling parameter and the sampling parameter threshold.

[0237] In an optional embodiment of the present invention, the difficulty threshold includes a first threshold and a second threshold, and the first threshold is greater than the second threshold; the updating unit 64 is used to update the sampling parameter based on the sum of the sampling parameter and the first value when the second parameter is greater than the first threshold; when the second parameter is less than the second threshold, update the sampling parameter based on the difference between the sampling parameter and the first value.

[0238] In an optional embodiment of the present invention, the sampling parameter threshold includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; the updating unit 64 is used to delete the elements corresponding to the updated sampling parameters in the database when the updated sampling parameters are greater than the third threshold or less than the fourth threshold; when the updated sampling parameters are less than or equal to the third threshold and greater than or equal to the fourth threshold, update the first parameter according to the updated sampling parameters.

[0239] In an optional embodiment of the present invention, the updating unit 64 is used to increase the first parameter when the updated sampling parameter is greater than the sampling parameter before updating; and to decrease the first parameter when the updated sampling parameter is less than the sampling parameter before updating.

[0240] In the implementation of the present invention, the first processing unit 61, the second processing unit 62, the third processing unit 63, the update unit 64 and the pre-processing unit in the device can all be implemented by a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA) in actual applications.

[0241] It should be noted that: when the above-mentioned data enhancement processing device performs data enhancement processing, only the division of the above-mentioned program modules is used as an example. In actual applications, the above-mentioned processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the above-mentioned processing. In addition, the data enhancement processing device provided in the above embodiment and the data enhancement processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0242] The embodiment of the present invention further provides a computer device, Fig.13 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention is shown in FIG. Fig.13As shown, the computer device includes a memory 72 , a processor 71 , and a computer program stored in the memory 72 and executable on the processor 71 .

[0243] Optionally, the processor 71 implements the steps of the data enhancement processing method of an embodiment of the present invention when executing the program.

[0244] Optionally, the various components in the communication device can be coupled together via a bus system 73. It is understood that the bus system 73 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 73 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.13 Various buses are labeled as bus system 73.

[0245] It can be understood that the memory 72 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 72 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0246] The method disclosed in the above embodiment of the present invention can be applied to the processor 71, or implemented by the processor 71. The processor 71 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 71 or the instruction in the form of software. The above processor 71 can be a general-purpose processor, DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 71 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiment of the present invention, it can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in the memory 72. The processor 71 reads the information in the memory 72 and completes the steps of the above method in combination with its hardware.

[0247] In an exemplary embodiment, the computer device can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), FPGA, general processor, controller, MCU, microprocessor, or other electronic components to execute the aforementioned method.

[0248] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored.

[0249] Optionally, the computer-readable storage medium can be applied to the data enhancement processing device of the embodiment of the present invention; then when the program is executed by the processor, the steps of the data enhancement processing method of the embodiment of the present invention are implemented.

[0250] The methods disclosed in several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0251] The features disclosed in several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0252] The features disclosed in several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments or device embodiments.

[0253] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0254] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0255] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0256] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0257] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0258] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A data enhancement processing method, characterized in that: The method comprises: Performing weighted sampling on elements in the database according to a first parameter to obtain a first enhancement strategy, wherein the first parameter represents a probability of extracting an element; Performing enhancement processing on the first training data based on the first enhancement strategy to obtain second training data; Using the second training data to train the first model, obtaining a second parameter, where the second parameter indicates the difficulty of the second training data for model training; The first parameter is updated and / or the elements in the database are updated according to the second parameter, and a second enhancement strategy is obtained based on the updated first parameter and / or the updated database to perform data enhancement processing according to the second enhancement strategy.

2. The method according to claim 1, characterized in that Before performing weighted sampling on the elements in the database according to the first parameter to obtain the first enhancement strategy, the method further includes: Initialize the database; the database includes: a first database including multiple enhancement operations, a second database including the enhancement range of each enhancement operation, and a third database including multiple training data; the enhancement operation includes an operation range and an operation value; Among them, the first database also includes a first update parameter corresponding to each enhancement operation; the second database also includes a second update parameter corresponding to each enhancement amplitude; the third database also includes a third update parameter corresponding to each training data; the first update parameter, the second update parameter and the third update parameter include: a first parameter, a sampling parameter, a sampling parameter threshold and a difficulty threshold.

3. The method according to claim 2, characterized in that The database also includes one or more fourth databases, and the fourth database includes at least one operation object and a fourth update parameter corresponding to each operation object. The fourth update parameter includes: a first parameter, a sampling parameter, a sampling parameter threshold, and a difficulty threshold.

4. The method according to claim 2, characterized in that: The step of performing weighted sampling on the elements in the database according to the first parameter to obtain the first enhancement strategy includes: Based on a first parameter in the first update parameter, acquiring enhancement operations from the first database; the number of the enhancement operations satisfies being greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; When the number of the enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining an enhancement amplitude corresponding to each enhancement operation from the second database; A first enhancement strategy is generated according to the enhancement operation and the corresponding enhancement amplitude.

5. The method according to claim 3, characterized in that: The step of performing weighted sampling on the elements in the database according to the first parameter to obtain the first enhancement strategy includes: Based on a first parameter in the first update parameter, acquiring enhancement operations from the first database; the number of the enhancement operations satisfies being greater than or equal to 0 and less than or equal to the total number of enhancement operations in the first database; When the number of the enhancement operations is greater than 0, based on the first parameter in the second update parameter, obtaining an enhancement amplitude corresponding to each enhancement operation from the second database; Based on a first parameter in the fourth update parameter, acquiring one or more operation objects from the fourth database; A first enhancement strategy is generated according to the enhancement operation, the corresponding enhancement amplitude and the one or more operation objects.

6. The method according to claim 2, characterized in that The performing enhancement processing on the first training data based on the first enhancement strategy includes: Based on a first parameter in the third update parameter, obtaining first training data from the third database; The first training data is enhanced based on the first enhancement strategy.

7. The method according to claim 2, characterized in that The updating of the first parameter and / or the element in the database according to the second parameter comprises: Update the sampling parameter according to the second parameter and the difficulty threshold; The first parameter is updated and / or the elements in the database are updated according to the updated sampling parameter and sampling parameter threshold.

8. The method according to claim 7, characterized in that The difficulty threshold includes a first threshold and a second threshold, and the first threshold is greater than the second threshold; and updating the sampling parameter according to the second parameter and the difficulty threshold includes: When the second parameter is greater than the first threshold, updating the sampling parameter based on the sum of the sampling parameter and the first value; When the second parameter is less than the second threshold, the sampling parameter is updated based on the difference between the sampling parameter and the first value.

9. The method according to claim 7, characterized in that: The sampling parameter threshold includes a third threshold and a fourth threshold, and the third threshold is greater than the fourth threshold; and updating the first parameter and / or updating the element in the database according to the updated sampling parameter and the sampling parameter threshold includes: When the updated sampling parameter is greater than the third threshold or less than the fourth threshold, deleting the element corresponding to the updated sampling parameter in the database; When the updated sampling parameter is less than or equal to the third threshold and greater than or equal to the fourth threshold, the first parameter is updated according to the updated sampling parameter.

10. The method according to claim 9, characterized in that The updating of the first parameter according to the updated sampling parameter comprises: When the updated sampling parameter is greater than the sampling parameter before updating, increasing the first parameter; When the updated sampling parameter is less than the sampling parameter before updating, the first parameter is reduced.

11. A data enhancement processing device, characterized in that: The device comprises: a first processing unit, a second processing unit, a third processing unit and an updating unit; wherein, The first processing unit is used to perform weighted sampling on elements in the database according to a first parameter to obtain a first enhancement strategy, wherein the first parameter represents a probability of extracting an element; The second processing unit is used to perform enhancement processing on the first training data based on the first enhancement strategy to obtain second training data; The third processing unit is used to train the first model using the second training data to obtain a second parameter, where the second parameter indicates the difficulty of the second training data for model training; The updating unit is used to update the first parameter and / or update the elements in the database according to the second parameter, and obtain a second enhancement strategy based on the updated first parameter and / or the updated database to perform data enhancement processing according to the second enhancement strategy.

12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that The computer program implements the method according to any one of claims 1 to 10 when executed by a processor.

Images