Neural network model training method, application method, device and electronic device

Abstract

The embodiments of the present application provide a training method, application method, device and electronic device for a neural network model, wherein the average loss value of each data enhancement operation in the current training cycle is determined based on the loss value of each new sample after training, and then the loss measurement parameter in the set table is updated, so that the loss measurement parameter of each data enhancement operation in the set table is the loss result accumulated over multiple training cycles, and the corresponding sampling probability parameter is updated based on the loss measurement parameter accumulated over multiple training cycles in the set table, so that the updated sampling probability parameter is also related to the loss result accumulated over multiple training cycles, thereby affecting the subsequent selection of data enhancement operations based on the sampling probability parameters in the set table, making the selected data enhancement operation more reasonable, and effectively improving the training speed and training effect of the neural network model.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a training method, application method, device and electronic equipment for a neural network model. Background Art

[0002] In the task of image classification based on neural networks, data augmentation (also known as data expansion) is a widely used technique to improve classification results. Data augmentation achieves the effect of expanding the data set by changing certain attributes of the image, such as flipping the image horizontally or vertically, or changing the color of the image, etc. There are many algorithms for data augmentation, and some algorithms have certain limitations on the images they are applicable to. For example, flipping a picture of the number 6 vertically will turn it into the number 9. This data augmentation algorithm changes the nature of the image.

[0003] In order to obtain better model training results, during the data augmentation process, some search algorithms can be used to search for data augmentation strategies that can maximize the model effect on a given set of data augmentation operations (wherein a strategy refers to a collection of multiple data augmentation operations, for example, horizontal flipping and changing colors can be combined into a data augmentation strategy). During the training of the target model, a proxy model is usually used to verify the effect of the data augmentation strategy. For example, under a data augmentation strategy, the proxy model is trained and updated, and then the effect of the proxy model on the validation set is used as the score of this data augmentation strategy. Because the proxy model is different from the target model that actually needs to be trained, the training effect on the proxy model cannot directly represent the training effect on the target model; in addition, the training and updating of the proxy model is time-consuming, which also affects the training efficiency of the target model. Summary of the invention

[0004] In view of this, an object of the present invention is to provide a training method, application method, device and electronic device for a neural network model to improve the training efficiency of the neural network model.

[0005] In a first aspect, an embodiment of the present invention provides a training method for a neural network model, wherein the method is applied to an electronic device, wherein the electronic device pre-stores a set table, the set table includes multiple data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to the data enhancement operations, including: for each training cycle, a sample group for the current training cycle is selected from the training sample set; based on the sampling probability parameters in the set table, the data enhancement operations in the set table are randomly applied to enhance the samples in the sample group to obtain a new sample group, and the correspondence between the new samples in the new sample group and the data enhancement operations is recorded; the new sample group is applied to train the neural network model, and based on the loss value and the corresponding relationship of each new sample, the mean loss value of the selected data enhancement operation in the current training cycle is calculated; for each selected data enhancement operation, the loss measurement parameter corresponding to the data enhancement operation in the set table of the current training cycle is updated based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the set table of the previous training cycle; training of the next training cycle is continued until the number of training cycles in this round reaches a preset value, the sampling probability parameter in the set table is updated by applying the loss measurement parameter in the set table, and subsequent training is continued until the training is completed.

[0006] In combination with the first aspect, an embodiment of the present invention provides a first possible implementation scheme of the first aspect, wherein the initial value of the sampling probability parameter corresponding to the data enhancement operation is 1 / m, wherein m is the number of data enhancement operations included in the set table; and the initial value of the loss measurement parameter is 0.

[0007] In combination with the first aspect, an embodiment of the present invention provides a second possible implementation method of the first aspect, wherein, based on the sampling probability parameters in the set table, the step of randomly applying the data enhancement operation in the set table to enhance the samples in the sample group includes: taking the samples in the sample group as target samples respectively, and for each target sample, performing the following operations: randomly selecting a data enhancement operation from the set table based on the sampling probability parameters in the set table; and applying the selected data enhancement operation to enhance the target sample to obtain a new sample.

[0008] In combination with the second possible implementation of the first aspect, an embodiment of the present invention provides a third possible implementation of the first aspect, wherein the step of randomly selecting a data enhancement operation from a set table based on the sampling probability parameter in the set table includes: generating a random number x between 0 and 1; if x is less than the sampling probability parameter of the first data enhancement operation in the set table, selecting the first data enhancement operation; if x is greater than the sampling probability parameter of the first data enhancement operation, performing the following calculation operation: starting from the first data enhancement operation in the set table, calculating the sum of the sampling probability parameter of the current data enhancement operation and the sampling probability parameters of all data enhancement operations before the current data enhancement operation, and obtaining a sum value; if x is greater than the sum value, continuing to perform the calculation operation until x is less than or equal to the sum value, and selecting the current data enhancement operation corresponding to x being less than or equal to the sum value.

[0009] In combination with the first aspect, an embodiment of the present invention provides a fourth possible implementation of the first aspect, wherein, based on the sampling probability parameters in the set table, the step of randomly applying the data enhancement operation in the set table to enhance the samples in the sample group includes: for each sample in the sample group, the following operations are performed: based on the sampling probability parameters in the set table, two different data enhancement operations are randomly selected from the set table; and the two selected different data enhancement operations are applied to enhance the samples respectively to obtain new samples.

[0010] In combination with the first aspect, an embodiment of the present invention provides a fifth possible implementation of the first aspect, wherein, based on the loss value and the corresponding relationship of each new sample, the step of calculating the mean loss value of the selected data enhancement operation in the current training cycle includes: taking each selected data enhancement operation as a target data enhancement operation, and performing the following operations for each target data enhancement operation: determining a target new sample set corresponding to the target data enhancement operation according to the corresponding relationship; calculating the mean loss value of the new samples in the target new sample set; and using the mean as the mean loss value of the target data enhancement operation in the current training cycle.

[0011] In combination with the first aspect, an embodiment of the present invention provides a sixth possible implementation of the first aspect, wherein, based on the average loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table, the step of updating the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table includes: updating the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table by the following formula: Avg i =K1·Avg i-1 +K2·Mean; where Avg iis the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table; i is the cycle identifier; K1 and K2 are preset weight coefficients between 0 and 1, where K1+K2=1, and K1 is greater than K2; Mean is the mean loss value of the data enhancement operation in the current training cycle.

[0012] In combination with the first aspect, an embodiment of the present invention provides a seventh possible implementation of the first aspect, wherein the step of applying the loss measurement parameters in the set table to update the sampling probability parameters in the set table includes: for each data enhancement operation in the set table, substituting the loss measurement parameter corresponding to the data enhancement operation into a preset positive correlation function to obtain the sampling probability parameter corresponding to the data enhancement operation.

[0013] In a second aspect, an embodiment of the present invention further provides an application method of a neural network model, which is applied to an electronic device, and the method comprises: obtaining a target image to be processed based on a current task; wherein the current task comprises a detection task, a recognition task or a tracking task for a target object; the target object comprises one of the following: a pedestrian, a human body part, a vehicle or a license plate; the target image is input into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model; wherein the neural network model is a model trained using the above-mentioned training method.

[0014] In a third aspect, an embodiment of the present invention further provides a training device for a neural network model, wherein the device is applied to an electronic device, the electronic device pre-stores a set table, the set table includes multiple data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to the data enhancement operations, and the device includes: a selection module, for selecting a sample group for the current training cycle from the training sample set for each training cycle; an enhancement module, for randomly applying the data enhancement operations in the set table to enhance the samples in the sample group based on the sampling probability parameters in the set table to obtain a new sample group, and recording the correspondence between the new samples in the new sample group and the data enhancement operations; a calculation module, for applying the new The sample group trains the neural network model, and based on the loss value and corresponding relationship of each new sample, calculates the mean loss value of the selected data enhancement operation in the current training cycle; the update module is used to update the loss measurement parameters corresponding to the data enhancement operation in the current training cycle set table for each selected data enhancement operation based on the mean loss value of the data enhancement operation and the loss measurement parameters corresponding to the data enhancement operation in the previous training cycle set table; the training module is used to continue the training of the next training cycle until the number of training cycles in this round reaches a preset value, and the loss measurement parameters in the set table are applied to update the sampling probability parameters in the set table, and continue subsequent training until the training is completed.

[0015] In a fourth aspect, an embodiment of the present invention further provides an application device of a neural network model, which is applied to an electronic device, and the device comprises: a target image acquisition module, used to acquire a target image to be processed based on a current task; wherein the current task comprises a detection task, a recognition task or a tracking task for a target object; the target object comprises one of the following: a pedestrian, a body part, a vehicle or a license plate; a model processing module, used to input the target image into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model; wherein the neural network model is a model trained using the above-mentioned training method.

[0016] In a fifth aspect, an embodiment of the present invention further provides an electronic device, wherein the electronic device includes: an image acquisition device, a processing device and a storage device; the image acquisition device is used to obtain an image to be detected; a computer program is stored on the storage device, and the computer program executes the training method or application method of the neural network model as described above when the processed device is running.

[0017] In a sixth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, wherein the computer program executes the steps of the training method or application method of the neural network model as described above when the computer program is executed by a processing device.

[0018] The embodiments of the present invention bring the following beneficial effects:

[0019] The embodiments of the present application provide a training method, application method, device and electronic device for a neural network model, wherein the average loss value of each data enhancement operation in the current training cycle is determined based on the loss value of each new sample after training, and then the loss measurement parameter in the set table is updated, so that the loss measurement parameter of each data enhancement operation in the set table is the loss result accumulated over multiple training cycles, and the corresponding sampling probability parameter is updated based on the loss measurement parameter accumulated over multiple training cycles in the set table, so that the updated sampling probability parameter is also related to the loss result accumulated over multiple training cycles, thereby affecting the subsequent selection of data enhancement operations based on the sampling probability parameters in the set table, making the selected data enhancement operation more reasonable, effectively improving the training speed and training effect of the neural network model, and thus ensuring the accuracy of the model during use.

[0020] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by implementing the above-mentioned technology of the present disclosure.

[0021] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0024] Figure 2 A flowchart of a training method for a neural network model provided by an embodiment of the present invention;

[0025] Figure 3 A flowchart of another neural network model training method provided by an embodiment of the present invention;

[0026] Figure 4 A flowchart of another neural network model training method provided by an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the structure of a training device for a neural network model provided by an embodiment of the present invention;

[0028] Figure 6 A flowchart of a method for applying a neural network model provided by an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of the structure of an application device of a neural network model provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the embodiment of the present invention, based on different computer vision tasks, a neural network model suitable for the task and a series of data enhancement operations (also called data augmentation operations) suitable for the task are selected, and these data enhancement operations can form a set or be stored in a list. The data enhancement operations in the embodiment of the present invention include but are not limited to: horizontal flipping, vertical flipping, rotation, adjusting image contrast, inverting color conversion, histogram equalization, tone separation, adjusting image saturation, adjusting image brightness, adjusting image sharpness, random occlusion of images, etc.

[0032] During use, one or more data enhancement operations can be selected from the above set or list, and the selected data enhancement operations can be combined into a data enhancement strategy. Different data enhancement strategies can be applied to training samples (for example, image samples), which can transform more training samples, thereby achieving the purpose of expanding the training samples, and training the neural network model based on these training samples. In order to improve the training speed and training effect of the model, a training method, application method, device and electronic device for a neural network model provided by an embodiment of the present invention determines the sampling probability parameters of the data enhancement operation by combining the loss measurement parameters accumulated on multiple samples by the data enhancement operation, and selects the data enhancement operation based on the sampling probability parameters corresponding to each data enhancement operation, so that the selected data enhancement operation can be made more reasonable, thereby effectively improving the training speed and training effect of the neural network model, and thereby improving the reliability of the model application. The following is described through embodiments.

[0033] The neural network model in the embodiment of the present invention can be applied to various application scenarios such as target detection and target recognition, for example, applying the neural network model to identify pedestrians or vehicles, applying the neural network model to track pedestrians or vehicles, applying the neural network model to identify human body parts or vehicle parts (such as license plates or vehicle logos), etc. The application scenarios of the neural network model are not limited here.

[0034] like Figure 1 The electronic device 100 includes one or more processing devices 102, one or more storage devices 104, an input device 106, an output device 108, and one or more image acquisition devices 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that Figure 1 The components and structures of the electronic device 100 shown are merely exemplary and non-limiting. The electronic device may also have other components and structures as required.

[0035] The processing device 102 can be a server, an intelligent terminal, or a device including a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities. It can process data of other components in the electronic device 100 and can also control other components in the electronic device 100 to perform neural network model training or application functions.

[0036] The storage device 104 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processing device 102 may run the program instructions to implement the client functions and / or other desired functions in the embodiments of the present invention (implemented by the processing device) described below. Various applications and various data, such as various data used and / or generated by the application, may also be stored in the computer-readable storage medium.

[0037] The input device 106 may be a device used by a user to input instructions, and may include one or more of a keyboard, a mouse, a microphone, a touch screen, and the like.

[0038] The output device 108 may output various information (eg, images or sounds) to the outside (eg, a user), and may include one or more of a display, a speaker, and the like.

[0039] The image acquisition device 110 may acquire training samples or target images, and store the acquired training samples or target images in the storage device 104 for use by other components.

[0040] Exemplarily, the components in the training method, application method, device and electronic device for implementing the neural network model according to the embodiment of the present invention can be integrated or dispersed, such as integrating the processing device 102, storage device 104, input device 106 and output device 108 into one, and setting the image acquisition device 110 at a designated position where the training sample or target image can be acquired. When the components in the above electronic device are integrated, the electronic device can be implemented as an intelligent terminal such as a camera, a smart phone, a tablet computer, a computer, a vehicle-mounted terminal, etc.

[0041] Based on the above electronic device, this embodiment provides a training method for a neural network model, which is applied to the above electronic device, and the electronic device pre-stores a set table, which includes multiple data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to each data enhancement operation. Among them, the sampling probability parameter is used to indicate the probability of the data enhancement operation being selected, and the loss measurement parameter is used to characterize the loss effect of the data enhancement operation in the neural network model training process. For ease of explanation, Table 1 shows a set table:

[0042] Table 1

[0043] Data augmentation operations Sampling probability parameters Loss measurement parameters <![CDATA[Data augmentation operation o1]]> A1 B1 <![CDATA[Data augmentation operation o2]]> A2 B2 <![CDATA[Data augmentation operation o3]]> A3 B3 … … … <![CDATA[Data augmentation operation o m > Am Bm

[0044] Table 1 shows m different data enhancement operations, each of which corresponds to a sampling probability parameter and a loss measurement parameter. In order to ensure the randomness when the enhancement operation is selected for the first time, in this embodiment, the initial value of the sampling probability parameter corresponding to each data enhancement operation is set to 1 / m, where the values ​​of the sampling probability parameters of the m data enhancement operations add up to 1; in this embodiment, the initial value of the loss measurement parameter corresponding to each data enhancement operation is set to 0.

[0045] Based on the above description, see Figure 2 A flowchart of a training method for a neural network model is shown, and the method specifically comprises the following steps:

[0046] Step S202, for each training cycle, a sample group of the current training cycle is selected from the training sample set;

[0047] The samples in the training sample set may be pre-collected by the electronic device, or obtained by the electronic device from a third-party device. During each training, the samples in the entire training sample set may be used as the sample group of this training cycle, or some samples in the training sample set may be randomly selected as the sample group of this training cycle, wherein the number of samples in the sample group of the current training cycle may be selected according to actual needs and is not limited here.

[0048] Step S204, based on the sampling probability parameters in the set table, randomly apply the data enhancement operation in the set table to enhance the samples in the sample group to obtain a new sample group, and record the corresponding relationship between the new samples in the new sample group and the data enhancement operation;

[0049] Among them, the sampling probability parameters in the set table of this embodiment can be updated at regular intervals or a certain number of training times, and the specific updating method of the sampling probability parameters can be based on the loss measurement parameters.

[0050] In this embodiment, for training samples before the sampling probability parameters are updated, since the sampling probability parameters corresponding to each data enhancement operation in the set table are all initial values, that is, the sampling probability parameters corresponding to each data enhancement operation are equal at this time, for these samples, the random application of the data enhancement operation in the set table to enhance the sample means that any one or more data enhancement operations can be selected from the set table to apply to the sample, and the selection is random. For example, there are 4 data enhancement operations in the set table, and the initial values ​​of their sampling probability parameters are all 0.25 (i.e. 1 / 4). The set table is shown in Table 2:

[0051] Table 2

[0052] Data augmentation operations Sampling probability parameters Loss measurement parameters Data augmentation operation 1 0.25 0 Data augmentation operation 2 0.25 0 Data enhancement operation 3 0.25 0 Data enhancement operation 4 0.25 0

[0053] For the first sample, in order to ensure the randomness of the selection, a random number between 0 and 1 is generated. Assume that the random number is 0.3, and 0.3 is between 0.25 and 0.5, so the data enhancement operation 2 is selected for the first sample. For the second sample, assume that the random number generated between 0 and 1 is 0.8, and 0.8 is between 0.75 and 1, so the data enhancement operation 4 is selected for the second sample.

[0054] For the training samples after the sampling probability parameters have been updated, since the sampling probability parameters corresponding to each data enhancement operation in the set table are updated values, that is, the sampling probability parameters corresponding to each data enhancement operation may not be equal at this time, based on the sampling probability parameters in the set table, the randomness of the machine applying the data enhancement operation in the set table to enhance the samples in the sample group will be affected by the sampling probability parameters, which will result in different probabilities of different data enhancement operations being selected.

[0055] When selecting a data enhancement operation, at least one data enhancement operation is selected for each sample in the above sample group. In this embodiment, the number of data enhancement operations selected for each sample in the sample group is not limited. When a sample selects multiple data enhancement operations, in order to avoid selecting the same data enhancement operation, the data enhancement operation can be marked when selecting the data enhancement operation or taken out from the set table. For example: for a sample, when the second data enhancement operation is selected from the set table, the second data enhancement operation in the set table can be set to gray or other colors to indicate that the second data enhancement operation is currently not selectable. In this way, when the next data enhancement operation is selected for the sample, the second data enhancement operation will not be selected. When selecting a data enhancement operation for the next sample, the data enhancement operations in the set table are restored to the default color to select a data enhancement operation for the next sample.

[0056] In actual use, in order to evaluate the training effect of the data augmentation operation, after performing the data augmentation operation on the sample, the correspondence between the new sample in the new sample group and the data augmentation operation can be recorded, such as the correspondence example of one sample corresponding to one data augmentation operation shown in Table 3.

[0057] Table 3

[0058] Sample identification Data augmentation operation identifier New Sample 1 Data augmentation operation 1 New Sample 2 Data augmentation operation 2 New Sample 3 Data enhancement operation 3 New Sample 4 Data enhancement operation 4

[0059] Table 4 shows an example of the correspondence between one sample and two data enhancement operations.

[0060] Table 4

[0061]

[0062] The above correspondence can clearly show which data enhancement operations are used to obtain each new sample. In addition to the forms other than those in Table 3 and Table 4, if two different data enhancement operations can be performed on the original sample in sequence, the correspondence can record the order in which the data enhancement operations are used. For example, if the correspondence is (new sample 1, o1, o3, o4), it means that the new sample 1 is obtained by performing data enhancement operations o1, data enhancement operations o3, and data enhancement operations o4 on the original sample 1 in sequence; if the correspondence is (new sample 3, o6, o3), it means that the new sample 3 is obtained by performing data enhancement operations o6 and data enhancement operations o3 on the original sample 3 in sequence.

[0063] Step S206, applying the new sample group to train the neural network model, and calculating the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value of each new sample and the above corresponding relationship;

[0064] Input each new sample in the new sample group into the neural network model to obtain the loss value of each new sample. The loss value can be a cross entropy loss or a mean square error, which is not limited here.

[0065] Based on the above correspondence between new samples and data augmentation operations, find out which new samples correspond to each selected data augmentation operation, and then perform mean calculation on the loss values ​​corresponding to these new samples to obtain the mean loss value of each data augmentation operation in the current training cycle. For example, the new samples corresponding to the above data augmentation operation o3 are new sample 1 and new sample 3, where the mean square error of new sample 1 is 0.01 and the mean square error of new sample 3 is 0.03, then the mean loss value of data augmentation operation o3 in the current training cycle is 0.02.

[0066] Step S208, for each selected data enhancement operation, based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table, update the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table;

[0067] In this embodiment, the loss measurement parameters in the set table can be updated once each time a cycle of sample training is completed. Each selected data enhancement operation can calculate the loss measurement parameters corresponding to the data enhancement operation of the current training cycle based on the mean loss value obtained in the current training cycle and the loss measurement parameters obtained in the previous training cycle. The calculation process can be implemented using a variety of functions. These functions only need to reflect that the loss measurement parameters corresponding to the data enhancement operation characterize the magnitude of the cumulative loss impact corresponding to the data enhancement operation. After obtaining the loss measurement parameters corresponding to the data enhancement operation of the current training cycle, the loss measurement parameters corresponding to the data enhancement operation of the previous training cycle in the set table are used to update the loss measurement parameters corresponding to the data enhancement operation obtained in the previous training cycle.

[0068] Step S210, continue to perform training for the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue subsequent training until the training is completed.

[0069] In actual application, the process from step S202 to step S208 can be executed repeatedly until the number of cycles reaches a preset value, and the sampling probability parameters in the set table are updated using the loss measurement parameters in the set table, that is, the update frequency of the sampling probability parameters in the set table is lower than the update frequency of the loss measurement parameters in the set table. For example, the loss measurement parameters in the set table are updated once in each training cycle, while the sampling probability parameters in the set table can be updated once every 10 or 50 training cycles.

[0070] In the process of training the neural network model, a data enhancement operation can be selected based on the sampling probability parameter in the updated set table. The sampling probability parameter is updated by applying the loss measurement parameter in the set table. In order to make the selected data enhancement operation more reasonable, the loss measurement parameter of this embodiment is positively correlated with the sampling probability parameter, that is, the larger the loss measurement parameter, the larger the sampling probability parameter. The larger the loss measurement parameter, the poorer the training effect of the neural network model under this data enhancement operation, that is, the neural network model needs to be trained under this data enhancement operation to achieve better results.

[0071] The training method of the above-mentioned neural network model provided in the embodiment of the present application determines the mean loss value of each data enhancement operation in the current training cycle based on the loss value of each new sample after training, and then updates the loss measurement parameters in the set table, so that the loss measurement parameters of each data enhancement operation in the set table are the loss results accumulated over multiple training cycles, and the corresponding sampling probability parameters are updated based on the loss measurement parameters accumulated over multiple training cycles in the set table, so that the updated sampling probability parameters are also related to the loss results accumulated over multiple training cycles, thereby affecting the subsequent selection of data enhancement operations based on the sampling probability parameters in the set table, making the selected data enhancement operations more reasonable, and effectively improving the training speed and training effect of the neural network model.

[0072] This embodiment provides another training method for a neural network model, which is implemented on the basis of the above embodiment; this embodiment focuses on describing a specific implementation method for enhancing samples in a sample group. Figure 3 FIG. 1 is a flowchart of another neural network model training method shown in FIG. 1 . The network model training method in this embodiment includes the following steps:

[0073] Step S302, for each training cycle, a sample group of the current training cycle is selected from the training sample set;

[0074] Step S304: taking samples in the sample group as target samples respectively, and performing the following operations S306 to S308 for each target sample:

[0075] Step S306, randomly selecting a data enhancement operation from the set table based on the sampling probability parameter in the set table;

[0076] Wherein, step S306 can be specifically implemented by steps A1 to A3:

[0077] Step A1, generate a random number x between 0 and 1;

[0078] In actual use, a random number x between 0 and 1 may be generated by a random generator in each training cycle for each target sample in the sample group.

[0079] Step A2: if x is less than the sampling probability parameter of the first data enhancement operation in the set table, select the first data enhancement operation;

[0080] Continuing with Table 1 as an example, the first data augmentation operation in Table 1 is data augmentation operation o1. If the random number x corresponding to the target sample is less than the sampling probability parameter A1 of the data augmentation operation o1, the data augmentation operation o1 is selected for the target sample.

[0081] Step A3: If x is greater than the sampling probability parameter of the first data enhancement operation, perform the following calculation operations: (1) starting from the first data enhancement operation in the set table, calculate the sum of the sampling probability parameter of the current data enhancement operation and the sampling probability parameters of all data enhancement operations before the current data enhancement operation to obtain a sum value;

[0082] From data enhancement operation o1 to data enhancement operation o in Table 1 m Taking the order as an example, if x is greater than the sampling probability parameter of the first data augmentation operation, start from the data augmentation operation o1, first use the data augmentation operation o2 as the current data augmentation operation, calculate A1+A2, and get the sum.

[0083] (2) If x is greater than the sum, continue to perform the calculation operation of (1) above until x is less than the calculated sum, and select the current data enhancement operation corresponding to x being less than or equal to the above sum.

[0084] If the above x is less than or equal to the sum of A1+A2, the data augmentation operation o2 is selected for the target sample.

[0085] If the above x is greater than the sum of A1+A2, then continue to execute the above (1), take the data augmentation operation o3 as the current data augmentation operation, calculate the sum of A1+A2+A3, if x is less than or equal to the sum of A1+A2+A3, then select the data augmentation operation o3 for the target sample. Otherwise, the same process is repeated until x is less than the calculated sum, which will not be described in detail here.

[0086] This method of selecting data augmentation operations based on random numbers can ensure the randomness of the selection of data augmentation operations, avoid human subjective factors interfering with the selection of data augmentation operations, and thus ensure the diversity of new samples, which is conducive to model training.

[0087] Step S308, applying the selected data enhancement operation to enhance the target sample to obtain a new sample, and recording the corresponding relationship between the new sample in the new sample group and the data enhancement operation;

[0088] If the number of data enhancement operations selected by the above step S306 is one, the target sample is enhanced by using the data enhancement operation; if the number of data enhancement operations selected is multiple, the target samples can be enhanced in sequence according to a preset enhancement order, or the target samples can be enhanced separately by applying multiple data enhancement operations, which is not limited here.

[0089] In this embodiment, in addition to using the random number x method to perform data enhancement operations to select target samples to enhance new samples, for each sample in the sample group, two different data enhancement operations can be randomly selected from the set table based on the sampling probability parameters in the set table; the two selected different data enhancement operations are applied to enhance the samples respectively to obtain new samples.

[0090] For each sample in the sample group, two different data enhancement operations can be randomly selected from the set table to enhance the sample. Specifically, the sample can be enhanced according to the order of selecting the data enhancement operations to obtain a new sample; or, two data enhancement operations can be applied to enhance the sample separately to obtain two new samples, which is not limited here.

[0091] Step S310, applying the new sample group to train the neural network model, and calculating the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value and corresponding relationship of each new sample;

[0092] Step S312, for each selected data enhancement operation, based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table, update the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table;

[0093] Step S314, continue to perform training for the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue subsequent training until the training is completed.

[0094] The training method of the above-mentioned neural network model provided by the embodiment of the present invention can use the size relationship between the random number x and the sampling probability parameter of the data enhancement operation to select the data enhancement operation of the sample, so as to perform the enhancement operation on the sample to obtain a new sample, and then determine the mean loss value of each data enhancement operation in the current training cycle through the loss value of the new sample, update the loss measurement parameter in the set table, and on this basis, update the corresponding sampling probability parameter according to the loss measurement parameter accumulated from multiple training cycles, so that the data enhancement operation selected based on the sampling probability parameter in the set table is more reasonable, which effectively improves the training speed and training effect of the neural network model.

[0095] This embodiment provides another training method for a neural network model, which is implemented on the basis of the above embodiment; this embodiment focuses on describing a specific implementation method for calculating the mean loss value of the current training cycle. Figure 4 FIG. 1 is a flowchart of another neural network model training method shown in FIG. 1 . The network model training method in this embodiment includes the following steps:

[0096] Step S402, for each training cycle, a sample group of the current training cycle is selected from the training sample set;

[0097] Step S404, based on the sampling probability parameters in the set table, randomly apply the data enhancement operation in the set table to enhance the samples in the sample group to obtain a new sample group, and record the corresponding relationship between the new samples in the new sample group and the data enhancement operation;

[0098] Step S406, using the new sample group to train the neural network model, taking each selected data enhancement operation as a target data enhancement operation, and performing the operations of steps S408 to S410 for each target data enhancement operation:

[0099] Step S408, determining a target new sample set corresponding to the target data enhancement operation according to the above correspondence;

[0100] In this embodiment, each new sample in the new sample group is obtained by enhancing the sample using the data enhancement operation selected in step S404. Therefore, the correspondence between the new sample and the data enhancement operation is recorded in the correspondence. Based on the correspondence, the target new samples corresponding to each target data enhancement operation are found out. For example, the target new samples corresponding to the target data enhancement operation o3 in the above embodiment are new sample 1 and new sample 3, the target new sample corresponding to the target data enhancement operation o1 is new sample 1, the target new sample corresponding to the target data enhancement operation o4 is new sample 1, and the target new sample corresponding to the target data enhancement operation o6 is new sample 3.

[0101] Step S410, calculating the mean of the loss values ​​of the new samples in the target new sample set, and using the mean as the mean of the loss values ​​of the target data enhancement operation in the current training cycle;

[0102] The loss value of the above new sample is obtained by inputting the new sample into the neural network model. In this embodiment, the new sample loss values ​​in the target new sample set corresponding to each target data enhancement operation are summed and averaged to obtain the average loss value of the target data enhancement operation in the current training cycle. For example, for the target data enhancement operation o3, since the mean square error of the new sample 1 is 0.01 and the mean square error of the new sample 3 is 0.03, the average loss value of the target data enhancement operation o3 in the current training cycle is 0.02; and the average loss value of the target data enhancement operation o1 in the current training cycle is 0.01.

[0103] Step S412, for each selected data enhancement operation, based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table, update the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table;

[0104] In specific implementation, the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table is updated through the following formula: Avg i =K1·Avg i-1 +K2·Mean; where Avg i is the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table; i is the cycle identifier; K1 and K2 are preset weight coefficients between 0 and 1, where K1+K2=1, and K1 is greater than K2; Mean is the mean loss value of the data enhancement operation in the current training cycle.

[0105] Among them, Avg i-1 is the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table, using Avg with weight coefficient K1 i-1 With Avg i-1 The sum of the Mean values ​​of the data enhancement operations in the current training cycle set table is updated with the loss measurement parameter Avg corresponding to the data enhancement operation. i .

[0106] The above formula Avg i =K1·Avg i-1 +K2·Mean is only used as an implementation method. In practical applications, there are many different ways to calculate Avg. i , this embodiment of the present invention does not limit this.

[0107] Step S414, continue to perform training for the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue subsequent training until the training is completed.

[0108] For each data enhancement operation in the set table, the loss measurement parameter corresponding to the data enhancement operation is substituted into a preset positive correlation function to obtain a sampling probability parameter corresponding to the data enhancement operation.

[0109] For example, the sampling probability parameters in the set table can be updated by the following positive correlation function: Among them, p i represents the sampling probability parameter of the ith data augmentation operation. In actual use, the positive correlation function can be determined according to actual needs and is not limited here.

[0110] The training method of the above-mentioned neural network model provided by the embodiment of the present invention can enhance the samples in the sample group based on the data enhancement operation in the set table. After obtaining the new sample group, the target new sample set corresponding to the target data enhancement operation is determined according to the corresponding relationship, and the mean of the loss values ​​of the new samples in the calculated target new sample set is used as the mean of the loss values ​​of the target data enhancement operation in the current training cycle. The loss measurement parameters in the set table are updated according to the mean of the loss values, and then the sampling probability parameters of each data enhancement operation in the set table are determined. The data enhancement operation is selected based on the updated sampling probability parameters in the set table, so that the selected data enhancement operation is more reasonable, and the training efficiency of the neural network model is effectively improved.

[0111] Based on the above embodiment, the training process of the face recognition neural network model is described below by taking face recognition as an example:

[0112] Step 1: Initialize a face recognition neural network model and construct an operation list containing operations suitable for face recognition enhancement. i ,i=1,....,m}, the elements in the list are pre-defined data augmentation operations, their sampling probabilities and the loss measurement parameters of the training loss corresponding to this data augmentation operation (also known as: training loss moving average). Assuming there are m data augmentation operations in total, initialize the sampling probability corresponding to each data augmentation operation And initialize the moving average Avg of the training loss corresponding to each data enhancement operation i =0, i=1, ..., m. The present embodiment introduces the moving average of the training loss to reduce the impact of randomness to a certain extent. Secondly, the moving average of the training loss is mainly used as the basis for updating the sampling probability later.

[0113] Step 2: In each model training update, a batch of samples (the number is assumed to be n) is sampled from the face sample set {x j ,j=1,....,n}. Perform the following operations on these n samples respectively: according to the sampling probability, perform two data enhancement operations without replacement from the operation list, and then apply these two data enhancement operations to this sample successively, and record the data enhancement operation applied to this sample as g(x j )={o t ,o k |t,k∈{1,....,m}}.

[0114] Step 3: Input a batch of samples obtained after data enhancement in step 2 into the face recognition neural network model for training, and record the training loss (either cross entropy loss or mean square error) generated by each sample as l j,j=1,....,n. Then update the moving average of the training loss corresponding to the data enhancement operation in the operation list as follows:

[0115] Avg i =0.9Avg i-1 +0.1Mean;

[0116] The actual meaning of this formula is to update the losses of all samples in this batch that use the i-th data augmentation operation to the moving average of the training loss of the i-th data augmentation operation. The Mean here refers to the mean of the losses caused by all samples that use the i-th data augmentation operation. If for the first data augmentation operation, 10 samples in this batch use this operation, then the loss values ​​corresponding to these 10 samples are averaged.

[0117] Step 4: After the moving average of the training loss has been updated a certain number of times (for example, 1000 times), the sampling probability p corresponding to the data enhancement operation can be updated i , as follows:

[0118]

[0119] Repeat steps 2, 3, and 4 until the training is completed.

[0120] Through the above training method of the face recognition neural network model, the training loss of the model training process can be directly applied to judge the effect of the data enhancement operation, and the loss measurement parameters in the set table can be updated based on the training loss, and then the sampling probability parameters of each data enhancement operation in the set table can be determined. The data enhancement operation is selected based on the updated sampling probability parameters in the set table, so that the selected data enhancement operation is more reasonable, and the training efficiency of the face recognition neural network model is effectively improved.

[0121] Corresponding to the above-mentioned training method embodiment of the neural network model, an embodiment of the present invention provides a training device for a neural network model, wherein the above-mentioned device is applied to an electronic device, and the electronic device pre-stores a set table, the set table includes multiple data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to the data enhancement operations, Figure 5 A schematic diagram of the structure of a training device for a neural network model is shown. Figure 5 As shown, the device comprises:

[0122] A selection module 502 is used to select a sample group of the current training cycle from the training sample set for each training cycle;

[0123] An enhancement module 504 is used to randomly apply the data enhancement operation in the set table to enhance the samples in the sample group based on the sampling probability parameters in the set table to obtain a new sample group, and record the corresponding relationship between the new samples in the new sample group and the data enhancement operation;

[0124] A calculation module 506 is used to apply the new sample group to train the neural network model, and calculate the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value and the corresponding relationship of each new sample;

[0125] An updating module 508 is used to update the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table for each selected data enhancement operation based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the previous training cycle set table;

[0126] The training module 510 is used to continue the training of the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue the subsequent training until the training is completed.

[0127] An embodiment of the present application provides a training device for a neural network model, wherein the mean loss value of each data enhancement operation in the current training cycle is determined based on the loss value of each new sample after training, and then the loss measurement parameter in the set table is updated, so that the loss measurement parameter of each data enhancement operation in the set table is the loss result accumulated over multiple training cycles, and the corresponding sampling probability parameter is updated based on the loss measurement parameter accumulated over multiple training cycles in the set table, so that the updated sampling probability parameter is also related to the loss result accumulated over multiple training cycles, thereby affecting the subsequent selection of the data enhancement operation based on the sampling probability parameter in the set table, making the selected data enhancement operation more reasonable, and effectively improving the training speed and training effect of the neural network model.

[0128] Among them, the initial value of the sampling probability parameter corresponding to the data enhancement operation is 1 / m, where m is the number of data enhancement operations included in the set table; the initial value of the loss measurement parameter is 0.

[0129] The enhancement module 504 is also used to respectively use samples in the sample group as target samples, and for each target sample, perform the following operations: randomly select a data enhancement operation from the set table based on the sampling probability parameters in the set table; and enhance the target sample by applying the selected data enhancement operation to obtain a new sample.

[0130] The enhancement module 504 is also used to generate a random number x between 0 and 1; if x is less than the sampling probability parameter of the first data enhancement operation in the set table, select the first data enhancement operation; if x is greater than the sampling probability parameter of the first data enhancement operation, perform the following calculation operation: starting from the first data enhancement operation in the set table, calculate the sum of the sampling probability parameter of the current data enhancement operation and the sampling probability parameters of all data enhancement operations before the current data enhancement operation to obtain the sum value; if x is greater than the sum value, continue to perform the calculation operation until x is less than or equal to the sum value, and select the current data enhancement operation corresponding to x being less than or equal to the sum value.

[0131] The enhancement module 504 is also used to perform the following operations for each sample in the sample group: based on the sampling probability parameters in the set table, randomly select two different data enhancement operations from the set table; and enhance the samples respectively by applying the two selected different data enhancement operations to obtain new samples.

[0132] The above-mentioned calculation module 506 is also used to take each selected data enhancement operation as a target data enhancement operation, and perform the following operations for each target data enhancement operation: determine the target new sample set corresponding to the target data enhancement operation according to the corresponding relationship; calculate the mean of the loss values ​​of the new samples in the target new sample set; and use the mean as the mean of the loss values ​​of the target data enhancement operation in the current training cycle.

[0133] The updating module 508 is also used to update the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table through the following formula: Avg i =K1·Avg i-1 +K2·Mean; where Avg i is the loss measurement parameter corresponding to the data enhancement operation in the current training cycle set table; i is the cycle identifier; K1 and K2 are preset weight coefficients between 0 and 1, where K1+K2=1, and K1 is greater than K2; Mean is the mean loss value of the data enhancement operation in the current training cycle.

[0134] The training module 510 is also used to substitute the loss measurement parameter corresponding to each data enhancement operation in the set table into a preset positive correlation function to obtain a sampling probability parameter corresponding to the data enhancement operation.

[0135] The training device for the neural network model provided in the embodiment of the present invention has the same technical features as the training method for the neural network model provided in the above embodiment, so it can also solve the same technical problems and achieve the same technical effects.

[0136] Corresponding to the training method embodiment of the neural network model, the embodiment of the present invention provides an application method of the neural network model, which is applied to the above electronic device, see Figure 6 A flowchart of an application method of a neural network model is shown, and the method specifically comprises the following steps:

[0137] Step S602, obtaining a target image to be processed based on a current task; wherein the current task includes a detection task, a recognition task or a tracking task for a target object; and the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate;

[0138] The target image may be pre-collected by the electronic device, or may be an image containing the target object obtained by the electronic device from a third-party device.

[0139] Step S604, inputting the target image into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model;

[0140] The neural network model is a model trained by applying the training method of the neural network model. The training method effectively improves the training speed and training effect of the neural network model, thereby ensuring the accuracy of the target image prediction during the use of the model.

[0141] Corresponding to the above-mentioned embodiment of the application method of the neural network model, the embodiment of the present invention further provides an application device of the neural network model, and the above-mentioned device is applied to electronic equipment. Figure 7 A schematic diagram of the structure of an application device of a neural network model is shown. Figure 7 As shown, the device comprises:

[0142] The target image acquisition module 702 is used to acquire a target image to be processed based on a current task; wherein the current task includes a detection task, a recognition task or a tracking task for a target object; and the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate;

[0143] The model processing module 704 is used to input the target image into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model; wherein the neural network model is a model trained using the above-mentioned training method.

[0144] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processing device, the computer program executes the above-mentioned network model training method or the above-mentioned neural network model application method, or executes the steps in the above-mentioned method.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the electronic equipment and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0146] The embodiments of the present invention provide a computer program product for a training method, an application method, an apparatus, and an electronic device for a neural network model, including a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the previous method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0147] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0149] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0150] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0151] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A training method for a neural network model, characterized in that: The method is applied to an electronic device, wherein the electronic device pre-stores a set table, wherein the set table includes a plurality of data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to the data enhancement operations, wherein the sampling probability parameters are used to indicate the probability of the data enhancement operation being selected, and the loss measurement parameters are used to characterize the loss impact of the data enhancement operation in the training process of a neural network model; include: For each training cycle, a sample group of the current training cycle is selected from a training sample set; the samples in the training sample set are image samples of a target object; the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate; Based on the sampling probability parameters in the set table, randomly apply the data enhancement operation in the set table to enhance the samples in the sample group to obtain a new sample group, and record the corresponding relationship between the new samples in the new sample group and the data enhancement operation; Apply the new sample group to train the neural network model, and calculate the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value of each new sample and the corresponding relationship; For each selected data enhancement operation, based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the set table in the previous training cycle, the loss measurement parameter corresponding to the data enhancement operation in the set table in the current training cycle is updated; Continue to perform training for the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue subsequent training until the training is completed.

2. The method according to claim 1, characterized in that The initial value of the sampling probability parameter corresponding to the data enhancement operation is 1 / m, where m is the number of data enhancement operations included in the set table; the initial value of the loss measurement parameter is 0.

3. The method according to claim 1, characterized in that The step of randomly applying the data enhancement operation in the set table to enhance the samples in the sample group based on the sampling probability parameters in the set table comprises: The samples in the sample group are respectively used as target samples, and for each of the target samples, the following operations are performed: Randomly select a data augmentation operation from the set table based on a sampling probability parameter in the set table; The selected data enhancement operation is applied to enhance the target sample to obtain a new sample.

4. The method according to claim 3, characterized in that The step of randomly selecting a data enhancement operation from the set table based on the sampling probability parameter in the set table comprises: Generate a random number x between 0 and 1; If x is less than the sampling probability parameter of the first data enhancement operation in the set table, select the first data enhancement operation; If x is greater than the sampling probability parameter of the first data augmentation operation, the following calculation operation is performed: Starting from the first data enhancement operation in the set table, calculating the sum of the sampling probability parameter of the current data enhancement operation and the sampling probability parameters of all data enhancement operations before the current data enhancement operation, and obtaining a sum value; If x is greater than the sum value, continue to perform the calculation operation until x is less than or equal to the sum value, and select the current data enhancement operation corresponding to x being less than or equal to the sum value.

5. The method according to claim 1, characterized in that The step of randomly applying the data enhancement operation in the set table to enhance the samples in the sample group based on the sampling probability parameters in the set table comprises: For each sample in the sample group, the following operations are performed: Based on the sampling probability parameters in the set table, randomly select two different data enhancement operations from the set table; The two different selected data enhancement operations are applied to enhance the samples respectively to obtain new samples.

6. The method according to claim 1, characterized in that The step of calculating the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value of each of the new samples and the corresponding relationship comprises: Each selected data enhancement operation is used as a target data enhancement operation, and the following operations are performed for each target data enhancement operation: Determine a target new sample set corresponding to the target data enhancement operation according to the corresponding relationship; Calculating the mean of the loss values ​​of the new samples in the target new sample set; The mean is used as the mean loss value of the target data enhancement operation in the current training cycle.

7. The method according to claim 1, characterized in that The step of updating the loss measurement parameter corresponding to the data enhancement operation in the set table of the current training cycle based on the loss value mean of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the set table of the previous training cycle includes: The loss measurement parameter corresponding to the data enhancement operation in the set table of the current training cycle is updated by the following formula: ； Among them, Avg i is the loss measurement parameter corresponding to the data enhancement operation in the set table of the current training cycle; i is the cycle identifier; K1 and K2 are preset weight coefficients between 0 and 1, where K1+K2=1, and K1 is greater than K2; Mean is the mean loss value of the data enhancement operation in the current training cycle.

8. The method according to claim 1, characterized in that The step of applying the loss measurement parameters in the set table to update the sampling probability parameters in the set table includes: For each data enhancement operation in the set table, the loss measurement parameter corresponding to the data enhancement operation is substituted into a preset positive correlation function to obtain a sampling probability parameter corresponding to the data enhancement operation.

9. A method for applying a neural network model, characterized in that: The method is applied to an electronic device, and the method comprises: Acquire a target image to be processed based on a current task; wherein the current task includes a detection task, a recognition task or a tracking task for a target object; the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate; The target image is input into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model; wherein the neural network model is a model trained by applying the training method described in any one of claims 1 to 8.

10. A training device for a neural network model, characterized in that: The device is applied to an electronic device, wherein the electronic device pre-stores a set table, wherein the set table includes multiple data enhancement operations, and sampling probability parameters and loss measurement parameters corresponding to the data enhancement operations, wherein the sampling probability parameters are used to indicate the probability of the data enhancement operation being selected, and the loss measurement parameters are used to characterize the loss impact of the data enhancement operation in the neural network model training process; the device includes: A selection module is used to select a sample group of the current training cycle from a training sample set for each training cycle; the samples in the training sample set are image samples of a target object; the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate; an enhancement module, configured to randomly apply the data enhancement operation in the set table to enhance the samples in the sample group based on the sampling probability parameters in the set table, obtain a new sample group, and record the corresponding relationship between the new samples in the new sample group and the data enhancement operation; A calculation module, used to apply the new sample group to train the neural network model, and calculate the mean loss value of the selected data enhancement operation in the current training cycle based on the loss value of each new sample and the corresponding relationship; An updating module, configured to update, for each selected data enhancement operation, the loss measurement parameter corresponding to the data enhancement operation in the set table of the current training cycle based on the mean loss value of the data enhancement operation and the loss measurement parameter corresponding to the data enhancement operation in the set table of the previous training cycle; The training module is used to continue the training of the next training cycle until the number of training cycles in this round reaches a preset value, apply the loss measurement parameters in the set table to update the sampling probability parameters in the set table, and continue the subsequent training until the training is completed.

11. An application device of a neural network model, characterized in that: The device is applied to electronic equipment, and comprises: A target image acquisition module, used to acquire a target image to be processed based on a current task; wherein the current task includes a detection task, a recognition task or a tracking task for a target object; the target object includes one of the following: a pedestrian, a human body part, a vehicle or a license plate; A model processing module is used to input the target image into a pre-trained neural network model to obtain a result corresponding to the target image output by the neural network model; wherein the neural network model is a model trained using the training method described in any one of claims 1 to 8.

12. An electronic device, characterized in that: The electronic device comprises: an image acquisition device, a processing device and a storage device; The image acquisition device is used to acquire the image to be detected; The storage device stores a computer program, which, when executed by the processing device, executes the training method of the neural network model as described in any one of claims 1 to 8 or the application method of the neural network model as described in claim 9.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the training method of the neural network model according to any one of claims 1 to 8 or the application method of the neural network model according to claim 9 are executed.

Images