A model training method, an object detection method, and related equipment

Abstract

This application discloses a model training method, an object detection method, and related equipment. This application can acquire a training sample set, including training samples and validation samples; acquire an enhancement strategy corresponding to the training samples; perform enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples; predict the enhanced training samples using an initial model; update the parameters of the initial model based on the prediction results of the enhanced training samples to obtain an updated model; predict the validation samples using the updated model; update the enhancement strategy based on the prediction results of the validation samples to obtain an updated enhancement strategy; and iteratively train the updated model according to the updated enhancement strategy to obtain a trained model. This application improves the accuracy and precision of model training.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a model training method, an object detection method, and related equipment. Background Technology

[0002] Data augmentation is a key technique to prevent overfitting in deep neural networks. Therefore, appropriate data augmentation strategies have a significant impact on the performance of deep neural network models. Data augmentation is usually performed by manually selecting data augmentation strategies, but the efficiency of manual selection is very low. As a result, automatic data augmentation has emerged. Automatic data augmentation refers to automatically finding the most suitable data augmentation strategy for different tasks or datasets through specific search optimization methods.

[0003] Existing automatic data augmentation techniques can be broadly categorized into two types: non-differentiable automatic data augmentation techniques and those that optimize data augmentation strategies (e.g., AutoAugment). AutoAugment primarily utilizes reinforcement learning, heuristic search algorithms, or Bayesian optimization to search for and optimize data augmentation strategies. Specifically, a data augmentation search space can be designed, where each strategy consists of multiple sub-strategies. Each sub-strategy contains two image processing operations, such as translation, rotation, or shearing. For each operation, a set of probabilities and magnitudes characterizes its usage. The search algorithm comprises a controller and an optimization algorithm. The controller generates the data augmentation strategy decision at each step and then uses the decision as the embedding vector for the next operation. At each step, a data augmentation strategy is sampled, and a model is trained using this strategy on a small proxy dataset sampled from the training set. The performance of the trained model on the proxy validation set is used as a reward signal to optimize the controller. This automatic data augmentation technique can achieve good data augmentation strategy search; however, due to limitations in sampling and optimization efficiency, the search time cost is often high, and the reliability of the searched data augmentations is low, thus reducing the accuracy and precision of model training.

[0004] Another type is differentiable automatic data augmentation techniques, which use approximation techniques such as Gumbel-Softmax to transform the data augmentation search into a differentiable optimization problem. A representative technique is Differentiable Automatic Data Augmentation (DADA). Specifically, the data augmentation policy search is first formalized as a categorical distribution sampling problem, with the probability of each operation in each sub-policy sampled as a Bernoulli distribution. Then, the optimization of the distribution parameters is relaxed to a differentiable optimization problem using Gumbel-Softmax, and the gradient of this distribution is estimated using a RELAX estimator, while second-order gradient estimation is used to improve search speed. DADA uses meta-learning based on one-step gradient updates to alternately optimize the gradients of model weights and data augmentation policy parameters. While differentiable automatic data augmentation techniques can improve search efficiency, the process of implementing differentiable search requires the use of many approximation techniques. For example, using gumbel-softmax to achieve differentiable relaxation, or using estimators such as REINFORCE to estimate the gradient of data augmentation parameters. These approximation techniques not only increase technical complexity and limit the widespread use of automatic data augmentation techniques, but may also introduce inaccurate approximations, resulting in weak performance of the searched data augmentation measures and reducing the accuracy and precision of model training. Summary of the Invention

[0005] This application provides a model training method, an object detection method, and related equipment. The related equipment may include a model training device, an object detection device, a computer device, and a computer-readable storage medium. This application can improve the accuracy and precision of model training.

[0006] To address the aforementioned technical problems, the embodiments of this application provide the following technical solutions:

[0007] This application provides a model training method, comprising: acquiring a training sample set, the training sample set including training samples and validation samples; acquiring an augmentation strategy corresponding to the training samples; performing augmentation operations on the training samples according to the augmentation strategy to obtain augmented training samples; predicting the augmented training samples using an initial model; updating the parameters of the initial model based on the prediction results of the augmented training samples to obtain an updated model; predicting validation samples using the updated model; updating the augmentation strategy based on the prediction results of the validation samples to obtain an updated augmentation strategy; and iteratively training the updated model according to the updated augmentation strategy to obtain a trained model.

[0008] According to one aspect of this application, a target detection method is also provided, comprising: acquiring an image to be detected; extracting feature information of the image through a trained target detection model, wherein the trained target detection model is obtained by iterative training based on an updated enhancement strategy, the updated enhancement strategy is obtained by updating the enhancement strategy according to the prediction results of the updated model based on validation samples, the updated model is obtained by updating the parameters of the initial model according to the prediction results of the enhanced training samples, and the enhanced training samples are obtained by enhancing training samples through the enhancement strategy; and performing target detection on the image based on the feature information using the trained target detection model.

[0009] According to one aspect of this application, a model training apparatus is also provided, comprising: a first acquisition module for acquiring a training sample set, the training sample set including training samples and validation samples; a second acquisition module for acquiring an enhancement strategy corresponding to the training samples; an enhancement module for performing enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples; a first prediction module for predicting the enhanced training samples using an initial model; a first update module for updating the parameters of the initial model based on the prediction results of the enhanced training samples to obtain an updated model; a second prediction module for predicting the validation samples using the updated model; a second update module for updating the enhancement strategy based on the prediction results of the validation samples to obtain an updated enhancement strategy; and a training module for iteratively training the updated model according to the updated enhancement strategy to obtain a trained model.

[0010] According to one aspect of this application, an object detection apparatus is also provided, comprising: an image acquisition module for acquiring an image to be detected; an extraction module for extracting feature information of the image through a trained object detection model, wherein the trained object detection model is obtained by iterative training based on an updated enhancement strategy, the updated enhancement strategy is obtained by updating the enhancement strategy based on the prediction results of the updated model based on validation samples, the updated model is obtained by updating the parameters of the initial model based on the prediction results of the enhanced training samples, and the enhanced training samples are obtained by enhancing the training samples through the enhancement strategy; and a detection module for performing object detection on the image based on the feature information using the trained object detection model.

[0011] According to one aspect of this application, a computer device is also provided, including a processor and a memory, wherein the memory stores a computer program, and when the processor invokes the computer program in the memory, it executes any model training method provided in the embodiments of this application, or executes any target detection method provided in the embodiments of this application.

[0012] According to one aspect of this application, a computer-readable storage medium is also provided for storing a computer program, which is loaded by a processor to execute any model training method provided in the embodiments of this application, or to execute any target detection method provided in the embodiments of this application.

[0013] This application embodiment can augment training samples based on augmentation strategies corresponding to the training samples in the training sample set, obtaining augmented training samples. Then, an initial model makes predictions based on the augmented training samples, and the parameters of the initial model are updated based on the prediction results of the augmented training samples, resulting in an updated model. Next, the updated model can be used to predict validation samples in the training sample set, and the augmentation strategy is automatically updated based on the prediction results of the validation samples, resulting in an updated augmentation strategy. Finally, the updated model can be iteratively trained based on the updated augmentation strategy to obtain a trained model. This scheme rapidly augments training samples through augmentation strategies, updates the parameters of the initial model based on the augmented training samples, and automatically updates the augmentation strategy based on validation samples using the updated model for iterative training, improving the efficiency of training sample augmentation and the accuracy and precision of model training. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0015] Figure 1 This is a schematic diagram illustrating an application scenario of the model training method provided in the embodiments of this application;

[0016] Figure 2 This is a flowchart illustrating the model training method provided in the embodiments of this application;

[0017] Figure 3 This is a schematic diagram of training sample enhancement provided in an embodiment of this application;

[0018] Figure 4 This is another schematic diagram of the model training method provided in the embodiments of this application;

[0019] Figure 5 This is another schematic diagram of training sample enhancement provided in the embodiments of this application;

[0020] Figure 6 This is another schematic diagram of the model training method provided in the embodiments of this application;

[0021] Figure 7 This is another schematic diagram of the model training method provided in the embodiments of this application;

[0022] Figure 8 This is a flowchart illustrating the target detection method provided in an embodiment of this application;

[0023] Figure 9 This is a schematic diagram of the model training device provided in the embodiments of this application;

[0024] Figure 10 This is a schematic diagram of the target detection device provided in the embodiments of this application;

[0025] Figure 11 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0026] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In the following description, the terms "first" and "second" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0028] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0029] This application provides a model training method, an object detection method, and related equipment, wherein the related equipment may include a model training device, an object detection device, a computer device, and a computer-readable storage medium, etc.

[0030] The model training method provided in this application embodiment can be implemented by the terminal or the server alone, or by the terminal and the server collaboratively. For example, the terminal can undertake the model training method described below alone, or the terminal can send a training request for model training to the server, and the server can execute the model training method according to the received training request and send the training results to the terminal. Similarly, the object detection method provided in this application embodiment can be implemented by the terminal or the server alone, or by the terminal and the server collaboratively.

[0031] The computer equipment for model training provided in this application embodiment can be various types of terminal devices or servers. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, but is not limited to these. The terminal can be a smartphone, tablet, laptop, desktop computer, camera, wearable device, or vehicle-mounted terminal, etc., which can be located in an autonomous vehicle, but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, and this application embodiment does not impose any limitations on this.

[0032] Please see Figure 1 , Figure 1This is a schematic diagram illustrating the application scenarios of the model training method and object detection method provided in this application. The model training method can be applied to a model training device, which can be integrated into a computer device such as a terminal or server. Similarly, the object detection method can be applied to an object detection device, which can be integrated into a computer device such as a terminal or server. The following will provide a detailed explanation using the example of the model training device being integrated into a server and the object detection device being integrated into a terminal. For instance, the server can perform augmentation operations on the training samples based on the augmentation strategy corresponding to the training samples in the training sample set, obtaining augmented training samples. Then, the server can make predictions based on the augmented training samples using the initial model, and update the parameters of the initial model based on the prediction results of the augmented training samples, obtaining an updated model. Then, the updated model can be used to predict the validation samples in the training sample set, and the augmentation strategy can be automatically updated based on the prediction results of the validation samples, obtaining an updated augmentation strategy. At this point, the updated model can be iteratively trained according to the updated augmentation strategy to obtain a trained model. The server rapidly augments training samples using augmentation strategies, updates the parameters of the initial model based on the augmented training samples, and automatically updates the augmentation strategy based on validation samples using the updated model for iterative training. This improves the efficiency of training sample augmentation and enhances the accuracy and precision of model training. After obtaining the trained model, the server can send it to the terminal. The specific type of the trained model can be flexibly configured according to actual needs. For example, the trained model could be a trained object detection model. In this case, the terminal can acquire the image to be detected, extract feature information from the image using the trained object detection model, and perform object detection based on the feature information, thus improving the accuracy of object detection.

[0033] It should be noted that, Figure 1 The schematic diagram illustrating the application scenario of the model training method is merely an example. The application and scenario of the model training method described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. As those skilled in the art will know, with the evolution of the application of model training methods and the emergence of new business scenarios, the technical solutions provided in this application are also applicable to similar technical problems.

[0034] The following sections provide detailed descriptions of each example. It should be noted that the order in which the embodiments are described is not intended to limit the preferred order of the embodiments.

[0035] In this embodiment, the description will be from the perspective of a model training device, which can be integrated into computer devices such as terminals or servers.

[0036] Please see Figure 2 , Figure 2 This is a schematic flowchart of a model training method provided in an embodiment of this application. The model training method may include:

[0037] S101. Obtain the training sample set, which includes training samples and validation samples.

[0038] The training sample set can be obtained from a local storage database or from a server. The training sample set may include multiple samples, which can be common data in the field such as images, videos, audio, and text; this application does not impose any limitations on this. Furthermore, a portion of the samples can be extracted from the training sample set as training samples, and another portion can be extracted as validation samples. The training samples can be used to make predictions for the initial model, and the validation samples can be used to validate the updated model. The training sample set may include one or more samples, and the validation sample set may include one or more samples.

[0039] For example, multiple samples can be selected from the training sample set as surrogate training sets and multiple samples can be selected as surrogate validation sets. One or more sets of samples can be selected from the surrogate training set as training samples, with each set of training samples including at least one sample. One or more sets of samples can be selected from the surrogate validation set as validation samples, with each set of validation samples including at least one sample.

[0040] The samples in the training sample set can include ground truth values, which can include the location of the target object or the correct data label. For example, when the samples in the training sample set are images, the images can include target objects, and the ground truth values ​​can be the location of the target objects. This allows the trained model, which is trained based on the image, to accurately detect the location of the target objects in the image. The target objects can include vehicles, people, buildings, animals, or plants.

[0041] S102. Obtain the enhancement strategy corresponding to the training samples.

[0042] In one embodiment, the enhancement strategy may include any one or more combinations of enhancement operations such as translation, rotation, cropping, flipping, mirroring, scaling, scaling, noise overlay, color transformation, and brightness adjustment.

[0043] In one embodiment, obtaining the augmentation strategy corresponding to the training sample may include: obtaining the augmentation weights and determining the augmentation strategy corresponding to the training sample based on the augmentation weights.

[0044] To improve the accuracy of augmentation strategy determination, the augmentation strategy corresponding to the training samples can be determined by the augmentation weights of the training samples. First, the augmentation weights corresponding to the training samples can be obtained, which may include the augmentation probability (also known as the overall distribution probability p). tp ) and execution probability (also known as operation distribution probability p) o ) etc., the overall probability distribution p tp This can represent the overall probability of data augmentation being applied to the training samples (i.e., the probability that the training samples will be augmented), and the operation probability p. o It can represent the probability of selecting each candidate enhancement operation.

[0045] In one embodiment, obtaining the enhancement weights may include: obtaining the current enhancement probability and obtaining the execution probability corresponding to different enhancement operations; and determining the enhancement weights corresponding to the enhancement strategy based on the enhancement probability and the execution probability.

[0046] For example, initial augmentation probabilities can be set for training samples in advance, as well as initial execution probabilities for different augmentation operations. When training starts, the initial augmentation probabilities of training samples and the initial execution probabilities of different augmentation operations can be obtained. During training, the augmentation probabilities of training samples and the execution probabilities of different augmentation operations can be adjusted. At this time, the adjusted augmentation probabilities of training samples and the adjusted execution probabilities of different augmentation operations can be obtained.

[0047] After obtaining the augmentation probability corresponding to the training sample and the execution probability corresponding to the augmentation operation, the augmentation weight corresponding to the training sample can be determined based on the augmentation probability and the execution probability. In one embodiment, determining the augmentation weight corresponding to the augmentation strategy based on the augmentation probability and the execution probability may include: multiplying the augmentation probability and the execution probabilities of multiple augmentation operations corresponding to the augmentation strategy to obtain the augmentation weight corresponding to the augmentation strategy.

[0048] For example, the augmentation weights corresponding to the training samples = augmentation probabilities p tp * Execution probability p o .

[0049] In one embodiment, determining the augmentation strategy corresponding to the training sample based on the augmentation weight may include: sampling multiple augmentation strategies based on the augmentation weight; performing augmentation operations on the training sample according to the augmentation strategy to obtain the augmented training sample, which includes: applying the sampled multiple augmentation strategies to different training samples to obtain the augmented training sample.

[0050] For example, different training samples can be augmented based on the augmentation operations included in the augmentation strategy to obtain augmented training samples. The augmentation strategy can also include the execution priority of each augmentation operation. For instance, augmentation strategy A can include augmentation operation 1, augmentation operation 2, and augmentation operation 3, as well as the execution priority order of augmentation operation 1, augmentation operation 2, and augmentation operation 3 in sequence: augmentation operation 3, augmentation operation 2, and augmentation operation 4 are executed sequentially on the training samples to obtain augmented training samples.

[0051] In one embodiment, the generation of multiple enhancement strategies may include: selecting multiple candidate enhancement operations that meet certain conditions from multiple enhancement operations; dividing the multiple candidate enhancement operations into multiple enhancement operation groups; and generating multiple enhancement strategies based on each enhancement operation group and the execution order of each enhancement operation in each enhancement operation group.

[0052] Specifically, these multiple enhancement operations can form an enhancement operation library. Assuming that an enhancement operation library has K enhancement operations, multiple candidate enhancement operations that meet the conditions can be selected from the K (K is greater than N) enhancement operations: O1, O2, ..., O No For example, N candidate augmentation operations can be randomly selected from K augmentation operations, or the top N candidate augmentation operations with higher execution probabilities can be selected from K augmentation operations, and so on. Then, these multiple candidate augmentation operations can be divided into multiple augmentation operation groups (e.g., L groups of augmentation operations). Each enhancement operation group can include at least one enhancement operation. The specific partitioning method can be random partitioning, average partitioning, or partitioning according to the execution probability of the enhancement operations, etc., which is not limited here. In this case, multiple enhancement strategies can be generated based on each enhancement operation group and the execution order of the enhancement operations in each enhancement operation group.

[0053] In one implementation, the initial value of the enhancement probability is a set value; if the enhancement operation library includes K enhancement operations, the initial value of the execution probability corresponding to each enhancement operation is 1 / K; the enhancement probability and the execution probability are updated after each iteration.

[0054] Generally, the augmentation probability can be related to the number of training iterations. The initial augmentation probability is a set value (e.g., 0.5 or 0.4). The initial execution probability value can be set as an average value based on the number of augmentation operations in the augmentation operation library. After each iteration, the augmentation probability value and the execution probability value are updated. For example, if there are 100 augmentation operations in the operation library, the initial execution probability of each augmentation operation is 0.01.

[0055] S103. Perform augmentation operations on the training samples according to the augmentation strategy to obtain augmented training samples.

[0056] After obtaining the enhancement strategy, the training samples can be enhanced based on the enhancement strategy, for example, such as... Figure 3 As shown, for training sample D, the boosting probability p of training sample D can be used as a basis. tp Based on the determination that an augmentation operation is needed, the augmentation probability p is used to determine that an augmentation operation is needed. tp and execution probability p o Determine the augmentation operations included in the augmentation strategy: O1, O2, ..., O No And perform augmentation operations on the training samples to obtain the augmented training samples D. a .

[0057] For example, such as Figure 4 As shown, N can be randomly selected from K enhancement operations. o One enhancement operation, based on N o Each enhancement operation generates L groups of enhancement strategies. Enhance L group strategy The enhancements are applied to L groups of training samples respectively, generating L groups of enhanced training samples. Figure 4 middle, V is the training sample. t To validate the sample, p tp To increase the probability, p o To perform the probability calculation, Neural Networkθ represents the model (e.g., a neural network) that needs to be trained. For training samples The obtained gradient value, These are training samples enhanced based on L groups. The obtained gradient value, For validation sample V t The obtained gradient values ​​can be used to analyze the training samples. L-group enhanced training samples and validation sample V t The input is the model to be trained (e.g., the initial model), and the gradient values ​​are based on the model's output. The model is trained iteratively, which will be explained in detail below.

[0058] The augmentation of training samples according to the augmentation strategy can be performed to obtain augmented training samples. This can include: when the augmentation probability of the training sample is greater than a preset probability threshold, determining the execution priority order of each augmentation operation in the augmentation strategy according to the execution probability of each augmentation operation in the augmentation strategy; and performing any one or more combinations of augmentation operations such as translation, rotation, cropping, flipping, mirroring, shrinking, enlarging, noise superposition, color transformation, and brightness adjustment on the training samples according to the execution priority order to obtain augmented training samples.

[0059] Optionally, after obtaining the augmentation probability of the training sample, it can be determined whether the augmentation probability corresponding to the training sample is greater than a preset probability threshold. The preset probability threshold can be flexibly set according to actual needs. For example, ... Figure 5 As shown, when the augmentation probability of a training sample is greater than a preset probability threshold, it indicates that augmentation operations need to be performed on the training sample. At this point, augmentation operations can be applied to the training sample. For example, the execution priority order of each augmentation operation in the augmentation strategy can be determined based on its execution probability. For instance, the higher the execution probability of an augmentation operation, the lower its execution priority; conversely, the lower the execution probability of an augmentation operation, the higher its execution priority. Then, augmentation operations such as translation, rotation, cropping, flipping, mirroring, shrinking, enlarging, noise superposition, color transformation, and brightness adjustment can be performed on the training sample according to the execution priority order to obtain augmented training samples. These augmented training samples can then be used to train the model. For example, when the enhancement strategy involves sequentially performing rotation, cropping, scaling, and brightness adjustment, the training samples can first be rotated to obtain rotated training samples. Then, the rotated training samples can be cropped to obtain cropped training samples. Next, the cropped training samples can be scaled up to obtain scaled training samples. Finally, the scaled training samples can be scaled up to obtain enhanced training samples. When the enhancement probability corresponding to a training sample is less than or equal to a preset probability threshold, it indicates that no enhancement operation is needed, and the training samples can be directly used to train the model.

[0060] It should be noted that during the augmentation of training samples according to the augmentation strategy, the augmentation operations included in the strategy can be performed on the training samples in parallel to obtain multiple augmented training samples. For example, when the augmentation strategy includes rotation, cropping, shrinking, and noise addition, the training samples can be rotated, cropped, shrunk, and noise added separately to obtain multiple augmented training samples, such as rotated training samples, cropped training samples, shrunk training samples, and noise-added training samples.

[0061] S104. Predict the augmented training samples using the initial model.

[0062] The type and structure of the initial model can be flexibly set according to actual needs, and there are no specific restrictions here. For example, the initial model can be a neural network model.

[0063] S105. Update the parameters of the initial model based on the prediction results of the enhanced training samples to obtain the updated model.

[0064] In one embodiment, the process of making predictions based on augmented training samples using an initial model and updating the parameters of the initial model based on the prediction results of the augmented training samples to obtain an updated model may include: making predictions based on augmented training samples using the initial model to obtain predicted values; obtaining a loss value based on the predicted value and the true value corresponding to the augmented training samples; obtaining a first gradient value based on the loss value; and updating the parameters of the initial model according to the first gradient value to obtain the updated model.

[0065] Specifically, the initial model can be used to make predictions based on the augmented training samples to obtain predicted values. For example, when the initial model needs to be trained based on sample images (i.e., training samples) to accurately detect the location of a target object in an image, the initial model can be used to predict the location of the target object based on the augmented sample images to obtain the predicted location. Furthermore, the ground truth value corresponding to the augmented training samples can be obtained; this ground truth value can be the actual location of the target object within the augmented training samples. Then, the loss value (also called the expectation value) between the predicted value obtained by the initial model based on the augmented training samples and the ground truth value corresponding to the augmented training samples can be obtained. Based on the loss value, the first gradient value is determined, i.e., the first gradient value is obtained by performing forward and backward propagation operations on the initial model based on the augmented training samples. When there are multiple augmented training samples, multiple first gradient values ​​can be obtained. At this point, the parameters of the initial model can be updated to appropriate values ​​based on the first gradient value, resulting in the updated model.

[0066] S106. Make predictions on the validation samples using the updated model.

[0067] S107. Update the enhancement strategy based on the prediction results of the validation samples to obtain the updated enhancement strategy.

[0068] In one embodiment, predicting validation samples using the updated model and updating the enhancement strategy based on the prediction results of the validation samples to obtain the updated enhancement strategy may include: predicting validation samples using the updated model to obtain a second gradient value based on the prediction results; updating the enhancement weights according to the second gradient value to obtain the updated enhancement weights; and determining the updated enhancement strategy according to the updated enhancement weights.

[0069] Specifically, the updated model can be used to predict the validation sample to obtain the predicted value corresponding to the validation sample. For example, when the validation sample is an image, in order for the trained model to accurately detect the location of the target object in the image, the updated model can be used to predict the location of the target object based on the validation sample to obtain the predicted location.

[0070] Furthermore, the ground truth value corresponding to the validation sample can be obtained, which may be the true position of the target object in the validation sample. Then, the loss value between the predicted value obtained by the updated model based on the validation sample and the ground truth value corresponding to the validation sample can be obtained. Based on this loss value, the second gradient value is determined, that is, the second gradient value is obtained by performing forward and backward propagation operations on the updated model based on the validation sample. When there are multiple validation samples, multiple second gradient values ​​can be obtained. At this point, the augmentation weights, such as the augmentation probability and execution probability, can be updated based on the second gradient values ​​to obtain the updated augmentation weights (e.g., the updated augmentation probability g). ptp and the probability of execution after update The specific update method can be flexibly set according to actual needs. Finally, based on the mapping relationship between different enhancement weights and enhancement strategies, the enhancement strategy that matches the updated enhancement weights can be determined, thus obtaining the updated enhancement strategy.

[0071] To improve the accuracy and reliability of updating model parameters, the first gradient value and the second gradient value can be normalized. In one embodiment, updating the parameters of the initial model based on the first gradient value to obtain the updated model may include: normalizing the first gradient value to obtain a normalized first gradient value; and updating the parameters of the initial model based on the normalized first gradient value to obtain the updated model.

[0072] In one embodiment, updating the augmented weights based on the second gradient value to obtain the updated augmented weights may include: normalizing the second gradient value to obtain a normalized second gradient value; and updating the augmented weights based on the normalized second gradient value to obtain the updated augmented weights.

[0073] For example, the boosting weights corresponding to the boosting strategy can be used. The first and second gradient values ​​are normalized as weights so that the calculated gradients are on the same preset scale. The augmentation weights may include the augmentation probability p. tp and execution probability p o The specific normalization method can be represented by the following formula:

[0074]

[0075]

[0076] in, It can represent the first gradient after normalization, g ptp This can represent the normalized second gradient. It can be represented based on the validation sample V t The obtained gradient value, It can be represented based on training samples The obtained gradient value, It can be represented based on the augmented training samples The resulting gradient value, p tp It can represent the augmentation probability of training samples. It can represent the enhancement weights corresponding to the enhancement strategy, and η can represent the coefficient. Its specific value can be flexibly set according to actual needs.

[0077] Where Z and Z g The normalization coefficient can be calculated using the following formula:

[0078]

[0079]

[0080] Based on the normalized first gradient and normalized second gradient, the data augmentation parameters can be searched and optimized. For example, the model parameters can be updated according to the normalized first gradient value to obtain the updated model, so as to predict the validation samples using the updated model. The augmentation weights can be updated according to the normalized second gradient value to obtain the updated augmentation weights, so as to determine the updated augmentation strategy based on the updated augmentation weights, and iteratively train the updated model according to the updated augmentation strategy.

[0081] S108. Iteratively train the updated model according to the updated enhancement strategy to obtain the trained model.

[0082] For example, such as Figure 6 As shown, after selecting training samples and determining the corresponding augmentation strategy for the training samples in the above manner, the augmentation strategy can be applied to the training samples to generate augmented training samples. Then, the first gradient can be calculated based on the augmented training samples to update the model parameters, resulting in an updated model. The second gradient value can be calculated based on the selected validation samples using the updated model, and the augmentation strategy can be updated based on the second gradient value to obtain an updated augmentation strategy. The updated model can then be iteratively trained based on the updated augmentation strategy to obtain a trained model.

[0083] In one embodiment, the training sample set is a partial sample set of the full training set. Iteratively training the updated model according to the updated enhancement strategy to obtain the trained model may include: iteratively training the updated model according to the updated enhancement strategy to obtain the final enhancement strategy; performing enhancement operations on the full training set using the final enhancement strategy; and iteratively training the updated model based on the enhanced full training set to obtain the trained model.

[0084] The full training set includes multiple training samples, which are part of the full training set. The updated model can be iteratively trained based on the updated enhancement strategy according to the iterative training method described above. Thus, the final enhancement strategy can be obtained based on the iterative training on the training sample set. The final enhancement strategy is then mapped to the full training set for enhancement operations. The updated model is then iteratively trained based on the enhanced full training set to obtain the trained model, which can improve the accuracy of model training.

[0085] In one embodiment, applying the final enhancement strategy to the full training set may include: applying the final enhancement strategy to every sample in the full training set; or, mapping the updated enhancement strategy obtained after each iteration to the corresponding full training set according to the ratio of the number of samples in the full training set and the proxy training set.

[0086] The final augmentation strategy can be the strategy obtained at the end of training on the training sample set, and then applied to the entire training set, meaning that every sample in the entire training set is augmented using the final augmentation strategy. Alternatively, the number of samples in the entire training set and the number of samples in the surrogate training set can be obtained. Based on the number of samples in the entire training set and the surrogate training set, the ratio of the number of samples in the entire training set to the number of samples in the surrogate training set can be determined (e.g., 1:s). The augmentation strategy used in the first batch of training sample sets is mapped to the first s samples in the entire set, the augmentation strategy used in the second batch of training sample sets is mapped to the second set of samples in the entire set (s~2s), and so on. This achieves the proportional mapping of the updated augmentation strategy obtained after each iteration to the corresponding sample in the entire training set, improving the flexibility and reliability of model training.

[0087] In one embodiment, the training samples include a first training sample and a second training sample. Augmenting the training samples according to an augmentation strategy to obtain augmented training samples may include: augmenting the first training sample according to the augmentation strategy to obtain augmented training samples. Iteratively training the updated model according to the updated augmentation strategy to obtain a trained model may include: augmenting the second training sample according to the updated augmentation strategy to obtain a target augmented training sample; using the target augmented training sample as the augmented training sample and the updated model as the initial model, and then performing a prediction operation based on the augmented training sample using the initial model, until the number of iterations reaches a preset number or the loss value predicted by the updated model is less than a preset threshold, thus obtaining the trained model.

[0088] Specifically, training samples can include multiple sets of training samples, such as a first training sample, a second training sample, ..., and an nth training sample. During the process of augmenting the training samples according to the augmentation strategy, the first training sample can be augmented to obtain an augmented training sample. Then, during iterative training of the updated model according to the updated augmentation strategy, the second training sample can be augmented to obtain a target augmented training sample. This target augmented training sample is used as the augmented training sample, and the updated model is used as the initial model. The process then returns to perform prediction based on the augmented training sample using the initial model, and updates the parameters of the initial model based on the prediction results of the augmented training sample to obtain the updated model. The updated model is then used to predict the validation samples, and the augmentation strategy is updated based on the prediction results of the validation samples to obtain the updated augmentation strategy. Iterative training of the updated model is then performed according to the updated augmentation strategy until the number of iterations reaches a preset number or the loss value predicted by the updated model is less than a preset threshold (i.e., the updated model converges), resulting in the trained model. The preset number of times or preset threshold can be flexibly set according to actual needs, and the specific value is not limited here.

[0089] For example, in this embodiment, the purpose of training the model is to make the model's parameters converge:

[0090]

[0091]

[0092] in, It can represent the parameters of the model. This can represent the loss of the validation samples. E can represent the loss of the training samples. p It can represent the expected value.

[0093] Among them, the expected value E p The calculation method can be shown below:

[0094]

[0095] Wherein, the probability of the l-th augmented training sample The calculation method can be shown below:

[0096]

[0097] Among them, the probability of training samples The calculation method can be shown below:

[0098]

[0099] The iterative update method for model parameters can be as follows:

[0100] θ t+1 =θ t -η·g t

[0101] Wherein, the gradient value g t The calculation method can be shown below:

[0102]

[0103] in, It can represent training samples. θ can represent the augmented training sample. t It can represent the parameters of the model. It can be represented based on the augmented training samples The obtained gradient value, It can be represented based on training samples The resulting gradient values. During the search process, the probability parameters can be transformed into weights in the loss function by taking the expectation of the loss function during training, thereby achieving a differentiable approximation.

[0104] In one embodiment, the trained model includes a trained object detection model or a trained image classification model. The model training method may further include: acquiring an image to be detected; extracting feature information of the image through the trained object detection model or the trained image classification model; and performing object detection or image classification on the image based on the feature information.

[0105] For example, when the training samples are sample images, the initial object detection model can be trained based on the sample images using the training method described above, and the location of the target object in the image can be accurately detected based on the trained object detection model. The type and structure of the object detection model can be flexibly set according to actual needs. The image to be detected can be a depth image acquired by LiDAR, or it can be an image acquired by a mobile phone, camera, or other similar device.

[0106] For example, when the trained model is a trained image classification model, the training samples can be sample images. The initial image classification model can be trained based on these sample images using the training method described above, and the trained image classification model can then accurately classify the images. The type and structure of the image classification model can be flexibly set according to actual needs. First, images to be detected can be acquired using devices such as LiDAR, mobile phones, cameras, or webcams. Then, the trained image classification model can extract the image's feature information. For example, the convolution operator of the trained image classification model can extract the feature information (e.g., feature maps) corresponding to the image to be detected. Then, the image to be detected can be classified based on this feature information to determine its category, improving the accuracy and convenience of image classification.

[0107] This application embodiment can augment training samples based on augmentation strategies corresponding to the training samples in the training sample set, obtaining augmented training samples. Then, an initial model makes predictions based on the augmented training samples, and the parameters of the initial model are updated based on the prediction results of the augmented training samples, resulting in an updated model. Next, the updated model can be used to predict validation samples in the training sample set, and the augmentation strategy is automatically updated based on the prediction results of the validation samples, resulting in an updated augmentation strategy. Finally, the updated model can be iteratively trained based on the updated augmentation strategy to obtain a trained model. This scheme rapidly augments training samples through augmentation strategies, updates the parameters of the initial model based on the augmented training samples, and automatically updates the augmentation strategy based on validation samples using the updated model for iterative training, improving the efficiency of training sample augmentation and the accuracy and precision of model training.

[0108] Based on the methods described in the above embodiments, the following will illustrate an exemplary application of the embodiments of this application in a practical application scenario, such as the application scenario of an autonomous vehicle. This embodiment takes the integration of a model training device into an autonomous vehicle as an example, and takes an initial model as an initial target detection model and a trained model as a trained target detection model for detailed explanation. The autonomous vehicle may be equipped with LiDAR or cameras for image acquisition, as well as a target detection model.

[0109] Please see Figure 7 , Figure 7 This is a schematic flowchart of a model training method provided in an embodiment of this application. The model training method may include:

[0110] S201. Obtain a sample image set, which includes training sample images and validation sample images.

[0111] Autonomous vehicles can obtain sample image sets from a local storage database or download sample image sets from a server. The sample image set may include multiple images. Multiple images can be selected from the sample image set as a proxy training set and multiple images can be selected as a proxy verification set. One or more sets of sample images can be selected from the proxy training set as training sample images, with each set of training sample images including at least one sample image. One or more sets of sample images can be selected from the proxy verification set as verification sample images, with each set of verification sample images including at least one sample image.

[0112] The sample images in the sample image set can include the target object and the corresponding real value of the target object. The real value can include the position of the target object in the sample image, the category of the target object, etc. The target object can include vehicles, people, buildings, animals or plants, etc.

[0113] S202. Obtain the augmentation weights corresponding to the training sample images, and determine the augmentation strategy corresponding to the training sample images based on the augmentation weights.

[0114] The enhancement weights can include the enhancement probability p. tp and execution probability p o For example, an initial augmentation probability can be set for the training sample image in advance, as well as an initial execution probability for different augmentation operations. When training starts, the initial augmentation probability of the training sample image and the initial execution probability of different augmentation operations can be obtained. During training, the augmentation probability of the training sample image and the execution probability of different augmentation operations can be adjusted. At this time, the adjusted augmentation probability of the training sample image and the adjusted execution probability of different augmentation operations can be obtained.

[0115] The enhancement operations can include translation, rotation, cropping, flipping, mirroring, shrinking, enlarging, noise overlay, color transformation, and brightness adjustment. The enhancement strategy can include any one or more combinations of these enhancement operations, as well as the execution priority of each enhancement operation. For example, enhancement strategy A can include enhancement operation 1, enhancement operation 2, and enhancement operation 3, and the execution priority order of these operations in sequence: enhancement operation 3, enhancement operation 2, and enhancement operation 4 are performed sequentially on the training sample image to obtain the enhanced training sample image.

[0116] After obtaining the augmentation probabilities corresponding to the training sample images and the execution probabilities corresponding to the augmentation operations, the autonomous vehicle can determine the augmentation weights corresponding to the training sample images based on the augmentation probabilities and execution probabilities. For example, the augmentation weights corresponding to the training samples = augmentation probabilities p tp * Execution probability p o Alternatively, the augmentation weights corresponding to the training samples = first coefficient * augmentation probability p tp +Second coefficient * Execution probability p o .

[0117] At this point, the autonomous vehicle can determine the augmentation strategy corresponding to the training sample image based on the augmentation weights. For example, it can obtain the pre-established mapping relationship between each augmentation strategy and the augmentation weights, and determine the augmentation strategy that matches the augmentation weights corresponding to the training sample image based on the mapping relationship, thus obtaining the augmentation strategy corresponding to the training sample image.

[0118] S203. Perform enhancement operations on the training sample images according to the enhancement strategy to obtain enhanced training sample images.

[0119] After obtaining the enhancement strategy, the autonomous vehicle can perform enhancement operations on the training sample images based on the enhancement strategy. For example, it can enhance the training sample images based on the enhancement probability p. tp Once it is determined that an enhancement operation is needed, it can be based on the enhancement probability p. tp and execution probability p o Determine the augmentation operations included in the augmentation strategy: O1, O2, ..., O No The training sample images are then augmented to obtain augmented training sample images.

[0120] Specifically, after obtaining the enhancement probability of the training sample image, the autonomous vehicle can determine whether the enhancement probability corresponding to the training sample image is greater than a preset probability threshold. When the enhancement probability corresponding to the training sample image is greater than the preset probability threshold, it means that the training sample image needs to be enhanced. At this time, according to the execution priority order of each enhancement operation in the enhancement strategy, any one or more combinations of enhancement operations such as translation, rotation, cropping, flipping, mirroring, shrinking, enlarging, noise superposition, color transformation, and brightness adjustment can be performed on the training sample image to obtain the enhanced training sample image.

[0121] For example, when the enhancement strategy involves sequentially performing rotation, cropping, magnification, and brightness adjustment, the training sample image can first be rotated to obtain a rotated training sample image. Then, the rotated training sample image can be cropped to obtain a cropped training sample image. Next, the cropped training sample image can be magnified to obtain a magnified training sample image. Finally, the magnified training sample image can be brightness adjusted to obtain an enhanced training sample image. When the enhancement probability corresponding to the training sample image is less than or equal to a preset probability threshold, it indicates that no enhancement operation is needed on the training sample image, and the model can be directly trained using the training sample image.

[0122] S204. The initial object detection model makes predictions based on the enhanced training sample images, and the parameters of the initial object detection model are updated based on the prediction results of the enhanced training sample images to obtain the updated object detection model.

[0123] For example, an autonomous vehicle can make predictions based on the enhanced training sample images using an initial object detection model to obtain predicted values ​​(i.e., predicted locations). It can then obtain a loss value based on the predicted values ​​and the ground truth values ​​corresponding to the enhanced training sample images, obtain a first gradient value based on the loss value, and update the parameters of the initial object detection model according to the first gradient value to obtain an updated object detection model.

[0124] S205. The updated target detection model is used to predict the validation sample image, and the enhancement strategy is updated based on the prediction results of the validation sample image to obtain the updated enhancement strategy.

[0125] For example, autonomous vehicles can use an updated object detection model to predict validation sample images and obtain a second gradient value based on the prediction result. For instance, the loss value between the predicted value obtained by the updated object detection model based on the validation sample image and the true value corresponding to the validation sample image can be obtained. The second gradient value is determined based on the loss value, the enhancement weights are updated based on the second gradient value, the updated enhancement weights are obtained, and the updated enhancement strategy is determined based on the updated enhancement weights.

[0126] S206. Iteratively train the updated target detection model according to the updated enhancement strategy to obtain the trained target detection model.

[0127] The autonomous vehicle can perform enhancement operations on the second training sample image according to the updated enhancement strategy to obtain the target enhanced training sample image. The target enhanced training sample image is used as the enhanced training sample image, and the updated target detection model is used as the initial target detection model. The vehicle then returns to perform the operation of making predictions based on the enhanced training sample image using the initial target detection model until the number of iterations reaches a preset number or the updated target detection model converges, thus obtaining the trained model.

[0128] The training sample images may include multiple sets of training sample images, such as a first training sample image, a second training sample image, ..., and an nth training sample image. In the process of enhancing the training sample images according to the enhancement strategy, the first training sample image may be enhanced according to the enhancement strategy to obtain the enhanced training sample image. When enhancing the training sample images according to the updated enhancement strategy, the second training sample image may be enhanced according to the updated enhancement strategy to obtain the target enhanced training sample image.

[0129] S207. Obtain the image to be detected, and perform object detection on the image using the trained object detection model.

[0130] The image to be detected can be a depth image acquired by a pre-set lidar on the autonomous vehicle, or it can be an image acquired by a pre-set camera or other camera on the autonomous vehicle.

[0131] After obtaining the image to be detected, the feature information of the image can be extracted by the trained object detection model. For example, the feature information (e.g., feature map) corresponding to the image to be detected can be extracted by the convolution operator of the trained object detection model. Specifically, the region within a preset neighborhood range can be sampled from the image to be detected by the convolution operator to obtain the sliding window region. The relative relationship (e.g., relative positional relationship) between the sampling reference point and the neighborhood points in the sliding window region can be obtained. The weight vector of each sampling point in the sliding window region can be obtained based on the relative relationship. The feature vector of each sampling point in the sliding window region can be extracted. Based on the weight vector and feature vector of each sampling point in the sliding window region, the feature information corresponding to the image to be detected can be obtained.

[0132] Then, the image to be detected can be detected based on this feature information. For example, the feature information (e.g., feature maps) can be detected based on the attribute information of each location point in the image to be detected, generating multiple detection boxes within a preset attribute interval corresponding to each feature map, as well as the classification confidence of each detection box. The classification confidence can be the degree of overlap between the detection box and the corresponding ground truth box of the same category. Afterward, multiple detection boxes can be merged using weighted non-maximum suppression to obtain the target box corresponding to the depth image to be detected.

[0133] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the model training method above, which will not be repeated here.

[0134] This application embodiment can obtain a sample image set including training sample images and validation sample images, and obtain the enhancement weights corresponding to the training sample images. Based on the enhancement weights, an enhancement strategy corresponding to the training sample images is determined. Then, the training sample images are enhanced according to the enhancement strategy to obtain enhanced training sample images, improving the efficiency of training sample image enhancement. An initial object detection model is then used to make predictions based on the enhanced training sample images, and the parameters of the initial object detection model are updated based on the prediction results of the enhanced training sample images to obtain an updated object detection model. Next, the updated object detection model can be used to predict validation sample images, and the enhancement strategy is updated based on the prediction results of the validation sample images to obtain an updated enhancement strategy. The updated object detection model is then iteratively trained according to the updated enhancement strategy to obtain a trained object detection model, improving the accuracy and precision of the object detection model training. At this point, the image to be detected can be obtained, and the trained object detection model can be used to perform object detection on the image, improving the accuracy of object detection.

[0135] In this embodiment, the description will be from the perspective of a target detection device, which can be integrated into computer devices such as terminals or servers.

[0136] Please see Figure 8 , Figure 8 This is a schematic flowchart of a target detection method provided in an embodiment of this application. The target detection method may include:

[0137] S301. Acquire the image to be detected.

[0138] S302. Feature information of the image is extracted through the trained object detection model. The trained object detection model is obtained through iterative training based on the updated enhancement strategy. The updated enhancement strategy is obtained by updating the enhancement strategy based on the prediction results of the updated model based on the validation samples. The updated model is obtained by updating the parameters of the initial model based on the prediction results of the enhanced training samples. The enhanced training samples are obtained by enhancing the training samples through the enhancement strategy. The detailed training process of this object detection model has been described. Figures 1-7 The details are disclosed in the description, so I will not go into them here.

[0139] S303. The trained target detection model performs target detection on the image based on feature information.

[0140] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the model training method above, which will not be repeated here.

[0141] To facilitate better implementation of the model training method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described model training method. The meanings of the terms used are the same as in the above-described model training method, and specific implementation details can be found in the descriptions in the method embodiments.

[0142] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of the model training device 400 provided in the embodiments of this application. The model training device 400 may include a first acquisition module 401, a second acquisition module 402, an enhancement module 403, a first prediction module 404, a first update module 405, a second prediction module 406, a second update module 407, and a training module 408, etc.

[0143] The first acquisition module 401 is used to acquire a training sample set, which includes training samples and validation samples.

[0144] The second acquisition module 402 is used to acquire the enhancement strategy corresponding to the training samples.

[0145] The enhancement module 403 is used to perform enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples.

[0146] The first prediction module 404 is used to predict the augmented training samples using the initial model.

[0147] The first update module 405 is used to update the parameters of the initial model based on the prediction results of the enhanced training samples, so as to obtain the updated model.

[0148] The second prediction module 406 is used to predict the validation samples using the updated model;

[0149] The second update module 407 is used to update the enhancement strategy based on the prediction results of the validation samples, and obtain the updated enhancement strategy.

[0150] Training module 408 is used to iteratively train the updated model according to the updated enhancement strategy to obtain the trained model.

[0151] In one embodiment, the training sample set is a partial sample set from the full training set, and the training module 408 includes:

[0152] The first training submodule is used to iteratively train the updated model based on the updated augmentation strategy to obtain the final augmentation strategy.

[0153] The augmentation submodule is used to perform augmentation operations on the entire training set using the final augmentation strategy;

[0154] The second training submodule is used to iteratively train the updated model based on the enhanced full training set to obtain the trained model.

[0155] In one embodiment, the enhancement submodule can be specifically used to: perform enhancement operations on each sample in the full training set using the final enhancement strategy; or, based on the ratio of the number of samples in the full training set and the proxy training set, map the updated enhancement strategy obtained after each iteration to the corresponding full training set in proportion.

[0156] In one embodiment, the second acquisition module 402 includes:

[0157] The `get` submodule is used to obtain the enhancement weights;

[0158] The determination submodule is used to determine the augmentation strategy corresponding to the training samples based on the augmentation weights.

[0159] In one implementation, the determining submodule may specifically be used to: sample multiple enhancement strategies based on enhancement weights;

[0160] Specifically, the enhancement module 403 can be used to apply multiple enhancement strategies obtained from sampling to different training samples to obtain enhanced training samples.

[0161] In one embodiment, the acquisition submodule can be specifically used to: acquire the current enhancement probability and the execution probability corresponding to different enhancement operations; and determine the enhancement weight corresponding to the enhancement strategy based on the enhancement probability and the execution probability.

[0162] In one implementation, the initial value of the enhancement probability is a set value; if the enhancement operation library includes K operations, the initial value of the execution probability corresponding to each enhancement operation is 1 / K; the enhancement probability and the execution probability are updated after each iteration.

[0163] In one embodiment, the acquisition submodule can be specifically used to: perform a multiplication operation on the enhancement probability and the execution probability of multiple enhancement operations corresponding to the enhancement strategy to obtain the enhancement weight corresponding to the enhancement strategy.

[0164] In one embodiment, the model training device 400 may include:

[0165] The filtering module is used to filter out multiple candidate enhancement operations that meet certain conditions from a number of enhancement operations.

[0166] The partitioning module is used to divide multiple candidate augmentation operations into multiple augmentation operation groups;

[0167] The generation module is used to generate multiple enhancement strategies based on each enhancement operation group and the execution order of each enhancement operation in each enhancement operation group.

[0168] In one embodiment, the enhancement strategy includes any one or more combinations of enhancement operations such as translation, rotation, cropping, flipping, mirroring, scaling, magnification, noise overlay, color transformation, and brightness adjustment.

[0169] In one embodiment, the first prediction module 404 can be specifically used to: make predictions based on the enhanced training samples using the initial model to obtain predicted values.

[0170] The first update module 405 can be used to: obtain the loss value based on the predicted value and the true value corresponding to the augmented training sample; obtain the first gradient value based on the loss value; update the parameters of the initial model according to the first gradient value to obtain the updated model.

[0171] In one embodiment, the first update module 405 may be used to: normalize the first gradient value to obtain a normalized first gradient value; and update the parameters of the initial model according to the normalized first gradient value to obtain the updated model.

[0172] In one embodiment, the second prediction module 406 may specifically be used to: predict the validation sample using the updated model, so as to obtain a second gradient value based on the prediction result;

[0173] The second update module 407 can be used to: update the enhancement weights according to the second gradient value to obtain the updated enhancement weights; and determine the updated enhancement strategy according to the updated enhancement weights.

[0174] In one embodiment, the second update module 407 can be specifically used to: normalize the second gradient value to obtain a normalized second gradient value; and update the enhancement weights according to the normalized second gradient value to obtain the updated enhancement weights.

[0175] In one embodiment, the training samples include a first training sample and a second training sample. The enhancement module 403 can be specifically used to: perform enhancement operations on the first training sample according to the enhancement strategy to obtain the enhanced training sample.

[0176] The training module 408 can be specifically used to: perform augmentation operations on the second training sample according to the updated augmentation strategy to obtain the target augmented training sample; use the target augmented training sample as the augmented training sample and the updated model as the initial model, and return to execute the operation of making predictions based on the augmented training sample using the initial model, until the number of iterations reaches a preset number or the loss value predicted by the updated model is less than a preset threshold, to obtain the trained model.

[0177] In one embodiment, the model training device 400 may further include:

[0178] The detection module is used to acquire the image to be detected, extract the feature information of the image through a trained object detection model or a trained image classification model, and perform object detection or image classification on the image based on the feature information.

[0179] In this embodiment, a first acquisition module 401 acquires a training sample set, which includes training samples and validation samples. A second acquisition module 402 acquires the enhancement strategies corresponding to the training samples. An enhancement module 403 performs enhancement operations on the training samples based on the enhancement strategies corresponding to the training samples in the training sample set to obtain enhanced training samples. A first prediction module 404 makes predictions based on the enhanced training samples using an initial model, and a first update module 405 updates the parameters of the initial model based on the prediction results of the enhanced training samples to obtain an updated model. Then, a second prediction module 406 makes predictions on the validation samples in the training sample set using the updated model, and a second update module 407 automatically updates the enhancement strategies based on the prediction results of the validation samples to obtain an updated enhancement strategy. At this time, a training module 408 iteratively trains the updated model according to the updated enhancement strategy to obtain a trained model. This scheme rapidly enhances training samples through augmentation strategies, updates the parameters of the initial model based on the augmented training samples, and automatically updates the augmentation strategy based on the updated model and validation samples to iteratively train the model. This improves the efficiency of training sample augmentation and the accuracy and precision of model training.

[0180] To facilitate better implementation of the target detection method provided in the embodiments of this application, the embodiments of this application also provide an apparatus based on the above-described target detection method. The meanings of the terms used are the same as in the target detection method described above, and specific implementation details can be found in the descriptions within the method embodiments.

[0181] Please see Figure 10 , Figure 10 This is a schematic diagram of the structure of the target detection device 500 provided in the embodiments of this application. The target detection device 500 may include an image acquisition module 501, an extraction module 502, and a detection module 503, etc.

[0182] The image acquisition module 501 is used to acquire the image to be detected.

[0183] The extraction module 502 is used to extract feature information of the image through the trained object detection model. The trained object detection model is obtained by iterative training based on the updated enhancement strategy. The updated enhancement strategy is obtained by updating the enhancement strategy according to the prediction results of the updated model based on the validation samples. The updated model is obtained by updating the parameters of the initial model according to the prediction results of the enhanced training samples. The enhanced training samples are obtained by performing enhancement operations on the training samples through the enhancement strategy.

[0184] The detection module 503 is used to perform target detection on the image based on feature information using a trained target detection model.

[0185] This application also provides a computer device, which may be a terminal or a server, etc. Figure 11 As shown, it illustrates a structural schematic diagram of the computer device involved in the embodiments of this application, specifically:

[0186] The computer device may include components such as a processor 601 with one or more processing cores, a memory 602 with one or more computer-readable storage media, a power supply 603, and an input unit 604. Those skilled in the art will understand that... Figure 11 The computer device structure shown does not constitute a limitation on the computer device and may include more or fewer components than shown, or combine certain components, or have different component arrangements. Wherein:

[0187] The processor 601 is the control center of the computer device. It connects various parts of the computer device via various interfaces and lines, and performs various functions and processes data by running or executing software programs and / or modules stored in the memory 602, and by calling data stored in the memory 602, thereby providing overall monitoring of the computer device. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 601.

[0188] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 602 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0189] The computer device also includes a power supply 603 that supplies power to the various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 603 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0190] The computer device may also include an input unit 604, which can be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0191] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 601 in the computer device loads the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 runs the application programs stored in the memory 602 to realize various functions, as follows:

[0192] Obtain a training sample set, including training samples and validation samples, and obtain the augmentation strategy corresponding to the training samples; perform augmentation operations on the training samples according to the augmentation strategy to obtain augmented training samples; make predictions using the initial model based on the augmented training samples, and update the parameters of the initial model based on the prediction results of the augmented training samples to obtain an updated model; make predictions using the updated model on the validation samples, and update the augmentation strategy based on the prediction results of the validation samples to obtain an updated augmentation strategy; iteratively train the updated model according to the updated augmentation strategy to obtain a trained model.

[0193] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the detailed description of the model training method above, which will not be repeated here.

[0194] According to one aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0195] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by computer instructions, or by controlling related hardware through computer instructions. These computer instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. Therefore, embodiments of this application provide a computer-readable storage medium storing a computer program, which may include computer instructions. This computer program can be loaded by a processor to execute any of the model training methods provided in the embodiments of this application.

[0196] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0197] The computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0198] Since the instructions stored in the computer-readable storage medium can execute any of the model training methods provided in the embodiments of this application, and thus can achieve the beneficial effects that any of the model training methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, will not be repeated here.

[0199] The foregoing has provided a detailed description of a model training method, model training apparatus, computer device, and computer-readable storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A model training method, characterized in that, include: Obtain a training sample set, which includes training samples and validation samples; Obtain the enhancement strategy corresponding to the training sample; The training samples are augmented according to the augmentation strategy to obtain augmented training samples. The enhanced training samples are predicted using the initial model; The parameters of the initial model are updated based on the prediction results of the enhanced training samples to obtain the updated model; The updated model is used to predict the validation samples; The enhancement strategy is updated based on the prediction results of the validation samples to obtain the updated enhancement strategy; as well as The updated model is iteratively trained according to the updated enhancement strategy to obtain the trained model. The step of obtaining the enhancement strategy corresponding to the training samples includes: Get the current enhancement probability, and get the execution probability corresponding to different enhancement operations; The enhancement weights corresponding to the enhancement strategy are determined based on the enhancement probability and the execution probability. The augmentation strategy corresponding to the training sample is determined based on the augmentation weights; The enhancement probability and execution probability are updated after each iteration; The trained model includes a trained object detection model or a trained image classification model, and the method further includes: Acquire the image to be detected; The image features are extracted using the trained object detection model or trained image classification model, and object detection or image classification is performed on the image based on the features.

2. The method according to claim 1, characterized in that, The training sample set is a partial sample set from the full training set. The step of iteratively training the updated model according to the updated enhancement strategy to obtain the trained model includes: The updated model is iteratively trained according to the updated enhancement strategy to obtain the final enhancement strategy; The final enhancement strategy is then applied to the entire training set for enhancement. The updated model is iteratively trained based on the enhanced full training set to obtain the trained model.

3. The method according to claim 2, characterized in that, The augmentation operation on the entire training set using the final augmentation strategy includes: Each sample in the full training set is augmented using the final augmentation strategy; or Based on the ratio of the number of samples in the full training set and the proxy training set, the updated enhancement strategy obtained after each iteration is mapped proportionally to the corresponding full training set.

4. The method according to claim 1, characterized in that, Also includes: Multiple augmentation strategies are applied to multiple training samples to generate multiple augmented training samples. The model is iteratively trained based on the gradient values ​​obtained from each group of enhanced training samples.

5. The method according to claim 4, characterized in that, The step of determining the augmentation strategy corresponding to the training sample based on the augmentation weights includes: Multiple enhancement strategies are sampled based on enhancement weights; The step of performing enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples includes: Multiple enhancement strategies obtained from sampling are applied to different training samples to obtain enhanced training samples.

6. The method according to claim 1, characterized in that, The initial value of the enhancement probability is a set value; if the enhancement operation library includes K enhancement operations, then the initial value of the execution probability corresponding to each enhancement operation is 1 / K.

7. The method according to claim 1, characterized in that, The step of determining the enhancement weight corresponding to the enhancement strategy based on the enhancement probability and the execution probability includes: The enhancement probability and the execution probabilities of multiple enhancement operations corresponding to the enhancement strategy are multiplied together to obtain the enhancement weight corresponding to the enhancement strategy.

8. The method according to claim 5, characterized in that, The method further includes: Select multiple candidate augmentation operations that meet the criteria from a range of augmentation operations; The multiple candidate enhancement operations are divided into multiple enhancement operation groups; Multiple enhancement strategies are generated based on each enhancement operation group and the execution order of each enhancement operation within each enhancement operation group.

9. The method according to claim 1, characterized in that, The enhancement strategies include any one or more combinations of enhancement operations such as translation, rotation, cropping, flipping, mirroring, shrinking, enlarging, noise overlay, color transformation, and brightness adjustment.

10. The method according to claim 1, characterized in that, The step of making predictions based on the enhanced training samples using an initial model, and updating the parameters of the initial model based on the prediction results of the enhanced training samples to obtain an updated model, includes: The predicted value is obtained by making a prediction based on the enhanced training samples using the initial model; The loss value is obtained based on the predicted value and the true value corresponding to the augmented training sample. The first gradient value is obtained based on the loss value; The parameters of the initial model are updated based on the first gradient value to obtain the updated model.

11. The method according to claim 10, characterized in that, The step of updating the parameters of the initial model based on the first gradient value to obtain the updated model includes: The first gradient value is normalized to obtain the normalized first gradient value; The parameters of the initial model are updated based on the normalized first gradient value to obtain the updated model.

12. The method according to claim 1, characterized in that, The step of predicting the validation samples using the updated model and updating the enhancement strategy based on the prediction results of the validation samples to obtain the updated enhancement strategy includes: The updated model is used to predict the validation samples, and a second gradient value is obtained based on the prediction results. The enhanced weights are updated according to the second gradient value to obtain the updated enhanced weights; The updated enhancement strategy is determined based on the updated enhancement weights.

13. The model training method according to claim 12, characterized in that, The step of updating the enhancement weights according to the second gradient value to obtain the updated enhancement weights includes: The second gradient value is normalized to obtain the normalized second gradient value; The enhanced weights are updated based on the normalized second gradient value to obtain the updated enhanced weights.

14. The method according to claim 1, characterized in that, The training samples include a first training sample and a second training sample. The step of performing enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples includes: performing enhancement operations on the first training sample according to the enhancement strategy to obtain enhanced training samples. The step of iteratively training the updated model according to the updated enhancement strategy to obtain the trained model includes: The second training sample is augmented according to the updated augmentation strategy to obtain the target augmented training sample. The enhanced training samples are used as the enhanced training samples, and the updated model is used as the initial model. The operation of making predictions based on the enhanced training samples using the initial model is returned until the number of iterations reaches a preset number or the loss value predicted by the updated model is less than a preset threshold, thus obtaining the trained model.

15. A target detection method, characterized in that, include: Acquire the image to be detected; Feature information of the image is extracted by a trained object detection model. The trained object detection model is obtained by iterative training based on an updated enhancement strategy. The updated enhancement strategy is obtained by updating the enhancement strategy according to the prediction results of the updated model based on the validation samples. The updated model is obtained by updating the parameters of the initial model according to the prediction results of the enhanced training samples. The enhanced training samples are obtained by enhancing the training samples through the enhancement strategy. as well as The trained target detection model performs target detection on the image based on the feature information; The enhancement strategy is obtained through the following method: Get the current enhancement probability, and get the execution probability corresponding to different enhancement operations; The enhancement weights corresponding to the enhancement strategy are determined based on the enhancement probability and the execution probability. The augmentation strategy corresponding to the training sample is determined based on the augmentation weights; The enhancement probability and execution probability are updated after each iteration.

16. A model training device, characterized in that, include: The first acquisition module is used to acquire a training sample set, which includes training samples and validation samples. The second acquisition module is used to acquire the enhancement strategy corresponding to the training sample; An enhancement module is used to perform enhancement operations on the training samples according to the enhancement strategy to obtain enhanced training samples; The first prediction module is used to predict the enhanced training samples using the initial model; The first update module is used to update the parameters of the initial model based on the prediction results of the enhanced training samples to obtain the updated model; The second prediction module is used to predict the validation samples using the updated model; The second update module is used to update the enhancement strategy based on the prediction results of the verification samples to obtain the updated enhancement strategy. as well as The training module is used to iteratively train the updated model according to the updated enhancement strategy to obtain the trained model. Wherein, the second acquisition module is further adapted to: Get the current enhancement probability, and get the execution probability corresponding to different enhancement operations; The enhancement weights corresponding to the enhancement strategy are determined based on the enhancement probability and the execution probability. The augmentation strategy corresponding to the training sample is determined based on the augmentation weights; The enhancement probability and execution probability are updated after each iteration; The trained model includes a trained object detection model or a trained image classification model, and the device further includes: The detection module is used to acquire the image to be detected, extract the feature information of the image through a trained object detection model or a trained image classification model, and perform object detection or image classification on the image based on the feature information.

17. A target detection device, characterized in that, include: The image acquisition module is used to acquire the image to be detected; The extraction module is used to extract feature information of the image through a trained target detection model. The trained target detection model is obtained by iterative training based on an updated enhancement strategy. The updated enhancement strategy is obtained by updating the enhancement strategy according to the prediction results of the updated model based on the validation samples. The updated model is obtained by updating the parameters of the initial model according to the prediction results of the enhanced training samples. The enhanced training samples are obtained by enhancing the training samples through the enhancement strategy. The detection module is used to perform target detection on the image based on the feature information using the trained target detection model; The extraction module is further adapted to obtain the enhancement strategy according to the following method: Get the current enhancement probability, and get the execution probability corresponding to different enhancement operations; The enhancement weights corresponding to the enhancement strategy are determined based on the enhancement probability and the execution probability. The augmentation strategy corresponding to the training sample is determined based on the augmentation weights; The enhancement probability and execution probability are updated after each iteration.

18. A computer device, characterized in that, The method includes a processor and a memory, wherein the memory stores a computer program, and the processor executes the method as described in any one of claims 1 to 15 when it invokes the computer program in the memory.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store a computer program, which is loaded by a processor to perform the method according to any one of claims 1 to 15.

Images