Model training method and apparatus, and electronic device

Abstract

The present disclosure provides a model training method and device and electronic equipment, relating to the technical field of artificial intelligence, and specifically relating to the technical field of deep learning. The specific implementation scheme is: obtaining a first data set under a target scene, the first data set comprising M historical multimedia data under the target scene; determining M first sampling probabilities corresponding to the M historical multimedia data based on first prediction data, the first prediction data being prediction data output in Kth iteration training of a first model based on a first training data set, the first training data set comprising historical multimedia data selected from the first data set; selecting N historical multimedia data from the first data set based on the M first sampling probabilities to obtain a second training data set; performing K+1th iteration training on the first model based on the second training data set to obtain second prediction data; and determining a target model based on the second prediction data.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of deep learning technology, specifically to a model training method, apparatus, and electronic device. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning technology has been widely applied. Utilizing massive amounts of data to drive the optimization of deep learning models is simpler and more efficient than manual design, and the results are more significant.

[0003] In deep learning models, data plays a crucial role in the entire optimization process. Therefore, selecting training data for deep learning models from massive datasets is of paramount importance during model training.

[0004] Currently, data sampling schemes during model training typically include random sampling, class sampling, and task sampling. Summary of the Invention

[0005] This disclosure provides a model training method, apparatus, and electronic device.

[0006] According to a first aspect of this disclosure, a model training method is provided, comprising:

[0007] Obtain the first dataset under the target scenario, the first dataset includes M historical multimedia data under the target scenario, where M is an integer greater than 1;

[0008] Based on the first prediction data, M first sampling probabilities corresponding to the M historical multimedia data are determined. The first prediction data is the prediction data output during the Kth iteration training of the first model based on the first training dataset. The first training dataset includes historical multimedia data selected from the first dataset. The first sampling probability indicates the probability that the historical multimedia data is selected in the first dataset, and K is a positive integer.

[0009] Based on the M first sampling probabilities, N historical multimedia data are selected from the first dataset to obtain the second training dataset, where N is an integer greater than 1;

[0010] Based on the second training dataset, the first model is trained for the K+1th iteration to obtain the second prediction data;

[0011] Based on the second prediction data, a target model trained by the first model is determined, and the target model is used to process multimedia data in the target scene.

[0012] According to a second aspect of this disclosure, a model training apparatus is provided, comprising:

[0013] The acquisition module is used to acquire a first dataset under the target scenario, the first dataset including M historical multimedia data under the target scenario, where M is an integer greater than 1;

[0014] The first determining module is used to determine M first sampling probabilities corresponding to the M historical multimedia data based on the first prediction data. The first prediction data is the prediction data output during the Kth iteration training of the first model based on the first training dataset. The first training dataset includes historical multimedia data selected from the first dataset. The first sampling probability indicates the probability that the historical multimedia data is selected in the first dataset, and K is a positive integer.

[0015] The selection module is used to select N historical multimedia data from the first dataset based on the M first sampling probabilities to obtain a second training dataset, where N is an integer greater than 1;

[0016] The iterative training module is used to perform the K+1th iteration training on the first model based on the second training dataset to obtain the second prediction data;

[0017] The second determining module is used to determine a target model trained by the first model based on the second prediction data, wherein the target model is used to process multimedia data in the target scene.

[0018] According to a third aspect of this disclosure, an electronic device is provided, comprising:

[0019] At least one processor; and

[0020] A memory that is communicatively connected to at least one processor; wherein,

[0021] The memory stores instructions that can be executed by at least one processor to enable the at least one processor to perform any of the methods in the first aspect.

[0022] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions for causing a computer to perform any of the methods in the first aspect.

[0023] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements any of the methods in the first aspect.

[0024] The technology disclosed herein solves the problem of poor model training performance caused by data sampling methods during model iterative training, and improves the model training performance for processing multimedia data in target scenarios.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0027] Figure 1 This is a schematic flowchart of a model training method according to the first embodiment of this disclosure;

[0028] Figure 2 This is a schematic diagram of the structure of a model training apparatus according to the second embodiment of the present disclosure;

[0029] Figure 3 This is a schematic block diagram of an example electronic device used to implement embodiments of the present disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] First Embodiment

[0032] like Figure 1 As shown, this disclosure provides a model training method, including the following steps:

[0033] Step S101: Obtain the first dataset under the target scene, the first dataset including M historical multimedia data under the target scene.

[0034] Where M is an integer greater than 1.

[0035] In this embodiment, the model training method relates to the field of artificial intelligence technology, particularly the field of deep learning technology, and can be widely applied in scenarios such as autonomous driving, smart cities, remote sensing detection, and industrial production. The model training method of this embodiment can be executed by the model training device of this embodiment. The model training device of this embodiment can be configured in any electronic device to execute the model training method of this embodiment. This electronic device can be a server or a terminal device; no specific limitation is made here.

[0036] The target scenario can be scenarios such as autonomous driving, smart cities, remote sensing detection, and industrial production. The first dataset can be a dataset within the target scenario, which may include at least two historical multimedia datasets within the target scenario. The historical multimedia datasets refer to multimedia data within the target scenario obtained before the deep learning model is trained; the multimedia data can be videos, images, or text, etc.

[0037] For example, in a smart city scenario, multiple RGB images captured historically by roadside equipment can be obtained to form a first dataset. The RGB images may include vehicle image content. In this embodiment, an image segmentation model can be trained based on the first training set. The trained image segmentation model (i.e., the target model) can be used to segment the RGB images captured in real time by the roadside equipment to achieve vehicle tracking and thus realize traffic monitoring and management.

[0038] For example, in a remote sensing detection scenario, multiple remote sensing images captured by satellites in the past can be acquired to obtain a first dataset. In this embodiment, an image detection model can be trained based on the first training set. The trained image detection model (i.e., the target model) can be used to perform target detection on remote sensing images captured by satellites in real time to determine whether there are targets such as airplanes and ships in the remote sensing images.

[0039] For example, in an industrial production scenario, images captured historically by the inspection equipment can be obtained to form a first dataset. These images may include the content of the production line, which is used to classify products on the production line. In this embodiment, an image classification model can be trained based on the first training set. The trained image classification model (i.e., the target model) can be used to classify products in real-time images captured by the inspection equipment, thereby achieving automated product classification in industrial production.

[0040] The methods for obtaining the first dataset may include, but are not limited to, the following: obtaining historical multimedia data pre-stored on an electronic device and adding it to the first dataset; or receiving historical multimedia data sent by other electronic devices and adding it to the first dataset.

[0041] Step S102: Based on the first prediction data, determine the M first sampling probabilities corresponding to the M historical multimedia data. The first prediction data is the prediction data output during the Kth iteration training of the first model based on the first training dataset. The first training dataset includes historical multimedia data selected from the first dataset. The first sampling probability indicates the probability that the historical multimedia data is selected in the first dataset.

[0042] Where K is a positive integer.

[0043] In related technologies, methods for sampling data from a first dataset to obtain the training dataset for iterative model training include random sampling, class sampling, and task sampling. In random sampling, each forward iteration randomly selects data from the first dataset to form a batch and feeds it into the deep learning model. Class sampling extracts data from each class according to a certain sampling ratio to form a batch and feed it into the deep learning model for training. Task sampling is a sampling scheme designed for multi-task model training. This scheme considers that a batch should contain data from different tasks simultaneously, so that the model training is not biased towards learning certain tasks. The above data sampling methods usually sample data randomly or according to the data distribution, but they do not consider the value of the data to the model training process.

[0044] In this context, "value" refers to the contribution of data to the loss function during model training. The larger the contribution, the more valuable the data is in model training; conversely, the smaller the contribution, the less valuable the data is in model training, as the information represented by the data has already been learned by the deep learning model.

[0045] However, the high-value data contained in a batch can influence the training efficiency and accuracy of a deep learning model. The more high-value data, the better the training efficiency and accuracy. The purpose of this embodiment is to consider the value of each historical multimedia data point in the training dataset during iterative model training, and automatically adjust the sampling probability of each historical multimedia data point in the first dataset in the next iteration based on its value. This aims to achieve better optimization of the deep learning model, thereby improving the accuracy of the deep learning model in processing multimedia data in the target scene.

[0046] The first model can be a deep learning model, trained on a first dataset. Once training is complete, i.e., the network parameters of the first model are updated, the target model can be obtained. This target model is used to process multimedia data in a target scenario. For example, in a smart city scenario, the target model trained on the first model is used to segment RGB images captured in real time by roadside equipment to achieve vehicle tracking; in a remote sensing detection scenario, the target model trained on the first model is used to detect objects in remote sensing images captured by satellites; and in an industrial production scenario, the target model trained on the first model is used to classify products in images captured by detection equipment.

[0047] It should be noted that the first model is trained iteratively, meaning it can be trained in multiple epochs. The first dataset can be referred to as the training data pool for the first model. Each epoch selects a batch of training data of size batchsize from the first dataset to form a batch. The batch is the input data for the first model. The batch can be input into the first model for processing, and the output is the predicted data for that batch. In other words, the predicted data is the output data of the first model.

[0048] The processing methods and predicted data vary depending on the target scenario. The predicted data can include prediction results from historical multimedia data. For example, in a smart city scenario, the processing involves image segmentation, resulting in vehicle identifiers and vehicle detection boxes. In a remote sensing scenario, the processing involves object detection, resulting in object categories and object detection boxes. In an industrial production scenario, the processing involves product classification, resulting in product categories.

[0049] During the iterative training of the first model, the Kth iteration of the first model training can be any one of these iterations, such as the first iteration, the second iteration, etc. The first training dataset is the input data for the first model during the Kth iteration of training, and it can include multiple historical multimedia data selected from the first dataset. By batch inputting the first training dataset into the first model and performing forward propagation, the first prediction data can be obtained, which can be the output data of the first model during the Kth iteration of training.

[0050] In one alternative implementation, historical multimedia data can be selected from the first dataset using a random sampling method to obtain the first training sample set. As the model iterates and trains, in the next iteration of model training, i.e., the (K+1)th iteration, the M first sampling probabilities corresponding to the M historical multimedia data in the (K+1)th iteration can be determined based on the prediction data output from the (K)th iteration.

[0051] In another optional implementation, historical multimedia data can be selected from the first dataset based on M third sampling probabilities corresponding to M historical multimedia data. The preset sampling function selects historical multimedia data from the first dataset according to the sampling probability. The larger the third sampling probability, the greater the probability that the corresponding historical multimedia data will be selected.

[0052] When K is 1, the M third sampling probabilities can be calculated by the following formula (1), that is, the initial sampling probability of each historical multimedia data is consistent.

[0053]

[0054] Among them, s i Let M be the initial sampling probability of the i-th sample, and M be the number of samples.

[0055] As the model iterates and trains, in the next iteration of model training, i.e., the sampling probabilities of the M historical multimedia data will be updated, i.e., from the M third sampling probabilities to the M first sampling probabilities.

[0056] The first sampling probability indicates the probability that historical multimedia data will be selected from the first dataset during the (K+1)th iteration of training of the first model. The higher the first sampling probability, the higher the probability that the corresponding historical multimedia data will be selected, and vice versa.

[0057] The value of each historical multimedia data point in the first training dataset can be determined based on the first prediction data. In an optional implementation, the error brought about by each historical multimedia data point in the first training dataset for this forward propagation can be determined based on the first prediction data. The first prediction data may include the prediction result output by each historical multimedia data point in the first training dataset for this forward propagation. Each historical multimedia data point in the first training dataset corresponds to one prediction result, and the number of prediction results in the first prediction data is the same as the number of historical multimedia data points in the first training dataset. The prediction result corresponding to the historical multimedia data can be compared with the label of the historical multimedia data to determine the error brought about by the historical multimedia data for this forward propagation. Based on the error, the value of the historical multimedia data can be determined. The larger the error, the greater the value of the historical multimedia data, and vice versa.

[0058] Accordingly, based on the value of each historical multimedia data point in the first training dataset, M first sampling probabilities corresponding to the M historical multimedia data points can be determined, and the method of determination is not limited here. Specifically, historical multimedia data points with high value in the first training dataset have a higher first sampling probability, while historical multimedia data points with low value in the first training dataset have a lower first sampling probability. This increases the sampling probability of high-value data during the iterative training of the first model, while simultaneously decreasing the sampling probability of low-value data, adaptively optimizing the training effect of the first model.

[0059] For example, in a smart city scenario, the first dataset may include RGB images 1 to RGB images 10, a total of 10 RGB images, with a batch size of 7. In the first iteration of training of the first model, the initial sampling probability of each historical multimedia data, i.e., the third sampling probability, is calculated as 1 / 10 by the above formula (1). Based on M third sampling probabilities, 7 RGB images are selected from the first dataset through a preset sampling function, which can be images 1 to 7 respectively, forming a batch (i.e., the first training dataset) and inputting it into the first model for image segmentation. Specifically, feature extraction can be performed on the RGB images to obtain image features, and image segmentation can be performed based on the image features to obtain vehicle detection box information (i.e., the first prediction data) in each image. The vehicle detection box information can be compared with the corresponding image label (the image label can be the vehicle detection box label in the image) to obtain the error of each image in this forward propagation. Based on this error, the value of each image in the batch can be determined. Correspondingly, the first sampling probability of each of images 1 to 10 can be determined based on the value of each image in the first training dataset.

[0060] For example, if the value of image 1 is 2 and the value of image 2 is 0.5, then the first sampling probability of image 1 should be greater than the first sampling probability of image 2.

[0061] The implementation process of step S102 in remote sensing detection scenarios and industrial production scenarios is similar to that in smart city scenarios, and will not be described in detail here.

[0062] Through the above step S102, the output data in the previous iteration of the first model training can be fed back to affect the input data in the next iteration of training. Specifically, the value of the sample can be determined based on the output data in the previous iteration of training, and the sampling probability of each sample in the first dataset can be determined based on the value. The sampling probability of the sample in the first dataset can affect the training sample selected in the next iteration of training, so that valuable samples can be sampled in real time with a high probability during each iteration of training of the first model.

[0063] Step S103: Based on the M first sampling probabilities, select N historical multimedia data from the first dataset to obtain the second training dataset.

[0064] Where N is an integer greater than 1.

[0065] In this step, each historical multimedia data in the first dataset maintains a first sampling probability. If the first sampling probability of the historical multimedia data is high, then the probability of that historical multimedia data being sampled is high, and vice versa.

[0066] Based on M first sampling probabilities, at least two historical multimedia data points can be selected from the first dataset using a preset sampling function (such as a random sampling function) to obtain the second training dataset.

[0067] For example, in a smart city scenario, the first sampling probabilities of images 1 to 10 obtained based on the first prediction data are as follows: images 1, 2 and 3 are all 0.2, and the probabilities of images 4 to 10 are 0.4 and 7 respectively. Since the first sampling probabilities of images 1, 2 and 3 are large, the images selected from the first dataset based on the M first sampling probabilities through the random sampling function can include images 1, 2, 3, and images 6 to 9, which are combined to form the second training dataset to obtain a batch.

[0068] It can be seen that since the first sampling probability of images 1, 2 and 3 is high, images 1, 2 and 3 can be selected from the first dataset with a high probability.

[0069] The implementation process of step S103 in remote sensing detection and industrial production scenarios is similar to that in smart city scenarios, and will not be elaborated here.

[0070] Step S104: Based on the second training dataset, train the first model for the K+1th iteration to obtain the second prediction data.

[0071] In this step, the second training dataset can be input into the first model for forward propagation to perform the K+1th iteration training of the first model and obtain the second prediction data.

[0072] In the smart city scenario, given a second training dataset, it can be input into the first model for image segmentation, yielding vehicle detection bounding box information for each image in the second training dataset (i.e., images 1, 2, 3, and 6 through 9). The implementation process of step S104 in remote sensing and industrial production scenarios is similar to that in the smart city scenario, and will not be elaborated upon here.

[0073] Step S105: Based on the second prediction data, determine the target model trained by the first model, the target model being used to process multimedia data in the target scene.

[0074] In this step, based on the second prediction data output in the K+1th iteration of training, it can be determined whether the first model has been trained successfully, or whether it has been trained to the point where the first preset condition is met.

[0075] In one optional implementation, if the difference between the second predicted data and the label data of the historical multimedia data in the second training dataset, i.e. the network loss value of the first model, is less than a certain threshold, it can be determined that the first model training is complete, or that the training has met a first preset condition. The first preset condition may include the network loss value being less than or equal to the first preset threshold.

[0076] In another optional implementation, if the number of iterations, i.e., epochs, is greater than or equal to a certain threshold, it can be determined that the training of the first model is complete, or that the training has met a first preset condition. The first preset condition may include the number of iterations of the first model being greater than or equal to a second preset threshold.

[0077] Accordingly, if the training has met the first preset condition, the model obtained from the (K+1)th iteration of training can be saved. Furthermore, once training is complete, the target model can be determined based on the model obtained from the (K+1)th iteration. For example, if only one model is saved, that model can be directly selected as the target model. If multiple models are saved, the merits of the saved models can be determined based on the validation set, and the target model can be selected from the saved models accordingly.

[0078] The target model is used to process multimedia data in the target scenario. For example, in a smart city scenario, if the first model has been iterated 10 times or more, the trained image segmentation model is saved as a candidate model. Training is considered complete when the network loss value is less than or equal to a threshold based on the second prediction data. Then, a validation set (which includes RGB images captured by roadside equipment) is obtained to validate the candidate models, and the applicable image segmentation model is determined from the candidate models. This image segmentation model (i.e., the target model) can segment RGB images captured in real-time by roadside equipment to achieve vehicle tracking, thereby realizing traffic monitoring and management.

[0079] It should be noted that if the first model iteration training is not completed, such as before the specified number of iterations, or if the network loss value is still greater than a certain threshold, then M first sampling probabilities can be updated based on the second prediction data to obtain M sampling probabilities corresponding to M historical multimedia data in the K+2th iteration training. This ensures that the next iteration training is also based on the value of the samples in the previous iteration training to select samples. That is, steps S102 to S105 are executed repeatedly until the iteration training is completed.

[0080] In this embodiment, the output data from the previous iteration of the first model's training can be fed back to influence the input data in the next iteration. Specifically, the value of a sample can be determined based on the output data from the previous iteration, and the sampling probability of each sample in the first dataset can be determined based on this value. The sampling probability of samples in the first dataset can affect the training samples selected in the next iteration, so that valuable samples can be sampled in real time with a higher probability during each iteration of the first model's training. This allows the first model to be optimized better and improves model training efficiency. Correspondingly, the better the accuracy of the target model in processing multimedia data in the target scene, for example, in a smart city scenario, the higher the accuracy of image segmentation of RGB images captured by roadside devices, the more accurate the vehicle detection box information can be.

[0081] Optionally, the first training dataset further includes first identifier data of historical multimedia data in the first training dataset, and step S102 specifically includes:

[0082] Determine the difference between the first predicted data and the first identified data;

[0083] Based on the difference value, the M first sampling probabilities are determined.

[0084] In this embodiment, the first identification data can be the label data of historical multimedia data, which may include the identifiers (i.e., labels) of each historical multimedia data in the first training dataset. In a smart city scenario, the first identification data may include vehicle detection box labels in an RGB image; in a remote sensing detection scenario, the first identification data may include target category labels and target detection box labels in a remote sensing image; and in an industrial production scenario, the first identification data may include product category labels in an image.

[0085] For each historical multimedia data point in the first training dataset, the prediction result corresponding to that historical multimedia data point can be compared with its corresponding identifier to determine the difference between the prediction result and the identifier. This difference can be called the error of that historical multimedia data point in this forward propagation. The larger the error corresponding to the historical multimedia data point, the larger the first sampling probability of that historical multimedia data point, and vice versa. The first sampling probability of historical multimedia data points in the first dataset other than the first training dataset can either continue the sampling probability from the previous iterations of training (i.e., the first K iterations of training) or be updated based on the sampling probability of historical multimedia data points in the first training dataset; no specific limitation is made here. In this way, M first sampling probabilities can be determined based on the difference between the first predicted data and the first identifier data.

[0086] Optionally, determining the M first sampling probabilities based on the difference values ​​includes:

[0087] Based on the difference value, the value of historical multimedia data in the first training dataset is determined;

[0088] Based on the value, the M first sampling probabilities are determined.

[0089] In this embodiment, the difference value can be normalized, and the value of each historical multimedia data in the first training dataset can be calculated using the following formula (2).

[0090]

[0091] In equation (2) above, In the Kth iteration of training, the normalized difference value (i.e., the normalized error) corresponding to the i-th sample (i.e., the i-th historical multimedia data) in the first training dataset is given. Let be the value of the i-th sample.

[0092] Based on the value, the sampling probability of each sample in the first training dataset is calculated using the following formula (3).

[0093]

[0094] The above formula Let be the sampling probability of the i-th sample in the first training dataset during the K-th iteration of training. In one optional implementation, the sampling probabilities of each sample determined by (3) above can be defined as the first sampling probability of that sample, and the first sampling probabilities of historical multimedia data in the first dataset other than the first training dataset can continue the sampling probabilities in previous iterations of training (i.e., the first K iterations of training). In another optional implementation, the M third sampling probabilities in the K-th iteration of training can also be updated based on the sampling probabilities of each sample determined by (3) above, without specific limitations here. In this way, the determination of the M first sampling probabilities can be achieved based on the values ​​of each sample determined in the previous iteration of training.

[0095] Optionally, determining the M first sampling probabilities based on the value includes:

[0096] Based on the value, a second sampling probability of historical multimedia data in the first training dataset is determined;

[0097] Based on the second sampling probability, the M third sampling probabilities corresponding to the M historical multimedia data are adjusted to obtain the M first sampling probabilities;

[0098] The M third sampling probabilities are used to obtain the first training dataset.

[0099] In this embodiment, the second sampling probability of each sample in the first training dataset can be determined based on the above (3), and based on the second sampling probability, the M third sampling probabilities determined in the previous iteration training can be adjusted by the following formula (4) to obtain M first sampling probabilities.

[0100]

[0101] In equation (4) above, Let be the first sampled probability of the i-th sample in the first training set after normalization and adjustment before the start of the (K+1)-th iteration of training, and Let be the sampling probability of the i-th sample in the first dataset after the K-th training iteration. After the K-th training iteration, the sampling probabilities of some samples in the first training set (i.e., samples in the first training dataset) have changed to the second sampling probability. If the i-th sample is in both the first dataset and the first training dataset, then after the K-th training iteration, the sampling probability of that sample changes to the second sampling probability.

[0102] Correspondingly, after adjustment by the above formula (4), the sum of the sampling probabilities of each sample in the first dataset is 1, which means that the normalization of the M sampling probabilities of the M historical multimedia data after the end of the Kth iteration training is achieved.

[0103] In other words, after the Kth iteration of training, the second sampling probability of the historical multimedia data in the first training dataset can be obtained. First, the third sampling probability of some samples in the first dataset (i.e., the historical multimedia data in the first training dataset) is updated, specifically updated to the second sampling probability. Then, the updated M sampling probabilities are normalized and adjusted. In this way, by updating the sampling probability of each sample in the first dataset as a whole, the accuracy of determining the sampling probability of each sample can be improved.

[0104] Optionally, before step S102, the method further includes:

[0105] When K is 1, the M third sampling probabilities are determined based on the number of historical multimedia data in the first dataset.

[0106] In this embodiment, when K is 1, M third sampling probabilities can be determined by the above formula (1). The third sampling probability is the initial sampling probability. In this way, the determination of M third sampling probabilities can be achieved.

[0107] Optionally, the second training dataset further includes second identifier data of historical multimedia data in the second training dataset, and step S105 specifically includes:

[0108] Based on the second prediction data and the second identification data, determine the network loss value of the first model in the K+1th iteration of training;

[0109] Based on the network loss value, update the network parameters of the first model;

[0110] If the network loss value determines that the first preset condition is met, the target model is determined based on the first model updated in the (K+1)th iteration of training.

[0111] The first preset condition includes at least one of the following:

[0112] The network loss value is less than or equal to a first preset threshold;

[0113] The number of iterations of the first model is greater than or equal to the second preset threshold.

[0114] In this embodiment, the difference between the second predicted data and the second labeled data can be determined. Based on the difference, an existing or new loss function is used to determine the network loss value of the first model in the K+1th iteration of training. The larger the difference, the larger the network loss value.

[0115] Based on the network loss value, backpropagation using the gradient descent method is used to update the network parameters of the first model. If the network loss value is less than or equal to a first preset threshold, or if the network loss value is greater than the first preset threshold but the number of iterations is greater than or equal to a second preset threshold, it can be determined that the training has met the first preset condition. Accordingly, the first model updated in the (K+1)th iteration of training can be saved, and the target model is determined based on the saved first model.

[0116] In one optional implementation, the first model can be determined as the target model if the network loss value is less than or equal to a first preset threshold and the number of iterations is greater than or equal to a second preset threshold. In another optional implementation, the target model can be obtained by filtering from the saved first models based on the validation set.

[0117] The first and second preset thresholds can be set according to the actual situation, and no specific limitations are made here.

[0118] In this way, the target model can be determined.

[0119] Optionally, determining the target model based on the first model updated in the (K+1)th iteration of training includes:

[0120] When the iterative training of the first model is completed, the first model that meets the second preset condition is determined as a candidate model, and at least two candidate models are obtained.

[0121] The target model is obtained by filtering from at least two candidate models;

[0122] The second preset condition includes at least one of the following:

[0123] The first model updated during iterative training when the number of iterations is greater than or equal to the second preset threshold;

[0124] The first model updated during iterative training where the network loss value is less than or equal to the first preset threshold.

[0125] In this embodiment, it can be determined whether the iterative training of the first model is complete. In an optional embodiment, it can be determined that the iterative training of the first model is complete when the number of iterations is greater than a certain threshold, such as a third preset threshold, and the network loss value is less than or equal to a certain threshold, such as a fourth preset threshold. The third preset threshold may be greater than or equal to a second preset threshold, and the fourth preset threshold may be less than or equal to a first preset threshold.

[0126] Once the iterative training of the first model is complete, the first model that meets the second preset condition can be identified as a candidate model, meaning these models can be saved to obtain at least two candidate models. The candidate models may include at least one of the following:

[0127] The first model updated during iterative training with an iteration number greater than or equal to the second preset threshold; for example, if the second preset threshold is 10 and the third preset threshold is 15, then the first model updated during iterative training with an iteration number of 10 to 15 are all candidate models.

[0128] The first model updated during iterative training when the network loss value is less than or equal to the first preset threshold; for example, if the first preset threshold is 0.2 and the fourth preset threshold is 0.01, then the first model updated during iterative training when the network loss value decreases from 0.2 to 0.01 is a candidate model.

[0129] These candidate models can be validated using a validation set to determine their merits. This allows for the selection of the best-performing candidate model from at least two options as the target model. This addresses the issue of overfitting and underfitting in the first model during iterative training, resulting in a target model that is proven to be the best performer for processing multimedia data.

[0130] Optionally, the step of selecting the target model from at least two candidate models includes:

[0131] Obtain the second dataset;

[0132] The second dataset is input into each of the at least two candidate models for processing to obtain at least two third prediction data;

[0133] The target model is obtained by screening from at least two candidate models based on at least two of the third prediction data.

[0134] In this embodiment, the second dataset serves as a validation set, which also includes historical multimedia data from the target scenario. The historical multimedia data from the target scenario can be divided into two parts: a first dataset for training and a second dataset for validation.

[0135] For example, in a smart city scenario, RGB images captured historically by roadside devices can be obtained and divided into two sets: the first dataset and the second dataset.

[0136] The second dataset can be input into each candidate model for processing to obtain at least two third prediction data. Specifically, candidate models can be loaded sequentially, the second dataset can be input into the candidate models for processing, and the third prediction data can be output.

[0137] The processing methods vary depending on the target scenario. For example, in a smart city scenario, processing may refer to image segmentation; in a remote sensing detection scenario, processing may refer to target detection; and in an industrial production scenario, processing may refer to product classification.

[0138] Next, the third predicted data is compared with the third identifier data of historical multimedia data in the second dataset to determine the difference between the third predicted data and the third identifier data. This difference is then compared with the difference values ​​corresponding to previously loaded candidate models. If the difference is smaller, the target model is replaced with the currently loaded candidate model. This process continues until all candidate models are loaded, at which point the target model is output. This target model is the one with the smallest difference value among all candidate models. This approach addresses the problem of poor model training performance caused by overfitting and underfitting during the iterative training of the first model, ensuring that the resulting target model is the one that has been verified to perform best in processing multimedia data.

[0139] Second Embodiment

[0140] like Figure 2 As shown, this disclosure provides a model training apparatus 200, comprising:

[0141] The acquisition module 201 is used to acquire a first dataset under the target scene, the first dataset including M historical multimedia data under the target scene, where M is an integer greater than 1;

[0142] The first determining module 202 is used to determine M first sampling probabilities corresponding to the M historical multimedia data based on the first prediction data. The first prediction data is the prediction data output during the Kth iteration training of the first model based on the first training dataset. The first training dataset includes historical multimedia data selected from the first dataset. The first sampling probability indicates the probability that the historical multimedia data is selected in the first dataset, and K is a positive integer.

[0143] The selection module 203 is used to select N historical multimedia data from the first dataset based on the M first sampling probabilities to obtain a second training dataset, where N is an integer greater than 1;

[0144] The iterative training module 204 is used to perform the (K+1)th iteration training on the first model based on the second training dataset to obtain the second prediction data.

[0145] The second determining module 205 is used to determine a target model trained by the first model based on the second prediction data, wherein the target model is used to process multimedia data in the target scene.

[0146] Optionally, the first training dataset further includes first identifier data of historical multimedia data in the first training dataset, and the first determining module 202 includes:

[0147] The first determining submodule is used to determine the difference value between the first predicted data and the first identification data;

[0148] The second determining submodule is used to determine the M first sampling probabilities based on the difference values.

[0149] Optionally, the second determining submodule includes:

[0150] The first determining unit is used to determine the value of historical multimedia data in the first training dataset based on the difference value.

[0151] The second determining unit is used to determine the M first sampling probabilities based on the value.

[0152] Optionally, the second determining unit is specifically used for:

[0153] Based on the value, a second sampling probability of historical multimedia data in the first training dataset is determined;

[0154] Based on the second sampling probability, the M third sampling probabilities corresponding to the M historical multimedia data are adjusted to obtain the M first sampling probabilities;

[0155] The M third sampling probabilities are used to obtain the first training dataset.

[0156] Optionally, the device further includes:

[0157] The third determining module is used to determine the M third sampling probabilities based on the number of historical multimedia data in the first dataset when K is 1.

[0158] Optionally, the second training dataset further includes second identifier data of historical multimedia data in the second training dataset, and the second determining module 205 includes:

[0159] The third determining submodule is used to determine the network loss value of the first model in the K+1th iteration of training based on the second predicted data and the second identification data.

[0160] The update submodule is used to update the network parameters of the first model based on the network loss value;

[0161] The fourth determining submodule is used to determine the target model based on the first model updated in the K+1th iteration training, provided that the network loss value satisfies the first preset condition.

[0162] The first preset condition includes at least one of the following:

[0163] The network loss value is less than or equal to a first preset threshold;

[0164] The number of iterations of the first model is greater than or equal to the second preset threshold.

[0165] Optionally, the fourth determining submodule includes:

[0166] The third determining unit is used to determine the first model that meets the second preset condition as a candidate model after the iterative training of the first model is completed, so as to obtain at least two candidate models.

[0167] A filtering unit is used to filter the target model from at least two candidate models;

[0168] The second preset condition includes at least one of the following:

[0169] The first model updated during iterative training when the number of iterations is greater than or equal to the second preset threshold;

[0170] The first model updated during iterative training where the network loss value is less than or equal to the first preset threshold.

[0171] Optionally, the filtering unit is specifically used for:

[0172] Obtain the second dataset;

[0173] The second dataset is input into each of the at least two candidate models for processing to obtain at least two third prediction data;

[0174] The target model is obtained by screening from at least two candidate models based on at least two of the third prediction data.

[0175] The model training apparatus 200 provided in this disclosure can implement all the processes implemented in the model training method embodiments and can achieve the same beneficial effects. To avoid repetition, it will not be described again here.

[0176] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0177] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0178] Figure 3 A schematic block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0179] like Figure 3 As shown, device 300 includes a computing unit 301, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 302 or a computer program loaded from storage unit 308 into random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via bus 304. Input / output (I / O) interface 305 is also connected to bus 304.

[0180] Multiple components in device 300 are connected to I / O interface 305, including: input unit 306, such as keyboard, mouse, etc.; output unit 307, such as various types of monitors, speakers, etc.; storage unit 308, such as disk, optical disk, etc.; and communication unit 309, such as network card, modem, wireless transceiver, etc. Communication unit 309 allows device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0181] The computing unit 301 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above, such as model training methods. For example, in some embodiments, the model training method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on device 300 via ROM 302 and / or communication unit 309. When the computer program is loaded into RAM 303 and executed by the computing unit 301, one or more steps of the model training method described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to perform model training methods by any other suitable means (e.g., by means of firmware).

[0182] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0186] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0187] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0188] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0189] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A model training method, comprising: obtaining a first data set in a target scene, the first data set comprising M historical multimedia data in the target scene, M being an integer greater than 1; determining M first sampling probabilities corresponding to the M historical multimedia data based on first prediction data, the first prediction data being prediction data output in Kth iteration training of a first model based on a first training data set, the first training data set comprising historical multimedia data selected from the first data set, the first sampling probability indicating a probability of the historical multimedia data being selected in the first data set, K being a positive integer; selecting N historical multimedia data from the first data set based on the M first sampling probabilities to obtain a second training data set, N being an integer greater than 1; performing (K+1)th iteration training of the first model based on the second training data set to obtain second prediction data; determining a target model trained by the first model based on the second prediction data, the target model being used for processing multimedia data in the target scene; the second training data set further comprising second identification data of historical multimedia data in the second training data set, and the determining of the target model trained by the first model based on the second prediction data comprises: determining a network loss value of the first model in the (K+1)th iteration training based on the second prediction data and the second identification data; updating network parameters of the first model based on the network loss value; in a case where it is determined that a first preset condition is met based on the network loss value, determining the target model based on the first model updated in the (K+1)th iteration training; wherein the first preset condition comprises at least one of: the network loss value is less than or equal to a first preset threshold value; an iteration number of the first model is greater than or equal to a second preset threshold value.

2. The method of claim 1, wherein, the first training data set further comprises first identification data of historical multimedia data in the first training data set, and the determining of the M first sampling probabilities based on the first prediction data comprises: determining a difference value between the first prediction data and the first identification data; determining the M first sampling probabilities based on the difference value.

3. The method of claim 2, wherein, The determining of the M first sampling probabilities based on the difference value comprises: determining a value of historical multimedia data in the first training data set based on the difference value; determining the M first sampling probabilities based on the value.

4. The method of claim 3, wherein, The determining of the M first sampling probabilities based on the value comprises: determining a second sampling probability of historical multimedia data in the first training data set based on the value; adjusting M third sampling probabilities corresponding to the M historical multimedia data based on the second sampling probability to obtain the M first sampling probabilities; wherein the M third sampling probabilities are used to obtain the first training data set.

5. The method of claim 4, before the determining the M first sampling probabilities corresponding to the M historical multimedia data based on the first prediction data, further comprising: in a case that K is 1, determining the M third sampling probabilities based on a quantity of historical multimedia data in the first data set.

6. The method of claim 1, wherein, the determining the target model based on the first model updated in the K+1th iteration training, comprising: in a case that the iteration training of the first model is completed, determining the first model satisfying a second preset condition as a candidate model, to obtain at least two candidate models; selecting the target model from the at least two candidate models; wherein the second preset condition comprises at least one of: the first model updated in iteration training with an iteration number greater than or equal to the second preset threshold value; the first model updated in iteration training with a network loss value less than or equal to the first preset threshold value.

7. The method of claim 6, wherein, the selecting the target model from the at least two candidate models, comprising: obtaining a second data set; inputting the second data set into each candidate model of the at least two candidate models respectively for processing, to obtain at least two third prediction data; selecting the target model from the at least two candidate models based on the at least two third prediction data.

8. A model training apparatus, comprising: an obtaining module, configured to obtain a first data set in a target scene, the first data set comprising M historical multimedia data in the target scene, M being an integer greater than 1; a first determining module, configured to determine M first sampling probabilities corresponding to the M historical multimedia data based on first prediction data, the first prediction data being prediction data output in Kth iteration training of a first model based on a first training data set, the first training data set comprising historical multimedia data selected from the first data set, the first sampling probability indicating a probability of the historical multimedia data being selected in the first data set, K being a positive integer; a selecting module, configured to select N historical multimedia data from the first data set based on the M first sampling probabilities, to obtain a second training data set, N being an integer greater than 1; an iteration training module, configured to perform K+1th iteration training of the first model based on the second training data set, to obtain second prediction data; a second determining module, configured to determine a target model trained by the first model based on the second prediction data, the target model being used for processing multimedia data in the target scene; the second training data set further comprising second identification data of historical multimedia data in the second training data set, the second determining module comprising: a third determining submodule, configured to determine a network loss value of the first model in the K+1th iteration training based on the second prediction data and the second identification data; an updating submodule, configured to update network parameters of the first model based on the network loss value; a fourth determining sub-module, configured to determine the target model based on the first model updated in the K+1th iteration training, in a case that the network loss value meets a first preset condition; The first preset condition includes at least one of the following: The network loss value is less than or equal to a first preset threshold value; The number of iterations of the first model is greater than or equal to a second preset threshold value.

9. The apparatus of claim 8, wherein, The first training data set further includes first identification data of historical multimedia data in the first training data set, and the first determining module includes: a first determining sub-module, configured to determine a difference value between the first prediction data and the first identification data; a second determining sub-module, configured to determine the M first sampling probabilities based on the difference value.

10. The apparatus of claim 9, wherein, The second determining sub-module includes: a first determining unit, configured to determine a value of the historical multimedia data in the first training data set based on the difference value; a second determining unit, configured to determine the M first sampling probabilities based on the value.

11. The apparatus of claim 10, wherein, The second determining unit is specifically configured to: determine a second sampling probability of the historical multimedia data in the first training data set based on the value; adjust M third sampling probabilities corresponding to the M historical multimedia data based on the second sampling probability, to obtain the M first sampling probabilities; The M third sampling probabilities are used to obtain the first training data set.

12. The apparatus of claim 11, further comprising: a third determining module, configured to determine the M third sampling probabilities based on a number of historical multimedia data in the first data set, in a case that K is 1.

13. The apparatus of claim 8, wherein, The fourth determining sub-module includes: a third determining unit, configured to determine a first model meeting a second preset condition as a candidate model in a case that the iteration training of the first model is completed, to obtain at least two candidate models; a screening unit, configured to screen the target model from the at least two candidate models; The second preset condition includes at least one of the following: a first model updated in iteration training with a number of iterations greater than or equal to the second preset threshold value; a first model updated in iteration training with a network loss value less than or equal to the first preset threshold value.

14. The apparatus of claim 13, wherein, The screening unit is specifically configured to: obtain a second data set; input the second data set into each of the at least two candidate models for processing, to obtain at least two third prediction data; screen the target model from the at least two candidate models based on the at least two third prediction data.

15. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.

Images