Image classification, model training method, device, storage medium and computer program

Abstract

The embodiment of the application discloses an image classification method, a model training method, equipment, a storage medium and a computer program, and belongs to the field of image processing. In the method, a target image is processed by a current neural network model to obtain a current classification result, the current neural network model is a neural network model i corresponding to the maximum probability in a selection result of a neural network model a, and the selection result includes probabilities corresponding to p neural network models in m neural network models. A current integration result is determined based on the current classification result, and the category of the target image is determined based on the current integration result. That is, the category of the target image is determined by selecting part of the neural network models from the m neural network models to process the target image, without obtaining the classification result of each neural network model in the m neural network models, thereby greatly reducing the computing cost and improving the image classification efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular to an image classification, model training method, device, storage medium, and computer program. Background Technology

[0002] In recent years, deep learning has been widely used in the field of image classification. As an important branch of deep learning, neural networks have greatly improved the accuracy of image classification results after applying them due to their super fitting ability.

[0003] A proposed image classification method involves pre-training multiple neural network models. During image classification, each of these models determines a classification result, resulting in multiple classification outcomes. These multiple results are then combined and processed to obtain the final classification result.

[0004] However, the previous method required obtaining the classification results of each neural network model, which multiplied the computational load and greatly reduced the efficiency of image classification. Summary of the Invention

[0005] This application provides an image classification and model training method, device, storage medium, and computer program, which can solve the problem of low image classification efficiency in related technologies. The technical solution is as follows:

[0006] Firstly, an image classification method is provided. In this method, a target image to be classified is processed by a current neural network model to obtain the current classification result output by the current neural network model. The current classification result includes the probability that the target image belongs to each of the multiple categories. The current neural network model is neural network model i, which corresponds to the highest probability among the selection results output by neural network model a. The selection results output by neural network model a include the probabilities corresponding to p neural network models among m trained neural network models. The p neural network models are neural network models allowed to be used to process the target image after neural network model a, including neural network model i. The m neural network models include neural network model a, where m is an integer greater than 1, p is an integer greater than or equal to 1, and p is less than m. Based on the current classification result, a current integration result is determined, which includes the integrated probability of the target image belonging to each of the multiple categories. Based on the current integration result, the category of the target image is determined.

[0007] In this embodiment of the application, among the m trained neural network models, one or more neural network models are allowed to be used to process the target image after neural network model a, and these one or more neural network models are other neural network models used to process the target image after neural network model a. That is, p is an integer greater than or equal to 1, and p is less than m.

[0008] Since the selection result output by neural network model a includes the probability of the neural network model that can be used to process the target image after neural network model a among the m trained neural network models, and the current neural network model is the neural network model i with the highest probability in the selection result output by neural network model a, it can be determined that the classification result output by neural network model a does not meet the conditions. Therefore, the neural network model i with the highest probability in the selection result output by neural network model a is determined as the current neural network model, and then the target image is processed by the current neural network model.

[0009] In other words, this embodiment processes the target image in an iterative manner. If the classification result obtained in the previous iteration does not meet the conditions, the selection result from the previous iteration can be used to determine the neural network model to be used for processing the target image next. Since the classification accuracy of the neural network model used for processing the target image next time is higher than that of the neural network model used for processing the target image in the previous iteration, this iterative processing improves the accuracy of image classification. That is, based on the properties of the target image, this embodiment can dynamically select a portion of the neural network models from the m neural network models for image classification, eliminating the need to classify the image using each neural network model, thus reducing the computational load. Furthermore, since only a portion of the neural network models are selected, only the parameters of a portion of the neural network models need to be read into memory, rather than the parameters of all neural network models, avoiding the time-consuming problem of memory access.

[0010] In some embodiments, the current neural network model can also output the current selection result, which includes the probabilities corresponding to d neural network models out of m neural network models. The d neural network models are those allowed to be used to process the target image after neural network model i, where d is an integer greater than or equal to 1 and less than m. That is, the embodiments of this application can process the target image using the current neural network model in two ways. These two implementation methods will be described below.

[0011] The first implementation involves inputting the target image into the current neural network model to obtain the current classification result and the current selection result output by the current neural network model.

[0012] In other words, in the first implementation, any one of the m neural network models is taken as the current neural network model. After the target image is input into the current neural network model, the current neural network model can not only output the current classification result of the target image, but also determine the probability of the neural network model that can be used to process the target image after the current neural network model, that is, the current selection result.

[0013] The higher the probability of the neural network model that can be used to process the target image after the current neural network model, the higher the classification accuracy of the target image by this neural network model. In general, the classification accuracy of neural network model i for the target image is higher than that of neural network model a for the target image.

[0014] The second implementation involves inputting the target image and the current iteration number into the current neural network model, and then processing the target image through the current neural network model.

[0015] There are two scenarios for the second implementation method, which will be introduced separately below.

[0016] In the first case, the target image and the current iteration number are input into the current neural network model to obtain the current classification result output by the current neural network model.

[0017] In other words, in the first case, the current iteration number input to the current neural network model is less than the iteration number threshold, and the difference between the current iteration number threshold and the threshold is 1. Thus, after processing the target image by the current neural network model, the current iteration number has reached the threshold, and no further iterations are needed. Therefore, the current neural network model can simply output the current classification result.

[0018] In the second case, the target image and the current iteration number are input into the current neural network model to obtain the current classification result and the current selection result output by the current neural network model.

[0019] In other words, in the second case, the current iteration number input to the current neural network model is less than the iteration threshold, and the difference between the current iteration number and the iteration threshold is greater than 1. Thus, after processing the target image by the current neural network model, the current iteration number has not yet reached the threshold, and further iterations may be needed. Therefore, the current neural network model needs to output the current classification result and the current selection result.

[0020] In some embodiments, the average probability of a corresponding category in multiple classification results can be determined as the probability of the corresponding category in the current integrated result. These multiple classification results include the classification results output by neural network model a and neural network model i. Alternatively, the current classification result can be directly determined as the current integrated result.

[0021] In other words, the probability of the first category in the current integration result includes the average probability of the first category among multiple classification results, including the classification results output by neural network model a and neural network model i. Alternatively, the probability of the first category in the current integration result includes the probability of the first category among the classification results output by neural network model i. Here, the first category is one of the multiple categories.

[0022] Based on the above description, there are two implementation methods for processing target images using the current neural network model. Under different implementation methods, the category of the target image is determined differently based on the current integration result. Therefore, the following will also explain these two implementation methods.

[0023] In the first implementation, when processing the target image using the current neural network model according to the first implementation method, determining the category of the target image based on the current integration result includes: if the current integration result satisfies the first convergence condition or the current neural network model satisfies the second convergence condition, then the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not satisfy the first convergence condition and the current neural network model does not satisfy the second convergence condition, then the neural network model corresponding to the highest probability in the current selection result is taken as the current neural network model, and the target image is processed again according to the above method.

[0024] In the first implementation, each neural network model outputs both classification and selection results. Therefore, after determining the current integration result based on the current classification result output by the current neural network model, it is necessary to determine whether the current integration result satisfies the first convergence condition and whether the current neural network model satisfies the second convergence condition. If the current integration result satisfies the first convergence condition or the current neural network model satisfies the second convergence condition, it indicates that the reliability of the current integration result is high, and the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not satisfy the first convergence condition and the current neural network model does not satisfy the second convergence condition, it indicates that the reliability of the current integration result is low. In this case, the neural network model corresponding to the highest probability in the current selection result can be used as the current neural network model, and the target image can be processed again according to the above method.

[0025] As an example, the current fusion result satisfies the first convergence condition if the confidence level of the current fusion result is greater than a confidence threshold. The current neural network model satisfies the second convergence condition if the number of iterations of the current neural network model is greater than or equal to an iteration threshold. Of course, the current fusion result satisfying the first convergence condition can also include other conditions, and similarly, the current neural network model satisfying the second convergence condition can also include other conditions.

[0026] If the current integration result satisfies the first convergence condition, including that the confidence level of the current integration result is greater than a confidence threshold, after determining the current integration result based on the current classification result, the confidence level of the current integration result can also be determined. The process of determining the confidence level of the current integration result includes: determining the highest probability in the current integration result as the confidence level of the current integration result. Alternatively, the target image and the current integration result can be input into a confidence network model to obtain the confidence level of the current integration result output by the confidence network model.

[0027] The second implementation method involves two scenarios when processing the target image using the current neural network model. These scenarios are divided into two cases, which will be explained below. These two scenarios correspond to the two scenarios in the second implementation method described above.

[0028] In the first case, the category corresponding to the highest probability in the current integration result is determined as the category of the target image.

[0029] In the first case, after processing the target image using the current neural network model, the current iteration number has reached the iteration number threshold, indicating that the reliability of the current integration result is high. Therefore, the category corresponding to the highest probability in the current integration result can be directly determined as the category of the target image, without needing to judge whether the current integration result meets the first convergence condition.

[0030] In the second scenario, if the current integration result meets the first convergence condition, then the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not meet the first convergence condition, then the neural network model corresponding to the highest probability in the current selection result is used as the current neural network model, and the target image is processed again according to the above method.

[0031] In the second scenario, since the current iteration count has not yet reached the threshold after processing the target image using the current neural network model, the reliability of the current integration result is uncertain. Therefore, it is necessary to determine whether the current integration result meets the first convergence condition. If the current integration result meets the first convergence condition, its reliability is considered high, and the category corresponding to the highest probability in the current integration result can be determined as the category of the target image. If the current integration result does not meet the first convergence condition, its reliability is considered low, and the neural network model corresponding to the highest probability in the current selection result can be used as the current neural network model, and the target image can be processed again using the method described above.

[0032] Optionally, before classifying the target image using the aforementioned m neural network models, m training neural network models can be trained to obtain the aforementioned m neural network models. That is, n sample images and n sample labels are obtained, with each sample label corresponding one-to-one with the n sample images; based on the n sample images and n sample labels, the gradients of each parameter in each of the m training neural network models are determined; based on the gradients of each parameter in the m training neural network models, the m training neural network models are updated to obtain the trained m neural network models.

[0033] For each of the m training neural network models, the method for determining the gradient of each parameter of each training neural network model is the same. The following explanation will take training neural network model k as an example.

[0034] In some embodiments, the gradients of each parameter of the neural network model k to be trained can be determined by: determining the training function value corresponding to the neural network model k to be trained based on n sample images and n sample labels; and determining the gradients of each parameter of the neural network model k to be trained based on the training function value corresponding to the neural network model k to be trained.

[0035] Based on the above description, the training function values ​​corresponding to the neural network model k to be trained include n classification function values, n weight function values, n label function values, n classification loss function values, and n selection weight function values. Each of the n classification function values, n weight function values, n label function values, n classification loss function values, and n selection weight function values ​​corresponds one-to-one with each of the n sample images. The process of determining these multiple function values ​​will be explained separately below.

[0036] The process of determining n classification function values

[0037] Since the classification function value for each sample image is determined in the same way, a sample image can be selected from the n sample images. The following steps are performed to determine the classification function value for the selected sample image, continuing until the classification function value for each sample image is determined: Input the selected sample image into the neural network model k to be trained, and obtain the classification result output by the neural network model k. Based on the sample label corresponding to the selected sample image and the classification result output by the neural network model k, determine the classification function value corresponding to the selected sample image.

[0038] The classification function value can be the cross-entropy loss function value, or also known as the task-related function.

[0039] The process of determining the values ​​of n weight functions

[0040] In this embodiment, the n weight function values ​​can be implemented based on rules or based on meta-learning. These two implementation methods will be described below.

[0041] In the first implementation, the n weight function values ​​are determined based on rules. This approach ensures that, when the next training neural network model to process an image is chosen as model k, model k will have higher weight function values ​​for that image.

[0042] Regarding the first implementation method, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (11)-(13).

[0043] (11) Using b neural network models to be trained, determine the standard probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t standard probabilities, t=mb, b neural network models to be trained are the models that are allowed to process n sample images before the m neural network models to be trained, and t neural network models to be trained include the model k and the models that are allowed to process n sample images after the model k in the m neural network models to be trained.

[0044] In some embodiments, using b training neural network models, the average probability that each of the t training neural network models is used to process each of the n sample images is determined, resulting in n*t average probabilities. The average probabilities corresponding to the same training neural network model among the n*t average probabilities are then grouped into a probability set, resulting in t probability sets. Based on the n*t average probabilities and the t probability sets, the n*t average probabilities are standardized to obtain n*t standard probabilities.

[0045] The method for determining the average probability that each of the t training neural network models is used to process each of the n sample images is the same, using b training neural network models. Therefore, a sample image can be selected from the n sample images, and a training neural network model can be selected from the t training neural network models. Using the b training neural network models, the average probability that the selected training neural network model is used to process the selected sample image is determined as follows, until n*t average probabilities are determined: For each of the b training neural network models, determine the probability that the selected training neural network model is used to process the selected sample image, thus obtaining b probabilities. The average of these b probabilities is then determined to obtain the average probability that the selected training neural network model is used to process the selected sample image.

[0046] As an example, for each of the b trainable neural network models, a selected sample image can be input into that model to obtain its output selection result. This selection result includes the probability that the selected model is used to process the selected sample image; that is, the probability that the next neural network model to classify the selected sample image is the selected model. Thus, for each of the b trainable neural network models, the probability that the selected model is used to process the selected sample image can be obtained, resulting in b probabilities. Then, the average of these b probabilities can be determined, and this average is taken as the average probability that the selected model is used to process the selected sample image.

[0047] The standardization process for each average probability within each of the t probability sets is the same. Therefore, a probability set can be selected from the t probability sets, and each average probability in the selected probability set can be standardized according to the following operations until n*t standard probabilities are determined: determine the mean and standard deviation of the average probabilities in the selected probability set, and determine the standard probability corresponding to each average probability in the selected probability set based on the average probability, mean, and standard deviation of the selected probability set.

[0048] (12) Based on n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images.

[0049] In some embodiments, the first weight corresponding to each of the n sample images can be determined by the following steps (a)-(g).

[0050] (a) Construct a standard probability set from n*t standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set.

[0051] In some embodiments, a standard probability can be selected from the set of standard probabilities, and the relative probability value corresponding to the selected standard probability can be determined by the following operations until the relative probability value corresponding to each standard probability is determined: from the other standard probabilities in the set of standard probabilities besides the selected standard probability, multiple standard probabilities corresponding to the same sample image as the selected standard probability are determined, the maximum standard probability among the multiple standard probabilities is determined, and the difference between the selected standard probability and the maximum standard probability is determined as the relative probability value corresponding to the selected standard probability.

[0052] (b) Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set.

[0053] The relative probability values ​​corresponding to the standard probabilities in the standard probability set are compared to determine the largest relative probability value among these relative probability values.

[0054] (c) Determine the sample image corresponding to the maximum relative probability value from n sample images, and determine the neural network model to be trained corresponding to the maximum relative probability value from t neural network models to be trained.

[0055] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, and the set of standard probabilities is composed of n*t standard probabilities, one standard probability in the set of standard probabilities corresponds to one sample image in the n sample images and one trainable neural network model in the t trainable neural network models. The relative probability value is determined based on the standard probabilities; therefore, each relative probability value also corresponds to one sample image in the n sample images and one trainable neural network model in the t trainable neural network models. Thus, after determining the maximum relative probability value, the sample image corresponding to the maximum relative probability value can be determined from the n sample images, and the trainable neural network model corresponding to the maximum relative probability value can be determined from the t trainable neural network models.

[0056] (d) Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value.

[0057] In some embodiments, if the neural network model to be trained corresponding to the maximum relative probability value is neural network model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as... If the neural network model to be trained corresponding to the maximum relative probability value is not model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as... Where a is a real number that satisfies a≥0 and a≤1.

[0058] (e) Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set.

[0059] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, a sample image will correspond to t standard probabilities. Thus, after determining the sample image corresponding to the maximum relative probability value, the t standard probabilities corresponding to that sample image can be removed from the standard probability set. At this point, the standard probability set still contains n*(t-1) standard probabilities.

[0060] (f) If the standard probability set is not empty after removal, update the number of model determinations, which refers to the number of times the model corresponding to the maximum relative probability value is determined from t models to be trained.

[0061] As an example, the number of model determinations can be increased by 1. That is, each time a maximum probability relative value is determined through one iteration, and the corresponding neural network model for that maximum probability relative value is selected from t trainable neural network models, the number of model determinations can be increased by 1. Of course, this is just one exemplary update method, and other methods are also possible.

[0062] (g) If the number of determinations of the updated model is greater than or equal to n / t, remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability from the standard probability set, and return to the step in step (a) to determine the relative probability value corresponding to each standard probability in the standard probability set.

[0063] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, one of the t trainable neural network models will correspond to n standard probabilities. Thus, the n standard probabilities corresponding to the trainable neural network model with the maximum relative probability can be removed from the set of standard probabilities. At this point, the set of standard probabilities still contains (n-1)*(t-1) standard probabilities.

[0064] (13) Normalize the first weight corresponding to each of the n sample images to obtain the weight function value corresponding to each of the n sample images.

[0065] The second implementation method uses meta-learning to determine the n weight function values. This method allows the neural network model k to have a lower classification loss on certain images.

[0066] Regarding the second implementation method, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (21)-(27).

[0067] (21) Using b neural network models to be trained, select h sample images from n sample images to be processed by the neural network model k next. The b neural network models to be trained are the models that are allowed to be used to process n sample images before the neural network model k among the m neural network models to be trained, and h is less than n.

[0068] In some embodiments, a sample image is selected from the n sample images, and a training neural network model is selected from the b training neural network models. The selected sample image is input into the selected training neural network model to obtain the selection result output by the selected training neural network model. This selection result includes the probability of the training neural network model among the m training neural network models that is allowed to process the selected sample image after the selected training neural network model. If the training neural network model corresponding to the highest probability in the selection result is training neural network model k, then the selected sample image is determined as the sample image to be processed by training neural network model k next. By traversing all the sample images and each of the b training neural network models in the above method, h sample images can be obtained.

[0069] (22) The prediction weights for each of the n sample images are determined by using the meta-network model.

[0070] In some embodiments, each of the n sample images can be input into the meta-network model to obtain the prediction weights corresponding to the sample image output by the meta-network model. That is, for any sample image among the n sample images, after inputting the sample image into the meta-network model, the prediction weights corresponding to the sample image output by the meta-network model can be obtained.

[0071] (23) Based on the prediction weights corresponding to each of the n sample images and the classification function values ​​corresponding to each of the n sample images, determine the weighted loss values ​​of the n sample images.

[0072] (24) Update the neural network model k to be trained based on the weighted loss values ​​of the n sample images.

[0073] (25) Input the selected h sample images into the updated neural network model k to obtain the total loss value of the updated neural network model k on the h sample images.

[0074] (26) Update the meta-network model based on the total loss value of the updated neural network model k on the h sample images.

[0075] (27) Determine the weight function value corresponding to each of the n sample images by using the updated meta-network model.

[0076] In some embodiments, each of the n sample images can be input into the updated meta-network model to obtain the weight function value corresponding to the corresponding sample image output by the updated meta-network model. That is, for any sample image among the n sample images, after inputting the sample image into the updated meta-network model, the weight function value corresponding to the sample image output by the updated meta-network model can be obtained.

[0077] The above method for determining the n weight function values ​​is only one of the meta-learning-based methods. In practical applications, there are other meta-learning-based methods. For example, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (31)-(37).

[0078] (31) Using b neural network models to be trained, select h sample images from n sample images to be processed by the neural network model k next. The b neural network models to be trained are the models that are allowed to be used to process n sample images before the neural network model k among the m neural network models to be trained, and h is less than n.

[0079] (32) Determine the initial weights for each of the n sample images.

[0080] In some embodiments, an initial weight can be set for each of the h sample images. For example, the initial weight can be 0.

[0081] (33) Determine the weighted loss value of the n sample images by using the initial weights corresponding to each sample image in the n sample images.

[0082] (34) Update the neural network model k to be trained based on the weighted loss values ​​of the n sample images.

[0083] (35) Input the selected h sample images into the updated neural network model k to obtain the total loss value of the updated neural network model k on the h sample images.

[0084] (36) Based on the total loss value of the updated neural network model k on the h sample images and the initial weights corresponding to each sample image in the n sample images, determine the second weights corresponding to each sample image in the n sample images.

[0085] (37) Normalize the second weight corresponding to each of the n sample images to obtain the weight function value corresponding to each of the n sample images.

[0086] The process of determining the n label function values

[0087] In this embodiment, the label function value corresponding to the sample image is related to the accuracy of the classification result of the sample image by the neural network model to be trained after the neural network model k that is allowed to process the sample image. That is, the neural network model to be trained is used to determine the label function value corresponding to the sample image, then the classification result of the sample image by that neural network model has higher accuracy, and the classification result of the sample image by other neural network models to be trained has lower accuracy. Among them, the n label function values ​​can be determined based on n sample images and n sample labels through the following steps (41)-(42).

[0088] (41) By using each of the s training neural network models, determine the standard probability that each sample image in the n sample images belongs to the corresponding sample label, so as to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0089] In some embodiments, the probability that each sample image in n sample images belongs to the corresponding sample label can be determined for each of the s training neural network models, resulting in n*s probabilities. The average probability corresponding to the same training neural network model among the n*s probabilities is then divided into a probability set, resulting in s probability sets. Based on the n*s probabilities and the s probability sets, the n*s probabilities are standardized to obtain n*s standard probabilities.

[0090] In this system, each of the *s* training neural network models determines the standard probability of each sample image belonging to its corresponding sample label from the *n* sample images in the same way. Therefore, we can select one training neural network model from the *s* models and one sample image from the *n* sample images. The following steps are performed to determine the probability of the selected sample image belonging to its corresponding sample label using the selected training neural network model, until *n**s probabilities are determined: The selected sample image is input into the selected training neural network model to obtain the classification result output by the selected training neural network model. This classification result includes the probability of the selected sample image belonging to each of multiple categories. These multiple categories can include the sample label corresponding to the selected sample image. In this way, the probability of the selected sample image belonging to its corresponding sample label can be determined using the selected training neural network model.

[0091] (42) Based on n*s standard probabilities, determine the label function value corresponding to each sample image in the n sample images.

[0092] In some embodiments, a standard probability set can be constructed from n*s standard probabilities, and the relative probability value corresponding to each standard probability in the standard probability set is determined. From the relative probability values ​​corresponding to the standard probabilities in the standard probability set, the maximum relative probability value is determined. The sample image corresponding to the maximum relative probability value is determined from n sample images, and the training neural network model corresponding to the maximum relative probability value is determined from s training neural network models. The position number of the training neural network model corresponding to the maximum relative probability value among the s training neural network models is determined as the label function value of the sample image corresponding to the maximum relative probability value. The standard probabilities corresponding to the sample images corresponding to the maximum relative probability value are removed from the standard probability set. If the standard probability set is not empty after removal, the model determination count is updated, which refers to the number of determinations made to determine the training neural network model corresponding to the maximum relative probability value from the s training neural network models. If the updated model determination count is greater than or equal to n / s, the standard probabilities corresponding to the training neural network models corresponding to the maximum relative probability value are removed from the standard probability set, and the process returns to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0093] The process of determining n classification loss function values

[0094] Since the classification loss function value for each sample image is determined in the same way, a sample image can be selected from n sample images. The classification loss function value for the selected sample image is determined by following these steps until the classification loss function value for each sample image is determined: using the training neural network model k, determine the probability that each of the s training neural network models is used to process the selected sample image, so as to obtain s probabilities; based on the label function value corresponding to the selected sample image and the s probabilities, determine the classification loss function value corresponding to the selected sample image.

[0095] In some embodiments, the selected sample image can be input into the neural network model k to be trained to obtain the selection result output by the neural network model k. The selection result includes the probability that each of the s neural network models to be trained is used to process the selected sample image. That is, the next neural network model to classify the selected sample image is the probability of each of the s neural network models to be trained, thus obtaining s probabilities.

[0096] The process of determining n selection weight function values

[0097] In this embodiment, the selection weight function value corresponding to the sample image is related to the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k. That is, the larger the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k, the larger the selection weight function value corresponding to that sample image; conversely, the smaller the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k, the smaller the selection weight function value corresponding to that sample image.

[0098] In some embodiments, the standard probability of each sample image belonging to the corresponding sample label in the n sample images can be determined by each of the s training neural network models, resulting in n*s standard probabilities. Based on the n*s standard probabilities, the selection weight function value corresponding to each sample image in the n sample images is determined.

[0099] The process of determining the selection weight function value for each of the n sample images based on n*s standard probabilities includes: dividing the probabilities corresponding to the same sample image among the n sample images into a probability set, resulting in n probability sets; determining the standard deviation of the standard probabilities included in each of the n probability sets, resulting in n standard deviations; and normalizing each of the n standard deviations to obtain the selection weight function value for each of the n sample images.

[0100] Secondly, a computer device is provided, which has the function of implementing the image classification method behavior described in the first aspect, or has the function of implementing the neural network model training method behavior described in the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described function.

[0101] In one possible implementation, the computer device includes:

[0102] The image processing module is used to process the target image to be classified using the current neural network model in order to obtain the current classification result output by the current neural network model.

[0103] The current classification result includes the probability of the target image belonging to each of the multiple categories. The current neural network model is the neural network model i with the highest probability in the selection result output by neural network model a. The selection result output by neural network model a includes the probability corresponding to p neural network models among the m trained neural network models. The p neural network models are the neural network models allowed to be used to process the target image after neural network model a. The m neural network models include neural network model i and neural network model a, where m is an integer greater than 1 and p is an integer greater than or equal to 1.

[0104] The integration result determination module is used to determine the current integration result based on the current classification result. The current integration result includes the probability of the target image belonging to each of the multiple categories after integration.

[0105] The image category determination module is used to determine the category of the target image based on the current integration results.

[0106] The image processing module, the integration result determination module, and the image category determination module described above can perform the corresponding functions in the image classification method example in the first aspect above. For details, please refer to the detailed description in the method example, which will not be repeated here.

[0107] In another possible implementation, the computer device includes:

[0108] The sample acquisition module is used to acquire n sample images and n sample labels, with each of the n sample labels corresponding one-to-one with the n sample images;

[0109] The gradient determination module is used to determine the gradient of each parameter of each of the m training neural network models based on n sample images and n sample labels.

[0110] The model update module is used to update the m neural network models to be trained based on the gradients of each parameter of the m neural network models to be trained, so as to obtain the m trained neural network models.

[0111] The sample acquisition module, gradient determination module, and model update module described above can perform the corresponding functions in the neural network model training method example in the first aspect above. For details, please refer to the detailed description in the method example, which will not be repeated here.

[0112] Thirdly, a computer device is provided, comprising a processor and a memory, the memory being used to store computer program instructions for performing the methods provided in the first aspect. The processor is configured to execute the computer program instructions stored in the memory to implement the methods described in the first aspect.

[0113] Optionally, the computer device may further include a communication bus for establishing a connection between the processor and the memory.

[0114] Fourthly, a computer-readable storage medium is provided, wherein instructions are stored therein, which, when executed on a computer, cause the computer to perform the steps of the method described in the first aspect.

[0115] Fifthly, a computer program product comprising instructions is provided, which, when executed on a computer, cause the computer to perform the steps of the method described in the first aspect. Alternatively, a computer program is provided that, when executed on a computer, causes the computer device to perform the steps of the method described in the first aspect.

[0116] The technical effects obtained by the second, third, fourth and fifth aspects mentioned above are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here.

[0117] The technical solutions provided in this application can bring at least the following beneficial effects:

[0118] In the embodiments of this application, for the m trained neural network models, when performing image classification, a portion of the neural network models can be selected from the m neural network models to process the target image based on the target image to be classified, thereby determining the category of the target image. It is not necessary to obtain the classification result of each of the m neural network models, which greatly reduces the computational overhead and improves the efficiency of image classification. Attached Figure Description

[0119] Figure 1 This is a schematic diagram of an implementation environment provided in an embodiment of this application;

[0120] Figure 2 This is a schematic diagram of an exemplary implementation environment provided in the embodiments of this application;

[0121] Figure 3 This is a schematic diagram of an interface for enabling the album categorization function provided in an embodiment of this application;

[0122] Figure 4 This is a schematic diagram of an interface for taking photos provided in an embodiment of this application;

[0123] Figure 5 This is a schematic diagram of an album categorization interface provided in an embodiment of this application;

[0124] Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application;

[0125] Figure 7 This is a flowchart of an image classification method provided in an embodiment of this application;

[0126] Figure 8 This is a schematic diagram of an operational example provided in an embodiment of this application;

[0127] Figure 9 This is a schematic diagram of another operating example provided in the embodiments of this application;

[0128] Figure 10 This is a flowchart illustrating an exemplary image classification method provided in an embodiment of this application;

[0129] Figure 11 This is a schematic diagram illustrating image classification accuracy under different computational loads provided in an embodiment of this application;

[0130] Figure 12 This is a flowchart of a neural network model training method provided in an embodiment of this application;

[0131] Figure 13 This is a flowchart illustrating an exemplary method for training a neural network model according to an embodiment of this application;

[0132] Figure 14 This is a schematic diagram of the structure of an image classification device provided in an embodiment of this application;

[0133] Figure 15 This is a schematic diagram of the structure of a training device for a neural network model provided in an embodiment of this application. Detailed Implementation

[0134] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0135] To facilitate understanding, the terms used in the embodiments of this application will be explained before providing a detailed explanation.

[0136] Neural network models are complex network systems formed by the extensive interconnection of a large number of simple processing units (called neurons). They are highly complex nonlinear dynamic learning systems. Neural network models possess massive parallelism, distributed storage and processing, self-organization, self-adaptation, and self-learning capabilities, making them particularly suitable for handling imprecise and fuzzy information processing problems that require simultaneous consideration of many factors and conditions.

[0137] Neural network models include convolutional neural network (CNN) models. A CNN is a type of feedforward neural network model that includes convolutional computation and has a deep structure; it is one of the representative algorithms of deep learning. The neural network model in the embodiments of this application can be a CNN model.

[0138] A convolutional neural network (CNN) model includes convolutional layers, pooling layers, fully connected layers, and non-linear activation layers. The convolutional layer is the core of the CNN model, its parameters consisting of a set of learnable kernels. During the feedforward phase, each kernel convolves the input to extract higher-level features. Pooling layers, also known as downsampling, reduce data processing while retaining useful information. Typically, pooling layers select the maximum value from every four neighboring pixels and convert it to a single pixel. The input and output of a fully connected layer are both one-dimensional vectors. Essentially, a fully connected operation is a matrix multiplication operation. The input is a vector, the matrix is ​​the parameters stored in the neural network model, and the output is the product of the vector and the matrix, also a vector. Non-linear activation layers include ReLU (Revised Linear Unit), Sigmoid, and Tanh (Hyperbolic Tangent), all commonly used and recognized non-linear activation functions. These are element-wise operations and do not change the size of the features.

[0139] Parameters of a neural network model: Any neural network model contains a large number of learnable parameters, which are called the parameters of the neural network model.

[0140] The structure of a neural network model is one of its important attributes, affecting its accuracy and efficiency. Representative structures include VGG, ResNets, and MobileNet.

[0141] Training a neural network: A neural network model contains a large number of learnable parameters. Training a neural network model refers to the process of determining the values ​​of the learnable parameters based on the dataset and the training algorithm.

[0142] Neural network inference: After the neural network model has been trained, it is applied to new data to obtain prediction results for the new data (such as the category of the new data). This process is called neural network model inference. In the embodiments of this application, neural network model inference specifically refers to image classification, that is, determining the category of the target image.

[0143] The method provided in this application can be applied to various scenarios. Image classification is just one example; it can also be applied to various inference scenarios such as video recognition, speech recognition, and image signal processing. For any inference scenario, the first data to be inferred can be processed by the current neural network model to obtain the current inference result output by the current neural network model. The current inference result varies depending on the inference scenario. The current neural network model is the neural network model i with the highest probability in the selection result output by neural network model a. The selection result output by neural network model a includes the probability of the neural network model that is allowed to be used to infer the first data after the current neural network model among the m trained neural network models. The m neural network models include neural network model i and neural network model a, where m is an integer greater than 1. The inference result of the first data is determined based on the current inference result.

[0144] Next, this application will use image classification as an example to introduce the implementation environment of this application embodiment.

[0145] Please refer to Figure 1 , Figure 1 This is a schematic diagram illustrating an implementation environment according to an embodiment of this application. The implementation environment includes multiple terminals 101 and a server (e.g., a cloud server) 102. Each of the multiple terminals 101 is connected to the server 102 wirelessly or via a wired connection for communication.

[0146] Among them, regarding Figure 1 As shown in the implementation environment, the embodiments of this application can perform image classification in a variety of different ways.

[0147] In the first implementation, server 102 trains m neural network models to obtain m trained neural network models. Then, server 102 deploys the m trained neural network models on each of the plurality of terminals 101. For each of the plurality of terminals 101, terminal 101 acquires the target image and processes the target image based on the m trained neural network models to determine the category of the target image.

[0148] The target image acquired by the terminal 101 can be an image stored on the terminal 101, or an image obtained by the terminal 101 from capturing images of the external environment. Of course, it can also be acquired through other means, and this embodiment does not limit this. For example, please refer to... Figure 2 The terminal can use its built-in camera to capture images of the external environment to obtain target images. For example... Figure 2Three images are obtained. Then, for each of the three images, each image can be processed based on the m neural network models to determine the category of each image.

[0149] In the second implementation, server 102 trains m neural network models to obtain m trained neural network models. For each of the multiple terminals 101, terminal 101 acquires the target image and sends it to server 102. Server 102 receives the target image sent by terminal 101 and processes the target image based on the m neural network models to determine the category of the target image.

[0150] Of course, besides the two implementation methods mentioned above, image classification can also be performed using other methods. For example, terminal 101 trains m neural network models to obtain m trained neural network models. Then, terminal 101 acquires the target image and processes the target image based on the m neural network models to determine the category of the target image.

[0151] It should be noted that the terminal 101 mentioned above can be any electronic product that can interact with the user through one or more methods such as a touchpad, touchscreen, voice interaction, or handwriting device, such as mobile phones, personal digital assistants (PDAs), wearable devices, tablets, smart car systems, smart TVs, etc. The server 102 mentioned above can be a single server or a server cluster composed of multiple servers.

[0152] The following describes an exemplary implementation process for image classification in the embodiments of this application.

[0153] The image classification method provided in this application can be applied to album classification. That is, for multiple images captured by the terminal, these multiple images can be divided into different albums according to different categories. However, before classifying albums according to the image classification method provided in this application, the album classification function needs to be enabled. Of course, the album classification function can also be enabled by default, and this application does not limit this. The following is an example of manually enabling the album classification function.

[0154] When the terminal detects a settings operation, it displays a settings interface, which includes multiple settings items, including album settings. When the terminal detects a selection operation for an album settings item, it can display an album settings interface, which includes multiple album settings items, including album category items. When the terminal detects an enabling operation for an album category item, it confirms that the album category function is currently enabled.

[0155] For example, please refer to Figure 3 The left image shows that when the terminal detects a click on the settings icon, the settings interface can be displayed, such as... Figure 3 The middle image shows the settings interface. This interface includes several settings items: wireless and network settings, desktop and wallpaper settings, sound settings, display settings, and photo album settings. When the terminal detects a selection in the photo album settings, the photo album settings interface can be displayed, as shown below. Figure 3 As shown in the right image, the album settings interface includes multiple settings items: enabling cloud backup and enabling automatic album generation. Each setting item has a corresponding toggle switch. When the terminal detects that the toggle switch for the automatic album generation setting is turned on, it determines that the album categorization function is currently enabled. In other words, when the user clicks... Figure 3 When the settings icon shown in the left image is displayed, Figure 3 The settings interface is shown in the middle image. When the user clicks on the album settings item in the settings interface, it displays... Figure 3 The right image shows the album settings interface. When the user clicks the switch corresponding to the "Automatically generate album settings" option in the album settings interface, it confirms that the album categorization function is currently enabled.

[0156] After enabling the album categorization function, when the terminal captures an image, it can determine the category of the image using any of the various implementation methods described above, and then add the image to the corresponding album. If no album of the corresponding category exists before this image, the terminal can create an album of the corresponding category and then add the image to that album.

[0157] For example, please refer to Figure 4 In the left image, when the terminal detects a click on the camera icon, the camera shooting interface can be displayed, such as... Figure 4 As shown in the right image, the camera's shooting interface includes several buttons: an album button, a shooting button, and a lens rotation button. When the terminal detects a click on the shooting button, it can capture the image within the viewfinder to obtain a photograph, such as... Figure 4 The building shown in the middle right image.

[0158] Please refer to Figure 5The left image shows that when the terminal detects a click on the album button, the album interface can be displayed, as shown below. Figure 5 The image on the right shows the album interface, which includes all captured photos. For each photo, its category can be determined using any of the various implementation methods described above, thus adding it to the corresponding album. For example, if a photo of a building is determined to be categorized as "building" using any of the methods described above, the photo can be added to the "buildings" album.

[0159] Please refer to Figure 6 , Figure 6 This is a schematic diagram of a computer device according to an embodiment of this application. The computer device can be either the terminal 101 described above or the server 102 described above. The computer device includes a processor 601, a memory 602, and a transceiver 603. These components can be connected via a communication bus 604 for communication.

[0160] Processor 601 can be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or one or more integrated circuits for implementing the solutions of this application, such as application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), generic array logic (GAL), or any combination thereof.

[0161] The memory 602 may be a read-only memory (ROM), a random access memory (RAM), an electrically erasable programmable read-only memory (EEPROM), an optical disc (including a compact disc read-only memory (CD-ROM), a compressed optical disc, a laser disc, a digital versatile optical disc, a Blu-ray disc, etc.), a magnetic disk storage medium, or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but not limited thereto. The memory 602 may exist independently and be connected to the processor 601 via a communication bus 604. The memory 602 may also be integrated with the processor 601.

[0162] Transceiver 603 uses any transceiver-like device for communicating with other devices or communication networks. Transceiver 603 includes a wired communication interface and may also include a wireless communication interface. The wired communication interface may be, for example, an Ethernet interface. The Ethernet interface may be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (WLAN) interface, a cellular network communication interface, or a combination thereof.

[0163] The communication bus 604 is used to transmit information between the aforementioned components. The communication bus 604 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, it is represented by only one thick line in the figure, but this does not indicate that there is only one bus or one type of bus.

[0164] As one embodiment, processor 601 may include one or more CPUs. A computer device may include multiple processors, each of which may be a single-core processor or a multi-core processor. Here, "processor" may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0165] As one embodiment, the computer device may also include output devices and input devices. The output device communicates with the processor 601 and can display information in various ways. For example, the output device may be a liquid crystal display (LCD), a light-emitting diode (LED) display device, a cathode ray tube (CRT) display device, or a projector, etc. The input device communicates with the processor 601 and can receive user input in various ways. For example, the input device may be a mouse, keyboard, touchscreen device, or sensing device, etc.

[0166] In some embodiments, memory 602 is used to store program code for executing the scheme of this application, and processor 601 can execute the program code stored in memory 602. The program code may include one or more software modules, and the computer device can implement the image classification method provided in the embodiments of this application through processor 601 and program code in memory 602.

[0167] Please refer to Figure 7 , Figure 7 This is a flowchart of an image classification method provided in an embodiment of this application, which is applied in a computer device. The method includes the following steps.

[0168] Step 701: Process the target image to be classified using the current neural network model to obtain the current classification result output by the current neural network model.

[0169] The current classification result includes the probability of the target image belonging to each of the multiple categories. The current neural network model is the neural network model i with the highest probability in the selection result output by neural network model a. The selection result output by neural network model a includes the probability corresponding to p neural network models among the m trained neural network models. The p neural network models are the neural network models allowed to be used to process the target image after neural network model a. The p neural network models include the neural network model i. The m neural network models include neural network model a. m is an integer greater than 1, p is an integer greater than or equal to 1, and p is less than m.

[0170] In some embodiments, the current neural network model can also output the current selection result, which includes the probabilities corresponding to d neural network models out of m neural network models. The d neural network models are those allowed to be used to process the target image after neural network model i, where d is an integer greater than or equal to 1 and less than m. That is, the embodiments of this application can process the target image using the current neural network model in two ways. These two implementation methods will be described below.

[0171] The first implementation involves inputting the target image into the current neural network model to obtain the current classification result and the current selection result output by the current neural network model.

[0172] In other words, in the first implementation, any one of the m neural network models is taken as the current neural network model. After the target image is input into the current neural network model, the current neural network model can not only output the current classification result of the target image, but also determine the probability of the neural network model that can be used to process the target image after the current neural network model, that is, the current selection result.

[0173] The higher the probability of the neural network model that can be used to process the target image after the current neural network model, the higher the classification accuracy of the target image by this neural network model. In general, the classification accuracy of neural network model i for the target image is higher than that of neural network model a for the target image.

[0174] The second approach involves inputting the target image and the current iteration number into the current neural network model, and then processing the target image using the current neural network model.

[0175] There are two scenarios for the second implementation method, which will be introduced separately below.

[0176] In the first case, the target image and the current iteration number are input into the current neural network model to obtain the current classification result output by the current neural network model.

[0177] In other words, in the first case, the current iteration number input to the current neural network model is less than the iteration number threshold, and the difference between the current iteration number threshold and the threshold is 1. Thus, after processing the target image by the current neural network model, the current iteration number has reached the threshold, and no further iterations are needed. Therefore, the current neural network model can simply output the current classification result.

[0178] In the second scenario, the target image and the current iteration number are input into the current neural network model to obtain the current classification result and the current selection result output by the current neural network model.

[0179] In other words, in the second case, the current iteration number input to the current neural network model is less than the iteration threshold, and the difference between the current iteration number and the iteration threshold is greater than 1. Thus, after processing the target image by the current neural network model, the current iteration number has not yet reached the threshold, and further iterations may be needed. Therefore, the current neural network model needs to output the current classification result and the current selection result.

[0180] Step 702: Determine the current integration result based on the current classification result. The current integration result includes the probability of the target image belonging to each of the multiple categories after integration.

[0181] In some embodiments, the average probability of a corresponding category in multiple classification results can be determined as the probability of the corresponding category in the current integrated result. These multiple classification results include the classification results output by neural network model a and neural network model i. Alternatively, the current classification result can be directly determined as the current integrated result.

[0182] In other words, the probability of the first category in the current integration result includes: the average probability of the first category among multiple classification results, including the classification results output by neural network model a and neural network model i. Alternatively, the probability of the first category in the current integration result includes: the probability of the first category among the classification results output by neural network model i. Here, the first category is one of the multiple categories.

[0183] For example, a target image has been processed using three neural network models to obtain three classification results: Classification Result 1, Classification Result 2, and Classification Result 3. Each of these three classification results includes the probability that the target image belongs to one of five categories. The average probability of the target image belonging to category 1 from these three classification results is determined as the probability of the target image belonging to category 1 in the integrated result; the average probability of the target image belonging to category 2 from these three classification results is determined as the probability of the target image belonging to category 2 in the integrated result; the average probability of the target image belonging to category 3 from these three classification results is determined as the probability of the target image belonging to category 3 in the integrated result; the average probability of the target image belonging to category 4 from these three classification results is determined as the probability of the target image belonging to category 4 in the integrated result; and the average probability of the target image belonging to category 5 from these three classification results is determined as the probability of the target image belonging to category 5 in the integrated result.

[0184] Step 703: Determine the category of the target image based on the current integration results.

[0185] Based on the description of step 701 above, there are two implementation methods for processing the target image using the current neural network model. Under different implementation methods, the category of the target image is determined differently based on the current integration result. Therefore, the following will also describe the two implementation methods.

[0186] In the first implementation, when processing the target image using the current neural network model in step 701, determining the category of the target image based on the current integration result includes: if the current integration result satisfies the first convergence condition or the current neural network model satisfies the second convergence condition, then the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not satisfy the first convergence condition and the current neural network model does not satisfy the second convergence condition, then the neural network model corresponding to the highest probability in the current selection result is taken as the current neural network model, and the target image is processed again according to the methods described in steps 701-703.

[0187] Since each neural network model outputs both classification and selection results in the first implementation, after determining the current integration result based on the current classification result output by the current neural network model, it is necessary to determine whether the current integration result satisfies the first convergence condition and whether the current neural network model satisfies the second convergence condition. If the current integration result satisfies the first convergence condition or the current neural network model satisfies the second convergence condition, it indicates that the reliability of the current integration result is high, and the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not satisfy the first convergence condition and the current neural network model does not satisfy the second convergence condition, it indicates that the reliability of the current integration result is low. In this case, the neural network model corresponding to the highest probability in the current selection result can be used as the current neural network model, and the target image can be processed again according to the methods described in steps 701-703 above.

[0188] As an example, the current fusion result satisfies the first convergence condition if the confidence level of the current fusion result is greater than a confidence threshold. The current neural network model satisfies the second convergence condition if the number of iterations of the current neural network model is greater than or equal to an iteration threshold. Of course, the current fusion result satisfying the first convergence condition can also include other conditions, and similarly, the current neural network model satisfying the second convergence condition can also include other conditions.

[0189] If the current integration result satisfies the first convergence condition, including that the confidence level of the current integration result is greater than a confidence threshold, after determining the current integration result based on the current classification result, the confidence level of the current integration result can also be determined. The process of determining the confidence level of the current integration result includes: determining the highest probability in the current integration result as the confidence level of the current integration result. Alternatively, the target image and the current integration result can be input into a confidence network model to obtain the confidence level of the current integration result output by the confidence network model.

[0190] The confidence network model can be pre-trained and can be a neural network model, or it can be a model with other structures.

[0191] Since confidence level is used to indicate the reliability of the integration result, and the embodiments of this application classify the target image through iterative iteration, if the confidence level of the current integration result is greater than the confidence level threshold, or if the current iteration count is greater than or equal to the iteration count threshold, it indicates that the reliability of the current integration result is high. In this case, the category corresponding to the highest probability in the current integration result can be directly determined as the category of the target image. If the confidence level of the current integration result is not greater than the confidence level threshold and the current iteration count is less than the iteration count threshold, it indicates that the reliability of the current integration result is low. In this case, the neural network model corresponding to the highest probability in the current selection result can be used as the current neural network model, and the target image can be reprocessed.

[0192] The confidence threshold is preset and can be adjusted according to different needs. The iteration number threshold can also be preset; it can be a specified number of iterations or a maximum number of iterations, and can be set according to different needs. This application embodiment does not limit this.

[0193] As can be seen from steps 701-703 above, this embodiment of the application determines the category of the target image through a cyclic iterative process. Each iteration of this cyclic iterative process includes steps 701-703. Furthermore, if the current integration result does not meet the first convergence condition and the current neural network model does not meet the second convergence condition, the cyclic iterative processing can continue. In this case, the neural network model corresponding to the highest probability in the current selection result is taken as the current neural network model, and the target image is processed again according to the above steps to execute the next iteration of the cyclic iterative process. That is, the neural network model for the next iteration can be determined using the previous neural network model, and the classification accuracy of the next neural network model is higher than that of the previous neural network model, thus gradually improving the reliability of the integration result.

[0194] Optionally, for the first iteration of this iterative method, the current neural network model is the first neural network model among the m neural network models. Furthermore, for the first method of determining the current integration result, the current integration result obtained in the first iteration is the current classification result output by the first neural network model.

[0195] The second implementation method, in step 701, when processing the target image through the current neural network model according to the second implementation method, the process of determining the category of the target image based on the current integration result is divided into two cases. The following will explain these two cases respectively, and these two cases correspond to the two cases in the second implementation method in step 701 above.

[0196] In the first case, the category corresponding to the highest probability in the current integration result is determined as the category of the target image.

[0197] In the first case, after processing the target image using the current neural network model, the current iteration number has reached the iteration number threshold, indicating that the reliability of the current integration result is high. Therefore, the category corresponding to the highest probability in the current integration result can be directly determined as the category of the target image, without needing to judge whether the current integration result meets the first convergence condition.

[0198] In the second scenario, if the current integration result meets the first convergence condition, then the category corresponding to the highest probability in the current integration result is determined as the category of the target image. If the current integration result does not meet the first convergence condition, then the neural network model corresponding to the highest probability in the current selection result is used as the current neural network model, and the target image is processed again according to the methods described in steps 701-703 above.

[0199] In the second scenario, after processing the target image using the current neural network model, the current iteration count has not yet reached the iteration threshold, making the reliability of the current integration result uncertain. Therefore, it is necessary to determine whether the current integration result satisfies the first convergence condition. If the current integration result satisfies the first convergence condition, its reliability is considered high, and the category corresponding to the highest probability in the current integration result can be determined as the category of the target image. If the current integration result does not satisfy the first convergence condition, its reliability is considered low, and the neural network model corresponding to the highest probability in the current selection result can be used as the current neural network model, and the target image can be processed again according to steps 701-703 described above.

[0200] The first convergence condition in the second implementation can be found in the relevant description of the first implementation above, and will not be repeated here in the embodiments of this application.

[0201] In this embodiment, after inputting the target image into the current neural network model, the current neural network model not only outputs the current classification result for the target image, but also determines the probability of the neural network model allowed to process the target image after the current neural network model. Furthermore, the higher the probability of the neural network model allowed to process the target image after the current neural network model, the higher the classification accuracy of that neural network model for the target image. Generally, the neural network model with the highest probability has a higher classification accuracy for the target image than the current neural network model. Therefore, by performing iterative processing in this manner, the accuracy of image classification can be improved. In other words, based on the properties of the target image, this embodiment can dynamically select a portion of the neural network models from the m neural network models for image classification, eliminating the need to perform image classification through each neural network model, thus reducing the computational load. Moreover, since only a portion of the neural network models are selected, only the parameters of a portion of the neural network models need to be read into memory, rather than the parameters of all neural network models, avoiding the problem of time-consuming memory read operations.

[0202] The image classification method provided in this application will be introduced next with two running examples.

[0203] Please refer to Figure 8 In Example 1, the m trained neural network models are denoted as Net1, Net2, Net3, ..., Net... m The target image is input into Net1 to obtain the classification result Pred1 and the selection result Sel1. The classification result Pred1 is then integrated to obtain the integrated result Pred. 11 And determine the current iteration number. If the integration result is Pred 11 If the confidence level is not greater than the confidence threshold and the current iteration number is less than the iteration number threshold, then the neural network model corresponding to the highest probability in the selection result Sel1 is determined. Assuming that the neural network model corresponding to the highest probability in the selection result Sel1 is Net2, then the target image is input into Net2 to obtain the classification result Pred2 and the selection result Sel2. The classification results Pred1 and Pred2 are integrated to obtain the integrated result Pred. 12 If the integration result is Pred 12 If the confidence level is greater than the confidence threshold or the current iteration number is greater than or equal to the iteration number threshold, then the integrated result Pred will be... 12If the category with the highest probability in the selection result Sel2 is determined as the category of the target image, then the neural network model with the highest probability in the selection result Sel2 is determined. The above steps are repeated until an integrated result with a confidence level greater than the confidence level threshold is obtained or the current iteration number is greater than or equal to the iteration number threshold.

[0204] Please refer to Figure 9 In Example 2, the m trained neural network models are denoted as Net1, Net2, Net3, ..., Net... m The target image is input into Net1 to obtain classification result Pred1 and selection result Sel1. The classification result Pred1 is then integrated to obtain the integrated result Pred. 11 And determine the current iteration number. If the integration result is Pred 11 If the confidence level is not greater than the confidence threshold and the current iteration number is less than the iteration number threshold, then the neural network model corresponding to the highest probability in the selection result Sel1 is determined. Assuming that the neural network model corresponding to the highest probability in the selection result Sel1 is Net2, then the target image is input into Net2 to obtain the classification result Pred2 and the selection result Sel2. The classification results Pred1 and Pred2 are integrated to obtain the integrated result Pred. 12 If the integration result is Pred 12 If the confidence level is not greater than the confidence threshold and the current iteration number is less than the iteration number threshold, then the neural network model corresponding to the highest probability in the selection result Sel2 is determined. Assuming that the neural network model corresponding to the highest probability in the selection result Sel2 is Net4, then the target image is input into Net4 to obtain the classification result Pred3 and the selection result Sel3. The classification results Pred1, Pred2 and Pred3 are integrated to obtain the integrated result Pred. 13 If the integration result is Pred 13 If the confidence level is greater than the confidence threshold or the current iteration number is greater than or equal to the iteration number threshold, then the integrated result Pred will be... 13 If the category with the highest probability in the selection result Sel3 is determined as the category of the target image, then the neural network model with the highest probability in the selection result Sel3 is determined. The above steps are repeated until an integrated result with a confidence level greater than the confidence level threshold is obtained or the current iteration number is greater than or equal to the iteration number threshold.

[0205] For example, please refer to Figure 10 The image classification method provided in this application embodiment can also be used through... Figure 10 It indicates. Among them, Figure 10In this context, q represents the current iteration number, and Q is the iteration threshold. That is, the m trained neural network models are denoted as Net1, Net2, Net3, ..., Net... m Set q=1. Use the neural network model Net1 as the current neural network model, and input the target image into the current neural network model to obtain the classification result Pred. q And the selection result Sel q , using {Pred1,...,Pred q}The integrated result Pred is obtained 1q Judge the integration result Pred 1q Does the confidence level exceed the confidence threshold? If the integration result is Pred 1q If the confidence level is greater than the confidence threshold, then the integrated result Pred will be... 1q The category corresponding to the highest probability in the merged result is determined as the category of the target image; otherwise, it is checked whether the current iteration number q is greater than Q. If the current iteration number q is greater than Q, the merged result Pred is then used. 1q The category with the highest probability is determined as the category of the target image; otherwise, let q = q + 1, and select the result Sel. q The neural network model with the highest probability is determined as the current neural network model, and the steps for inputting the target image into the current neural network model are returned.

[0206] The method provided in this application has different computational loads depending on the aforementioned confidence threshold. Please refer to... Figure 11 , Figure 11 The graphs show the image classification accuracy under different computational loads. Furthermore, Figure 11 This document not only illustrates the image classification accuracy curves of the embodiments of this application under different computational loads, but also illustrates the image classification accuracy curves of other related technologies (BasisNet, MobileNetV3) under different computational loads. Through comparison, it can be determined that the method provided by the embodiments of this application has higher image classification accuracy. Furthermore, the method provided by the embodiments of this application was validated on the authoritative academic dataset ImageNet. The method provided by the embodiments of this application significantly outperforms non-dynamic network algorithms in performance, overcoming the low performance problem of non-dynamic network algorithms. For example, compared to the current mainstream lightweight model MobileNetV3, the embodiments of this application reduce the computational load by 2.8 times (i.e., from 216M to 77M), and improve the image classification accuracy by 1.08% (i.e., from 75.2% to 76.28%). Moreover, compared to the current mainstream heavyweight model ResNeXt-101, the embodiments of this application reduce the computational load by 178 times (i.e., from 32B to 180M), while maintaining essentially the same image classification accuracy (i.e., from 80.9% to 81.04%).

[0207] Optionally, before classifying the target image using the aforementioned m neural network models, m additional neural network models can be trained to obtain the m neural network models described above. Please refer to [link / reference]. Figure 12 The process of training the m neural network models to be trained includes the following steps.

[0208] Step 1201: Obtain n sample images and n sample labels, with each of the n sample labels corresponding to one of the n sample images.

[0209] Step 1202: Based on the n sample images and the n sample labels, determine the gradient of each parameter of each of the m neural network models to be trained.

[0210] For each of the m training neural network models, the method for determining the gradient of each parameter of each training neural network model is the same. The following explanation will take training neural network model k as an example.

[0211] In some embodiments, the gradients of each parameter of the neural network model k to be trained can be determined by: determining the training function value corresponding to the neural network model k to be trained based on n sample images and n sample labels; and determining the gradients of each parameter of the neural network model k to be trained based on the training function value corresponding to the neural network model k to be trained.

[0212] As an example, the training function value corresponding to the neural network model k to be trained can be represented by the following formulas (1) and (2):

[0213]

[0214]

[0215] In formula (1) above, L k This refers to the training function value corresponding to the neural network model k to be trained. This refers to the model selection function value corresponding to the training neural network model k. It can be determined by selecting the training neural network model k from among m training neural network models that is allowed to process n sample images after model k. Furthermore, the model selection function value ensures that the selection result output by training neural network model k assigns a given image to the training neural network model k with the highest classification accuracy among the subsequent training neural network models. In other words, after inputting an image into training neural network model k, the training neural network model with the highest probability among the selection results output by training neural network model k is the training neural network model k with the highest classification accuracy among the subsequent training neural network models.

[0216] ω k (j,Net1,...,Net k-1 () refers to the sample image x among n sample images when determining the training function value corresponding to the neural network model k to be trained. j The corresponding weight function value, a real number, can be determined by considering the m training neural network models that are allowed to process n sample images before training neural network model k. Furthermore, the weight function value allows training neural network model k to achieve higher classification accuracy on a particular image, while reducing its classification accuracy on other images. That is, when training neural network model k processes an image next, it achieves higher classification accuracy for that image but lower accuracy for other images. Task (x j ,label j () refers to the sample image x among n sample images when determining the training function value corresponding to the neural network model k to be trained. j The corresponding classification function value can be, for example, the cross-entropy loss function value, or also known as the task-related function value.

[0217] In the above formula (2), ψ k (j,Net k+1 ,...,Net m () refers to the sample image x among n sample images when determining the training function value corresponding to the neural network model k to be trained. j The corresponding selection weight function value, which is a real number, can be determined by selecting the training neural network model that is allowed to process n sample images after training neural network model k from among the m training neural network models. k (j,Net k+1 ,...,Net m () refers to the sample image x among n sample images when determining the training function value corresponding to the neural network model k to be trained. jThe corresponding label function value is a positive integer, which can be the sample image x. j The corresponding label selection, that is, labeling the sample image x using a label function. j The labels are defined. Furthermore, the label function value can be determined from among m trainable neural network models, specifically the one that, after trainable neural network model k, is allowed to process n sample images. L CE This refers to determining the training function value corresponding to the n sample images x when training the neural network model k. j The corresponding classification loss function value, such as the cross-entropy loss function value, can be determined by the training neural network model that is allowed to process n sample images after the training neural network model k among the m training neural network models.

[0218] It should be noted that the gradients of each parameter of the neural network model k can be determined using the backpropagation algorithm based on the training function values ​​corresponding to the model. Backpropagation is a supervised learning algorithm; for details on using it to determine the gradients of the parameters of the neural network model, please refer to relevant techniques, which will not be elaborated upon here.

[0219] Based on the above description, the training function values ​​corresponding to the neural network model k to be trained include n classification function values, n weight function values, n label function values, n classification loss function values, and n selection weight function values. Each of the n classification function values, n weight function values, n label function values, n classification loss function values, and n selection weight function values ​​corresponds one-to-one with each of the n sample images. The process of determining these multiple function values ​​will be explained separately below.

[0220] The process of determining n classification function values

[0221] Since the classification function value for each sample image is determined in the same way, a sample image can be selected from the n sample images. The following steps are performed to determine the classification function value for the selected sample image, continuing until the classification function value for each sample image is determined: Input the selected sample image into the neural network model k to be trained, and obtain the classification result output by the neural network model k. Based on the sample label corresponding to the selected sample image and the classification result output by the neural network model k, determine the classification function value corresponding to the selected sample image.

[0222] In some embodiments, the classification function value corresponding to the selected sample image can be determined according to the following formula (3) based on the sample label corresponding to the selected sample image and the classification result output by the neural network model k to be trained.

[0223] in,

[0224] In formula (3) above, L Task (x j ,label j () refers to the classification function value corresponding to the selected sample image, x j This refers to the selected sample image, label. j This refers to the sample label corresponding to the selected sample image, l c It refers to the probability that the selected sample image belongs to category c in the classification results output by the neural network model k to be trained. This refers to the summation of ylog(l) corresponding to all probabilities in the classification results output by the neural network model k to be trained.

[0225] For example, the classification result output by the neural network model k to be trained includes 5 probabilities, each of which represents the probability that the selected sample image belongs to one of the 5 categories. In this case, based on these 5 categories and the sample labels corresponding to the selected sample images, we can determine the y corresponding to each category. Then, based on the y corresponding to each of the 5 categories and the probability corresponding to each category in the classification result output by the neural network model k to be trained, we can sum them up as described above to obtain the classification function value corresponding to the selected sample image.

[0226] The process of determining the values ​​of n weight functions

[0227] In this embodiment, the n weight function values ​​can be implemented based on rules or based on meta-learning. These two implementation methods will be described below.

[0228] In the first implementation, the n weight function values ​​are determined based on rules. This approach ensures that, when the next training neural network model to process an image is chosen as model k, model k will have higher weight function values ​​for that image.

[0229] Regarding the first implementation method, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (11)-(13).

[0230] (11) Using b neural network models to be trained, determine the standard probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t standard probabilities, t=mb, b neural network models to be trained are the models that are allowed to process n sample images before the m neural network models to be trained, and t neural network models to be trained include the model k and the models that are allowed to process n sample images after the model k in the m neural network models to be trained.

[0231] In some embodiments, using b training neural network models, the average probability that each of the t training neural network models is used to process each of the n sample images is determined, resulting in n*t average probabilities. The average probabilities corresponding to the same training neural network model among the n*t average probabilities are then grouped into a probability set, resulting in t probability sets. Based on the n*t average probabilities and the t probability sets, the n*t average probabilities are standardized to obtain n*t standard probabilities.

[0232] The method for determining the average probability that each of the t training neural network models is used to process each of the n sample images is the same, using b training neural network models. Therefore, a sample image can be selected from the n sample images, and a training neural network model can be selected from the t training neural network models. Using the b training neural network models, the average probability that the selected training neural network model is used to process the selected sample image is determined as follows, until n*t average probabilities are determined: For each of the b training neural network models, determine the probability that the selected training neural network model is used to process the selected sample image, thus obtaining b probabilities. The average of these b probabilities is then determined to obtain the average probability that the selected training neural network model is used to process the selected sample image.

[0233] As an example, for each of the b trainable neural network models, a selected sample image can be input into that model to obtain its output selection result. This selection result includes the probability that the selected model is used to process the selected sample image; that is, the probability that the next neural network model to classify the selected sample image is the selected model. Thus, for each of the b trainable neural network models, the probability that the selected model is used to process the selected sample image can be obtained, resulting in b probabilities. Then, the average of these b probabilities can be determined, and this average is taken as the average probability that the selected model is used to process the selected sample image.

[0234] The standardization process for each average probability within each of the t probability sets is the same. Therefore, a probability set can be selected from the t probability sets, and each average probability in the selected probability set can be standardized according to the following operations until n*t standard probabilities are determined: determine the mean and standard deviation of the average probabilities in the selected probability set, and determine the standard probability corresponding to each average probability in the selected probability set based on the average probability, mean, and standard deviation of the selected probability set.

[0235] As an example, the standard probability corresponding to each average probability in the selected probability set can be determined according to the following formula (4) based on the average probability, mean, and standard deviation in the selected probability set.

[0236]

[0237] In formula (4) above, P j,h It refers to the average probability in the set of possible choices. It refers to the average probability P in the set of chosen probabilities. j,h The corresponding standard probability, Ω h It refers to the set of probabilities of choice, Mean(Ω) h Std(Ω) refers to the mean of the set of probabilities of selection. h ) refers to the standard deviation of the chosen probability set.

[0238] (12) Based on n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images.

[0239] In some embodiments, the first weight corresponding to each of the n sample images can be determined by the following steps (a)-(g).

[0240] (a) Construct a standard probability set from n*t standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set.

[0241] In some embodiments, a standard probability can be selected from the set of standard probabilities, and the relative probability value corresponding to the selected standard probability can be determined by the following operations until the relative probability value corresponding to each standard probability is determined: from the other standard probabilities in the set of standard probabilities besides the selected standard probability, multiple standard probabilities corresponding to the same sample image as the selected standard probability are determined, the maximum standard probability among the multiple standard probabilities is determined, and the difference between the selected standard probability and the maximum standard probability is determined as the relative probability value corresponding to the selected standard probability.

[0242] The method for determining the relative probability value corresponding to each standard probability in the standard probability set can be expressed by the following formula (5).

[0243]

[0244] In the above formula (5), Standard probability The corresponding relative probability value, It refers to the set of standard probabilities excluding the standard probability. Other standard probabilities besides the standard probability The maximum standard probability among multiple standard probabilities corresponding to the same sample image.

[0245] (b) Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set.

[0246] The relative probability values ​​corresponding to the standard probabilities in the standard probability set are compared to determine the largest relative probability value among these relative probability values.

[0247] (c) Determine the sample image corresponding to the maximum relative probability value from n sample images, and determine the neural network model to be trained corresponding to the maximum relative probability value from t neural network models to be trained.

[0248] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, and the set of standard probabilities is composed of n*t standard probabilities, one standard probability in the set of standard probabilities corresponds to one sample image in the n sample images and one trainable neural network model in the t trainable neural network models. The relative probability value is determined based on the standard probabilities; therefore, each relative probability value also corresponds to one sample image in the n sample images and one trainable neural network model in the t trainable neural network models. Thus, after determining the maximum relative probability value, the sample image corresponding to the maximum relative probability value can be determined from the n sample images, and the trainable neural network model corresponding to the maximum relative probability value can be determined from the t trainable neural network models.

[0249] (d) Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value.

[0250] In some embodiments, if the neural network model to be trained corresponding to the maximum relative probability value is neural network model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as... If the neural network model to be trained corresponding to the maximum relative probability value is not model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as... Where a is a real number that satisfies a≥0 and a≤1.

[0251] (e) Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set.

[0252] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, a sample image will correspond to t standard probabilities. Thus, after determining the sample image corresponding to the maximum relative probability value, the t standard probabilities corresponding to that sample image can be removed from the standard probability set. At this point, the standard probability set still contains n*(t-1) standard probabilities.

[0253] (f) If the standard probability set is not empty after removal, update the number of model determinations, which refers to the number of times the model corresponding to the maximum relative probability value is determined from t models to be trained.

[0254] As an example, the number of model determinations can be increased by 1. That is, each time a maximum probability relative value is determined through one iteration, and the corresponding neural network model for that maximum probability relative value is selected from t trainable neural network models, the number of model determinations can be increased by 1. Of course, this is just one exemplary update method, and other methods are also possible.

[0255] Furthermore, if the standard probability set is empty after removal, the operation ends. At this point, the first weight corresponding to each sample image in the n sample images has been determined.

[0256] (g) If the number of determinations of the updated model is greater than or equal to n / t, remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability from the standard probability set, and return to the step in step (a) to determine the relative probability value corresponding to each standard probability in the standard probability set.

[0257] Since the n*t standard probabilities are determined by b trainable neural network models, and each of the t trainable neural network models is used to process the standard probability of each sample image in the n sample images, one of the t trainable neural network models will correspond to n standard probabilities. Thus, the n standard probabilities corresponding to the trainable neural network model with the maximum relative probability can be removed from the set of standard probabilities. At this point, the set of standard probabilities still contains (n-1)*(t-1) standard probabilities.

[0258] (13) Normalize the first weight corresponding to each of the n sample images to obtain the weight function value corresponding to each of the n sample images.

[0259] In some embodiments, the first weight corresponding to each of the n sample images can be normalized according to the following formula (6) to obtain the weight function value corresponding to each of the n sample images.

[0260]

[0261] In the above formula (6), ω j Refers to the sample image x j The corresponding weight function value, Refers to the sample image x j The corresponding first weight, It refers to the first weight corresponding to the g-th sample image among n sample images.

[0262] The second implementation method uses meta-learning to determine the n weight function values. This method allows the neural network model k to have a lower classification loss on certain images.

[0263] Regarding the second implementation method, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (21)-(27).

[0264] (21) Using b neural network models to be trained, select h sample images from n sample images to be processed by the neural network model k next. The b neural network models to be trained are the models that are allowed to be used to process n sample images before the neural network model k among the m neural network models to be trained, and h is less than n.

[0265] In some embodiments, a sample image is selected from the n sample images, and a training neural network model is selected from the b training neural network models. The selected sample image is input into the selected training neural network model to obtain the selection result output by the selected training neural network model. This selection result includes the probability of the training neural network model among the m training neural network models that is allowed to process the selected sample image after the selected training neural network model. If the training neural network model corresponding to the highest probability in the selection result is training neural network model k, then the selected sample image is determined as the sample image to be processed by training neural network model k next. By traversing all the sample images and each of the b training neural network models in the above method, h sample images can be obtained.

[0266] (22) The prediction weights for each of the n sample images are determined by using the meta-network model.

[0267] In some embodiments, each of the n sample images can be input into the meta-network model to obtain the prediction weights corresponding to the sample image output by the meta-network model. That is, for any sample image among the n sample images, after inputting the sample image into the meta-network model, the prediction weights corresponding to the sample image output by the meta-network model can be obtained.

[0268] The meta-network model can be a pre-trained model, and this application does not limit this.

[0269] (23) Based on the prediction weights corresponding to each of the n sample images and the classification function values ​​corresponding to each of the n sample images, determine the weighted loss values ​​of the n sample images.

[0270] As an example, the weighted loss value of the n sample images can be determined according to the following formula (7) based on the prediction weight corresponding to each sample image in the n sample images and the classification function value corresponding to each sample image in the h sample images.

[0271]

[0272] In formula (7) above, L train This refers to the weighted loss value of the n sample images. L refers to the prediction weight corresponding to the g-th sample image among the n sample images. Task (x g ,label g ) refers to the classification function value corresponding to the g-th sample image among the n sample images.

[0273] (24) Update the neural network model k to be trained based on the weighted loss values ​​of the n sample images.

[0274] In some embodiments, the backpropagation algorithm can be used to update the neural network model k to be trained based on the weighted loss values ​​of the n sample images. The method of updating the neural network model k using the backpropagation algorithm can refer to related technologies, and this application embodiment does not limit it in this regard.

[0275] (25) Input the selected h sample images into the updated neural network model k to obtain the total loss value of the updated neural network model k on the h sample images.

[0276] (26) Update the meta-network model based on the total loss value of the updated neural network model k on the h sample images.

[0277] In some embodiments, the meta-network model can be updated using the backpropagation algorithm based on the total loss value of the updated neural network model k on the h sample images. The method of updating the meta-network model using the backpropagation algorithm can refer to related technologies, and this application does not limit it in this regard.

[0278] (27) Determine the weight function value corresponding to each of the n sample images by using the updated meta-network model.

[0279] In some embodiments, each of the n sample images can be input into the updated meta-network model to obtain the weight function value corresponding to the corresponding sample image output by the updated meta-network model. That is, for any sample image among the n sample images, after inputting the sample image into the updated meta-network model, the weight function value corresponding to the sample image output by the updated meta-network model can be obtained.

[0280] The above method for determining the n weight function values ​​is only one of the meta-learning-based methods. In practical applications, there are other meta-learning-based methods. For example, the n weight function values ​​can be determined based on n sample images and n sample labels through the following steps (31)-(37).

[0281] (31) Using b neural network models to be trained, select h sample images from n sample images to be processed by the neural network model k next. The b neural network models to be trained are the models that are allowed to be used to process n sample images before the neural network model k among the m neural network models to be trained, and h is less than n.

[0282] The implementation process of step (31) can refer to the implementation process of step (21) above, and will not be repeated here.

[0283] (32) Determine the initial weights for each of the n sample images.

[0284] In some embodiments, an initial weight can be set for each of the h sample images. For example, the initial weight can be 0.

[0285] (33) Determine the weighted loss value of the n sample images by using the initial weights corresponding to each sample image in the n sample images.

[0286] As an example, the weighted loss value of the n sample images can be determined according to the following formula (8) based on the initial weights corresponding to each sample image in the n sample images and the classification function value corresponding to each sample image in the n sample images.

[0287]

[0288] In formula (8) above, L train This refers to the weighted loss value of the n sample images. L refers to the initial weight corresponding to the g-th sample image among the n sample images. Task (x g ,label g ) refers to the classification function value corresponding to the g-th sample image among the n sample images.

[0289] (34) Update the neural network model k to be trained based on the weighted loss values ​​of the n sample images.

[0290] In some embodiments, the backpropagation algorithm can be used to update the neural network model k to be trained based on the weighted loss values ​​of the n sample images. The method of updating the neural network model k using the backpropagation algorithm can refer to related technologies, and this application embodiment does not limit it in this regard.

[0291] (35) Input the selected h sample images into the updated neural network model k to obtain the total loss value of the updated neural network model k on the h sample images.

[0292] (36) Based on the total loss value of the updated neural network model k on the h sample images and the initial weights corresponding to each sample image in the n sample images, determine the second weights corresponding to each sample image in the n sample images.

[0293] As an example, the second weights corresponding to each of the n sample images can be determined based on the total loss value of the updated neural network model k on the h sample images and the initial weights corresponding to each sample image in the n sample images, using the following formula (9).

[0294]

[0295] In the above formula (9), It refers to sample image x among the n sample images. j The corresponding second weight, This refers to the total loss value of the updated neural network model k on those h sample images. This refers to the total loss value of the updated neural network model k on the h sample images relative to the sample image x. j The gradient of the initial weights.

[0296] (37) Normalize the second weight corresponding to each of the n sample images to obtain the weight function value corresponding to each of the n sample images.

[0297] As an example, the second weight corresponding to each of the n sample images can be normalized according to the following formula (10) to obtain the weight function value corresponding to each of the n sample images.

[0298]

[0299] In the above formula (10), ω j Refers to the sample image x j The corresponding weight function value, Refers to the sample image x j The corresponding second weight, It refers to the second weight corresponding to the g-th sample image among n sample images.

[0300] The process of determining the n label function values

[0301] In this embodiment, the label function value corresponding to the sample image is related to the accuracy of the classification result of the sample image by the neural network model to be trained after the neural network model k that is allowed to process the sample image. That is, the neural network model to be trained is used to determine the label function value corresponding to the sample image, then the classification result of the sample image by that neural network model has higher accuracy, and the classification result of the sample image by other neural network models to be trained has lower accuracy. Among them, the n label function values ​​can be determined based on n sample images and n sample labels through the following steps (41)-(42).

[0302] (41) By using each of the s training neural network models, determine the standard probability that each sample image in the n sample images belongs to the corresponding sample label, so as to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0303] In some embodiments, the probability that each sample image in n sample images belongs to the corresponding sample label can be determined for each of the s training neural network models, resulting in n*s probabilities. The average probability corresponding to the same training neural network model among the n*s probabilities is then divided into a probability set, resulting in s probability sets. Based on the n*s probabilities and the s probability sets, the n*s probabilities are standardized to obtain n*s standard probabilities.

[0304] In this system, each of the *s* training neural network models determines the standard probability of each sample image belonging to its corresponding sample label from the *n* sample images in the same way. Therefore, we can select one training neural network model from the *s* models and one sample image from the *n* sample images. The following steps are performed to determine the probability of the selected sample image belonging to its corresponding sample label using the selected training neural network model, until *n**s probabilities are determined: The selected sample image is input into the selected training neural network model to obtain the classification result output by the selected training neural network model. This classification result includes the probability of the selected sample image belonging to each of multiple categories. These multiple categories can include the sample label corresponding to the selected sample image. In this way, the probability of the selected sample image belonging to its corresponding sample label can be determined using the selected training neural network model.

[0305] The method of standardizing the n*s probabilities based on n*s probabilities and s probability sets is similar to the method of standardizing the n*t average probabilities based on n*t average probabilities and t probability sets described above, and will not be repeated in this embodiment.

[0306] (42) Based on n*s standard probabilities, determine the label function value corresponding to each sample image in the n sample images.

[0307] In some embodiments, a standard probability set can be constructed from n*s standard probabilities, and the relative probability value corresponding to each standard probability in the standard probability set is determined. From the relative probability values ​​corresponding to the standard probabilities in the standard probability set, the maximum relative probability value is determined. The sample image corresponding to the maximum relative probability value is determined from n sample images, and the training neural network model corresponding to the maximum relative probability value is determined from s training neural network models. The position number of the training neural network model corresponding to the maximum relative probability value among the s training neural network models is determined as the label function value of the sample image corresponding to the maximum relative probability value. The standard probabilities corresponding to the sample images corresponding to the maximum relative probability value are removed from the standard probability set. If the standard probability set is not empty after removal, the model determination count is updated, which refers to the number of determinations made to determine the training neural network model corresponding to the maximum relative probability value from the s training neural network models. If the updated model determination count is greater than or equal to n / s, the standard probabilities corresponding to the training neural network models corresponding to the maximum relative probability value are removed from the standard probability set, and the process returns to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0308] For example, after determining the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set, we can determine the sample image corresponding to the maximum relative probability value from n sample images as sample image 5, and determine the 8th neural network model to be trained from s neural network models to be trained as the model corresponding to the maximum relative probability value. Then, we can determine that the label function value of sample image 5 is 30.

[0309] The content of the above implementation method can be referred to the relevant description in step (12) above, and will not be repeated here.

[0310] The process of determining n classification loss function values

[0311] Since the classification loss function value for each sample image is determined in the same way, a sample image can be selected from n sample images. The classification loss function value for the selected sample image is determined by following these steps until the classification loss function value for each sample image is determined: using the training neural network model k, determine the probability that each of the s training neural network models is used to process the selected sample image, so as to obtain s probabilities; based on the label function value corresponding to the selected sample image and the s probabilities, determine the classification loss function value corresponding to the selected sample image.

[0312] In some embodiments, the selected sample image can be input into the neural network model k to be trained to obtain the selection result output by the neural network model k. The selection result includes the probability that each of the s neural network models to be trained is used to process the selected sample image. That is, the next neural network model to classify the selected sample image is the probability of each of the s neural network models to be trained, thus obtaining s probabilities.

[0313] In some embodiments, the classification loss function value corresponding to the selected sample image can be determined according to the following formula (11) based on the label function value corresponding to the selected sample image and the s probabilities.

[0314] in,

[0315] In the above formula (11), L * This refers to the selected sample image x j The corresponding label function value, L CE (x j ,L * ) refers to the classification loss function value corresponding to the selected sample image, r eThis refers to the probability that the selected sample image belongs to category e among the s probabilities output by the neural network model k to be trained. This refers to the summation of ylog(l) corresponding to the s probabilities output by the neural network model k to be trained.

[0316] The process of determining n selection weight function values

[0317] In this embodiment, the selection weight function value corresponding to the sample image is related to the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k. That is, the larger the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k, the larger the selection weight function value corresponding to that sample image; conversely, the smaller the accuracy variance of the classification result of the n sample images by the neural network model to be trained after model k, the smaller the selection weight function value corresponding to that sample image.

[0318] In some embodiments, the standard probability of each sample image belonging to the corresponding sample label in the n sample images can be determined by each of the s training neural network models, resulting in n*s standard probabilities. Based on the n*s standard probabilities, the selection weight function value corresponding to each sample image in the n sample images is determined.

[0319] The process of determining the standard probability of each sample image belonging to the corresponding sample label in n sample images by using each of the s training neural network models can be referred to the description in step (41) above, and will not be repeated here.

[0320] The process of determining the selection weight function value for each of the n sample images based on n*s standard probabilities includes: dividing the probabilities corresponding to the same sample image among the n sample images into a probability set, resulting in n probability sets; determining the standard deviation of the standard probabilities included in each of the n probability sets, resulting in n standard deviations; and normalizing each of the n standard deviations to obtain the selection weight function value for each of the n sample images.

[0321] The method of normalizing each of the n standard deviations is similar to the method of normalizing the first weight corresponding to each of the n sample images in step (13) above, and will not be repeated in this embodiment.

[0322] Step 1203: Based on the gradients of each parameter of the m neural network models to be trained, update the m neural network models to be trained to obtain m neural network models.

[0323] For each of the m trainable neural network models, the gradient descent algorithm is used to update the model based on the gradients of its parameters, resulting in a new neural network model. After updating each model using gradient descent, m neural network models are obtained.

[0324] It should be noted that the method for updating the m training neural network models using the gradient descent algorithm can be found in relevant techniques, and will not be elaborated here. For example, the gradient descent algorithm can be stochastic gradient descent (SGD).

[0325] For example, please refer to Figure 13 The training method for the neural network model provided in this application embodiment can also be achieved through... Figure 13 The m neural network models to be trained are denoted as Net1, Net2, Net3, ..., Net... m Set k=1. Obtain n sample images and n sample labels, with each sample label corresponding one-to-one with a sample image. Determine the neural network model Net. k The corresponding training function value. Based on the Net neural network model. k The corresponding training function value determines the neural network model Net. k The gradients of each parameter. Determine if k is less than m; if k is not less than m, then apply the gradients to the neural network models Net1, Net2, Net3, ..., Net... m Update the dataset to obtain m trained neural network models. If k is less than m, let k = k + 1 and return the determined neural network model Net. k The steps for obtaining the corresponding training function values.

[0326] Figure 14 This is a schematic diagram of the structure of an image classification device provided in an embodiment of this application. The image classification device can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be... Figure 1 The terminal or server shown. See also Figure 14 The device includes: an image processing module 1401, an integration result determination module 1402, and an image category determination module 1403.

[0327] The image processing module 1401 is used to process the target image to be classified through the current neural network model in order to obtain the current classification result output by the current neural network model.

[0328] The current classification result includes the probability of the target image belonging to each of the multiple categories. The current neural network model is the neural network model i with the highest probability in the selection result output by neural network model a. The selection result output by neural network model a includes the probability corresponding to p neural network models among the m trained neural network models. The p neural network models are the neural network models allowed to be used to process the target image after neural network model a. The p neural network models include the neural network model i. The m neural network models include neural network model a. m is an integer greater than 1. p is an integer greater than or equal to 1. p is less than m.

[0329] The integration result determination module 1402 is used to determine the current integration result based on the current classification result. The current integration result includes the probability of the target image belonging to each category among the multiple categories after integration.

[0330] Image category determination module 1403 is used to determine the category of the target image based on the current integration results.

[0331] Optionally, the probability of the first category in the current integration result includes: the average probability of the first category among multiple classification results, where the first category is one of the multiple categories, and the multiple classification results include the classification result output by neural network model a and the classification result output by neural network model i; or,

[0332] The probability of the first category in the current integration result includes: the probability of the first category in the classification result output by neural network model i.

[0333] Optionally, the image category determination module 1403 is specifically used to: determine the category corresponding to the highest probability in the current integration result as the category of the target image; or,

[0334] The current neural network model also outputs the current selection result, which includes the probabilities of d neural network models out of m neural network models. The d neural network models are the neural network models that are allowed to be used to process the target image after neural network model i, and d is an integer greater than or equal to 1. The image category determination module 1403 is specifically used to: if the current integration result satisfies the first convergence condition or the current neural network model satisfies the second convergence condition, then determine the category corresponding to the highest probability in the current integration result as the category of the target image.

[0335] Optionally, the current integration result satisfies the first convergence condition, including: the confidence level of the current integration result is greater than the confidence level threshold.

[0336] Optionally, the device further includes:

[0337] The first confidence level determination module is used to determine the highest probability in the current integration result as the confidence level of the current integration result; or

[0338] The second confidence determination module is used to input the target image and the current integration result into the confidence network model to obtain the confidence of the current integration result output by the confidence network model.

[0339] Optionally, the current neural network model satisfies the second convergence condition, which includes: the number of iterations of the current neural network model is greater than or equal to the number of iterations threshold.

[0340] Optionally, the device further includes:

[0341] The sample acquisition module is used to acquire n sample images and n sample labels, with each of the n sample labels corresponding one-to-one with the n sample images;

[0342] The gradient determination module is used to determine the gradient of each parameter of each of the m training neural network models based on n sample images and n sample labels.

[0343] The model update module is used to update the m neural network models to be trained based on the gradients of each parameter of the m neural network models to be trained, so as to obtain the m trained neural network models.

[0344] Optionally, the gradient determination module includes:

[0345] The gradients of each parameter of the training neural network model k among the m training neural network models are determined according to the following sub-modules, until the gradients of each parameter of each training neural network model are determined:

[0346] The function value determination submodule is used to determine the training function value corresponding to the neural network model k to be trained based on n sample images and n sample labels;

[0347] The gradient determination submodule is used to determine the gradients of each parameter of the neural network model k to be trained based on the training function value corresponding to the model k.

[0348] Optionally, the training function values ​​include n classification function values, and each of the n classification function values ​​corresponds one-to-one with one of the n sample images;

[0349] The function value determination submodule is specifically used for:

[0350] Select one sample image from n sample images, and determine the classification function value corresponding to the selected sample image by performing the following operations, until the classification function value corresponding to each sample image is determined:

[0351] Input the selected sample image into the neural network model k to be trained, so as to obtain the classification result output by the neural network model k;

[0352] Based on the sample labels corresponding to the selected sample images and the classification results output by the neural network model k to be trained, the classification function value corresponding to the selected sample images is determined.

[0353] Optionally, the training function values ​​include n weight function values, and the n weight function values ​​correspond one-to-one with the n sample images;

[0354] The function value determination submodule is specifically used for:

[0355] Using b neural network models to be trained, determine the standard probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t standard probabilities, where t = mb. The b neural network models to be trained are the models that are allowed to be used to process n sample images before the m neural network models to be trained, and the t neural network models to be trained include the model to be trained, model k, and the models that are allowed to be used to process n sample images after the model to be trained, model k.

[0356] Based on n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images;

[0357] The first weight corresponding to each of the n sample images is normalized to obtain the weight function value corresponding to each of the n sample images.

[0358] Optionally, the function value determination submodule is specifically used for:

[0359] Using b neural network models to be trained, determine the average probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t average probabilities.

[0360] Divide the average probabilities of the n*t average probabilities into a probability set corresponding to the average probabilities of the same neural network model in the t training neural network models, so as to obtain t probability sets;

[0361] Based on n*t average probabilities and t probability sets, the n*t average probabilities are standardized to obtain n*t standard probabilities.

[0362] Optionally, the function value determination submodule is specifically used for:

[0363] From n sample images, select one sample image; from t trainable neural network models, select one to train; using b trainable neural network models, determine the average probability that the selected model will be used to process the selected sample image, until n*t average probabilities are determined:

[0364] For each of the b trainable neural network models, determine the probability that the selected trainable neural network model is used to process the selected sample image, so as to obtain b probabilities.

[0365] Determine the average of b probabilities to obtain the average probability that the selected neural network model to be trained is used to process the selected sample images.

[0366] Optionally, the function value determination submodule is specifically used for:

[0367] Choose one probability set from the t probability sets, and standardize each average probability in the selected probability set according to the following operation until n*t standard probabilities are determined:

[0368] Determine the mean and standard deviation of the average probability in the selected probability set;

[0369] Based on the average probability, mean, and standard deviation of the selected probability set, determine the standard probability corresponding to each average probability in the selected probability set.

[0370] Optionally, the function value determination submodule is specifically used for:

[0371] Construct a standard probability set from n*t standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set;

[0372] Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set;

[0373] From n sample images, determine the sample image corresponding to the maximum relative probability value; and from t neural network models to be trained, determine the neural network model to be trained corresponding to the maximum relative probability value.

[0374] Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value;

[0375] Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set;

[0376] If the standard probability set is not empty after removal, the number of model determinations is updated. The number of model determinations refers to the number of times the model corresponding to the maximum relative probability value is determined from t models to be trained.

[0377] If the model determines the number of times greater than or equal to n / t, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0378] Optionally, the function value determination submodule is specifically used for:

[0379] If the neural network model to be trained corresponding to the maximum relative probability value is model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as...

[0380] If the neural network model to be trained corresponding to the maximum relative probability value is not model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as...

[0381] Where a is a real number that satisfies a≥0 and a≤1.

[0382] Optionally, the training function values ​​include n label function values, and the n label function values ​​correspond one-to-one with the n sample images;

[0383] The function value determination submodule is specifically used for:

[0384] By determining the standard probability of each sample image belonging to the corresponding sample label in n sample images through each of the s training neural network models, n*s standard probabilities are obtained. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0385] Based on n*s standard probabilities, determine the label function value corresponding to each sample image in the n sample images.

[0386] Optionally, the function value determination submodule is specifically used for:

[0387] Construct a standard probability set from n*s standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set;

[0388] Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set;

[0389] From n sample images, determine the sample image corresponding to the maximum relative probability value, and from s neural network models to be trained, determine the neural network model to be trained corresponding to the maximum relative probability value;

[0390] The position number of the neural network model to be trained corresponding to the maximum probability relative value among the s neural network models to be trained is determined as the label function value of the sample image corresponding to the maximum probability relative value;

[0391] Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set;

[0392] If the standard probability set is not empty after removal, the number of model determinations is updated. The number of model determinations refers to the number of times the model corresponding to the maximum relative probability value is determined from s models to be trained.

[0393] If the model determines the number of times greater than or equal to n / s, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0394] Optionally, the function value determination submodule is specifically used for:

[0395] Select a standard probability from the set of standard probabilities, and determine the relative probability value corresponding to the selected standard probability by performing the following operations until the relative probability value corresponding to each standard probability is determined:

[0396] From the standard probability set other than the selected standard probability, determine multiple standard probabilities that correspond to the same sample image as the selected standard probability;

[0397] Determine the largest standard probability among these multiple standard probabilities;

[0398] The difference between the selected standard probability and the maximum standard probability is determined as the relative probability value corresponding to the selected standard probability.

[0399] Optionally, the training function values ​​include n classification loss function values, and each of the n classification loss function values ​​corresponds one-to-one with one of the n sample images;

[0400] The function value determination submodule is specifically used for:

[0401] Select one sample image from the n sample images, and determine the classification loss function value corresponding to the selected sample image by performing the following operations, until the classification loss function value corresponding to each sample image is determined:

[0402] Using the neural network model k to be trained, determine the probability that each of the s neural network models to be trained is used to process the selected sample image, so as to obtain s probabilities;

[0403] Based on the label function value and s probabilities corresponding to the selected sample image, determine the classification loss function value corresponding to the selected sample image.

[0404] Optionally, the training function values ​​include n model selection weight function values, and the n model selection weight function values ​​correspond one-to-one with the n sample images;

[0405] The function value determination submodule is specifically used for:

[0406] By determining the standard probability of each sample image belonging to the corresponding sample label in n sample images through each of the s training neural network models, n*s standard probabilities are obtained. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0407] Based on n*s standard probabilities, determine the model selection weight function value for each sample image among the n sample images.

[0408] Optionally, the function value determination submodule is specifically used for:

[0409] By using each of the s training neural network models, determine the probability that each sample image in the n sample images belongs to the corresponding sample label, so as to obtain n*s probabilities;

[0410] Divide the probability of n*s probability into a probability set corresponding to the same neural network model in s training neural network models, so as to obtain s probability sets;

[0411] Based on n*s probabilities and s probability sets, the n*s probabilities are standardized to obtain n*s standard probabilities.

[0412] Optionally, the function value determination submodule is specifically used for:

[0413] Divide the probability of the same sample image in the n sample images into a probability set from the n*s standard probabilities to obtain n probability sets;

[0414] Determine the standard deviation of the standard probability included in each of the n probability sets to obtain n standard deviations;

[0415] Normalize each of the n standard deviations to obtain the model selection weight function value for each of the n sample images.

[0416] In this embodiment, after inputting the target image into the current neural network model, the current neural network model not only outputs the current classification result for the target image, but also determines the probability of the neural network model allowed to process the target image after the current neural network model. Furthermore, the higher the probability of the neural network model allowed to process the target image after the current neural network model, the higher the classification accuracy of that neural network model for the target image. Generally, the neural network model with the highest probability has a higher classification accuracy for the target image than the current neural network model. Therefore, by performing iterative processing in this manner, the accuracy of image classification can be improved. In other words, based on the properties of the target image, this embodiment can dynamically select a portion of the neural network models from the m neural network models for image classification, eliminating the need to perform image classification through each neural network model, thus reducing the computational load. Moreover, since only a portion of the neural network models are selected, only the parameters of a portion of the neural network models need to be read into memory, rather than the parameters of all neural network models, avoiding the problem of time-consuming memory read operations.

[0417] It should be noted that the image classification device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image classification device and the image classification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0418] Figure 15 This is a schematic diagram of the structure of a neural network model training device provided in an embodiment of this application. The device can be implemented as part or all of a computer device by software, hardware, or a combination of both. The computer device can be... Figure 1 The terminal or server shown. See also Figure 15 The device includes: a sample acquisition module 1501, a gradient determination module 1502, and a model update module 1503.

[0419] The sample acquisition module 1501 is used to acquire n sample images and n sample labels, with each of the n sample labels corresponding to one of the n sample images.

[0420] The gradient determination module 1502 is used to determine the gradient of each parameter of each of the m training neural network models based on n sample images and n sample labels.

[0421] The model update module 1503 is used to update the m neural network models to be trained based on the gradients of each parameter of the m neural network models to be trained, so as to obtain the m trained neural network models.

[0422] Optionally, the gradient determination module 1502 includes:

[0423] The gradients of each parameter of the training neural network model k among the m training neural network models are determined according to the following sub-modules, until the gradients of each parameter of each training neural network model are determined:

[0424] The function value determination submodule is used to determine the training function value corresponding to the neural network model k to be trained based on n sample images and n sample labels;

[0425] The gradient determination submodule is used to determine the gradients of each parameter of the neural network model k to be trained based on the training function value corresponding to the model k.

[0426] Optionally, the training function values ​​include n classification function values, and each of the n classification function values ​​corresponds one-to-one with one of the n sample images;

[0427] The function value determination submodule is specifically used for:

[0428] Select one sample image from n sample images, and determine the classification function value corresponding to the selected sample image by performing the following operations, until the classification function value corresponding to each sample image is determined:

[0429] Input the selected sample image into the neural network model k to be trained, so as to obtain the classification result output by the neural network model k;

[0430] Based on the sample labels corresponding to the selected sample images and the classification results output by the neural network model k to be trained, the classification function value corresponding to the selected sample images is determined.

[0431] Optionally, the training function values ​​include n weight function values, and the n weight function values ​​correspond one-to-one with the n sample images;

[0432] The function value determination submodule is specifically used for:

[0433] Using b training neural network models, determine the standard probability that each of the t training neural network models is used to process each of the n sample images, so as to obtain n*t standard probabilities, where t = mb. The b training neural network models are the models that are allowed to process n sample images before the training neural network model k among the m training neural network models. The t training neural network models include the training neural network model k and the models that are allowed to process n sample images after the training neural network model k among the m training neural network models.

[0434] Based on n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images;

[0435] The first weight corresponding to each of the n sample images is normalized to obtain the weight function value corresponding to each of the n sample images.

[0436] Optionally, the function value determination submodule is specifically used for:

[0437] Using b neural network models to be trained, determine the average probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t average probabilities.

[0438] Divide the average probabilities of the n*t average probabilities into a probability set corresponding to the average probabilities of the same neural network model in the t training neural network models, so as to obtain t probability sets;

[0439] Based on n*t average probabilities and t probability sets, the n*t average probabilities are standardized to obtain n*t standard probabilities.

[0440] Optionally, the function value determination submodule is specifically used for:

[0441] From n sample images, select one sample image; from t trainable neural network models, select one to train; using b trainable neural network models, determine the average probability that the selected model will be used to process the selected sample image, and continue this process until n*t average probabilities are determined:

[0442] For each of the b trainable neural network models, determine the probability that the selected trainable neural network model is used to process the selected sample image, so as to obtain b probabilities.

[0443] Determine the average probability of b to obtain the average probability that the selected neural network model to be trained is used to process the selected sample images.

[0444] Optionally, the function value determination submodule is specifically used for:

[0445] Choose one probability set from the t probability sets, and standardize each average probability in the selected probability set according to the following operation until n*t standard probabilities are determined:

[0446] Determine the mean and standard deviation of the average probability in the selected probability set;

[0447] Based on the average probability, mean, and standard deviation of the selected probability set, determine the standard probability corresponding to each average probability in the selected probability set.

[0448] Optionally, the function value determination submodule is specifically used for:

[0449] Construct a standard probability set from n*t standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set;

[0450] Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set;

[0451] From n sample images, determine the sample image corresponding to the maximum relative probability value; and from t neural network models to be trained, determine the neural network model to be trained corresponding to the maximum relative probability value.

[0452] Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value;

[0453] Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set;

[0454] If the standard probability set is not empty after removal, the number of model determinations is updated. The number of model determinations refers to the number of times the model corresponding to the maximum relative probability value is determined from t models to be trained.

[0455] If the model determines the number of times greater than or equal to n / t, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0456] Optionally, the function value determination submodule is specifically used for:

[0457] If the neural network model to be trained corresponding to the maximum relative probability value is model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as...

[0458] If the neural network model to be trained corresponding to the maximum relative probability value is not model k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as...

[0459] Where a is a real number that satisfies a≥0 and a≤1.

[0460] Optionally, the training function values ​​include n label function values, and the n label function values ​​correspond one-to-one with the n sample images;

[0461] The function value determination submodule is specifically used for:

[0462] By determining the standard probability of each sample image belonging to the corresponding sample label in n sample images through each of the s training neural network models, n*s standard probabilities are obtained. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0463] Based on n*s standard probabilities, determine the label function value corresponding to each sample image in the n sample images.

[0464] Optionally, the function value determination submodule is specifically used for:

[0465] Construct a standard probability set from n*s standard probabilities, and determine the relative probability value corresponding to each standard probability in the standard probability set;

[0466] Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set;

[0467] From n sample images, determine the sample image corresponding to the maximum relative probability value, and from s neural network models to be trained, determine the neural network model to be trained corresponding to the maximum relative probability value;

[0468] The position number of the neural network model to be trained corresponding to the maximum probability relative value among the s neural network models to be trained is determined as the label function value of the sample image corresponding to the maximum probability relative value;

[0469] Remove the standard probabilities corresponding to the sample images with the highest relative probabilities from the standard probability set;

[0470] If the standard probability set is not empty after removal, the number of model determinations is updated. The number of model determinations refers to the number of times the model corresponding to the maximum relative probability value is determined from s models to be trained.

[0471] If the model determines the number of times greater than or equal to n / s, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

[0472] Optionally, the function value determination submodule is specifically used for:

[0473] Select a standard probability from the set of standard probabilities, and determine the relative probability value corresponding to the selected standard probability by performing the following operations until the relative probability value corresponding to each standard probability is determined:

[0474] From the standard probability set other than the selected standard probability, determine multiple standard probabilities that correspond to the same sample image as the selected standard probability;

[0475] Determine the largest standard probability among these multiple standard probabilities;

[0476] The difference between the selected standard probability and the maximum standard probability is determined as the relative probability value corresponding to the selected standard probability.

[0477] Optionally, the training function values ​​include n classification loss function values, and each of the n classification loss function values ​​corresponds one-to-one with one of the n sample images;

[0478] The function value determination submodule is specifically used for:

[0479] Select one sample image from the n sample images, and determine the classification loss function value corresponding to the selected sample image by performing the following operations, until the classification loss function value corresponding to each sample image is determined:

[0480] Using the neural network model k to be trained, determine the probability that each of the s neural network models to be trained is used to process the selected sample image, so as to obtain s probabilities;

[0481] Based on the label function value and s probabilities corresponding to the selected sample image, determine the classification loss function value corresponding to the selected sample image.

[0482] Optionally, the training function values ​​include n model selection weight function values, and the n model selection weight function values ​​correspond one-to-one with the n sample images;

[0483] The function value determination submodule is specifically used for:

[0484] By determining the standard probability of each sample image belonging to the corresponding sample label in n sample images through each of the s training neural network models, n*s standard probabilities are obtained. The s training neural network models are the models that are allowed to be used to process n sample images after the training neural network model k in the m training neural network models.

[0485] Based on n*s standard probabilities, determine the model selection weight function value for each sample image among the n sample images.

[0486] Optionally, the function value determination submodule is specifically used for:

[0487] By using each of the s training neural network models, determine the probability that each sample image in the n sample images belongs to the corresponding sample label, so as to obtain n*s probabilities;

[0488] Divide the probability of n*s probability into a probability set corresponding to the same neural network model in s training neural network models, so as to obtain s probability sets;

[0489] Based on n*s probabilities and s probability sets, the n*s probabilities are standardized to obtain n*s standard probabilities.

[0490] Optionally, the function value determination submodule is specifically used for:

[0491] Divide the probability of the same sample image in the n sample images into a probability set from the n*s standard probabilities to obtain n probability sets;

[0492] Determine the standard deviation of the standard probability included in each of the n probability sets to obtain n standard deviations;

[0493] Normalize each of the n standard deviations to obtain the model selection weight function value for each of the n sample images.

[0494] In this embodiment, after inputting the target image into the current neural network model, the current neural network model not only outputs the current classification result for the target image, but also determines the probability of the neural network model allowed to process the target image after the current neural network model. Furthermore, the higher the probability of the neural network model allowed to process the target image after the current neural network model, the higher the classification accuracy of that neural network model for the target image. Generally, the neural network model with the highest probability has a higher classification accuracy for the target image than the current neural network model. Therefore, by performing iterative processing in this manner, the accuracy of image classification can be improved. In other words, based on the properties of the target image, this embodiment can dynamically select a portion of the neural network models from the m neural network models for image classification, eliminating the need to perform image classification through each neural network model, thus reducing the computational load. Moreover, since only a portion of the neural network models are selected, only the parameters of a portion of the neural network models need to be read into memory, rather than the parameters of all neural network models, avoiding the problem of time-consuming memory read operations.

[0495] It should be noted that the image classification device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the image classification device and the image classification method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0496] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital versatile disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)). It is worth noting that the computer-readable storage medium mentioned in the embodiments of this application can be a non-volatile storage medium; in other words, it can be a non-transient storage medium.

[0497] It should be understood that "multiple" as mentioned herein refers to two or more. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. In addition, to facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first," "second," etc., are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and the terms "first," "second," etc., do not necessarily imply that they are different.

[0498] The above descriptions are embodiments provided in this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. An image classification method, characterized in that, The method includes: The target image to be classified is processed by the current neural network model to obtain the current classification result output by the current neural network model; Wherein, the current classification result includes the probability that the target image belongs to each of the multiple categories, the current neural network model is the neural network model i corresponding to the highest probability in the selection result output by neural network model a, the selection result includes the probability corresponding to p neural network models among the m trained neural network models, the p neural network models are the neural network models allowed to process the target image after neural network model a, the p neural network models include neural network model i, the m neural network models include neural network model a, m is an integer greater than 1, p is an integer greater than or equal to 1, and p is less than m; The current integration result is determined based on the current classification result, and the current integration result includes the probability of the target image belonging to each of the multiple categories after integration; The category of the target image is determined based on the current integration results.

2. The method as described in claim 1, characterized in that, The probability of the first category in the current integration result includes: the average probability of the first category among multiple classification results, where the first category is one of the multiple categories, and the multiple classification results include the classification result output by neural network model a and the classification result output by neural network model i; or, The probability of the first category in the current integration result includes: the probability of the first category in the classification result output by the neural network model i.

3. The method as described in claim 1, characterized in that, Determining the category of the target image based on the current integration result includes: determining the category corresponding to the highest probability in the current integration result as the category of the target image; or, The current neural network model also outputs a current selection result, which includes the probabilities corresponding to d neural network models out of the m neural network models. The d neural network models are those allowed to be used to process the target image after the neural network model i, where d is an integer greater than or equal to 1 and less than m. Determining the category of the target image based on the current integration result includes: if the current integration result satisfies a first convergence condition or the current neural network model satisfies a second convergence condition, then the category corresponding to the highest probability in the current integration result is determined as the category of the target image.

4. The method as described in claim 3, characterized in that, The current integration result satisfies the first convergence condition as follows: the confidence level of the current integration result is greater than the confidence level threshold.

5. The method as described in claim 4, characterized in that, After determining the current integration result based on the current classification result, the process also includes: The highest probability in the current integration result is determined as the confidence level of the current integration result; or The target image and the current integration result are input into the confidence network model to obtain the confidence level of the current integration result output by the confidence network model.

6. The method according to any one of claims 3-5, characterized in that, The current neural network model satisfies the second convergence condition by: the number of iterations of the current neural network model being greater than or equal to the number of iterations threshold.

7. The method according to any one of claims 1-5, characterized in that, Before processing the target image to be classified using the current neural network model, the process also includes: Obtain n sample images and n sample labels, wherein the n sample labels correspond one-to-one with the n sample images; Based on the n sample images and the n sample labels, determine the gradient of each parameter of each of the m neural network models to be trained; Based on the gradients of each parameter of the m neural network models to be trained, the m neural network models to be trained are updated to obtain the m trained neural network models.

8. The method as described in claim 7, characterized in that, The step of determining the gradient of each parameter of each of the m neural network models to be trained, based on the n sample images and the n sample labels, includes: The gradients of each parameter of the training neural network model k among the m training neural network models are determined by the following steps, until the gradients of each parameter of each training neural network model are determined: Based on the n sample images and the n sample labels, determine the training function value corresponding to the neural network model k to be trained; Based on the training function value corresponding to the neural network model k to be trained, the gradient of each parameter of the neural network model k to be trained is determined.

9. The method as described in claim 8, characterized in that, The training function values ​​include n classification function values, and each of the n classification function values ​​corresponds one-to-one with one of the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Select one sample image from the n sample images, and determine the classification function value corresponding to the selected sample image by performing the following operations, until the classification function value corresponding to each sample image is determined: The selected sample image is input into the neural network model k to be trained, so as to obtain the classification result output by the neural network model k to be trained; Based on the sample label corresponding to the selected sample image and the classification result output by the neural network model k to be trained, the classification function value corresponding to the selected sample image is determined.

10. The method as described in claim 8, characterized in that, The training function values ​​include n weight function values, and each of the n weight function values ​​corresponds one-to-one with one of the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Using b training neural network models, determine the standard probability that each of the t training neural network models is used to process each of the n sample images, to obtain n*t standard probabilities, where t=mb. The b training neural network models are the models among the m training neural network models that are allowed to process the n sample images before the training neural network model k. The t training neural network models include the training neural network model k and the models among the m training neural network models that are allowed to process the n sample images after the training neural network model k. Based on the n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images; The first weight corresponding to each of the n sample images is normalized to obtain the weight function value corresponding to each of the n sample images.

11. The method as described in claim 10, characterized in that, The step of determining the standard probability of each of the t training neural network models used to process each of the n sample images from the b training neural network models, to obtain n*t standard probabilities, includes: Using the b neural network models to be trained, determine the average probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t average probabilities; The average probabilities of the n*t average probabilities that correspond to the same neural network model in the t neural network models to be trained are divided into a probability set to obtain t probability sets; Based on the n*t average probabilities and the t probability sets, the n*t average probabilities are standardized to obtain the n*t standard probabilities.

12. The method as described in claim 11, characterized in that, The step of determining the average probability that each of the b trainable neural network models is used to process each of the n sample images, in order to obtain n*t average probabilities, includes: From the n sample images, select one sample image; from the t trainable neural network models, select one trainable neural network model; using the b trainable neural network models, determine the average probability that the selected trainable neural network model is used to process the selected sample image, until the n*t average probabilities are determined: By using each of the b training neural network models, the probability that the selected training neural network model is used to process the selected sample image is determined, so as to obtain b probabilities; The average of the b probabilities is determined to obtain the average probability that the selected neural network model to be trained is used to process the selected sample image.

13. The method as described in claim 11, characterized in that, The standardization process, based on the n*t average probabilities and the t probability sets, to obtain the n*t standard probabilities includes: From the t probability sets, select one probability set, and standardize each average probability in the selected probability set according to the following operation until the n*t standard probabilities are determined: Determine the mean and standard deviation of the average probability in the selected probability set; Based on the average probability, the mean, and the standard deviation in the selected probability set, the standard probability corresponding to each average probability in the selected probability set is determined.

14. The method as described in claim 10, characterized in that, The step of determining the first weight corresponding to each sample image among the n sample images based on the n*t standard probabilities includes: The n*t standard probabilities are used to form a standard probability set, and the relative probability value corresponding to each standard probability in the standard probability set is determined. Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set; The sample image corresponding to the maximum relative probability value is determined from the n sample images, and the neural network model to be trained corresponding to the maximum relative probability value is determined from the t neural network models to be trained; Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value; Remove the standard probability corresponding to the sample image that corresponds to the maximum relative probability value from the standard probability set; If the standard probability set is not empty after removal, the number of model determinations is updated, whereby the number of model determinations refers to the number of times the neural network model corresponding to the maximum relative probability value is determined from the t neural network models to be trained. If the model determines that the number of occurrences is greater than or equal to n / t, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

15. The method as described in claim 14, characterized in that, The step of determining the first weight of the sample image corresponding to the maximum probability relative value based on the neural network model to be trained includes: If the neural network model to be trained corresponding to the maximum relative probability value is the neural network model to be trained k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as follows: ; If the neural network model to be trained corresponding to the maximum relative probability value is not the neural network model to be trained k, then the first weight of the sample image corresponding to the maximum relative probability value is determined as... ; in, For a given real number, satisfying ,and .

16. The method as described in claim 8, characterized in that, The training function values ​​include n label function values, and each of the n label function values ​​corresponds one-to-one with one of the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: By using each of the s training neural network models, the standard probability of each sample image belonging to the corresponding sample label in the n sample images is determined to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process the n sample images after the training neural network model k among the m training neural network models. Based on the n*s standard probabilities, determine the label function value corresponding to each of the n sample images.

17. The method as described in claim 16, characterized in that, The step of determining the label function value corresponding to each of the n sample images based on the n*s standard probabilities includes: The n*s standard probabilities are used to form a standard probability set, and the relative probability value corresponding to each standard probability in the standard probability set is determined. Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set; The sample image corresponding to the maximum relative probability value is determined from the n sample images, and the neural network model to be trained corresponding to the maximum relative probability value is determined from the s neural network models to be trained; The position number of the neural network model to be trained corresponding to the maximum probability relative value among the s neural network models to be trained is determined as the label function value of the sample image corresponding to the maximum probability relative value; Remove the standard probability corresponding to the sample image that corresponds to the maximum relative probability value from the standard probability set; If the standard probability set is not empty after removal, the number of model determinations is updated, whereby the number of model determinations refers to the number of times the neural network model corresponding to the maximum relative probability value is determined from the s neural network models to be trained; If the model determines that the number of occurrences is greater than or equal to n / s, then the standard probability corresponding to the neural network model to be trained corresponding to the maximum relative probability value is removed from the standard probability set, and the step of determining the relative probability value corresponding to each standard probability in the standard probability set is returned.

18. The method as described in claim 14 or 17, characterized in that, Determining the relative probability value corresponding to each standard probability in the standard probability set includes: Select a standard probability from the set of standard probabilities, and determine the relative probability value corresponding to the selected standard probability by performing the following operations, until the relative probability value corresponding to each standard probability is determined: From the standard probability set other than the selected standard probability, determine multiple standard probabilities that correspond to the same sample image as the selected standard probability; Determine the maximum standard probability among the plurality of standard probabilities; The difference between the selected standard probability and the maximum standard probability is determined as the relative probability value corresponding to the selected standard probability.

19. The method as described in claim 16, characterized in that, The training function values ​​include n classification loss function values, and each of the n classification loss function values ​​corresponds one-to-one with one of the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Select one sample image from the n sample images, and determine the classification loss function value corresponding to the selected sample image by performing the following operations, until the classification loss function value corresponding to each sample image is determined: Using the neural network model k to be trained, determine the probability that each of the s neural network models to be trained is used to process the selected sample image, so as to obtain s probabilities; Based on the label function value corresponding to the selected sample image and the s probabilities, the classification loss function value corresponding to the selected sample image is determined.

20. The method as described in claim 8, characterized in that, The training function values ​​include n model selection weight function values, and the n model selection weight function values ​​correspond one-to-one with the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: By using each of the s training neural network models, the standard probability of each sample image belonging to the corresponding sample label in the n sample images is determined to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process the n sample images after the training neural network model k among the m training neural network models. Based on the n*s standard probabilities, determine the model selection weight function value for each of the n sample images.

21. The method as described in claim 16 or 20, characterized in that, The step of determining the standard probability that each sample image in the n sample images belongs to the corresponding sample label through each of the s training neural network models to obtain n*s standard probabilities includes: By using each of the s training neural network models, the probability that each sample image in the n sample images belongs to the corresponding sample label is determined, so as to obtain n*s probabilities; The probabilities of the n*s probabilities that correspond to the same neural network model in the s neural network models to be trained are divided into a probability set to obtain s probability sets; Based on the n*s probabilities and the s probability sets, the n*s probabilities are standardized to obtain n*s standard probabilities.

22. The method as described in claim 20, characterized in that, The step of determining the model selection weight function value for each of the n sample images based on the n*s standard probabilities includes: The probabilities corresponding to the same sample image in the n sample images among the n*s standard probabilities are divided into a probability set to obtain n probability sets; Determine the standard deviation of the standard probability included in each of the n probability sets to obtain n standard deviations; Normalize each of the n standard deviations to obtain the model selection weight function value corresponding to each of the n sample images.

23. A method for training a neural network model, characterized in that, The method includes: Obtain n sample images and n sample labels, wherein the n sample labels correspond one-to-one with the n sample images; The gradients of each parameter of the training neural network model k among the m training neural network models are determined by the following steps until the gradients of each parameter of each training neural network model are determined: Based on the n sample images and the n sample labels, the training function value corresponding to the training neural network model k is determined; Based on the training function value corresponding to the training neural network model k, the gradients of each parameter of the training neural network model k are determined. Based on the gradients of each parameter of the m neural network models to be trained, the m neural network models to be trained are updated to obtain the m trained neural network models. The training function values ​​include at least one of the following: n weight function values, n label function values, n classification loss function values, and n model selection weight function values. The n weight function values ​​are determined by the model among the m neural network models to be trained that is allowed to process the n sample images before the neural network model k to be trained. The n label function values, the n classification loss function values, and the n model selection weight function values ​​are determined by the model among the m neural network models to be trained that is allowed to process the n sample images after the neural network model k to be trained.

24. The method as described in claim 23, characterized in that, The training function values ​​include n classification function values, and each of the n classification function values ​​corresponds one-to-one with one of the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Select one sample image from the n sample images, and determine the classification function value corresponding to the selected sample image by performing the following operations, until the classification function value corresponding to each sample image is determined: The selected sample image is input into the neural network model k to be trained, so as to obtain the classification result output by the neural network model k to be trained; Based on the sample label corresponding to the selected sample image and the classification result output by the neural network model k to be trained, the classification function value corresponding to the selected sample image is determined.

25. The method as described in claim 23, characterized in that, The training function values ​​include the n weight function values, and the n weight function values ​​correspond one-to-one with the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Using b training neural network models, determine the standard probability that each of the t training neural network models is used to process each of the n sample images, to obtain n*t standard probabilities, where t=mb. The b training neural network models are the models among the m training neural network models that are allowed to process the n sample images before the training neural network model k. The t training neural network models include the training neural network model k and the models among the m training neural network models that are allowed to process the n sample images after the training neural network model k. Based on the n*t standard probabilities, determine the first weight corresponding to each sample image in the n sample images; The first weight corresponding to each of the n sample images is normalized to obtain the weight function value corresponding to each of the n sample images.

26. The method as described in claim 25, characterized in that, The step of determining the standard probability of each of the t training neural network models used to process each of the n sample images from the b training neural network models, to obtain n*t standard probabilities, includes: Using the b neural network models to be trained, determine the average probability that each of the t neural network models to be trained is used to process each of the n sample images, so as to obtain n*t average probabilities; The average probabilities of the n*t average probabilities that correspond to the same neural network model in the t neural network models to be trained are divided into a probability set to obtain t probability sets; Based on the n*t average probabilities and the t probability sets, the n*t average probabilities are standardized to obtain the n*t standard probabilities.

27. The method as described in claim 26, characterized in that, The step of determining the average probability that each of the b trainable neural network models is used to process each of the n sample images, in order to obtain n*t average probabilities, includes: From the n sample images, select one sample image; from the t trainable neural network models, select one trainable neural network model; using the b trainable neural network models, determine the average probability that the selected trainable neural network model is used to process the selected sample image, until the n*t average probabilities are determined: By using each of the b training neural network models, the probability that the selected training neural network model is used to process the selected sample image is determined, so as to obtain b probabilities; The average of the b probabilities is determined to obtain the average probability that the selected neural network model to be trained is used to process the selected sample image.

28. The method as described in claim 25, characterized in that, The step of determining the first weight corresponding to each sample image among the n sample images based on the n*t standard probabilities includes: The n*t standard probabilities are used to form a standard probability set, and the relative probability value corresponding to each standard probability in the standard probability set is determined. Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set; The sample image corresponding to the maximum relative probability value is determined from the n sample images, and the neural network model to be trained corresponding to the maximum relative probability value is determined from the t neural network models to be trained; Based on the neural network model to be trained corresponding to the maximum probability relative value, determine the first weight of the sample image corresponding to the maximum probability relative value; Remove the standard probability corresponding to the sample image that corresponds to the maximum relative probability value from the standard probability set; If the standard probability set is not empty after removal, the number of model determinations is updated, whereby the number of model determinations refers to the number of times the neural network model corresponding to the maximum relative probability value is determined from the t neural network models to be trained. If the model determines that the number of occurrences is greater than or equal to n / t, then remove the standard probability corresponding to the neural network model to be trained that corresponds to the maximum relative probability value from the standard probability set, and return to the step of determining the relative probability value corresponding to each standard probability in the standard probability set.

29. The method as described in claim 23, characterized in that, The training function values ​​include the n label function values, and the n label function values ​​correspond one-to-one with the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: By using each of the s training neural network models, the standard probability of each sample image belonging to the corresponding sample label in the n sample images is determined to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process the n sample images after the training neural network model k among the m training neural network models. Based on the n*s standard probabilities, determine the label function value corresponding to each of the n sample images.

30. The method as described in claim 29, characterized in that, The step of determining the label function value corresponding to each of the n sample images based on the n*s standard probabilities includes: The n*s standard probabilities are used to form a standard probability set, and the relative probability value corresponding to each standard probability in the standard probability set is determined. Determine the maximum relative probability value from the relative probability values ​​corresponding to the standard probabilities in the standard probability set; The sample image corresponding to the maximum relative probability value is determined from the n sample images, and the neural network model to be trained corresponding to the maximum relative probability value is determined from the s neural network models to be trained; The position number of the neural network model to be trained corresponding to the maximum probability relative value among the s neural network models to be trained is determined as the label function value of the sample image corresponding to the maximum probability relative value; Remove the standard probability corresponding to the sample image that corresponds to the maximum relative probability value from the standard probability set; If the standard probability set is not empty after removal, the number of model determinations is updated, whereby the number of model determinations refers to the number of times the neural network model corresponding to the maximum relative probability value is determined from the s neural network models to be trained; If the model determines that the number of occurrences is greater than or equal to n / s, then the standard probability corresponding to the neural network model to be trained corresponding to the maximum relative probability value is removed from the standard probability set, and the step of determining the relative probability value corresponding to each standard probability in the standard probability set is returned.

31. The method as described in claim 23, characterized in that, The training function values ​​include the n classification loss function values, and the n classification loss function values ​​correspond one-to-one with the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: Select one sample image from the n sample images, and determine the classification loss function value corresponding to the selected sample image by performing the following operations, until the classification loss function value corresponding to each sample image is determined: Using the neural network model k to be trained, determine the probability that each of the s neural network models to be trained is used to process the selected sample image, so as to obtain s probabilities. The s neural network models to be trained are the models among the m neural network models to be trained that are allowed to be used to process the n sample images after the neural network model k. Based on the label function value corresponding to the selected sample image and the s probabilities, the classification loss function value corresponding to the selected sample image is determined.

32. The method as described in claim 23, characterized in that, The training function values ​​include the n model selection weight function values, and the n model selection weight function values ​​correspond one-to-one with the n sample images; The step of determining the training function value corresponding to the neural network model k to be trained based on the n sample images and the n sample labels includes: By using each of the s training neural network models, the standard probability of each sample image belonging to the corresponding sample label in the n sample images is determined to obtain n*s standard probabilities. The s training neural network models are the models that are allowed to be used to process the n sample images after the training neural network model k among the m training neural network models. Based on the n*s standard probabilities, determine the model selection weight function value for each of the n sample images.

33. A computer device, characterized in that, The computer device includes a memory and a processor; The memory is used to store computer program instructions, and the processor is configured to execute the computer program instructions stored in the memory to implement the steps of the method according to any one of claims 1-32.

34. A computer-readable storage medium, characterized in that, The storage medium stores instructions that, when executed on a computer device, cause the computer device to perform the steps of the method described in any one of claims 1-32.

35. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a computer device, cause the computer device to perform the steps of the method according to any one of claims 1-32.

Images