Task processing model training, task processing method, apparatus, and device

Abstract

The application provides a task processing model training method, a task processing method, a device and equipment. The task processing model training method comprises: obtaining a training data set, the training data set comprising a plurality of training data, each training data corresponding to a label, the label being related to a function possessed by a user demand task processing model; iteratively training a preset model using the training data set to obtain a task processing model; a weight distribution matrix used by the preset model in the i-th iteration training is obtained by adjusting a weight distribution matrix updated based on a gradient descent method in the i-1-th iteration training of the preset model, the task processing model is used for processing a target task, the target task is a function possessed by the user demand task processing model, i is an integer greater than or equal to 2; and the task processing model is output. The method reduces the overfitting phenomenon of the task processing model obtained by training, and improves the prediction accuracy of the task processing model.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to a task processing model training, task processing method, apparatus and device. Background Technology

[0002] Bayesian convolutional neural networks have been applied in many fields due to their uncertainty estimation properties.

[0003] A Bayesian convolutional neural network (CNN) applies the Bayesian method to the convolutional layers of a convolutional neural network, representing the weight parameters of the convolutional layers as a probability distribution. This probability distribution can well express uncertain prediction results. The training process of a Bayesian CNN is similar to that of a regular neural network. First, an objective function is determined, and the value of the objective function is determined based on the results and labels obtained in each iteration of training. Then, the weight parameters of the current iteration of training are adjusted according to the objective function value until training is completed, resulting in a trained model.

[0004] However, this training method can lead to overfitting in the trained model, resulting in lower prediction accuracy. Summary of the Invention

[0005] This application provides a task processing model training, task processing method, apparatus, and device to solve the problem that the trained model in existing training methods for Bayesian convolutional neural networks suffers from overfitting, resulting in low prediction accuracy.

[0006] In a first aspect, this application provides a method for training a task processing model, comprising: acquiring a training dataset, wherein the training dataset includes multiple training data, each training data corresponding to a label, the label being related to the functions of the task processing model required by the user; iteratively training a preset model using the training dataset to obtain a task processing model; wherein the weight distribution matrix used by the preset model in the i-th iteration of training is obtained by adjusting the weight distribution matrix used by the preset model in the (i-1)-th iteration of training, the task processing model being used to process a target task, the target task being the functions of the task processing model required by the user, wherein i is an integer greater than or equal to 2; and outputting the task processing model.

[0007] Secondly, this application provides a task processing method, comprising: receiving an image corresponding to a task to be processed; inputting the image corresponding to the task to be processed into a task processing model obtained based on the task processing model training method described in the first aspect, to obtain a task processing result; and outputting the task processing result.

[0008] Thirdly, this application provides a task processing model training device, comprising: an acquisition module for acquiring a training dataset, the training dataset including multiple training data, each training data corresponding to a label, the label being related to the functions of the task processing model required by the user; a training module for iteratively training a preset model using the training dataset to obtain a task processing model; the weight distribution matrix used by the preset model in the i-th iteration of training is obtained by adjusting the weight distribution matrix used by the preset model in the (i-1)-th iteration of training, the task processing model being used to process a target task, the target task being the functions of the task processing model required by the user, wherein i is an integer greater than or equal to 2; and an output module for outputting the task processing model.

[0009] Fourthly, this application provides a task processing apparatus, comprising: a receiving module for receiving an image corresponding to a task to be processed; an input module for inputting the image corresponding to the task to be processed into a task processing model obtained based on the task processing model training method described in the first aspect, to obtain a task processing result; and an output module for outputting the task processing result.

[0010] Fifthly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect.

[0011] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method described in the first aspect.

[0012] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0013] The task processing model training, task processing method, apparatus, and device provided in this application involve acquiring a training dataset, which includes multiple training data points, each with a corresponding label. The label is related to the functionality required by the user's task processing model. The training dataset is used to iteratively train a pre-defined model to obtain the task processing model. The weight distribution matrix used by the pre-defined model in the i-th iteration is obtained by adjusting the weight distribution matrix updated using gradient descent in the (i-1)-th iteration. The task processing model is used to process the target task, which is the functionality required by the user's task processing model, where i is an integer greater than or equal to 2. The task processing model is then output. Because the weight distribution matrix used by the pre-defined model in the i-th iteration is obtained by adjusting the weight distribution matrix updated using gradient descent in the (i-1)-th iteration, the weight distribution curve in the weight distribution matrix obtained in each iteration is smooth. Reducing higher weights achieves regularization, preventing overfitting and improving the prediction accuracy of the trained pre-defined model in the task processing process. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0015] Figure 1 A neural network architecture diagram provided for embodiments of this application;

[0016] Figure 2 Example diagram of a convolutional neural network provided in the embodiments of this application;

[0017] Figure 3 A schematic diagram illustrating the principle of a convolutional neural network provided in an embodiment of this application;

[0018] Figure 4 Example diagrams for uncertainty estimation provided in embodiments of this application;

[0019] Figure 5 Example diagram of a Bayesian convolutional neural network provided in the embodiments of this application;

[0020] Figure 6 A schematic diagram illustrating the principle of the Bayesian convolutional neural network provided in the embodiments of this application;

[0021] Figure 7 Application scenario diagrams provided for embodiments of this application;

[0022] Figure 8 The flowchart of the task processing model training method provided in the embodiments of this application Figure 1 ;

[0023] Figure 9 The flow of the task processing method provided in the embodiments of this application Figure 2 ;

[0024] Figure 10 A schematic diagram of the structure of a task processing model training device provided in an embodiment of this application;

[0025] Figure 11 This is a schematic diagram of the structure of the task processing device provided in the embodiments of this application;

[0026] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0027] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0028] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0029] Artificial neural networks (ANNs), or neural networks (NNs) for short, are a hot research topic in the field of artificial intelligence. They are abstractions of the human brain's neuronal network from an information processing perspective, establishing a simple model and forming different networks with different connection methods. They are mathematical or computational models that mimic the structure and function of biological neural networks. With the continuous and in-depth research on artificial neural networks, they have successfully solved many practical problems that modern computers struggle with in fields such as pattern recognition, intelligent robotics, automatic control, prediction and estimation, biology, medicine, and economics, demonstrating excellent intelligent characteristics.

[0030] Figure 1 This is an architecture diagram of the neural network provided in an embodiment of this application. (For example...) Figure 1 As shown, a neural network includes an input layer, hidden layers, and an output layer; each of these layers contains multiple neurons (i.e., the circles shown in the diagram). Through... Figure 1As can be seen, the neural network is a fully connected neural network, meaning that each neuron in the input layer is connected to each neuron in the hidden layer through network parameters. However, for some multidimensional data, such as images, the computational cost of neural networks can be extremely high. For example, an n*n image can be represented as an n... 2 A vector. When applying a fully connected neural network to image processing, n is required. 2 There are input layers and n 2 If there are n hidden layers, then the network parameters between the input layer and the hidden layers need to be n. 2 *n 2 =n 4 Too many parameters can make the calculation process very slow, making training virtually impossible.

[0031] Convolutional Neural Networks (CNNs) effectively address the challenges of fully connected neural networks in handling multidimensional data. A CNN consists of an input layer, convolutional layers, pooling layers, and fully connected layers. The convolutional layer is the most crucial part of the entire CNN. Unlike fully connected layers, the input to each neuron in a convolutional layer is the output of a subset of neurons from the previous layer. Convolutional layers are primarily used for convolution operations. The principle of convolution operations is explained below with reference to the accompanying diagram:

[0032] Figure 2 An example diagram of a convolutional neural network provided in an embodiment of this application. For example... Figure 2 As shown, for 4*4*3 input data, the convolution operation uses the convolution kernel to extract features. Figure 2 The feature extraction was performed using a 2x2 convolutional kernel (i.e., a weight matrix), resulting in a 2x2 feature map. It can be seen that the convolutional neural network uses a fixed value for both feature extraction and output. This is related to the network parameters of the convolutional neural network.

[0033] Figure 3 This is a schematic diagram illustrating the principle of a convolutional neural network provided in an embodiment of this application. Figure 3 As shown, X represents the input, H1, H2, and H3 represent neurons, Y represents the output, and I represents the bias. The network parameters in a convolutional neural network are specific parameter values ​​(the values ​​shown on each arrow in the figure), and therefore the extracted features are also specific feature values.

[0034] The training process of a convolutional neural network (CNN) can be understood as including forward propagation (feedforward) and backpropagation (backward). Forward propagation refers to the process of inputting the training dataset into the CNN and obtaining the network output; each training data point in the dataset corresponds to one network output. Backpropagation involves adjusting the CNN's parameters based on the difference between the network output and the labels of the training data, until the CNN converges. In backpropagation, the network output is input into the objective function (also called the loss function) to calculate the error between the network output and the labels of the training data. This error is then used to adjust the network parameters, obtaining an optimal estimate of the network parameters. This optimal estimate provides a fixed value for the network parameters, supporting deterministic outputs. CNNs trained based on this optimal parameter estimation can fit the training dataset well, but are prone to overfitting. They fit the training dataset well but not other data not present in the training dataset. Regularization methods, such as early stopping, weight decay, L1-L2 regularization, and dropout, can address overfitting. However, the neural networks trained in this way all output deterministic results and cannot provide the degree of certainty about the prediction results, that is, they cannot measure uncertainty.

[0035] From the perspective of data distribution, this uncertainty arises because the training process fits a distribution that reflects the features of the training set, while the features of the test samples to be inferred may not conform to this distribution. This uncertainty mainly stems from the neural network's failure to learn enough samples; that is, the observed data cannot fully cover the features of the unobserved data (test samples). In this situation, the learned network parameters introduce significant uncertainty into the prediction.

[0036] This uncertainty is essentially because the observed data (training samples) is always limited, and the features learned by the network model are insufficient, which makes the network model's predictions about unobserved data (test samples) uncertain.

[0037] Figure 4 Example diagrams illustrating uncertainty estimation provided for embodiments of this application. (e.g.) Figure 4 As shown, because the network model was not trained on "non-hot dog" images, it predicts objects with ketchup (such as apples or bananas with ketchup) as hot dogs. In reality, it is impossible to exhaustively list all "non-hot dog" images, so this uncertainty needs to be addressed by the network model itself, rather than by increasing the dataset.

[0038] Bayesian methods excel on small datasets (sets of training data with limited quantity). They add a prior distribution to each weight and bias parameter of the network model and approximate the posterior distribution during training using a limited number of samples. This posterior distribution more accurately reflects the characteristics of the overall sample. BCNN models learn the distribution of network parameters on limited training samples, making the network parameter distribution approximate the distribution of the overall sample, thus enabling prediction of uncertainties in unobserved data. Bayesian methods transform the optimal point estimation of network parameters into a distribution estimation, allowing for uncertainty estimation during inference. Due to the ability of Bayesian convolutional neural networks to measure uncertainty, they perform well in scenarios requiring high model prediction accuracy, such as medical imaging, security, and fraudulent transactions. For example, in these fields, the datasets are highly imbalanced; for instance, fraudulent transactions may account for only one ten-thousandth of all transactions, and spam may account for less than one percent of all emails. This imbalance can lead to overfitting of the model to the sampled categories.

[0039] Bayesian neural networks excel at learning from small datasets, providing further uncertainty estimates through their parameters in the form of probability distributions. Furthermore, by integrating parameters using prior probability distributions and calculating averages across many models during training, the network gains a regularization effect, preventing overfitting.

[0040] The following section will explain the differences between Bayesian convolutional neural networks with the accompanying diagrams to help readers better understand the distinction between convolutional neural networks and Bayesian convolutional neural networks.

[0041] Figure 5 An example diagram of a Bayesian convolutional neural network provided for embodiments of this application. (See diagram below.) Figure 5 As shown, for 4*4*3 input data, the convolution operation uses the convolution kernel to extract features. Figure 5 The algorithm uses 2x2 convolutional kernels (i.e., weight matrices) for feature extraction, resulting in a 2x2 feature map. It can be seen that the convolutional neural network uses a probability distribution in both feature extraction and output. This is because in a Bayesian convolutional neural network, the weights in the network parameters represent a probability distribution of those weights. In other words, the optimal estimation of network parameters in a convolutional neural network is transformed into an estimation of the probability distribution of those parameters. The weights are random variables, not fixed values ​​given in the convolutional neural network.

[0042] Figure 6 This is a schematic diagram illustrating the principle of a Bayesian convolutional neural network provided in an embodiment of this application. Figure 6As shown, X represents the input, H1, H2, and H3 represent neurons, Y represents the output, and I represents the bias. It can be seen that the features extracted based on the Bayesian convolutional neural network are feature distributions, and its final output is also the probability distribution of the prediction result. Figure 6 The curves shown in the figure represent a probability distribution. Therefore, Bayesian convolutional neural networks can effectively support uncertainty estimation.

[0043] The principle of a Bayesian convolutional neural network is as follows: given a prior distribution for the network parameters, gradient approximation is performed using variational inference. Then, based on the training dataset, a posterior distribution of the network parameters is learned and fitted, thus learning the posterior distribution from the training dataset. Finally, inference is performed on the test dataset based on this posterior distribution. The following concrete example illustrates the principle of a Bayesian convolutional neural network:

[0044] Assuming the posterior distribution of the Bayesian convolutional neural network is p(w|x,y), then the prediction of the test data based on this posterior distribution can be expressed as the following formula (1):

[0045]

[0046] In equation (1), The first and second terms in the equation represent the input and label of the test data, respectively. The prediction of the test data is based on the expectation E of the learned posterior distribution p(w|x,y), which is equivalent to the ensemble of countless convolutional layers. Due to the integral nature of the expectation, the above formula (1) is difficult to solve and cannot be embedded into the convolutional neural network for feedforward and backpropagation. Therefore, a posterior distribution q(w|θ) is found based on the approximate variational method to approximate the true posterior distribution p(w|x,y), where θ is the parameter of the distribution of weights w. The distance between the two distributions can be characterized by KL divergence. Therefore, the objective function of the Bayesian convolutional neural network can be expressed as the following formula (2):

[0047]

[0048] In formula (2) above, each term after the equal sign represents the derivation process of the objective function from top to bottom. The last equal sign corresponds to the final objective function of the Bayesian convolutional neural network. This objective function aims to minimize the first term and maximize the second term. The first term is the posterior distribution q(w|θ) and prior p(w) learned from the data (x,y). priorThe distance between q(w|θ) and the expectation of the prediction probability model p(y|x,w) based on the learned posterior distribution q(w|θ). This objective function has integral properties and is difficult to handle. Therefore, some literature constructs an objective function that can be embedded in a Bayesian convolutional neural network by approximating it with the unbiased Monte Carlo gradient, and applies the Monte Carlo sampling method and the expectation that the derivative of the expectation is equal to the expectation of the derivative to derive an approximate solution to the above objective function. That is, the objective function shown in formula (2) can be approximated as the following formula (3):

[0049] l(w,θ)≈logq(w|θ)-logp(w prior )-logp(y|x,w) (3)

[0050] In equation (3), the prior p(w) prior The posterior weights q(w|θ) are predefined, while the posterior weights w are sampled from the learned posterior distribution q(w|θ). prior The distribution parameter θ of the weight w is independent of the distribution parameter θ, so the derivative is zero. In this case, there is no prior constraint, which means that the regularization loses its meaning and thus cannot improve the model's generalization ability, that is, the model has a low predictive ability for unknown data.

[0051] Therefore, some literature also retains the first term of the original objective function, namely the posterior distribution q(w|θ) and the prior p(w). prior The KL divergence of the weights w is used as a regularization term to constrain the prior distribution. However, this method has two main problems: first, it is difficult to define a good prior; second, as an independent convolutional layer, the objective function with added KL divergence is difficult to integrate into the entire network for backpropagation to calculate gradients and update the parameters θ of the posterior distribution of the weights w.

[0052] In summary, the training methods for Bayesian convolutional neural networks in related technologies still cannot effectively solve the overfitting problem, resulting in low prediction accuracy when performing tasks. For example, in the field of facial recognition, this leads to low accuracy, which can cause errors in security monitoring if the facial recognition results are used. On the other hand, in traffic scene object detection, high accuracy leads to high accuracy, but errors in object detection can cause errors in safe driving warnings, resulting in traffic accidents.

[0053] In summary, the limitations of Bayesian convolutional neural networks (CNNs) have hindered their widespread application in scenarios requiring high prediction accuracy. Therefore, this application aims to address these limitations to improve the prediction accuracy of Bayesian networks, enabling their effective application in areas demanding high accuracy. The technical concept is as follows: the convolution kernel in a Bayesian CNN can be understood as a matrix relating the weight distribution. By constraining each weight distribution within this matrix through its maximum singular value, excessively high weight distributions are suppressed, thus achieving regularization, reducing overfitting, and improving the prediction accuracy of the trained model. Furthermore, this method does not require a prior distribution, allowing for the flexible embedding of Bayesian convolutions within the CNN to support uncertainty estimation.

[0054] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0055] Figure 7 Application scenario diagrams provided for embodiments of this application, such as Figure 7 As shown, the application scenario includes an image acquisition device 71 and a computing device 72; the image acquisition device 71 and the computing device 72 are communicatively connected, and the computing device 72 is equipped with a Bayesian convolutional neural network model.

[0056] Image acquisition device 71 is used to acquire images and send them to computing device 72. The Bayesian convolutional neural network model in computing device 72 performs task processing based on the received images and outputs the task processing results.

[0057] The tasks can include body part recognition, face recognition, image classification, image segmentation, and other tasks.

[0058] Taking body part recognition as an example, the image acquisition device 71 acquires a body image, and the computing device 72 performs body part recognition based on the body image, such as chest recognition, lung recognition, and body joint recognition, and outputs the recognition results of the body parts. For example, the recognized chest is labeled and output.

[0059] Taking facial recognition as an example, the image acquisition device 71 acquires facial images, and the computing device 72 performs facial recognition based on these images. It then compares the recognized faces with pre-stored facial images in a facial image database and provides the facial recognition result. The facial recognition result can be applied in scenarios requiring identity verification, such as access control systems and mobile phone screen unlocking.

[0060] Taking image classification as an example, the image acquisition device 71 acquires images to be classified, and the computing device 72 performs image classification based on these images. The image classification task refers to predicting the category label of a given image. For example, given an image of a cat, the task is to predict whether it is a cat.

[0061] Taking image segmentation as an example, the image acquisition device 71 acquires the image to be segmented, and the computing device 72 performs image segmentation based on the image to be segmented. Image segmentation includes ordinary segmentation, semantic segmentation, and instance segmentation. Ordinary segmentation refers to separating pixel regions belonging to different objects, such as separating the foreground and background regions in an image. Semantic segmentation, based on ordinary segmentation, classifies the semantics of each region, that is, what kind of object the region is, such as identifying the category of all objects in the image. Instance segmentation, based on semantic segmentation, assigns a number to each object. For example, if the image contains two dogs, the two dogs are numbered Dog A and Dog B respectively.

[0062] The image acquisition device 71 and computing device 72 mentioned above can be integrated into one electronic device, such as a camera in a mobile phone, tablet computer, laptop computer, desktop computer with camera, access control system, etc., or they can be deployed in different locations, such as the image acquisition device 71 located on the user side and the computing device 72 located in the cloud.

[0063] based on Figure 7 As shown in the application scenario, this application proposes a task processing model training method. Figure 8 The flowchart of the task processing model training method provided in the embodiments of this application Figure 1 .like Figure 8 As shown, the training method for this task processing model includes:

[0064] S801. Obtain the training dataset. The training dataset includes multiple training data, each with a corresponding label. The label is related to the functions of the task processing model required by the user.

[0065] The execution subject of the method in this embodiment can be Figure 7 The computing device shown in the figure.

[0066] The labels are the annotation information of the training data, which are usually related to the purpose of the training task, that is, the functions that the task processing model needs to possess. For example, if the training task is an image classification task, the label is the classification of the training image; if the training task is a body part recognition task, the label is the body part to be recognized; if the training task is a face recognition task, the label is the face recognition result; if the training task is an image segmentation task, the label is the identifier of each category of object included in the image.

[0067] S802. Use the training dataset to iteratively train the preset model to obtain the task processing model. The weight distribution matrix used by the preset model in the i-th iteration of training is obtained by adjusting the weight distribution matrix updated by the gradient descent method in the (i-1)-th iteration of training. The task processing model is used to process the target task, which is the function required by the user's task processing model. Here, i is an integer greater than or equal to 2.

[0068] The target task can be understood as the technical problem that the task processing model needs to solve. For example, the target task could be: a Bayesian convolutional neural network model that can perform face recognition, or a Bayesian convolutional neural network model that can accurately identify images containing various objects (such as animals, fruits, etc.).

[0069] In some specific embodiments, step S802 includes:

[0070] Step A1: For the i-th iteration of training, based on the weight distribution matrix updated by gradient descent in the i-1th iteration of training of the preset model, determine the weight adjustment parameters corresponding to the i-1th iteration of the preset model.

[0071] Before step A1, it is necessary to determine the weight distribution matrix of the preset model after updating it using gradient descent during the (i-1)th iteration of training. The specific determination process includes the following steps:

[0072] a1. Use gradient descent to determine the updated value of the weight distribution matrix used in the (i-1)th iteration of training.

[0073] In a Bayesian convolutional neural network, the weights are probability distributions, which can be represented by the mean and variance. The updated value of the weight distribution matrix used in the (i-1)th iteration of training includes the updated value of the mean and variance of each weight in the weight distribution matrix used in the (i-1)th iteration of training.

[0074] The following example illustrates in detail how to obtain the updated mean and variance values ​​of each weight in the weight distribution matrix used in the (i-1)th training iteration:

[0075] In this embodiment, the true posterior distribution can be approximated using a distribution controlled by a set of parameters according to the variational inference method. For example, if a Gaussian distribution is used for approximation, the parameters are the mean and variance of the weight distribution, i.e., the Gaussian variational posterior distribution. The process of the Gaussian variational posterior distribution is as follows:

[0076] Suppose that the posterior distribution q(w|θ) of the weights w is a Gaussian function, i.e., θ=(μ,ρ), where μ is the mean and ρ is a function of the variance σ, where variance σ=log(1+e ρ The role of ρ is to ensure that the variance σ is always non-negative. Then, the weight w sampled from the posterior distribution q(w|θ) can be approximated by the following formula (4):

[0077]

[0078] A Bayesian convolutional neural network can be represented as f(x,w)=logp(y|x,w), where f(x,w)=x·w in the convolutional layer, x is the training dataset, and w is the convolutional kernel (weight matrix). The gradient of the mean (the updated value of the mean) is calculated as shown in formula (5):

[0079]

[0080] And the gradient of the variance (the updated value of the variance) is given by the following formula (6):

[0081]

[0082] a2. Update the weight distribution matrix used in the (i-1)th iteration of training based on the updated value of the weight distribution matrix used in the (i-1)th iteration of training, and obtain the updated weight distribution matrix in the (i-1)th iteration of training.

[0083] Specifically, the gradient based on the mean can be used to update the mean according to the following formula (7):

[0084] μ←μ-αΔμ; (7)

[0085] The variance can then be updated according to the gradient of the variance using the following formula (8):

[0086] ρ←ρ-αΔρ; (8)

[0087] a3. Based on the updated mean and variance, we can determine the updated weights in the weight distribution matrix used in the (i-1)th iteration of training, thus obtaining the updated weight distribution matrix in the (i-1)th iteration of training.

[0088] Specifically, step A1 includes:

[0089] Step A11: Determine the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training of the preset model. Specifically, step A11 involves determining the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training of the preset model.

[0090] Step a: Based on the first parameter vector of the weight distribution matrix updated by the gradient descent method in the (i-1)th iteration of the preset model during training, perform a left multiplication operation on the weight distribution matrix updated by the gradient descent method in the (i-1)th iteration of the preset model to obtain the second parameter vector of the weight distribution matrix updated by the gradient descent method in the (i-1)th iteration of the preset model during training, where j is an integer greater than or equal to 2;

[0091] Step b: Multiply the weight distribution matrix updated by gradient descent in the i-1th iteration of the preset model by the second parameter vector of the weight distribution matrix updated by gradient descent in the i-1th iteration of the preset model on the left to obtain the first parameter vector of the weight distribution matrix updated by gradient descent in the i-1th iteration of the preset model.

[0092] Step c: Determine whether the value of j reaches the preset value;

[0093] Step d: If the value of j reaches the preset value, then determine the maximum singular value of the weight distribution matrix updated by the gradient descent method in the (i-1)th iteration of the preset model based on the first parameter vector of the jth iteration and the second parameter vector of the jth iteration.

[0094] Step e: If the value of j does not reach the preset value, then increment j by 1 and return to step a.

[0095] Step A12: Take the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training as the weight adjustment parameter for the (i-1)th iteration.

[0096] Step A2: Based on the weight adjustment parameters of the (i-1)th iteration, adjust the weight distribution matrix of the preset model after updating it using the gradient descent method during the (i-1)th iteration of training to obtain the weight distribution matrix used by the preset model during the (i-1)th iteration of training. The weight adjustment parameters of the (i-1)th iteration are used to constrain the update range of each weight in the weight distribution matrix after updating it using the gradient descent method during the (i-1)th iteration of training.

[0097] Specifically, the weight adjustment parameter in the (i-1)th iteration is used to suppress the larger weights in the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training. In some implementations, this can be understood as reducing the weights in the weight distribution matrix that are greater than a preset value, which can be the mean of all weights in the weight distribution matrix.

[0098] Specifically, step A2 includes: step A21, dividing each weight in the weight distribution matrix used by the preset model in the (i-1)th iteration of training by the weight adjustment parameter in the (i-1)th iteration, to obtain the weight distribution matrix used by the preset model in the (i-1)th iteration of training.

[0099] For example, suppose the updated weight distribution matrix W is obtained during the (i-1)th iteration of training. i-1 As shown in formula (9):

[0100]

[0101] Combining the above formula (9), the weight distribution matrix used in the i-th iteration of training obtained in this step is shown in formula (10):

[0102]

[0103] In equation (10), ║W i-1 ║2 is matrix W i-1 The weight adjustment parameters.

[0104] Step A3: Using the weight distribution matrix of the i-th iteration and the training dataset, perform the i-th iteration training on the preset model.

[0105] Among them, ║W i-1 ║2 is matrix W i-1 The maximum singular value involves a large amount of computation. This application uses the power iteration method to find the maximum singular value of ║W. i-1 ║2 is solved approximatingly, and the principle is as follows: first, v is initialized randomly. j-1 When j is 2, a random vector is randomly sampled from the Gaussian distribution and used as the weight distribution matrix W for the (i-1)th iteration of training. i -1 The first parameter vector v1 of the first iteration is obtained by calculating the second parameter vector u of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model using the following formula (11). j Based on u j The first parameter vector v of the weight distribution matrix updated by gradient descent in the i-1th iteration of the preset model is calculated using formula (12). jThe value of j determines whether the constraint condition is met. If the constraint condition is met, the iterative calculation stops. The preset condition can be that the number of power iterations reaches a preset iteration threshold.

[0106] u j ←W i-1 ·v j-1 (11)

[0107] v j ←W i-1 ·u j (12)

[0108] In formulas (11) and (12), j represents the power iteration number. The principle of the power iteration method lies in obtaining v through random initialization. j-1 For example, it is sampled from a Gaussian distribution and W is processed using formula (11). i-1 Left multiply v j-1 The second parameter vector u is obtained. j Then, using formula (12), the second parameter vector u is... j With matrix W i-1 Perform a left multiplication to obtain the first parameter vector v. j Then, after adding 1 to j, return to formula (11) and repeat the cycle until the constraint conditions are met, then stop the power iteration calculation. Finally, calculate matrix W based on v and u when the constraint conditions are met. i-1 The weight adjustment parameters are specifically calculated based on formula (13) to obtain matrix W. i-1 Weight adjustment parameters:

[0109]

[0110] In step d, the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of the preset model is determined based on the first parameter vector and the second parameter vector of the jth iteration. This can be calculated using formula (13) to obtain the weight distribution matrix W. i-1 The maximum singular value, i.e., the matrix W i-1 The weight adjustment parameters can be used to adjust W. i -1 Each weight in the matrix is ​​constrained, or can be understood as normalized, thereby suppressing the weight matrix W. i-1 Excessively high weights in the middle can have a regularization effect.

[0111] S803, Output Task Processing Model.

[0112] Specifically, the task processing model is output to the user's computing device, which is then used for task processing. This computing device can be a device other than a computing unit. Alternatively, it can output to... Figure 7 The computing device shown is used for task processing.

[0113] Taking an access control system as an example, the task processing model is output to the access control system. Specifically, it can be understood as storing the task processing model in the access control system, so that the access control system can apply the task processing model to perform facial recognition and verify identity based on the facial recognition results.

[0114] Of course, this embodiment is not limited to applying the face recognition results to the access control system; face recognition can be used in any application scenario.

[0115] This embodiment obtains a training dataset, which includes multiple training data points, each with a label related to the functionality required by the user's task processing model. The training dataset is used to iteratively train a preset model to obtain the task processing model. The weight distribution matrix used by the preset model in the i-th iteration is obtained by adjusting the weight distribution matrix updated using gradient descent in the (i-1)-th iteration. The task processing model is used to process the target task, which is the functionality required by the user's task processing model, where i is an integer greater than or equal to 2. The task processing model is then output. Because the weight distribution matrix used by the preset model in the i-th iteration is obtained by adjusting the weight distribution matrix updated using gradient descent in the (i-1)-th iteration, the weight distribution curve in the weight distribution matrix obtained in each iteration is smooth. The higher weights are reduced, thus achieving regularization and preventing overfitting, thereby improving the prediction accuracy of the trained preset model in the task processing process.

[0116] Figure 9 The flow of the task processing method provided in the embodiments of this application Figure 2 ,like Figure 9 As shown, this task processing method includes the following steps:

[0117] S901: Receive the image corresponding to the task to be processed.

[0118] The execution subject of the method in this embodiment can be Figure 7 The computing device shown receives images acquired by the image acquisition device.

[0119] S902. Input the image corresponding to the task to be processed into the task processing model obtained based on the task processing model training method to obtain the task processing result.

[0120] S903, Output the task processing results.

[0121] In some scenarios, the task to be processed may include body part recognition, face recognition, image classification, or image segmentation. The result of the task processing may include body part recognition, face recognition, image classification, or image segmentation.

[0122] In the security field, taking access control systems as an example, facial recognition results can be used to determine whether passage is permitted. Furthermore, in some identity verification scenarios based on facial recognition, the verification result can be provided, along with whether further action is allowed, such as whether login to a business system is permitted.

[0123] This embodiment receives the image corresponding to the task to be processed, inputs the image into the task processing model obtained based on the task processing model training method, obtains the task processing result, and outputs it. Since the task processing model trained by the task processing model training method has high prediction accuracy, it can improve the accuracy of the task processing result, thus enabling the Bayesian convolutional neural network to be well applied in fields with high requirements for prediction accuracy.

[0124] Based on the above method embodiments, Figure 10 A schematic diagram of the structure of a task processing model training device provided in an embodiment of this application. (See attached diagram.) Figure 10 As shown, the task processing model training device includes: an acquisition module 101, a training module 102, and an output module 103;

[0125] The acquisition module 101 is used to acquire a training dataset, which includes multiple training data, each training data corresponding to a label, and the label is related to the functions of the task processing model required by the user.

[0126] Training module 102 is used to iteratively train a preset model using the training dataset to obtain a task processing model; the weight distribution matrix used by the preset model in the i-th iteration of training is obtained by adjusting the weight distribution matrix updated by the gradient descent method in the (i-1)-th iteration of training; the task processing model is used to process a target task, the target task being the function required by the user for the task processing model, where i is an integer greater than or equal to 2;

[0127] Output module 103 is used to output the task processing model.

[0128] In some embodiments, the training module 102 uses the training dataset to iteratively train a preset model to obtain a task processing model. Specifically, this includes: for the i-th iteration of training, determining the weight adjustment parameters corresponding to the preset model for the (i-1)th iteration based on the weight distribution matrix updated by the gradient descent method during the (i-1)th iteration of training; adjusting the weight distribution matrix updated by the gradient descent method during the (i-1)th iteration of training based on the weight adjustment parameters to obtain the weight distribution matrix used by the preset model during the i-th iteration of training, wherein the weight adjustment parameters for the (i-1)th iteration are used to constrain the update range of each weight in the weight distribution matrix updated by the gradient descent method during the (i-1)th iteration of training; and using the weight distribution matrix of the i-th iteration and the training dataset to perform the i-th iteration of training on the preset model.

[0129] In some embodiments, the training module 102 determines the weight adjustment parameters for the (i-1)th iteration of the preset model based on the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training. Specifically, this includes: determining the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training; and using the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration as the weight adjustment parameters for the (i-1)th iteration.

[0130] In some embodiments, determining the maximum singular value of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model includes: step a, performing a left multiplication operation on the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model based on the first parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model, to obtain the second parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model, where j is an integer greater than or equal to 2; step b, performing a left multiplication operation on the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model based on the first parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model, to obtain the second parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the preset model, where j is an integer greater than or equal to 2; The second parameter vector of the updated weight distribution matrix at the j-th iteration is left-multiplied by the weight distribution matrix updated by the preset model in the (i-1)-th iteration of training using the gradient descent method, to obtain the first parameter vector of the weight distribution matrix updated by the preset model in the (i-1)-th iteration of training using the gradient descent method; Step c: Determine whether the value of j reaches the preset value; Step d: If the value of j reaches the preset value, determine the maximum singular value of the weight distribution matrix updated by the preset model in the (i-1)-th iteration of training using the first parameter vector of the j-th iteration and the second parameter vector of the j-th iteration; Step e: If the value of j does not reach the preset value, increment j by 1 and return to step a.

[0131] In some embodiments, when the value of j is 2, the left multiplication result of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training is a random vector obtained by random sampling from the Gaussian distribution.

[0132] In some embodiments, the training module 102 adjusts the weight distribution matrix of the preset model updated by gradient descent in the (i-1)th iteration of training according to the weight adjustment parameter of the (i-1)th iteration, to obtain the weight distribution matrix used by the preset model in the i-th iteration of training. Specifically, this includes dividing each weight in the weight distribution matrix of the preset model updated by gradient descent in the (i-1)th iteration of training by the weight adjustment parameter of the (i-1)th iteration, to obtain the weight distribution matrix used by the preset model in the i-th iteration of training.

[0133] The task processing model training device provided in this application embodiment can be used to execute the technical solution of the task processing model training method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0134] Based on the above method embodiments, Figure 11 This is a schematic diagram of the structure of the task processing device provided in an embodiment of this application. Figure 11As shown, the task processing device includes: a receiving module 111, an input module 112, and an output module 113;

[0135] The receiving module 111 is used to receive the image corresponding to the task to be processed;

[0136] Input module 112 is used to input the image corresponding to the task to be processed into the task processing model obtained based on the task processing model training method according to any one of claims 1-5, so as to obtain the task processing result;

[0137] Output module 113 is used to output the task processing result.

[0138] In some embodiments, the task to be processed includes a body part recognition task, a face recognition task, an image classification task, or an image segmentation task; the task processing result includes a body part recognition result, a face recognition result, an image classification result, or an image segmentation result.

[0139] In some embodiments, the receiving module receives an image corresponding to a task to be processed, specifically including: receiving the image acquired by an image acquisition device.

[0140] The task processing device provided in this application embodiment can be used to execute the technical solution of the task processing method in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0141] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the training module 102 can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its functions can be called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0142] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 12As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0143] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the scheme in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0144] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0145] Transceiver 121 can be used to receive images corresponding to the task to be processed, or to obtain training datasets.

[0146] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0147] The electronic device provided in this application embodiment can be the computing device described in the above embodiments.

[0148] This application also provides a chip for executing instructions, which is used to execute the technical solutions of the task processing model training method or task processing method in the above embodiments.

[0149] This application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed on a computer, the computer performs the technical solution of the task processing model training method or task processing method described in the above embodiments.

[0150] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solutions of the task processing model training method or task processing method in the above embodiments.

[0151] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0152] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0153] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0154] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for training a task processing model, characterized in that, include: Obtain a training dataset, which includes multiple training data points, each of which has a corresponding label. The label is related to the functionality of the task processing model required by the user. The Bayesian convolutional neural network model is iteratively trained using the training dataset to obtain a task processing model. The processing model is used for body part recognition, face recognition, image classification, or image segmentation. The weight distribution matrix used by the Bayesian convolutional neural network model in the i-th iteration of training is obtained by adjusting the weight distribution matrix updated by gradient descent in the (i-1)-th iteration of training. The task processing model is used to process a target task, which is the function required by the user for the task processing model. Here, i is an integer greater than or equal to 2. Output the task processing model; The step of iteratively training the Bayesian convolutional neural network model using the training dataset to obtain the task processing model includes: For the i-th iteration of training, based on the weight distribution matrix updated by the gradient descent method during the (i-1)-th iteration of training of the Bayesian convolutional neural network model, the weight adjustment parameters for the (i-1)-th iteration of the Bayesian convolutional neural network model are determined; wherein, the weights are probability distributions. Based on the weight adjustment parameters of the (i-1)th iteration, the weight distribution matrix of the Bayesian convolutional neural network model updated by gradient descent during the (i-1)th iteration of training is adjusted to obtain the weight distribution matrix used by the Bayesian convolutional neural network model during the i-th iteration of training. The weight adjustment parameters of the (i-1)th iteration are used to constrain the update range of each weight in the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training. The Bayesian convolutional neural network model is trained in the i-th iteration using the weight distribution matrix of the i-th iteration and the training dataset.

2. The method according to claim 1, characterized in that, The step of determining the weight adjustment parameters for the (i-1)th iteration of the Bayesian convolutional neural network model based on the updated weight distribution matrix using gradient descent during the (i-1)th iteration of training includes: Determine the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training of the Bayesian convolutional neural network model; The maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training is used as the weight adjustment parameter for the (i-1)th iteration.

3. The method according to claim 2, characterized in that, Determining the maximum singular value of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training of the Bayesian convolutional neural network model includes: Step a: Multiply the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the Bayesian convolutional neural network model by the first parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the training to obtain the second parameter vector of the weight distribution matrix updated by gradient descent in the (i-1)th iteration of the training, where j is an integer greater than or equal to 2; Step b: Multiply the second parameter vector of the weight distribution matrix updated by the gradient descent method in the i-1th iteration of the Bayesian convolutional neural network model by left multiplication to obtain the first parameter vector of the weight distribution matrix updated by the gradient descent method in the i-1th iteration of the Bayesian convolutional neural network model. Step c: Determine whether the value of j reaches the preset value; Step d: If the value of j reaches a preset value, then determine the maximum singular value of the weight distribution matrix updated by gradient descent method during the (i-1)th iteration of training of the Bayesian convolutional neural network model based on the first parameter vector of the jth iteration and the second parameter vector of the jth iteration. Step e: If the value of j does not reach the preset value, then increment j by 1 and return to step a.

4. The method according to claim 3, characterized in that, When j is 2, the left multiplication result of the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training is a random vector obtained by random sampling from the Gaussian distribution.

5. The method according to any one of claims 1-4, characterized in that, The step of adjusting the weight distribution matrix of the Bayesian convolutional neural network model updated by gradient descent during the (i-1)th iteration of training, based on the weight adjustment parameters of the (i-1)th iteration, to obtain the weight distribution matrix used by the Bayesian convolutional neural network model during the i-th iteration of training, includes: Divide each weight in the weight distribution matrix updated by gradient descent during the (i-1)th iteration of the Bayesian convolutional neural network model by the weight adjustment parameter of the (i-1)th iteration to obtain the weight distribution matrix used by the Bayesian convolutional neural network model during the i-th iteration of training.

6. A task processing method, characterized in that, include: Receive the image corresponding to the task to be processed; The image corresponding to the task to be processed is input into the task processing model obtained based on the task processing model training method according to any one of claims 1-4 to obtain the task processing result; Output the task processing results.

7. The method according to claim 6, characterized in that, The tasks to be processed include body part recognition tasks, face recognition tasks, image classification tasks, or image segmentation tasks. The task processing results include body part recognition results, face recognition results, image classification results, or image segmentation results.

8. The method according to claim 6 or 7, characterized in that, The receiving of the image corresponding to the task to be processed includes: Receive the image acquired from the image acquisition device.

9. A task processing model training device, characterized in that, include: The acquisition module is used to acquire a training dataset, which includes multiple training data, each of which has a corresponding label. The label is related to the functions of the task processing model required by the user. The training module is used to iteratively train the Bayesian convolutional neural network model using the training dataset to obtain a task processing model. The processing model is used for body part recognition, face recognition, image classification, or image segmentation. The weight distribution matrix used by the Bayesian convolutional neural network model in the i-th iteration of training is obtained by adjusting the weight distribution matrix updated by the gradient descent method in the (i-1)-th iteration of training. The task processing model is used to process the target task, which is the function required by the user for the task processing model. Here, i is an integer greater than or equal to 2. The output module is used to output the task processing model; The training module is specifically used to determine the weight adjustment parameters for the (i-1)th iteration of the Bayesian convolutional neural network model based on the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training, for the i-th iteration of training; wherein the weights are probability distributions. Based on the weight adjustment parameters of the (i-1)th iteration, the weight distribution matrix of the Bayesian convolutional neural network model updated by gradient descent during the (i-1)th iteration of training is adjusted to obtain the weight distribution matrix used by the Bayesian convolutional neural network model during the i-th iteration of training. The weight adjustment parameters of the (i-1)th iteration are used to constrain the update range of each weight in the weight distribution matrix updated by gradient descent during the (i-1)th iteration of training. The Bayesian convolutional neural network model is trained in the i-th iteration using the weight distribution matrix of the i-th iteration and the training dataset.

10. A task processing device, characterized in that, include: The receiving module is used to receive the image corresponding to the task to be processed; The input module is used to input the image corresponding to the task to be processed into the task processing model obtained based on the task processing model training method according to any one of claims 1-4, so as to obtain the task processing result; The output module is used to output the task processing results.

11. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

13. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images