An artificial intelligence (AI) model precision configuration method, apparatus, and device

Abstract

An artificial intelligence (AI) model precision configuration method, device and equipment, in the present application, the AI model M layer network layer is divided into N sub-models, the input data is input into the first sub-model and the precision residual model in the N sub-models, the first output data of the first sub-model, the second output data of the precision residual model are obtained. The precision residual model simulates the difference between the output data obtained by the first sub-model and the second sub-model for the same input data, and the second sub-model is obtained by converting the precision of the first sub-model from the first precision to the second precision. According to the first output data and the second output data, the precision measurement value is obtained, which can determine the loss degree of the output data after the first sub-model is adjusted to the second sub-model; according to the precision measurement value, it is determined whether the precision of the first sub-model is configured as the second precision. The precision of the first sub-model is related to the loss degree of the output data after the precision of the first sub-model is changed, which improves the flexibility of the precision configuration.

Description

Technical Field

[0001] This application relates to the field of communication technology, and in particular to a method, apparatus and device for configuring the accuracy of an artificial intelligence (AI) model. Background Technology

[0002] As AI models become increasingly larger, improving their performance has become a crucial research issue. The accuracy of each network layer in a neural network is a key factor affecting the performance of an AI model.

[0003] The precision of a network layer refers to the precision of the data used by the network layer (such as the weights, inputs, and outputs). Common precisions include single-precision and double-precision. Since the data used by the network is typically floating-point numbers, single and double precision represent floating-point precision. Single-precision floating-point numbers require 32 bits for storage, which occupies less space, but network layers using single-precision floating-point numbers have lower precision. Double-precision floating-point numbers require 64 bits for storage, which occupies more space, but network layers using double-precision floating-point numbers have higher precision.

[0004] Currently, blacklists and whitelists are commonly used to determine the accuracy of network layers in AI models. These lists record the appropriate accuracy for different types of network layers. When an AI model is applied to different scenarios using this method, the accuracy of each network layer remains fixed. However, different scenarios have different accuracy requirements for AI models, making this method of determining the accuracy of network layers in an inflexible approach. Summary of the Invention

[0005] This application provides an AI model precision configuration method, apparatus, and device to improve the flexibility of precision configuration for each network layer in an AI model.

[0006] In a first aspect, embodiments of this application provide a precision configuration method for an AI model. This method is executed by a precision configuration device. In this method, the precision configuration device divides an M-layer network in the AI ​​model into N sub-models. Each sub-model includes at least one network layer, where N is less than or equal to M, and both N and M are positive integers.

[0007] The precision configuration device inputs input data into the first sub-model and the precision residual model respectively, and obtains the output data of the first sub-model and the precision residual model. To easily distinguish the output data of these two models, the output data of the first sub-model is called the first output data, and the output data of the precision residual model is called the second output data. The input data can be the output data of the preceding sub-model in the AI ​​model. The precision residual model is used to simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data, after transforming the first sub-model into the second sub-model. The second sub-model is obtained by converting the precision of the first sub-model from the first precision to the second precision.

[0008] The accuracy configuration device obtains an accuracy metric value based on the first output data and the second output data. The accuracy metric value is used to determine the degree of loss of the output data of the two after adjusting the first sub-model to the second sub-model. The accuracy configuration device determines whether to set the accuracy of the first sub-model to the second accuracy based on the accuracy metric value.

[0009] Using the above method, when configuring the accuracy of the first sub-model, the accuracy configuration device analyzes the output data of the first sub-model and the accuracy residual model to obtain an accuracy metric value, and then determines whether to set the accuracy of the first sub-model based on the accuracy metric value. In this way, the accuracy configuration of the first sub-model is no longer fixed, but is related to the degree of loss of output data after the accuracy of the first sub-model changes, thus improving the flexibility of accuracy configuration.

[0010] In one possible implementation, the precision configuration device inputs the input data into the second sub-model to obtain the third output data. When calculating the precision metric, the precision configuration device obtains the precision metric based on the first output data, the second output data, and the third output data.

[0011] By using the above method, the accuracy metric obtained from the first output data, the second output data, and the third output data can more accurately characterize the degree of loss of the output data of the two after the first sub-model is adjusted to the second sub-model, so as to ensure that the appropriate accuracy can be configured for the first sub-model in the end.

[0012] In one possible implementation, the precision configuration device can train the precision residual model. During the training process, the precision configuration device inputs the input data into the first sub-model, the second sub-model, and the precision residual model respectively, and obtains the first output data of the first sub-model, the third output data of the second sub-model, and the second output data of the precision residual model; then, the difference between the first output data and the third output data is used as the first loss value to train the precision residual model.

[0013] Through the above method, the precision configuration device trains the precision residual model, enabling the precision residual model to more accurately simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data. This allows the precision configuration device to obtain a more effective precision metric value to set the precision of the first sub-model.

[0014] In one possible implementation, when the precision configuration device trains the precision residual model using the difference between the first output data and the third output data as the loss value, it can alternately train the second sub-model and the combined model until the loss function converges. The combined model includes the first sub-model and the precision residual model. The loss function includes the first loss value and the second loss value. The second loss value is used to characterize the difference between the target output of the AI ​​model and the reference output. The target output of the AI ​​model is the output obtained by the AI ​​model using the second output data or the combined output data. The combined output data is the sum of the first output data and the second output data.

[0015] By using the above method, and by training the second sub-model and the combined model alternately, we can prevent the processes of the second sub-model and the combined model from affecting each other, so as to ensure the accuracy of the precision residual model obtained in the final training.

[0016] In one possible implementation, when the accuracy configuration device obtains the accuracy metric value based on the first output data and the second output data, it calculates the average value of the Hessian matrix spectrum of the accuracy residual model based on the first output data and the second output data. After obtaining the average value of the Hessian matrix spectrum, it obtains the accuracy metric value based on the average value of the Hessian matrix spectrum of the accuracy residual model and the third output data.

[0017] Using the above method, the average value of the Hessian matrix spectrum can reflect the sensitivity of the accuracy residual model. The accuracy metric obtained based on the average value of the Hessian matrix spectrum can better reflect the degree of loss of the output data of the two after adjusting the first sub-model to the second sub-model.

[0018] In one possible implementation, when the precision configuration device determines whether to configure the precision of the first sub-model to the second precision based on the precision metric value, if all N sub-models need to be configured with precision, the precision configuration device obtains the precision metric values ​​of the N sub-models. The precision metric value of any sub-model is obtained in a similar manner to the precision metric value of the first sub-model, as detailed in the foregoing description, and will not be repeated here.

[0019] The precision configuration device configures the precision of the first K sub-models after sorting them from largest to smallest according to their precision metric values ​​as the second precision, where K is a positive integer less than N.

[0020] Using the above method, the precision configuration device determines the sub-models that need to be set to the second precision based on the ranking results of the precision metric values. This method is simple, efficient, and suitable for scenarios where the precision of multiple sub-models in an AI model needs to be set.

[0021] In one possible implementation, when the accuracy configuration device determines whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric value, it determines whether to set the accuracy of the first sub-model to the second accuracy based on the comparison result between the accuracy metric value and the accuracy threshold. For example, when the accuracy metric value is greater than the accuracy threshold, the accuracy of the first sub-model is not set, that is, the first sub-model maintains the first accuracy; when the accuracy metric value is less than the accuracy threshold, the accuracy of the first sub-model is set to the second accuracy.

[0022] Using the above method, the precision configuration device only needs to compare the precision metric value with the precision threshold to determine whether to configure the precision of the first sub-model. This method is simple to operate and is suitable for scenarios where the precision of a small number of sub-models in the AI ​​model needs to be set.

[0023] Secondly, embodiments of this application also provide a precision configuration device, which has the function of implementing the behavior in the first aspect or the method example of the first aspect described above. The beneficial effects can be found in the description of the first aspect and will not be repeated here. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. In one possible design, the precision configuration device includes a partitioning module, an acquisition module, and a configuration module. These modules can perform the corresponding functions in the method example of the first aspect described above, as detailed in the method example, and will not be repeated here.

[0024] Thirdly, this application also provides a computing device, which includes a processor and a memory, and may further include a communication interface. The processor executes program instructions in the memory to perform the method provided in the first aspect or any possible implementation thereof. The memory is coupled to the processor and stores computer program instructions and data necessary for determining the anomaly detection process. The communication interface is used for communicating with other devices.

[0025] Fourthly, this application provides a computing device system including at least one computing device. Each computing device includes a memory and a processor. The processor of at least one computing device is used to access code in the memory to execute the methods provided in the first aspect or any possible implementation thereof.

[0026] Fifthly, this application provides a computer-readable storage medium that, when executed by a computing device, allows the computing device to perform the method provided in the first aspect or any possible implementation thereof. The storage medium stores computer program instructions. The storage medium includes, but is not limited to, volatile memory, such as random access memory, and non-volatile memory, such as flash memory, hard disk drive (HDD), and solid-state drive (SSD).

[0027] Sixthly, this application provides a computing device program product, which includes computer program instructions. When executed by a computing device, the computing device performs the methods provided in the first aspect or any possible implementation thereof. The computer program product can be a software installation package, and when it is necessary to use the methods provided in the first aspect or any possible implementation thereof, the computer program product can be downloaded and executed on the computing device.

[0028] In a seventh aspect, this application also provides a computer chip connected to a memory, the chip being used to read and execute computer program instructions stored in the memory, and to execute the methods described in the first aspect and various possible implementations of the first aspect.

[0029] For the technical effects that can be achieved in the second to seventh aspects mentioned above, please refer to the description of the technical effects that can be achieved by the corresponding design scheme in the first aspect mentioned above. This application will not repeat them here. Attached Figure Description

[0030] Figure 1 A schematic diagram of the structure of an AI model provided in this application;

[0031] Figure 2 A schematic diagram of a precision configuration system provided in this application;

[0032] Figure 3 A schematic diagram of a precision configuration method provided in this application;

[0033] Figure 4 A schematic diagram of the structure of an AI model provided in this application;

[0034] Figure 5 A schematic diagram of a structure in an AI model provided in this application that replaces the precision configuration model;

[0035] Figure 6A A schematic diagram of the structure of an AI model provided in this application;

[0036] Figure 6BA schematic diagram of a structure in an AI model provided in this application that replaces the precision configuration model;

[0037] Figure 7 A schematic diagram of the structure of a precision configuration device provided in this application;

[0038] Figures 8-9 A schematic diagram of the result of a computing device provided in this application. Detailed Implementation

[0039] This application provides a method for configuring the precision of an artificial intelligence (AI) model. In this method, the precision configuration device 100 can configure the precision of sub-models in a neural network. To better understand the solution of this application, the relevant concepts of AI models are introduced below.

[0040] (1) Artificial intelligence (AI) model.

[0041] Artificial intelligence (AI), also known as machine intelligence, refers to machines created by humans that can exhibit intelligence. Generally, AI refers to the technology of using ordinary computer programs to represent human intelligence. An AI model can be understood as a computer program running on a machine.

[0042] Machine learning (ML) is the core of artificial intelligence. AI models can also be called machine learning models. Machine learning theory mainly involves designing and analyzing algorithms that allow computers to learn automatically. Machine learning algorithms are a class of algorithms that automatically analyze data to obtain patterns and use these patterns to predict unknown data. Therefore, the core of machine learning is data, algorithms (models), and computing power (computer processing capabilities). Machine learning has a wide range of applications, including data mining, data classification, computer vision, natural language processing (NLP), biometrics, search engines, medical diagnosis, credit card fraud detection, securities market analysis, DNA sequencing, speech and handwriting recognition, strategy games, and robotics. Machine learning involves designing an algorithmic model to process data and output the desired results. Users can continuously optimize the algorithmic model to achieve more accurate data processing capabilities.

[0043] There are many types of machine learning models, such as neural network models, clustering models, reinforcement learning models, and natural language processing models. The following explanation uses neural network models as an example to illustrate the structure of machine learning models.

[0044] Neural networks (NN) are a common type of machine learning model. In this embodiment, the neural network model will be simply referred to as a neural network.

[0045] A neural network is an algorithmic mathematical model that mimics the behavioral characteristics of animal neural networks to perform distributed parallel information processing. By adjusting the connections between neurons and the parameters of the neurons, the neural network acquires the ability to process information. Neural networks possess self-learning and adaptive capabilities. Neural networks are commonly used in the fields of model training and data inference processing in artificial intelligence.

[0046] like Figure 1 The diagram shown illustrates the working principle of a neural network. First, a feature term X is input into the neural network, such as... Figure 1 The diagram shows X1, X2, X3, ... Within the neural network, a series of intermediate features are constructed based on the feature term X, ultimately yielding the final output. These "intermediate features" refer to the "features" learned by the neural network based on the feature term X, and these intermediate features may not have any practical meaning. Internally, this series of intermediate features can include one or more layers.

[0047] The basic concepts related to neural networks are as follows:

[0048] 1. Neuron: The basic unit of a neural network, similar to a neuron in the human brain. Each neuron processes the input signal and produces intermediate output features. The core of the neuron's processing of the input signal lies in adjusting the input signal based on weights and biases to capture the input signal. The signal input to a neuron is the intermediate feature output by the neuron in the previous layer.

[0049] 2. Layer, also known as a network layer: A layer consists of multiple neurons. A neural network typically includes an input layer, hidden layers, and an output layer. Each layer of neurons is responsible for extracting features from the data at different levels.

[0050] 3. Weights and biases: The input signal of each neuron is multiplied by a weight and then a bias is added. Weights and biases are parameters that need to be adjusted during the learning process of a neural network.

[0051] 4. Activation function: Used to introduce non-linearity. Common activation functions include ReLU (rectified linear unit), sigmoid, and tanh. Neural networks can contain activation functions. The presence of activation functions ensures that the neural network can fit non-linear data distributions, no longer limited to linear data distributions, thus improving the generalization ability of the neural model.

[0052] 5. Loss function: Used to evaluate the difference between the prediction result of the neural network and the reference value. Common loss functions include mean squared error and cross-entropy. The reference value is the expected output of the neural network; in this embodiment, the reference value can also be called the reference output. The prediction result of the neural network can also be called the output of the neural network.

[0053] 6. Backpropagation: An algorithm for training neural networks that calculates the gradients of parameters by computing a loss function and uses these gradients to update weights and biases.

[0054] 7. Optimizer: Used to update network parameters in each iteration. Common optimizers include stochastic gradient descent (SGD) and Adam. Based on the gradient descent principle, the parameters of neurons are gradually adjusted to minimize the loss function and approximate the optimal solution.

[0055] The training of neural networks will be explained below.

[0056] The training process of a neural network is an iterative optimization process using data from the training set. This includes a closed-loop process from randomly initializing parameters, performing forward propagation to obtain predictions, calculating the loss, performing backpropagation to obtain gradients, and finally updating the parameters using an optimization algorithm. The training of a neural network mainly includes the following steps:

[0057] Step 1. Initialization: Randomly initialize the weights and biases of each neuron in the neural network.

[0058] Step 2. Forward Propagation: Signals are input to the neurons in the input layer of the neural network. These signals are processed through each layer of the neural network, and finally, the output layer outputs the final prediction result. The signals input to the input layer carry data from the training set.

[0059] Step 3. Calculate the loss: Use the loss function to calculate the difference between the predicted result and the reference value.

[0060] Step 4. Backpropagation: Calculate the gradient of the parameters based on the results of the loss function.

[0061] Step 5. Parameter Update: Use the optimizer to update the parameters (i.e., weights and biases) of each neuron in the neural network based on the gradient.

[0062] Step 6. Iteration: Repeat steps 1 to 5 until a cutoff condition is met. This cutoff condition can be the convergence of the loss function or the number of iterations reaching a threshold.

[0063] The accuracy configuration method for AI models provided in this application is applicable to AI models that include network layers.

[0064] (2) Sub-model.

[0065] In this application embodiment, the concept of "sub-model" is introduced, such as the first sub-model, second sub-model, and third sub-model mentioned in this application embodiment.

[0066] The AI ​​model is divided into multiple sub-models. This application does not limit the method of dividing the AI ​​model. For example, it can be divided according to the function of each network layer in the AI ​​model, with one or more adjacent network layers having the same function forming a sub-model. Alternatively, the network layers in the AI ​​model can be divided according to a set number of layers, with one or more adjacent network layers having the same number of layers forming a sub-model.

[0067] In this embodiment, the M-layer network of the AI ​​model can be divided into N sub-models. Taking the first sub-model among the N sub-models as an example, the precision of the first sub-model is first precision. By adjusting the precision of the first sub-model, a second sub-model with second precision is obtained. The first precision and the second precision are different, but the first and second sub-models have the same function. The precision of the first sub-model refers to the fact that the input, output, weights, and biases of the neurons in each layer of the sub-model all use first precision. The parameter precision of the first sub-model is the same, and this first precision can be single precision, double precision, half precision, etc. For example, assuming the first sub-model is a double precision sub-model, then the input, output, weights, and biases of the neurons in each layer of the first sub-model all use double precision floating-point numbers. Converting a double precision floating-point number to a binary number requires 64 bits, and the numerical range that a double precision floating-point number can represent is approximately ±1.7 × 10⁻⁶. -30到 ±1.7×10 30 The precision is approximately 15 significant digits. Assuming the second sub-model is an egg-precision sub-model, then the input, output, weights, and biases of neurons in each layer of this second sub-model all use single-precision floating-point numbers. Converting a single-precision floating-point number to binary requires 32 bits, and the numerical range that a single-precision floating-point number can represent is approximately ±3.4 × 10⁻⁶. -38 Up to ±3.4×1038 The precision is approximately 7 significant digits. The single-precision sub-model has lower precision but higher computational efficiency, while the double-precision sub-model has higher precision and better accuracy.

[0068] like Figure 2 As shown, an accuracy configuration system is provided in an embodiment of this application. The accuracy configuration system includes an accuracy configuration device 100 and a storage device 200.

[0069] The storage device 200 can provide storage space to the precision configuration device 100, which can be used to store data or data generated or required by the precision configuration device 100 during the execution of the precision configuration method for the AI ​​model provided in this application embodiment, such as storing the AI ​​model, or storing parameter files of each network layer (such as network layer weights and biases) generated during the training of the AI ​​model. It can also store the training set required for training the AI ​​model.

[0070] Storage device 200 can be a standalone physical server, desktop computer, portable computing device, or a computer cluster or distributed system composed of multiple computing devices. Any device or system with storage capabilities can be used as storage device 200.

[0071] The precision configuration device 100 can determine the precision of any one of the N sub-models in an AI model. Taking the first sub-model among the N sub-models as an example, in this embodiment, the precision configuration device 100 adjusts the precision of the first sub-model to obtain a second sub-model. Input data is input into the first sub-model and the precision residual model respectively to obtain the first output data of the first sub-model and the second output data of the precision residual module. The precision residual model is used to simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data after the first sub-model is transformed into the second sub-model.

[0072] Since the precision residual model can simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data, the precision residual device can determine whether the precision of the first sub-model needs to be configured to the second precision by analyzing the first output data and the second output data.

[0073] For example, the accuracy configuration device 100 obtains an accuracy metric value based on the first output data and the second output data. The accuracy metric value is used to determine the degree of loss of the output data of the two after adjusting the first sub-model to the second sub-model. The accuracy configuration device 100 determines whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric value.

[0074] For example, the accuracy configuration device 100 inputs input data to the second sub-module to obtain third output data, and obtains an accuracy metric value based on the first output data, the second output data, and the third output data. Then, the accuracy configuration device 100 determines whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric value.

[0075] In order to enable the precision residual model to accurately simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data, the precision configuration device 100 can train the precision residual model. The precision configuration device 100 can use the difference between the first output data and the third output data as the first loss value to train the precision residual model.

[0076] This application does not limit the specific type of AI model. For example, the AI ​​model could be a data cleaning model for cleaning large datasets, a language model for processing human dialogue, a large-scale meteorological model for predicting weather, or a large-scale biological model for predicting protein structures. Furthermore, the AI ​​model could be a convolutional neural network (CNN) model or a deep convolutional neural network (DCNN) model with object recognition capabilities.

[0077] This application does not limit the specific form of the precision configuration device 100. The precision configuration device 100 can be a hardware device, such as a processor, a computing device, or a cluster of multiple computing devices. The precision configuration device 100 can also be a software module, such as an application deployed on a computing device.

[0078] like Figure 3 The diagram illustrates a method for configuring the accuracy of an AI model according to an embodiment of this application. This method includes an AI model pre-training process and an accuracy configuration process. During the AI ​​model pre-training process, the accuracy configuration device 100 trains the accuracy residual model of a sub-model within the AI ​​model; details can be found in steps 301 to 304. During the accuracy configuration process, the accuracy configuration device 100 configures the accuracy of the sub-models and determines whether the accuracy of the sub-models needs adjustment; details can be found in steps 305 to 307.

[0079] Step 301: The precision configuration device 100 determines the sub-model in the AI ​​model whose precision needs to be configured.

[0080] An AI model comprises N sub-models, each of which can exist with varying degrees of precision. For any given AI model, the precision of each sub-model affects the overall accuracy and computational efficiency of the AI ​​model. Therefore, it is necessary to configure the precision of each sub-model within the AI ​​model.

[0081] In step 301, the precision configuration device 100 can select one sub-model from the AI ​​model as the sub-model whose precision is to be configured, or it can select multiple sub-models as the sub-models whose precision is to be configured.

[0082] For ease of explanation, the sub-model whose accuracy to be configured is determined by the accuracy configuration device 100 is referred to as the first sub-model. The process of configuring the accuracy of the first sub-model is explained below.

[0083] Step 302: The precision configuration device 100 adjusts the precision of the first sub-model from the first precision to the second precision to obtain the second sub-model, wherein the second precision is different from the first precision.

[0084] For any given first sub-model, the first sub-model can only present one level of precision, namely, a first level of precision. In this embodiment, the precision configuration device 100 needs to know the impact on the AI ​​model when transforming the first sub-model into a sub-model with other levels of precision. Therefore, for any given first sub-model, the precision configuration device 100 can adjust the precision of the first sub-model, converting it into a second sub-model with different levels of precision.

[0085] Step 303: In the AI ​​model, the precision configuration device 100 replaces the first sub-model in the AI ​​model with a precision configuration model. The precision configuration model includes a second sub-model running in parallel and a combined model. The combined model includes the first sub-model and a precision residual model.

[0086] For any first sub-model, the precision configuration device 100 executes step 303, replacing the first sub-model in the AI ​​model with a precision configuration model. The position and connection relationship of the precision configuration model in the AI ​​model are the same as those of the first sub-model in the AI ​​model.

[0087] For example, such as Figure 4 As shown, assume that the sub-model preceding the first sub-model in the AI ​​model is the third sub-model, and the first sub-model receives the output of the third sub-model; the sub-model following the first sub-model in the AI ​​model is the fourth sub-model, and the fourth sub-model receives the output of the first model. After replacing the first sub-model with the precision configuration model, the precision configuration model receives the output of the third sub-model. The fourth sub-model receives the output of the precision configuration model.

[0088] like Figure 5The diagram shows a structural schematic of a precision configuration model provided in an embodiment of this application. This precision configuration model includes two parallel parts: a second sub-model and a combined model. These two parts operate in parallel; that is, both parts can receive the output of the third sub-model and transmit their respective outputs to the fourth sub-model. In other words, the fourth sub-model can receive the output of the second sub-model and also the output of the combined model.

[0089] The second sub-model has the second highest precision. The second sub-model processes the input data (i.e., the output data of the third sub-model) to obtain the third output data, and then transmits the third output data to the fourth sub-model.

[0090] The combined model includes a first sub-model and a precision residual model. The precision residual model simulates the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data, after transforming the first sub-model into the second sub-model. In the combined model, the first sub-model and the precision residual model run in parallel. Both the first sub-model and the precision residual model can receive the output of the third sub-model and process the input data (i.e., the output data of the third sub-model) respectively. In the combined model, the first sub-model outputs the first output data, and the precision residual model outputs the third output model. The first output data and the second output data are added together and then transmitted to the fourth sub-model. In the embodiments of this application, the data obtained by adding the first output data and the second output data is called the combined output data.

[0091] like Figure 5 As shown, assuming the output data of the third sub-model is x, the input data received by the second sub-model is x, and it processes x to output the third output data y3, which is then transmitted to the fourth sub-model. In the combined model, the input data received by the first sub-model is x, and it processes x to output the first output data y1. The precision residual model receives the input data x, processes x, and outputs the second output data y2. The combined output data y1+y2 is transmitted to the third sub-model.

[0092] Because the first and second sub-models use different levels of precision, their output data differs, meaning there is a difference between y1 and y3. The precision residual model is set up to simulate the difference between the output data obtained by the first and second sub-models after training on the same input data, after the first sub-model is adjusted to the second sub-model. In other words, y2 needs to fit the difference between y1 and y3, and y2 needs to be close to or equal to y3-y1.

[0093] In order for the precision residual model to simulate the difference between the output data of the first sub-model and the output data of the second sub-model, the AI ​​model that replaced the precision configuration model is trained, i.e., step 304 is executed.

[0094] Step 304: The precision configuration device 100 calls the training set to train the AI ​​model that replaced the precision configuration model until the loss function of the AI ​​model converges. The loss function represents the difference between the target output and the reference output of the AI ​​model. The loss function also represents the difference between the first output data of the first sub-model and the third output data of the second sub-model.

[0095] The training process for AI models is similar to that for neural networks; please refer to the aforementioned explanation of neural networks for details. The training process will not be discussed further here. It is worth noting that the key objective of the training process performed in step 304 is to train the precision residual model so that the trained precision residual model can simulate the difference between the output data of the first sub-model and the output data of the second sub-model, that is, to make y2 approximately equal to y3-y1. Therefore, the loss function used in the AI ​​model training process needs to have the following characteristics:

[0096] Feature 1: The loss function characterizes the difference between the AI ​​model's target output and the reference output. The AI ​​model's target output is the final actual output after processing the input signal. The reference output is the expected output of the AI ​​model or parameters related to the expected output of the AI ​​model.

[0097] Since the first sub-model in this AI model is replaced by a second sub-model and a combined model running in parallel, the target output of this AI model is the output obtained by the AI ​​model using third output data or combined output data, where the combined output data is the sum of the first output data and the second output data. In the embodiment of the application, the loss function includes a second loss value, which characterizes the difference between the target output of the AI ​​model and the reference output, where the target output of the AI ​​model is the output obtained by the AI ​​model using third output data or combined output data.

[0098] Because of the characteristic of this loss function, after the AI ​​model is trained, the first and second sub-models in the precision configuration model can cooperate with other parts of the AI ​​model, making the actual output of the AI ​​model closer to the reference output.

[0099] Feature 2: The loss function represents the difference between the first output data of the first sub-model and the third output data of the second sub-model.

[0100] In other words, the difference between the first output data of the first sub-model and the third output data of the second sub-model in the accuracy configuration model can be used as a part of the loss function. In the embodiment of the application, the loss function includes a first loss value, which is the difference between the first output data and the third output data.

[0101] Since the training process involves the gradual convergence of the loss function, and given that this loss function possesses property two, training the AI ​​model is a process of making the second output data of the precision residual model gradually approach the difference. By training this AI model, the precision residual model can effectively simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data.

[0102] During step 304, the AI ​​model training process requires training both the second sub-model and the combined model. This AI model training process can be divided into two sub-processes: one for training the second sub-model, where only the weights or biases of the second sub-model are adjusted according to the loss function, while keeping the weights or biases of the combined model unchanged. The goal of this sub-process is to ensure the accuracy of the second sub-model. For ease of explanation, this sub-process is referred to as the first sub-process. In this case, the target output of the AI ​​model involved in the loss function is the output obtained by the AI ​​model using the third output data. The other sub-process is for training the combined model, where only the weights or biases of the combined model are adjusted according to the loss function, while keeping the weights or biases of the second sub-model unchanged. In this case, the target output of the AI ​​model involved in the loss function is the output obtained by the AI ​​model using the combined output data. The goal of this sub-process is to ensure the accuracy of both the first sub-model and the accuracy residual model. For ease of explanation, this sub-process is referred to as the second sub-process.

[0103] When the precision configuration device 100 executes step 303, it may execute the first sub-process first and then the second sub-process; or it may execute the second sub-process first and then the third sub-process; or it may execute the first sub-process and the second sub-process alternately, that is, it may train the first sub-model and the combined model alternately.

[0104] For example, when training the first sub-model and the combined model alternately, the precision configuration device 100 can split the training set into multiple subsets. For each subset, the first sub-process is executed first using that subset, and then the second sub-process is executed using that subset. This achieves the effect of alternating training of the first sub-model and the combined model. By alternating training of the second sub-model and the combined model, the accuracy of the second sub-model and the combined model can be guaranteed, avoiding mutual interference between the training of the second sub-model and the combined model. This ensures that the precision residual model can accurately simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data.

[0105] The internal structure of the precision residual model is not limited in this embodiment. To simplify the training process, the precision residual model can adopt a simple structure, such as including linear layers and activation functions. A linear layer is a network layer containing one or more neurons. A neuron in a linear layer can be understood as a linear function, where the input and output of the neuron satisfy the linear function. The number of linear layers is not limited in this embodiment. For a description of the activation function, please refer to the foregoing description; it will not be repeated here. The precision residual model may include one linear layer or multiple linear layers.

[0106] In the foregoing description, the number of sub-models (i.e., first sub-models) in the AI ​​model requiring precision configuration was not specifically emphasized. Regardless of whether there is one or multiple sub-models in the AI ​​model requiring precision configuration, the precision configuration device 100 must execute steps 302 to 304. When there are multiple sub-models in the AI ​​model requiring precision configuration, the precision configuration device 100 first obtains the second sub-models of each sub-model using the second precision, and replaces each sub-model in the AI ​​model with the corresponding precision configuration model. The precision configuration model corresponding to any sub-model includes the second sub-model of that sub-model and the combined model of that sub-model. The combined model of the sub-model includes the sub-model (i.e., the sub-model can be understood as the aforementioned first sub-model) and the precision residual model. When the precision configuration device 100 executes step 304, the training process of the AI ​​model includes two training sub-processes: the first training sub-process is to train each of the second sub-models in the AI ​​model. That is, in the first training sub-process, each of the second sub-models in the AI ​​model is trained together. The second training sub-process involves training each combined model within the AI ​​model. Specifically, in this second training sub-process, the combined models of each of the first sub-models within the AI ​​model are trained together.

[0107] For ease of understanding, the following explanation will use the example of configuring precision for all sub-models in the AI ​​model to illustrate how the precision configuration device 100 performs steps 302 and 304.

[0108] Here, we assume that the AI ​​model includes four sub-models: sub-model A, sub-model B, sub-model C, and sub-model D. For example... Figure 6A The diagram shown is a structural schematic of the AI ​​model.

[0109] First, the accuracy configuration device 100 replaces each sub-model with an accuracy configuration model. For example, the accuracy configuration model of sub-model A includes a second sub-model A2 running in parallel and a combined model AP, wherein the combined model AP includes a first sub-model A1 running in parallel and an accuracy residual model A-PRLM. The accuracy configuration model of sub-model B includes a second sub-model B2 running in parallel and a combined model BP, wherein the combined model BP includes a first sub-model B1 running in parallel and an accuracy residual model B-PRLM. The accuracy configuration model of sub-model C includes a second sub-model C2 running in parallel and a combined model CP, wherein the combined model CP includes a first sub-model C1 running in parallel and an accuracy residual model C-PRLM. The accuracy configuration model of sub-model D includes a second sub-model D2 running in parallel and a combined model DP, wherein the combined model DP includes a first sub-model D1 running in parallel and an accuracy residual model D-PRLM.

[0110] like Figure 6B The diagram shown illustrates the structure of the AI ​​model after the precision configuration model was replaced. (See also...) Figure 6B The second sub-model A2 in the precision configuration model of sub-model A is connected to the second sub-model B2 in the precision configuration model of sub-model B, and the second sub-model B2 receives the output of the second sub-model A2. The second sub-model B2 in the precision configuration model of sub-model B is connected to the second sub-model C2 in the precision configuration model of sub-model C, and the second sub-model C2 receives the output of the second sub-model B2. The second sub-model C2 in the precision configuration model of sub-model C is connected to the second sub-model D2 in the precision configuration model of sub-model D, and the second sub-model D2 receives the output of the second sub-model C2. The combined model AP in the precision configuration model of sub-model A is connected to the combined model BP in the precision configuration model of sub-model B, and the combined model BP receives the output of the combined model AP. The combined model BP in the precision configuration model of sub-model B is connected to the combined model CP in the precision configuration model of sub-model C, and the combined model CP receives the output of the combined model BP. The combined model CP in the precision configuration model of sub-model C is connected to the combined model DP in the precision configuration model of sub-model D, and the combined model DP receives the output of the combined model CP.

[0111] After replacing the precision configuration model, the precision configuration device 100 trains the AI ​​model with the replaced precision configuration model. The training process of the AI ​​model includes two training sub-processes: The first training sub-process trains the second sub-models A1, B2, C2, and D2 in the AI ​​model. That is, in the second training sub-process, the second sub-models of each sub-model in the AI ​​model are trained together, while keeping the weights and biases of each combined model in the AI ​​model unchanged in the first training sub-process. The second training sub-process trains the combined models AP, BP, CP, and DP in the AI ​​model. That is, in the second training sub-process, each combined model in the AI ​​model is trained together, while keeping the weights and biases of each second sub-model in the AI ​​model unchanged in the second training sub-process.

[0112] The precision configuration device 100 points to the first training sub-process and the second training sub-process respectively. The execution order of the second training sub-process and the first training sub-process can be found in the foregoing description. The specific content of the second training sub-process and the first training sub-process can be found in the relevant description of the AI ​​model, and will not be repeated here.

[0113] The precision configuration device 100 uses a loss function during AI model training to characterize the difference between the target output and the reference output of the AI ​​model. The loss function also characterizes the difference between the output data of the second sub-model and the first sub-model trained on the same input data. For example, this loss function includes four loss values: loss value A, loss value B, loss value C, and loss value D. Loss value A is the difference between the output data of the second sub-model A2 and the output data of the first sub-model A1. Since the output data of the first sub-model A1 is at first precision, it can be converted to second precision for easier difference calculation. Loss value A is also the difference between the output data of the second sub-model A2 and the second-precision output data of the first sub-model A1, where the second-precision output data of the first sub-model A1 is the output after converting the first sub-model A1's output to second precision. Similarly, loss value B is the difference between the output data of the second sub-model B2 and the output data of the first sub-model B1; loss value C is the difference between the output data of the second sub-model C2 and the output data of the first sub-model C1; loss value D is the difference between the output data of the second sub-model D2 and the output data of the first sub-model D1.

[0114] After training the AI ​​model, the precision residual model can simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data.

[0115] It should be noted that the above explanation uses the example of AI model training being completed when the loss function converges. In practical applications, other cutoff conditions can also be used to determine whether AI model training is complete, such as when the number of iterations reaches a threshold.

[0116] Step 305: The accuracy configuration device 100 obtains an accuracy metric value based on the first output data and the second output data. The accuracy metric value is used to determine the degree of loss of the output data of the two after the first sub-model is adjusted to the second sub-model.

[0117] After the AI ​​model is trained, the output of the precision residual model is close to or equal to the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data. The precision configuration device 100 can determine the degree of loss of the output data of the first sub-model and the precision residual module (i.e., the first output data and the second output data) by analyzing the output data of the first sub-model and the precision residual module.

[0118] In this embodiment of the application, the accuracy configuration device 100 obtains an accuracy metric value based on the first output data and the second output data. The accuracy configuration device 100 uses the accuracy metric value to characterize the degree of loss of the output data of the two after the first sub-model is adjusted to the second sub-model.

[0119] There are many ways to obtain the accuracy metric value based on the first output data and the second output data. Two methods for obtaining the accuracy metric value are listed below.

[0120] Method 1: The accuracy configuration device 100 calculates the average value of the Hessian matrix spectrum of the accuracy residual model based on the first output data and the second output data, and uses the average value of the Hessian matrix spectrum of the accuracy residual model as the accuracy metric.

[0121] The Hessian matrix is ​​a square matrix containing second-order partial derivatives of the functional relationship represented by the precision residual model. The spectral mean of the Hessian matrix is ​​the average of its eigenvalues. The eigenvalues ​​of the Hessian matrix are calculated when the Hessian matrix is ​​treated as a matrix; their description is consistent with that of matrices in linear algebra and will not be elaborated upon here. Since the Hessian matrix contains second-order partial derivatives, the average of its eigenvalues ​​characterizes the sensitivity of the precision residual model to the AI ​​model. The relationship between the sensitivity of the precision residual model and its impact on the final output of the AI ​​model can be found in the published article "HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks".

[0122] This section explains how each element in the Hessian matrix is ​​calculated, assuming that all trainable parameters in the precision residual model are {w}. i}, i=1,…,P. Then the Hessian matrix H is a P×P matrix, where each element of the precision residual model is the second-order partial derivative of the loss function L of the AI ​​model with respect to the parameters of the precision residual model, i.e., H=(h ij ), i = 1, ..., P, j = 1, ..., P. Here

[0123] To further illustrate the relationship between the Hessian matrix and the first output data y1 and the second output data y2, we assume that the AI ​​model contains only one sub-model, whose accuracy residual model is considered as a mapping function f. Then, y2 = f(x; w), and y2 + y1 and the reference output y are used to calculate the final loss value through the loss function L. The loss function L and the mapping function f satisfy the following:

[0124] L(y;y1+y2)=L(y;y1+f(x;w))

[0125] Based on this, there exists Any element h in the Hessian matrix ij satisfy:

[0126]

[0127] This formula can also be called the chain rule formula, and each element in the Hessian matrix is ​​obtained by the above chain rule formula.

[0128] The Hessian matrix can be calculated using the above method. After obtaining the Hessian matrix, the spectral average value of the Hessian matrix can be further calculated.

[0129] Since the computation of the average value of the Hessian matrix spectrum is quite large, the precision configuration device 100 can solve for an approximate value Tr(H) of the average value of the Hessian matrix spectrum by randomly sampling noise. Est (H):

[0130] Tr(H)=Tr(HI)=Tr(HE[zz T ])=E[Tr(Hzz T )]=E[z T Hz]

[0131]

[0132] Where H represents the Hessian matrix, I is the identity matrix, z is a random vector obtained through random sampling noise, and E is the expression for the random vector z. T The expectation, zT It is the transpose of z, zz T Represent z and z T Perform matrix multiplication.

[0133] The sensitivity of the precision residual model refers to its impact on the final output of the AI ​​model. The stronger the sensitivity of the precision residual model, the greater its impact on the final output of the AI ​​model. In other words, after the first sub-model is transformed into the second sub-model, the difference in the output of the entire AI model becomes significant. In this case, a network with higher precision can be selected.

[0134] For example, the weaker the sensitivity of the precision residual model, the smaller its impact on the final output of the AI ​​model. That is, the difference between the output of the entire AI model using the first sub-model and the output using the second sub-model is smaller. In this case, using the second sub-model will not have a significant impact on the AI ​​model. However, the sub-model with lower precision can improve the computational efficiency of the AI ​​model. For instance, assuming the first sub-model is a single-precision sub-model and the second sub-model is a double-precision sub-model, the single-precision sub-model is more suitable. Conversely, the stronger the sensitivity of the precision residual model, the greater its impact on the final output of the AI ​​model. That is, the difference between the output of the entire AI model using the first sub-model and the output using the second sub-model is larger. In this case, assuming the first sub-model is a single-precision sub-model and the second sub-model is a double-precision sub-model, using the more precise second sub-model will improve the accuracy of the AI ​​model.

[0135] Method 2: The precision configuration device 100 calculates the average value of the Hessian matrix spectrum and obtains the precision metric value based on the average value of the Hessian matrix spectrum and the third output data.

[0136] For example, the accuracy metric Ω of the first sub-model i i satisfy:

[0137]

[0138] Wherein, Tr(P i () represents the mean value of the Hessian matrix spectrum or an approximation of the mean value of the Hessian matrix spectrum for the accuracy residual model of the first sub-model i. The third output data of the second sub-model obtained after adjusting the precision of the first sub-model i. This is the first output data of the first sub-model i. Since in actual training... as well as The value of can be multiple, therefore, This refers to a multidimensional vector, where each element is obtained during each iteration of the AI ​​model training process. as well as The difference between them. For multidimensional vectors The norm of . It can represent the difference between the output data of the first sub-model and the output data of the second sub-model.

[0139] It should be noted that the above-described method for calculating the accuracy metric is merely an example, and the embodiments of this application do not limit the method of obtaining the accuracy metric from the average value of the Hessian matrix spectrum and the third output data.

[0140] Step 306: The accuracy configuration device 100 determines whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric value.

[0141] In practical applications, when determining whether to configure the precision of the first sub-model, the precision of the first sub-model can be determined by comparing the precision metric value with a precision threshold, which can be an empirical value. Here, we take the example that a larger precision metric value indicates a greater loss of output data after adjusting the first sub-model to the second sub-model. When the precision metric value is greater than the precision threshold, the precision of the first sub-model is kept unchanged; when the precision metric value is less than the precision threshold, the precision of the first sub-model is configured to the second precision. If there are multiple sub-models in the AI ​​model that require precision configuration, the precision configuration model can calculate the precision metric value in each sub-model and determine the precision of each sub-model based on the ranking of the precision metric values. For example, the first K sub-models with precision metric values ​​sorted from largest to smallest are configured to the second precision, while the precision of the remaining sub-models remains unchanged. The value of K can be set according to actual needs, and the embodiments of this application do not limit the method of determining the value of K.

[0142] The case where a larger accuracy metric value indicates a smaller loss of output data after adjusting the first sub-model to the second sub-model is similar to the previous method. The difference is that the relationship between the accuracy metric value and the accuracy threshold is reversed (that is, if the accuracy metric value is greater than the accuracy threshold, the accuracy of the first sub-model is configured as the second accuracy), and the sorting order based on the accuracy metric value is reversed (that is, the first K sub-models after sorting the accuracy metric values ​​from smallest to largest are configured as the second accuracy).

[0143] Step 307: After determining the accuracy of the first sub-model, the accuracy configuration device 100 reconfigures the AI ​​model.

[0144] After determining the accuracy of the first sub-model, if the accuracy of the first sub-model remains unchanged, then the accuracy configuration device 100 retains the first sub-model and removes the accuracy residual model and the second sub-model in the AI ​​model that replaces the accuracy configuration model. If the first sub-model uses the second sub-model, then the accuracy configuration device 100 removes the first sub-model and retains the second sub-model in the AI ​​model that replaces the accuracy configuration model.

[0145] After reconfiguring the AI ​​model, it can continue to be trained. This training process takes place after the accuracy of the first sub-model has been configured. This training process ensures that the AI ​​model can accurately process the input data and that the output of the AI ​​model is closer to the real result.

[0146] Based on the same inventive concept as the method embodiments, this application also provides a precision configuration device, wherein the data processing engine is used to execute the method executed by the precision configuration device 100 in the above method embodiments. For example... Figure 7 As shown, the precision configuration device 700 includes a division module 701, an acquisition module 702, and a configuration module 703. In the precision configuration device 700, the modules are connected through a communication path.

[0147] The partitioning module 701 is used to divide the M network layers in the AI ​​model into N sub-models. Each sub-model includes at least one network layer, where N is less than or equal to M and N and M are positive integers.

[0148] The acquisition module 702 is used to input the input data into the first sub-model and the precision residual model among the N sub-models respectively, and to acquire the first output data of the first sub-model and the second output data of the precision residual model. The precision residual model is used to simulate the difference between the output data obtained by the first sub-model and the second sub-model after training on the same input data after the first sub-model is transformed into the second sub-model. The second sub-model is obtained by converting the precision of the first sub-model from the first precision to the second precision.

[0149] The configuration module 703 is used to obtain an accuracy metric value based on the first output data and the second output data. The accuracy metric value is used to determine the degree of loss of the output data of the two after adjusting the first sub-model to the second sub-model. Based on the accuracy metric value, it is determined whether to configure the accuracy of the first sub-model to the second accuracy.

[0150] As one possible implementation, the acquisition module 702 can also input the input data into the second sub-model to obtain the third output data. The configuration module obtains the accuracy metric value based on the first output data, the second output data, and the third output data.

[0151] As one possible implementation, the acquisition module 702 can also train the precision residual model. The training method is as follows: the acquisition module 702 inputs the input data into the first sub-model, the second sub-model, and the precision residual model respectively, and acquires the first output data of the first sub-model, the third output data of the second sub-model, and the second output data of the precision residual model. The acquisition module 702 uses the difference between the first output data and the third output data as the first loss value to train the precision residual model.

[0152] As one possible implementation, when training the precision residual model, the acquisition module 702 alternately trains the second sub-model and the combined model until the loss function converges. The combined model includes the first sub-model and the precision residual model. The loss function includes a first loss value and a second loss value. The second loss value is used to characterize the difference between the target output of the AI ​​model and the reference output. The target output of the AI ​​model is the output obtained by the AI ​​model using the second output data or the combined output data. The combined output data is the sum of the first output data and the second output data.

[0153] As one possible implementation, when determining whether to configure the accuracy of the first sub-model, the configuration module 703 calculates the average value of the Hessian matrix spectrum of the accuracy residual model based on the first output data and the third output data; then, the configuration module 703 obtains the accuracy metric value based on the average value of the Hessian matrix spectrum of the accuracy residual model and the second output data.

[0154] As one possible implementation, for the N sub-modules in the AI ​​module, the configuration module 703 obtains the accuracy metrics of N sub-models, and the method for obtaining the accuracy metrics of each sub-model is the same as that for the first sub-model. The configuration module 703 sets the accuracy of the first K sub-models after sorting the N sub-models in descending order of their accuracy metrics as the second accuracy, where K is a positive integer less than N.

[0155] As one possible implementation, the configuration module 703 can also determine whether to configure the accuracy of the first sub-model to the second accuracy based on the comparison result between the accuracy metric and the accuracy threshold.

[0156] The module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0157] If the integrated module is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a terminal device (which may be a personal computer, mobile phone, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0158] This application also provides, for example Figure 8 The computing device 800 shown includes a bus 801, a processor 802, a communication interface 803, and a memory 804. The processor 802, the memory 804, and the communication interface 803 communicate with each other via the bus 801.

[0159] The processor 802 can be a central processing unit (CPU) or other specific integrated circuits. The processor 802 can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0160] The memory 804 can typically be dynamic random access memory (DRAM). Besides DRAM, the memory 804 can also be other types of random access memory, such as static random access memory (SRAM) or storage class memory (SCM). Additionally, the memory 804 can also be read-only memory (ROM). For example, read-only memory can be programmable read-only memory (PROM) or erasable programmable read-only memory (EPROM). The memory 804 can also be a dual in-line memory module (DIMM), flash memory, hard disk drive (HDD), or solid-state drive (SSD).

[0161] The memory 804 stores computer program instructions, and the processor 802 executes these computer program instructions to perform the aforementioned tasks. Figure 3 The steps performed by the precision configuration device 100 in the described method. The memory 804 may also include software modules required for other running processes, such as an operating system (e.g., multiple modules in the precision configuration device 700). The operating system may be LINUX. TM UNIX TM WINDOWS TM wait.

[0162] This application also provides a computing device system, the computing device system including at least one such as Figure 9 The computing device 900 shown includes a bus 901, a processor 902, a communication interface 903, and a memory 904. The processor 902, memory 904, and communication interface 903 communicate with each other via the bus 901. At least one computing device 900 in the computing device system communicates with each other via a communication path.

[0163] The specific types of processor 902 and memory 904 can be found in the relevant descriptions of processor 802 and memory 804, and will not be repeated here. Processor 902 executes the computer program instructions stored in memory 904 to perform the aforementioned tasks. Figure 3The described method includes some or all of the steps performed by the precision configuration device 100. The memory may also include other software modules required for running processes, such as an operating system. The operating system may be Linux. TM UNIX TM WINDOWS TM wait.

[0164] At least one computing device 900 in the computing device system establishes communication with each other through a communication network, and each computing device 900 runs any one or any multiple modules of the precision configuration device 700.

[0165] The descriptions of the processes corresponding to the above-mentioned figures each have their own emphasis. For parts of a process that are not described in detail, please refer to the relevant descriptions of other processes.

[0166] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product. A computer program product includes computer program instructions, which, when loaded and executed on a computer, generate, in whole or in part, the product according to the embodiments of the present invention. Figure 3 The process or function described.

[0167] The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., SSD).

[0168] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for configuring the precision of an artificial intelligence (AI) model, characterized in that, The method includes: The M-layer network in the AI ​​model is divided into N sub-models, each sub-model including at least one network layer, where N is less than or equal to M; Input data is input into the first sub-model and the precision residual model among the N sub-models respectively, and the first output data of the first sub-model and the second output data of the precision residual model are obtained. The precision residual model is used to simulate the difference between the output data of the first sub-model and the second sub-model after training on the same input data after the first sub-model is transformed into the second sub-model. The second sub-model is obtained by converting the precision of the first sub-model from the first precision to the second precision. Based on the first output data and the second output data, an accuracy metric is obtained. The accuracy metric is used to determine the degree of loss of the output data of the two after the first sub-model is adjusted to the second sub-model. Based on the accuracy metric, determine whether to configure the accuracy of the first sub-model to the second accuracy.

2. The method as described in claim 1, characterized in that, The method further includes: inputting the input data into the second sub-model to obtain third output data; The step of obtaining the accuracy metric value based on the first output data and the second output data includes: The accuracy metric is obtained based on the first output data, the second output data, and the third output data.

3. The method as described in claim 1 or 2, characterized in that, The method further includes: The input data is input into the first sub-model, the second sub-model, and the precision residual model respectively, and the first output data of the first sub-model, the third output data of the second sub-model, and the second output data of the precision residual model are obtained. The difference between the first output data and the third output data is used as the first loss value to train the accuracy residual model.

4. The method according to any one of claims 1 to 3, characterized in that, The step of using the difference between the first output data and the third output data as the first loss value to train the accuracy residual model includes: The second sub-model and the combined model are trained alternately until the loss function converges. The combined model includes the first sub-model and the accuracy residual model. The loss function includes the first loss value and the second loss value. The second loss value is used to characterize the difference between the target output and the reference output of the AI ​​model. The target output of the AI ​​model is the output obtained by the AI ​​model using the third output data or the combined output data. The combined output data is the sum of the first output data and the second output data.

5. The method as described in claim 2, characterized in that, The step of obtaining the accuracy metric value based on the first output data, the second output data, and the third output data includes: Based on the first output data and the second output data, the average value of the Hessian matrix spectrum of the precision residual model is calculated. The accuracy metric is obtained based on the average value of the Hessian matrix spectrum of the accuracy residual model and the third output data.

6. The method according to any one of claims 1 to 5, characterized in that, The step of determining whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric includes: Obtain the accuracy metric values ​​of the N sub-models; The accuracy of the first K sub-models after sorting the N sub-models in descending order of accuracy metric value is configured as the second accuracy, where K is a positive integer less than N.

7. The method according to any one of claims 1 to 5, characterized in that, The step of determining whether to configure the accuracy of the first sub-model to the second accuracy based on the accuracy metric includes: Based on the comparison between the accuracy metric and the accuracy threshold, it is determined whether to configure the accuracy of the first sub-model to the second accuracy.

8. A precision configuration device, characterized in that, The precision configuration device includes: A partitioning module is used to divide the M network layers in the AI ​​model into N sub-models, each sub-model including at least one network layer, wherein N is less than or equal to M; The acquisition module is used to input input data into the first sub-model and the precision residual model among the N sub-models respectively, and to acquire the first output data of the first sub-model and the second output data of the precision residual model. The precision residual model is used to simulate the difference between the output data of the first sub-model and the second sub-model after training on the same input data after the first sub-model is transformed into the second sub-model. The second sub-model is obtained by converting the precision of the first sub-model from the first precision to the second precision. The configuration module is used to obtain an accuracy metric value based on the first output data and the second output data. The accuracy metric value is used to determine the degree of loss of the output data of the two after adjusting the first sub-model to the second sub-model. Based on the accuracy metric value, it is determined whether to configure the accuracy of the first sub-model to the second accuracy.

9. The apparatus as claimed in claim 8, characterized in that, The acquisition module is further configured to: input the input data into the second sub-model to obtain the third output data; The configuration module is used for: The accuracy metric is obtained based on the first output data, the second output data, and the third output data.

10. The apparatus as claimed in claim 8 or 9, characterized in that, The acquisition module is also used for: The input data is input into the first sub-model, the second sub-model, and the precision residual model respectively, and the first output data of the first sub-model, the third output data of the second sub-model, and the second output data of the precision residual model are obtained. The difference between the first output data and the third output data is used as the first loss value to train the accuracy residual model.

11. The apparatus according to any one of claims 8 to 9, characterized in that, The acquisition module is used for: The second sub-model and the combined model are trained alternately until the loss function converges. The combined model includes the first sub-model and the accuracy residual model. The loss function includes the first loss value and the second loss value. The second loss value is used to characterize the difference between the target output and the reference output of the AI ​​model. The target output of the AI ​​model is the output obtained by the AI ​​model using the third output data or the combined output data. The combined output data is the sum of the first output data and the second output data.

12. The apparatus as claimed in claim 9, characterized in that, The configuration module is used for: Based on the first output data and the second output data, the average value of the Hessian matrix spectrum of the precision residual model is calculated. The accuracy metric is obtained based on the average value of the Hessian matrix spectrum of the accuracy residual model and the third output data.

13. The apparatus according to any one of claims 8 to 12, characterized in that, The configuration module is used for: Obtain the accuracy metric values ​​of the N sub-models; The accuracy of the first K sub-models after sorting the N sub-models in descending order of accuracy metric value is configured as the second accuracy, where K is a positive integer less than N.

14. A computing device, characterized in that, The computing device includes a processor and memory; The memory is used to store computer program instructions; The processor executes computer program instructions in the memory to perform the method as described in any one of claims 1 to 7.

15. A computer-readable storage medium, characterized in that, When the computer-readable storage medium is executed by a computing device, the computing device performs the method according to any one of claims 1 to 7.

Images