Model deployment method, neural network quantization-aware training method, and electronic device

Abstract

This disclosure provides a method for deploying a model, a method for training a neural network with quantization perception, and an electronic device, relating to the field of artificial intelligence. The main technical features include: inputting a network to be processed into a target model for quantization processing to obtain a quantized network to be processed; wherein the target model is obtained by training a target neural network, the target neural network is obtained by fusing a rotation matrix with the weights of the target network layer of the neural network samples, the rotation matrix being used to process abnormal data in the target network layer; and deploying the quantized network to be processed.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and in particular to a method for deploying a model, a method for training a neural network with quantitative perception, and an electronic device. Background Technology

[0002] Deep learning models are increasingly widely used in various fields, but due to the limited hardware resources of devices (such as computing power, memory, storage, etc.), traditional high-precision deep learning models face many challenges in deployment. Therefore, quantizing neural network models and deploying them to resource-constrained hardware devices is a common way to implement model deployment.

[0003] However, during the model quantization training process, abnormal data may appear in the weights or activation values ​​of the network layers, resulting in poor stability of the quantization training process and difficulty in model convergence, which directly affects the quantization quality and deployment stability of the model.

[0004] Currently, the various technical means adopted to alleviate the above problems can mitigate the negative impact of abnormal data to a certain extent, but they generally suffer from high computational overhead and complex training processes during the training phase, which further affects the stability of the quantized model deployment. Summary of the Invention

[0005] This disclosure provides a method for deploying a model, a method for training a neural network using quantization perception, and an electronic device. Its main purpose is to address the problems in existing technologies where the quantization training process is unstable and difficult to converge due to abnormal data in the network layers, and where existing mitigation methods suffer from high computational overhead and complex training processes during the training phase, resulting in poor deployment stability of the quantized model.

[0006] According to a first aspect of this disclosure, a method for deploying a model is provided, comprising:

[0007] The network to be processed is input into the target model for quantization to obtain the quantized network to be processed; wherein, the target model is obtained by training the target neural network, and the target neural network is obtained by fusing the rotation matrix with the weights of the target network layer of the neural network sample, and the rotation matrix is ​​used to process abnormal data in the target network layer; The quantized network to be processed is then deployed.

[0008] In some embodiments, the training method for the target model includes: The neural network samples are preprocessed to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network samples, the target network layer being a network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. The rotation matrix is ​​fused with the weights of the target network layer to obtain the target neural network; The target neural network is subjected to quantization perception training to obtain the target model.

[0009] In some embodiments, the preprocessing of the neural network samples to obtain the processed neural network includes: Determine whether the data input to the input terminal of the target network layer has undergone a linear transformation using the first rotation matrix; When the data input to the input end of the target network layer is linearly transformed by the first rotation matrix, a second rotation matrix is ​​inserted at the output end of the target network layer to obtain the processed neural network; wherein the first rotation matrix and the second rotation matrix are inverse matrices of each other.

[0010] In some embodiments, the target network layer includes a linear transformation layer in the neural network sample, and the linear transformation layer includes at least one of a projection transformation layer in the attention mechanism module and a fully connected transformation layer in the feedforward calculation module.

[0011] In some embodiments, processing the anomalous data in the network layer to be quantized for training includes: A linear transformation is performed on the weights and / or activation values ​​of the target network layer to eliminate outliers in the weights and / or activation values ​​of the target network layer.

[0012] In some embodiments, before preprocessing the neural network samples to obtain the processed neural network, the method further includes: The training data is processed to obtain processed training data; wherein the data processing includes at least one of the following methods: pruning, normalization, and data augmentation.

[0013] In some embodiments, the step of performing quantization-sensory training on the target neural network to obtain a target model includes: A pseudo-quantization operator is inserted at the position to be quantized in the target neural network; The quantization parameters in the pseudo-quantization operator are initialized and calibrated. Based on the processed training data, the target neural network after inserting the pseudo-quantization operator is iteratively trained to update the network parameters and the quantization parameters, thereby obtaining the target model.

[0014] In some embodiments, after performing quantization-sensory training on the target neural network to obtain the target model, the method further includes: The target model is exported as a model file in a preset format; wherein, the pseudo-quantization operators in the model file are converted into real quantization operators.

[0015] According to a second aspect of this disclosure, a method for training a neural network with quantitative perception is provided, comprising: The neural network to be processed is preprocessed to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network to be processed, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. The rotation matrix is ​​fused with the weights of the target network layer to obtain the target neural network; The target neural network is subjected to quantization perception training to obtain a quantized neural network.

[0016] According to a third aspect of this disclosure, a model deployment apparatus is provided, comprising: The processing unit is used to input the network to be processed into the target model for quantization processing to obtain the quantized network to be processed; wherein, the target model is obtained by training a target neural network, the target neural network is obtained by fusing a rotation matrix with the weights of the target network layer of the neural network samples, and the rotation matrix is ​​used to process abnormal data in the target network layer; The deployment unit is used to deploy the quantized network to be processed.

[0017] According to a fourth aspect of this disclosure, a neural network quantization perception training apparatus is provided, comprising: A preprocessing unit is used to preprocess the neural network to be processed to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network to be processed, the target network layer being a network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. A fusion unit is used to fuse the rotation matrix with the weights of the target network layer to obtain the target neural network. The training unit is used to perform quantization-sensory training on the target neural network to obtain a quantized neural network.

[0018] In some embodiments, the preprocessing unit includes: The judgment module is used to determine whether the data input to the input terminal of the target network layer has undergone linear transformation by the first rotation matrix; The first insertion module is used to insert a second rotation matrix into the output of the target network layer when the data input to the input of the target network layer has undergone a linear transformation by the first rotation matrix, so as to obtain the processed neural network; wherein the first rotation matrix and the second rotation matrix are inverse matrices of each other.

[0019] In some embodiments, the target network layer includes a linear transformation layer in the neural network to be processed, and the linear transformation layer includes at least one of a projection transformation layer in the attention mechanism module and a fully connected transformation layer in the feedforward calculation module.

[0020] In some embodiments, processing the abnormal data in the network layer to be quantized for training includes: performing a linear transformation on the weights and / or activation values ​​of the target network layer to eliminate abnormal values ​​in the weights and / or activation values ​​of the target network layer.

[0021] In some embodiments, the apparatus further includes: A data processing unit is configured to perform data processing on the training data before the preprocessing unit preprocesses the neural network to be processed to obtain the processed neural network, thereby obtaining processed training data; wherein the data processing includes at least one of the following processing methods: pruning, normalization, and data augmentation.

[0022] In some embodiments, the training unit includes: The second insertion module is used to insert a pseudo-quantization operator at the position to be quantized in the target neural network; The calibration module is used to initialize and calibrate the quantization parameters in the pseudo-quantization operator. The training module is used to iteratively train the target neural network after inserting the pseudo-quantization operator based on the processed training data, update the network parameters and the quantization parameters, and obtain the quantized neural network.

[0023] In some embodiments, the apparatus further includes: The export unit is used to export the quantized neural network as a model file in a preset format after the training unit performs quantization perception training on the target neural network to obtain the quantized neural network; wherein, the pseudo-quantization operator in the model file is converted into a real quantization operator.

[0024] According to a fifth aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first or second aspect above.

[0025] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first or second aspect above.

[0026] According to a seventh aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method as described in the first or second aspect above.

[0027] In summary, the model deployment method, neural network quantization perception training method, and electronic device provided in this disclosure include: inputting the network to be processed into a target model for quantization processing to obtain a quantized network to be processed; wherein the target model is obtained by training a target neural network, the target neural network is obtained by fusing a rotation matrix with the weights of the target network layer of the neural network samples, and the rotation matrix is ​​used to process abnormal data in the target network layer; and deploying the quantized network to be processed. Compared with related technologies, the solution of this disclosure can reduce the training computational overhead during the training phase by pre-fusing the rotation matrix with the weights of the target network layer, simplifying the training process, and improving the stability and convergence of the training process; during the deployment phase, since the effect of the rotation matrix has been absorbed into the weights, there is no need to retain additional transformation operators or perform independent computational operations during the inference process, thereby simplifying the structure of the deployed model, reducing the computational overhead during the inference phase, and improving the deployment stability on resource-constrained hardware devices.

[0028] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0029] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A flowchart illustrating a method for deploying a model provided in an embodiment of this disclosure; Figure 2A flowchart illustrating another deployment method for a model provided in this embodiment of the disclosure; Figure 3 A flowchart illustrating another deployment method for a model provided in this embodiment of the disclosure; Figure 4 This is a schematic diagram of the structure of a Transformer module in a Vision Transformer network provided in an embodiment of the present disclosure; Figure 5 A flowchart illustrating another deployment method for a model provided in this embodiment of the disclosure; Figure 6 A flowchart illustrating another deployment method for a model provided in this embodiment of the disclosure; Figure 7 A flowchart illustrating another deployment method for a model provided in this embodiment of the disclosure; Figure 8 This is a flowchart illustrating a neural network quantization perception training method provided in an embodiment of the present disclosure. Figure 9 A schematic diagram of the structure of a deployment device for a model provided in an embodiment of this disclosure; Figure 10 A schematic diagram of the structure of a neural network quantization perception training device provided in an embodiment of this disclosure; Figure 11 A schematic diagram of another neural network quantization perception training device provided in an embodiment of this disclosure; Figure 12 This is a schematic block diagram of an example electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0031] The product involved in this invention is a quantitative toolchain.

[0032] The following description, with reference to the accompanying drawings, outlines an embodiment of the present disclosure, including a model deployment method, a neural network quantization perception training method, and an electronic device.

[0033] Figure 1 This is a flowchart illustrating a method for deploying a model provided in an embodiment of this disclosure.

[0034] like Figure 1 As shown, the method includes the following steps: Step 101: Input the network to be processed into the target model for quantization processing to obtain the quantized network to be processed; wherein, the target model is obtained by training the target neural network, the target neural network is obtained by fusing the rotation matrix with the weights of the target network layer of the neural network sample, and the rotation matrix is ​​used to process abnormal data in the target network layer.

[0035] In some embodiments, the network to be processed refers to the original neural network model that needs to be quantized, including but not limited to image classification models, object detection models, or natural language processing models. The target model is a trained neural network model that can be used for quantization. Its training process includes: fusing the rotation matrix with the weights of the target network layer of the neural network samples to obtain the target neural network, and then performing quantization-aware training on the target neural network. The rotation matrix is ​​an invertible linear transformation matrix used to process anomalous data in the target network layer. Anomalous data refers to data points with abnormal numerical distributions in the weights or activation values ​​of the target network layer. Fusion refers to merging the rotation matrix with the weight matrix of the target network layer, so that the adjustment effect of the rotation matrix on the data distribution is absorbed into the weights.

[0036] In practical implementation, inputting the network to be processed into the target model for quantization processing means using the quantization parameters already contained in the target model to perform low-bit-width quantization operations on the weights and / or activation values ​​of the network to be processed. Through quantization processing, the numerical accuracy of the network to be processed is reduced, and the model size is reduced.

[0037] The above method utilizes a target model pre-integrated with a rotation matrix to quantize the network to be processed, eliminating the need to introduce additional transformation operators during the quantization process and simplifying the quantization operation.

[0038] Step 102: Deploy the quantized network to be processed.

[0039] In some embodiments, deployment refers to installing or loading the quantized network to be processed onto a resource-constrained hardware device (e.g., a mobile terminal, embedded device, or edge computing node) so that it can perform inference tasks on that hardware device. The deployment process may include: converting the quantized network to be processed into a model format supported by the target hardware platform; transferring the converted model file to the storage medium of the hardware device; loading the model on the hardware device and allocating memory resources; and configuring input / output interfaces so that the model can receive actual data and output inference results.

[0040] Using the above method, the quantized network to be processed is deployed to resource-constrained hardware devices, enabling the devices to run neural network models with low computational overhead, thus meeting the practical application requirements of low latency and low power consumption.

[0041] In summary, the method provided in this disclosure can reduce training computation overhead, simplify the training process, and improve the stability and convergence of the training process by pre-integrating the rotation matrix with the weights of the target network layer. In the deployment phase, since the effect of the rotation matrix has been absorbed into the weights, there is no need to retain additional transformation operators or perform independent computational operations during inference, thereby simplifying the structure of the deployed model, reducing the computational overhead in the inference phase, and improving the deployment stability on resource-constrained hardware devices.

[0042] Optionally, such as Figure 2 As shown, based on any of the above embodiments or optional methods, the training method of the target model includes the following steps: Step 201: Preprocess the neural network sample to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network sample, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training.

[0043] In some embodiments, the neural network samples are initial neural network models that require quantization (such as image classification models, object detection models, or natural language processing models). The network layer to be quantized for perceptual training refers to the network layer in the neural network samples that requires low-bit-width quantization (such as fully connected layers or convolutional layers). The rotation matrix is ​​an invertible linear transformation matrix whose dimension matches the input or output data dimension of the target network layer. The rotation matrix is ​​used to process anomalous data in the target network layer. Anomalous data refers to data points with abnormal numerical distributions in the weights or activation values ​​of the target network layer, such as values ​​much larger or smaller than the normal range.

[0044] In practice, preprocessing includes inserting rotation matrices at the input and / or output of the target network layer. Inserting rotation matrices at the input allows for a linear transformation of the input data to the target network layer, adjusting its numerical distribution. Similarly, inserting rotation matrices at the output allows for a linear transformation of the output data, adjusting its numerical distribution. The inserted rotation matrices are used to process outlier data in the target network layer, suppressing the impact of outliers on subsequent quantization processes.

[0045] The above method preprocesses the neural network samples before quantization perception training, and processes abnormal data in the target network layer by inserting a rotation matrix, thus providing a network model with a balanced data distribution for subsequent quantization perception training.

[0046] Step 202: Fuse the rotation matrix with the weights of the target network layer to obtain the target neural network.

[0047] In some embodiments, fusion refers to merging the rotation matrix with the weight matrix of the target network layer to obtain a new weight matrix, which replaces the original weight matrix.

[0048] Specifically, when a first rotation matrix is ​​inserted at the input of the target network layer and a second rotation matrix is ​​inserted at the output, and the first and second rotation matrices are inverses of each other, the forward computation process of the target network layer can be expressed as: Output = Second rotation matrix × (Target network layer weights × (First rotation matrix × Input)). According to the associative law of matrix multiplication, this calculation is equivalent to: Output = (Second rotation matrix × Target network layer weights × First rotation matrix) × Input. Let the new weight matrix... =Second rotation matrix × weights of target network layer × first rotation matrix, then the forward calculation simplifies to: Output = × Input. In this way, the first rotation matrix, the weights of the target network layer, and the second rotation matrix can be combined into a new weight matrix. The original weight matrix is ​​replaced. After fusion, the calculation of the target network layer can be performed directly using the new weight matrix, without the need to perform the linear transformations corresponding to the first and second rotation matrices separately.

[0049] It should be noted that the fusion operation is completed before the start of quantization-aware training. Essentially, it pre-incorporates the adjustment effect of the rotation matrix on the data distribution into the weights of the target network layer. The fused target neural network no longer contains independent rotation matrix operators or parameters. Subsequent quantization-aware training directly optimizes the fused weights without requiring additional joint optimization or alternating updates to the rotation matrix.

[0050] By incorporating the rotation matrix with the weights of the target network layer before quantization training begins, the effect of the rotation matrix on adjusting the data distribution is pre-absorbed into the weights. This eliminates the need for independent optimization of the rotation matrix during subsequent training, reducing computational overhead and simplifying the training process.

[0051] Step 203: Perform quantization perception training on the target neural network to obtain the target model.

[0052] In some embodiments, quantization-aware training is a training method that simulates low-bit-width quantization computation during the training process. Quantization-aware training of the target neural network refers to simulating low-bit-width (e.g., INT8, INT4, etc.) quantization computation during training, enabling the model to actively adapt to the quantization environment and thus maintain high accuracy after quantization.

[0053] In practical implementation, quantization-aware training may include inserting pseudo-quantization operators at the locations to be quantized in the target neural network. These pseudo-quantization operators are used to simulate quantization and dequantization operations during forward propagation. The quantization parameters in the pseudo-quantization operators are initialized and calibrated based on the statistical characteristics of the data in the target neural network. The network after inserting the pseudo-quantization operators is iteratively trained using training data, with network parameters and quantization parameters updated synchronously during training, until the model converges, yielding the target model. The target model refers to a neural network model that has completed quantization-aware training and can be used to quantize the network to be processed.

[0054] By using the above method, the target neural network is trained with quantization perception, enabling the model to learn feature representations in a low bit-width quantization environment, thereby obtaining a quantized neural network that can run efficiently on resource-constrained hardware devices.

[0055] Figure 3 A flowchart illustrating a deployment method for a model provided in an embodiment of this disclosure is further shown, such as... Figure 3 As shown, the method includes steps 301-302.

[0056] Step 301: Determine whether the data input to the input terminal of the target network layer has undergone a linear transformation by the first rotation matrix.

[0057] In some embodiments, when preprocessing neural network samples, it is necessary to traverse each target network layer. For the currently processed target network layer, it is first necessary to determine whether the input data at the input end of the target network layer has already undergone a linear transformation using the first rotation matrix. The determination can be based on: checking whether the input data of the target network layer comes from the output of the previous network layer, and whether a rotation matrix has been inserted at the output end of the previous network layer; or checking whether the input data of the target network layer has an indication of having undergone a rotation matrix transformation in the computation graph. Specifically, by analyzing the computation graph structure of the neural network, it is possible to identify whether a rotation matrix operator exists on the input data path, thereby determining whether the input data has undergone a linear transformation using the first rotation matrix.

[0058] If the data input to the target network layer has not undergone any linear transformation using the first rotation matrix, then the subsequent insertion of the second rotation matrix is ​​not performed on the target network layer, i.e., the output of the target network layer remains unchanged. If the data input to the target network layer has already undergone a linear transformation using the first rotation matrix, then proceed to step 302.

[0059] Step 302: Insert a second rotation matrix at the output of the target network layer to obtain the processed neural network; wherein the first rotation matrix and the second rotation matrix are inverses of each other.

[0060] In some embodiments, when it is determined that the data input to the target network layer has undergone a linear transformation using the first rotation matrix, a second rotation matrix needs to be inserted at the output of the target network layer. The second rotation matrix is ​​the inverse of the first rotation matrix; that is, the second rotation matrix is ​​the inverse of the first rotation matrix. By inserting the second rotation matrix, the data distribution after computation by the target network layer can be restored to its original distribution state, thereby ensuring that the data distribution is consistent with the data distribution before rotation transformation when entering subsequent network layers, and avoiding deviation of the data distribution from its original characteristics due to the continuous effect of the rotation matrix.

[0061] It should be noted that the first and second rotation matrices, which are inverses of each other, can be interchanged. That is, the one inserted at the input end can be called the second rotation matrix, and the one inserted at the output end can be called the first rotation matrix, as long as they are inverses of each other.

[0062] Using the above method, during the preprocessing process, for target network layers that already have rotation matrices at the input end, the corresponding inverse matrix is ​​inserted at the output end, thereby ensuring that the rotation matrices appear in pairs, realizing the reversible adjustment of the data distribution, and providing a network structure with balanced and recoverable data distribution for subsequent quantization-sensory training.

[0063] Optionally, the target network layer includes a linear transformation layer in the neural network sample, and the linear transformation layer includes at least one of a projection transformation layer in the attention mechanism module and a fully connected transformation layer in the feedforward calculation module.

[0064] To clearly illustrate the embodiments of this disclosure, a Vision Transformer network is used as an example. Figure 4 This is a schematic diagram of the structure of a Transformer module in a Vision Transformer network provided in an embodiment of this disclosure, used to exemplify the specific type of the target network layer. For example... Figure 4 As shown, the Transformer module includes a self-attention submodule and a feedforward computation module.

[0065] In some embodiments, the target network layer includes a linear transformation layer in the neural network samples. A linear transformation layer is a network layer that performs matrix multiplication (i.e., linear projection) on the data to be processed. Linear transformation layers are the basic units for feature transformation in neural networks and are the primary objects requiring low-bit-width quantization in quantization-aware training.

[0066] Specifically, the linear transformation layer includes at least one of the projection transformation layer in the attention mechanism module and the fully connected transformation layer in the feedforward computation module. The projection transformation layer in the attention mechanism module is used to perform linear projection on the input data to generate queries, keys, values, or to project weighted values ​​onto the output, such as... Figure 4 The self-attention submodule shown includes a query projection layer, a key projection layer, a value projection layer, and an output projection layer. The fully connected transformation layer in the feedforward computation module is used to further transform features and perform nonlinear mappings on the output of the attention mechanism module, such as... Figure 4 The feedforward computation module shown contains a first fully connected layer and a second fully connected layer. The first fully connected layer maps the input features to a higher-dimensional space; the second fully connected layer maps the higher-dimensional features back to the original dimension.

[0067] It should be noted that, Figure 4 The linear layer shown is the entry linear transformation layer of the Transformer module. The first and second rotation matrices within the dashed box are inverses of each other, as are the third and fourth rotation matrices. The root mean square normalization layer is the root mean square normalization operation unit, the matrix multiplication layer is the matrix multiplication calculation unit, the normalized exponential function layer is the normalized exponential function calculation unit, the Gaussian error linear unit layer is the Gaussian error linear unit activation calculation unit, and the residual addition layer is the residual connection calculation unit. These units together constitute the computation flow of the Transformer module. The query projection layer, key projection layer, value projection layer, output projection layer, first fully connected layer, and second fully connected layer are all linear transformation layers and can be used as the target network layers to which the rotation matrices are to be inserted in this embodiment. It should be noted that... Figure 4 The diagram shown is merely an exemplary network structure. Linear transformation layers such as convolutional layers and fully connected layers in other types of neural networks (such as convolutional neural networks) are also applicable to the embodiments of this disclosure.

[0068] To clearly illustrate the embodiments of this disclosure, based on Figure 2 The illustrated embodiment further explains step 201, which involves processing the abnormal data in the network layer to be quantized for training, including: A linear transformation is performed on the weights and / or activation values ​​of the target network layer to eliminate outliers in the weights and / or activation values ​​of the target network layer.

[0069] In some embodiments, linear transformation of the weights and / or activation values ​​of a target network layer refers to performing matrix multiplication on the weight matrix or activation value tensor using a rotation matrix. A rotation matrix is ​​an invertible linear transformation matrix that acts as a rotation or scaling of the coordinate space containing the data, thereby altering the numerical distribution characteristics of the data. Specifically, when a rotation matrix is ​​applied to the weight matrix, it is equivalent to linearly combining the rows or columns of the weight matrix, causing the weight values, originally scattered in different directions, to redistribute under the new basis vectors. When a rotation matrix is ​​applied to the activation values, it is equivalent to performing a coordinate transformation on each activation value vector, causing outliers that deviate from the mainstream distribution to be compressed or stretched on the new coordinate axes, thus reducing their difference from normal values.

[0070] Outliers typically appear as isolated points far from the data center in the original coordinate system. A rotation matrix can rotate the data coordinate axes to the direction of maximum variance, making the variance distribution of the data more uniform in the new coordinate system. This reduces the extreme prominence of outlier projections on the main dimensions, thus suppressing outliers. Furthermore, rotation matrices can be combined with scaling operations (such as rotation via orthogonal matrices) to adjust the numerical range, making the overall distribution more compact.

[0071] By using the above method, the weights and / or activation values ​​of the target network layer are linearly transformed by the rotation matrix, which can effectively eliminate or suppress outliers, making the numerical distribution of weights and activation values ​​more balanced. This provides a network model with a numerical range friendly for subsequent quantization-aware training, thereby improving the stability of quantization training and the accuracy of the quantized model.

[0072] Figure 5 A flowchart illustrating a deployment method for a model provided in an embodiment of this disclosure is further shown, such as... Figure 5 As shown, the method includes steps 401-404.

[0073] Step 401: Perform data processing on the training data to obtain processed training data; wherein, the data processing includes at least one of the following processing methods: pruning, normalization, and data augmentation.

[0074] In some embodiments, training data refers to the original dataset used to train the neural network. The purpose of data processing is to improve the quality and diversity of the data, making the training data suitable for quantitative perception training, thereby improving the generalization ability and robustness of the model.

[0075] Specifically, data processing can include at least one of the following methods: cropping, normalization, and data augmentation. Cropping refers to resizing or cropping the original data. For example, in image processing, an image is randomly cropped to a fixed size (e.g., 224×224 pixels) to remove irrelevant information from image edges and make the input data uniform in size. Normalization refers to mapping the numerical range of data to a specific interval (e.g., [0,1]), or making its mean 0 and variance 1, to eliminate the influence of different dimensions between samples and accelerate model convergence.

[0076] It should be noted that the above data processing methods can be flexibly selected and combined according to specific tasks and data types. For example, for image classification tasks, cropping, normalization, and data augmentation are usually used simultaneously; for text classification tasks, data processing may include word segmentation, padding and truncation, normalization, and other operations. The specific implementation of data processing is not limited to the examples above, and any preprocessing operation that can improve data quality and diversity falls within the protection scope of this disclosure.

[0077] Step 402: Preprocess the neural network sample to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network sample, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training.

[0078] Step 403: Fuse the rotation matrix with the weights of the target network layer to obtain the target neural network.

[0079] Step 404: Perform quantization perception training on the target neural network to obtain the target model.

[0080] For explanations of steps 402-404, please refer to [link / reference needed]. Figure 2 The detailed descriptions of the relevant embodiments are not repeated here.

[0081] By using the above method, the training data is processed before the neural network samples are preprocessed to obtain high-quality and diverse training data, which provides a data foundation for subsequent quantitative perception training, thereby further improving the performance and generalization ability of the quantitative neural network.

[0082] Figure 6 A flowchart illustrating a deployment method for a model provided in this disclosure embodiment is further illustrated, based on... Figure 5 The illustrated embodiment further explains step 203. Figure 6 This may include the following steps: Step 501: Insert a pseudo-quantization operator at the position to be quantized in the target neural network.

[0083] In some embodiments, the location to be quantized refers to a network layer or computation node in the neural network that requires low-bit-width quantization. A pseudo-quantization operator is a function module used to simulate low-bit-width quantization operations during training. It contains quantization parameters and quantizes high-precision floating-point numbers (such as FP32) into low-bit-width integers (such as INT8) during forward propagation, then dequantizes them back to floating-point numbers, thus introducing quantization error. This allows the network to learn and adapt to the precision loss caused by quantization during training. A specific implementation of the pseudo-quantization operator can use a pass-through estimator to handle the non-differentiability of the quantization function, allowing the gradient to be approximately propagated backward through the pseudo-quantization operator. In practice, pseudo-quantization operators are typically inserted into the original network structure as network layers, for example, inserting a pseudo-quantization node after each layer to be quantized. The number and location of inserted pseudo-quantization operators can be determined according to the quantization strategy.

[0084] Step 502: Initialize and calibrate the quantization parameters in the pseudo-quantization operator.

[0085] In some embodiments, quantization parameters refer to parameters used in the pseudo-quantization operator to control the quantization process, including but not limited to scaling factors and zeros. The scaling factor determines the mapping ratio from floating-point numbers to integers, and the zeros align zeros in floating-point numbers with zeros in integers. Initial calibration refers to initially determining the values ​​of the quantization parameters by statistically analyzing the range of activation values ​​and weights in the network using a small amount of calibration data before quantization-aware training. The calibration process includes: inputting calibration data into the network for forward inference, collecting the numerical distribution (e.g., minimum, maximum, histogram, etc.) at each position to be quantized, and then calculating the scaling factor and zeros based on this statistical information. Calibration methods include, but are not limited to, symmetric quantization based on maximum and minimum values, and asymmetric quantization based on histogram KL divergence. The purpose of calibration is to enable the pseudo-quantization operator to reasonably simulate quantization operations in the early stages of training, avoiding training instability due to excessive deviations in the quantization parameters.

[0086] Step 503: Based on the processed training data, iteratively train the target neural network after inserting the pseudo-quantization operator, update the network parameters and the quantization parameters, and obtain the target model.

[0087] In some embodiments, iterative training refers to performing multiple rounds of forward and backward propagation on a network with inserted pseudo-quantization operators using processed training data, aiming to minimize the loss function while simultaneously updating the network weight parameters and the quantization parameters in the pseudo-quantization operators. During training, the pseudo-quantization operator performs quantization and dequantization operations on the data during forward propagation, introducing quantization noise. During backward propagation, a pass-through estimator or other gradient approximation methods are used to approximate the gradient of the quantization function to 1 or correct it according to the quantization interval, allowing the gradient to be backpropagated through the pseudo-quantization operator, thereby updating the network parameters. Simultaneously, the quantization parameters can also be used as learnable parameters in training (e.g., scaling factors can be updated through gradient descent). Training continues until the model converges (e.g., the loss no longer decreases or a preset number of iterations is reached). The final model obtained is the target model, which is adapted to low-bit-width quantization environments and can be used for quantization processing of the network to be processed and subsequent deployment.

[0088] By using the above method, quantization-aware training is performed on the target neural network, enabling the model to actively learn and adapt to quantization noise during the training process, thereby obtaining a target model that can maintain high accuracy when deployed with low bit width.

[0089] Figure 7 A flowchart illustrating a deployment method for a model provided in an embodiment of this disclosure is further shown, such as... Figure 7 As shown, the method includes steps 601-605.

[0090] Step 601: Perform data processing on the training data to obtain processed training data; wherein, the data processing includes at least one of the following processing methods: pruning, normalization, and data augmentation.

[0091] For a description of step 601, please refer to [link / reference needed]. Figure 5 The detailed descriptions of the relevant embodiments are not repeated here.

[0092] Step 602: Preprocess the neural network sample to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network sample, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training.

[0093] Step 603: Fuse the rotation matrix with the weights of the target network layer to obtain the target neural network.

[0094] Step 604: Perform quantization perception training on the target neural network to obtain the target model.

[0095] For explanations of steps 602-604, please refer to [link / reference needed]. Figure 2 The detailed descriptions of the relevant embodiments are not repeated here.

[0096] Step 605: Export the target model as a model file in a preset format; wherein, the pseudo-quantization operator in the model file is converted into a real quantization operator.

[0097] In some embodiments, the preset format refers to a model file format supported by a general or specific inference framework used for model deployment (e.g., Open Neural Network Exchange (ONNX), TensorRT, TFLite, etc.). Exporting the target model as a model file in a preset format facilitates the deployment of the model to resource-constrained hardware devices for inference.

[0098] During the export process, pseudo-quantization operators in the model file are converted into true quantization operators. Pseudo-quantization operators are used during training to simulate quantization operations. They contain quantization parameters and perform quantization and dequantization operations during forward propagation, but their form is designed for training and is not suitable for direct inference. True quantization operators are the operators that actually perform low-bit-width computations in the inference framework, such as the QuantizeLinear and DequantizeLinear operators in ONNX, or the INT8 convolution operator in TensorRT.

[0099] The conversion process includes: identifying all pseudo-quantization operators in the model, generating corresponding true quantization operators based on their quantization parameters (scaling factor, zero point), replacing the pseudo-quantization operators with true quantization operators, and adjusting network connections to ensure correct data flow. In the converted model file, the quantization parameters are embedded in operator properties or separate tensors for the inference engine to load and use at runtime.

[0100] Using the above method, after quantization-aware training is completed, the model is exported in a deployment-ready format, and the pseudo-quantization operator is converted into a real quantization operator. This enables the model to run efficiently on resource-constrained hardware devices while maintaining the accuracy obtained from quantization-aware training, providing a foundation for subsequent quantization processing and deployment of the network to be processed.

[0101] Figure 8 This is a flowchart illustrating a neural network quantization perception training method provided in an embodiment of the present disclosure.

[0102] like Figure 8 As shown, the method includes the following steps: Step 701: Preprocess the neural network to be processed to obtain the processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network to be processed, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training.

[0103] In some embodiments, the neural network to be processed and Figure 2 The neural network samples in the related embodiments are the same, referring to the initial neural network model (such as an image classification model, object detection model, or natural language processing model) that needs to undergo quantization-based perceptual training. The network layer to be quantized for perceptual training refers to the network layer in the neural network to be processed that requires low-bit-width quantization (such as a fully connected layer or convolutional layer). The rotation matrix is ​​an invertible linear transformation matrix whose dimension matches the input or output data dimension of the target network layer. The rotation matrix is ​​used to process anomalous data in the target network layer. Anomalous data refers to data points with abnormal numerical distributions in the weights or activation values ​​of the target network layer, such as values ​​much larger or smaller than the normal range.

[0104] In practice, preprocessing includes inserting rotation matrices at the input and / or output of the target network layer. Inserting rotation matrices at the input allows for a linear transformation of the input data to the target network layer, adjusting its numerical distribution. Similarly, inserting rotation matrices at the output allows for a linear transformation of the output data, adjusting its numerical distribution. The inserted rotation matrices are used to process outlier data in the target network layer, suppressing the impact of outliers on subsequent quantization processes.

[0105] The above method preprocesses the neural network to be processed before quantization perception training. By inserting a rotation matrix, abnormal data in the target network layer is processed, providing a network model with balanced data distribution for subsequent quantization perception training.

[0106] Step 702: Fuse the rotation matrix with the weights of the target network layer to obtain the target neural network.

[0107] In some embodiments, fusion refers to merging the rotation matrix with the weight matrix of the target network layer to obtain a new weight matrix, which replaces the original weight matrix.

[0108] Specifically, when a first rotation matrix is ​​inserted at the input of the target network layer and a second rotation matrix is ​​inserted at the output, and the first and second rotation matrices are inverses of each other, the forward computation process of the target network layer can be expressed as: Output = Second rotation matrix × (Target network layer weights × (First rotation matrix × Input)). According to the associative law of matrix multiplication, this calculation is equivalent to: Output = (Second rotation matrix × Target network layer weights × First rotation matrix) × Input. Let the new weight matrix... =Second rotation matrix × weights of target network layer × first rotation matrix, then the forward calculation simplifies to: Output = × Input. In this way, the first rotation matrix, the weights of the target network layer, and the second rotation matrix can be combined into a new weight matrix. The original weight matrix is ​​replaced. After fusion, the calculation of the target network layer can be performed directly using the new weight matrix, without the need to perform the linear transformations corresponding to the first and second rotation matrices separately.

[0109] It should be noted that the fusion operation is completed before the start of quantization-aware training. Essentially, it pre-incorporates the adjustment effect of the rotation matrix on the data distribution into the weights of the target network layer. The fused target neural network no longer contains independent rotation matrix operators or parameters. Subsequent quantization-aware training directly optimizes the fused weights without requiring additional joint optimization or alternating updates to the rotation matrix.

[0110] By using the above method, the rotation matrix is ​​fused with the weights of the target network layer before the start of quantization perception training. This allows the adjustment effect of the rotation matrix on the data distribution to be pre-absorbed into the weights, so that the rotation matrix does not need to be optimized independently in the subsequent training process, reducing the computational overhead of the training phase and simplifying the training process.

[0111] Step 703: Perform quantization perception training on the target neural network to obtain the quantized neural network.

[0112] In some embodiments, quantization-aware training is a training method that simulates low-bit-width quantization computation during the training process. Quantization-aware training of the target neural network refers to simulating low-bit-width (e.g., INT8, INT4, etc.) quantization computation during training, enabling the model to actively adapt to the quantization environment and thus maintain high accuracy after quantization.

[0113] In practical implementation, quantization-aware training may include inserting pseudo-quantization operators at the positions to be quantized in the target neural network. These pseudo-quantization operators are used to simulate quantization and dequantization operations during forward propagation. The quantization parameters in the pseudo-quantization operators are initialized and calibrated based on the statistical characteristics of the data in the target neural network. The network with the inserted pseudo-quantization operators is then iteratively trained using training data, with network parameters and quantization parameters updated synchronously during training, until the model converges, resulting in the quantized neural network. The quantized neural network refers to a neural network model that has completed quantization-aware training and can be used for low-bit-width deployment, or for quantizing the network to be processed.

[0114] By using the above method, the target neural network is trained with quantization perception, enabling the model to learn feature representations in a low bit-width quantization environment, thereby obtaining a quantized neural network that can run efficiently on resource-constrained hardware devices.

[0115] The method provided in this disclosure can process abnormal data in the target network layer by inserting a rotation matrix, effectively suppressing the adverse effects of outliers on the initialization of quantization parameters, ensuring the stability of the quantization-aware training process, and solving the problem of training convergence difficulty caused by abnormal data. At the same time, the fusion of the rotation matrix and the weights of the target network layer is completed before quantization-aware training, eliminating independent rotation matrix operators and related additional calculation operations. There is no need to perform joint optimization related to the rotation matrix during the training phase, which simplifies the overall process of quantization-aware training, reduces the computational overhead during the training phase, and avoids the impact of additional calculation operations on training stability.

[0116] Corresponding to the model deployment method and neural network quantization perception training method described above, this invention also proposes a model deployment device and a neural network quantization perception training device. Since the device embodiments of this invention correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to in the method embodiments described above, and will not be repeated here.

[0117] Figure 9 This is a schematic diagram of the structure of a model deployment device provided in an embodiment of the present disclosure, as shown below. Figure 9 As shown, it includes: Processing unit 61 is used to input the network to be processed into the target model for quantization processing to obtain the quantized network to be processed; wherein, the target model is obtained by training a target neural network, the target neural network is obtained by fusing a rotation matrix with the weights of the target network layer of the neural network samples, and the rotation matrix is ​​used to process abnormal data in the target network layer; Deployment unit 62 is used to deploy the quantized network to be processed.

[0118] The deployment apparatus for the model described in this disclosure can reduce the computational overhead of training during the training phase, simplify the training process, and improve the stability and convergence of the training process by pre-integrating the rotation matrix with the weights of the target network layer. During the deployment phase, since the effect of the rotation matrix has been absorbed into the weights, there is no need to retain additional transformation operators or perform independent computational operations during the inference process, thereby simplifying the structure of the deployed model, reducing the computational overhead of the inference phase, and improving the deployment stability on resource-constrained hardware devices.

[0119] Figure 10 This is a schematic diagram of the structure of a neural network quantization perception training device provided in an embodiment of this disclosure, as shown below. Figure 10 As shown, it includes: The preprocessing unit 71 is used to preprocess the neural network to be processed to obtain the processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input end and / or output end of the target network layer of the neural network to be processed, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. Fusion unit 72 is used to fuse the rotation matrix with the weights of the target network layer to obtain the target neural network; Training unit 73 is used to perform quantization perception training on the target neural network to obtain a quantized neural network.

[0120] The neural network quantization perception training device described in this embodiment can process abnormal data in the target network layer by inserting a rotation matrix, effectively suppressing the adverse effects of outliers on the initialization of quantization parameters, ensuring the stability of the quantization perception training process, and solving the problem of training convergence difficulty caused by abnormal data. At the same time, the fusion of the rotation matrix and the weights of the target network layer is completed before quantization perception training, eliminating independent rotation matrix operators and related additional calculation operations. There is no need to perform joint optimization related to the rotation matrix during the training phase, simplifying the overall process of quantization perception training, reducing the computational overhead during the training phase, and avoiding the impact of additional calculation operations on training stability.

[0121] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 11 As shown, the preprocessing unit 71 includes: The judgment module 711 is used to determine whether the data input to the input end of the target network layer has undergone linear transformation by the first rotation matrix; The first insertion module 712 is used to insert a second rotation matrix at the output of the target network layer when the data input to the input of the target network layer has undergone a linear transformation by the first rotation matrix, so as to obtain the processed neural network; wherein the first rotation matrix and the second rotation matrix are inverse matrices of each other.

[0122] Furthermore, in one possible implementation of this disclosure embodiment, the target network layer includes a linear transformation layer in the neural network to be processed, and the linear transformation layer includes at least one of a projection transformation layer in the attention mechanism module and a fully connected transformation layer in the feedforward calculation module.

[0123] Furthermore, in one possible implementation of this disclosure, the processing of abnormal data in the network layer to be quantized for training includes: performing a linear transformation on the weights and / or activation values ​​of the target network layer to eliminate abnormal values ​​in the weights and / or activation values ​​of the target network layer.

[0124] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 11 As shown, the device further includes: The data processing unit 74 is used to process the training data before the preprocessing unit 71 preprocesses the neural network to be processed to obtain the processed neural network, thereby obtaining the processed training data; wherein the data processing includes at least one of the following processing methods: pruning, normalization, and data augmentation.

[0125] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 11 As shown, the training unit 73 includes: The second insertion module 731 is used to insert a pseudo-quantization operator at the position to be quantized in the target neural network; Calibration module 732 is used to initialize and calibrate the quantization parameters in the pseudo-quantization operator; The training module 733 is used to iteratively train the target neural network after inserting the pseudo-quantization operator based on the processed training data, update the network parameters and the quantization parameters, and obtain the quantized neural network.

[0126] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 11 As shown, the device further includes: The export unit 75 is used to export the quantized neural network as a model file in a preset format after the training unit 73 performs quantization perception training on the target neural network to obtain the quantized neural network; wherein, the pseudo-quantization operator in the model file is converted into a real quantization operator.

[0127] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of the embodiments of this disclosure, and the principle is the same. Therefore, the embodiments of this disclosure are not limited thereto.

[0128] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0129] Figure 12 A schematic block diagram of an example electronic device 1200 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0130] like Figure 12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in ROM (Read-Only Memory) 1202 or loaded from storage unit 1208 into RAM (Random Access Memory) 1203. The RAM 1203 may also store various programs and data required for the operation of the electronic device 1200. The computing unit 1201, ROM 1202, and RAM 1203 are interconnected via bus 1204. An I / O (Input / Output) interface 1205 is also connected to bus 1204.

[0131] Multiple components in electronic device 1200 are connected to I / O interface 1205, including: input unit 1206, such as keyboard, mouse, etc.; output unit 1207, such as various types of displays, speakers, etc.; storage unit 1208, such as disk, optical disk, etc.; and communication unit 1209, such as network card, modem, wireless transceiver, etc. Communication unit 1209 allows electronic device 1200 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0132] The computing unit 1201 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 1201 performs the various methods and processes described above, such as model deployment methods or neural network quantization perception training methods. For example, in some embodiments, the model deployment methods or neural network quantization perception training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 1208. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 1200 via ROM 1202 and / or communication unit 1209. When the computer program is loaded into RAM 1203 and executed by computing unit 1201, one or more steps of the method described above can be performed. Alternatively, in other embodiments, computing unit 1201 can be configured by any other suitable means (e.g., by means of firmware) to perform the deployment method of the aforementioned model or the neural network quantization perception training method.

[0133] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0134] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0135] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0136] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0137] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0138] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0139] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0140] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0141] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for deploying a model, characterized in that, include: The network to be processed is input into the target model for quantization to obtain the quantized network to be processed; wherein, the target model is obtained by training the target neural network, and the target neural network is obtained by fusing the rotation matrix with the weights of the target network layer of the neural network sample, and the rotation matrix is ​​used to process abnormal data in the target network layer; The quantized network to be processed is then deployed.

2. The method according to claim 1, characterized in that, The training methods for the target model include: The neural network samples are preprocessed to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network samples, the target network layer being a network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. The rotation matrix is ​​fused with the weights of the target network layer to obtain the target neural network; The target neural network is subjected to quantization perception training to obtain the target model.

3. The method according to claim 2, characterized in that, The preprocessing of the neural network samples to obtain the processed neural network includes: Determine whether the data input to the input terminal of the target network layer has undergone a linear transformation using the first rotation matrix; When the data input to the input end of the target network layer is linearly transformed by the first rotation matrix, a second rotation matrix is ​​inserted at the output end of the target network layer to obtain the processed neural network; wherein the first rotation matrix and the second rotation matrix are inverse matrices of each other.

4. The method according to claim 3, characterized in that, The target network layer includes a linear transformation layer in the neural network sample, and the linear transformation layer includes at least one of a projection transformation layer in the attention mechanism module and a fully connected transformation layer in the feedforward calculation module.

5. The method according to claim 2, characterized in that, The processing of abnormal data in the network layer to be quantized for training includes: A linear transformation is performed on the weights and / or activation values ​​of the target network layer to eliminate outliers in the weights and / or activation values ​​of the target network layer.

6. The method according to claim 2, characterized in that, Before preprocessing the neural network samples to obtain the processed neural network, the method further includes: The training data is processed to obtain processed training data; wherein the data processing includes at least one of the following methods: pruning, normalization, and data augmentation.

7. The method according to claim 6, characterized in that, The step of performing quantization-sensory training on the target neural network to obtain the target model includes: A pseudo-quantization operator is inserted at the position to be quantized in the target neural network; The quantization parameters in the pseudo-quantization operator are initialized and calibrated. Based on the processed training data, the target neural network after inserting the pseudo-quantization operator is iteratively trained to update the network parameters and the quantization parameters, thereby obtaining the target model.

8. The method according to claim 7, characterized in that, After performing quantized perceptual training on the target neural network to obtain the target model, the method further includes: The target model is exported as a model file in a preset format; wherein, the pseudo-quantization operators in the model file are converted into real quantization operators.

9. A method for training a neural network based on quantitative perception, characterized in that, include: The neural network to be processed is preprocessed to obtain a processed neural network; wherein, the preprocessing includes inserting a rotation matrix at the input and / or output of the target network layer of the neural network to be processed, the target network layer being the network layer to be quantized for perceptual training, and the rotation matrix being used to process abnormal data in the network layer to be quantized for perceptual training. The rotation matrix is ​​fused with the weights of the target network layer to obtain the target neural network; The target neural network is subjected to quantization perception training to obtain a quantized neural network.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or claim 9.

Images