Model parameter quantification method and device, computing equipment and storage medium

Abstract

The invention provides a model parameter quantification method and device, a model operation method and device, computing equipment and a storage medium. When the model parameters are quantized, abnormal channels corresponding to the layer structures included in the target model are identified, the positions of the parameters corresponding to the abnormal channels in the parameter matrix are switched, the parameter matrix is divided into an abnormal channel matrix and a non-abnormal channel matrix, the non-abnormal channel matrix is subjected to parameter quantization, and the model parameters are quantized; fusion calculation between different parameter matrixes is replaced by fusion calculation between integer matrixes corresponding to the different parameter matrixes and fusion calculation between abnormal channel matrixes corresponding to the different parameter matrixes, more efficient integer calculation units in calculation hardware are fully utilized as much as possible to accelerate reasoning, the efficiency of model reasoning is improved, and the reasoning accuracy is improved. And meanwhile, the influence of abnormal parameters on the overall calculation precision in the operation of the low-bit integer matrix is avoided, and the accuracy of a calculation result is ensured.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, and in particular to a model parameter quantification method, model operation method, device, computing equipment, storage medium and computer program product. Background Art

[0002] Large models usually refer to machine learning models with a large number of parameters and complex structures. They are usually built by deep neural networks and learn complex patterns and relationships by processing large amounts of data. They are currently used in many fields such as natural language processing, computer vision, speech processing, and recommendation systems. They perform well in text generation, video generation, and feature extraction. The wide application and powerful performance of large models make them one of the core technologies in modern artificial intelligence research and application.

[0003] As large models grow exponentially, their demand for hardware resources also increases. Larger models increase accuracy, but this also means more computing resources and higher computational costs. Quantizing large models can reduce model size and computational requirements, but this also reduces inference accuracy. Therefore, it is necessary to address the severe loss of accuracy in large models after low-bit quantization. Summary of the Invention

[0004] Embodiments of the present application provide a model parameter quantification method, a model operation method, an apparatus, a computing device, a storage medium, and a computer program product to solve one or more of the above-mentioned technical problems.

[0005] In a first aspect, an embodiment of the present application provides a method for quantizing model parameters, including: identifying abnormal channels of at least one layer structure in a target model; for a parameter matrix corresponding to each layer structure, switching the position of the parameters corresponding to the abnormal channel in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, performing parameter quantization processing on the non-abnormal channel matrix to obtain an integer matrix corresponding to the non-abnormal channel matrix; replacing the fusion calculation between different parameter matrices with a fusion calculation between integer matrices corresponding to different parameter matrices, and a fusion calculation between abnormal channel matrices corresponding to different parameter matrices.

[0006] In a second aspect, an embodiment of the present application provides a model operation method, including: obtaining a target model, wherein the model parameters of the target model are quantized, and the fusion calculation between different parameter matrices has been replaced by the fusion calculation between the integer matrices corresponding to the different parameter matrices, and the fusion calculation between the abnormal channel matrices corresponding to the different parameter matrices. The integer matrix corresponding to the non-abnormal channel matrix is obtained by performing parameter quantization processing on the non-abnormal channel matrix, and the non-abnormal channel matrix and the abnormal channel matrix are obtained by switching the positions of the parameters corresponding to the abnormal channel in the parameter matrix; calling the integer calculation unit in the computing hardware to process the fusion calculation between the integer matrices corresponding to the different parameter matrices.

[0007] On the third side, an embodiment of the present application provides a method for quantizing model parameters, including: before the model is run, identifying abnormal channels of at least one layer structure in the target model; performing transposition transformation processing and parameter quantization processing on the weight parameter matrix of the layer structure according to the identified abnormal channels of the layer structure; during the model operation stage, performing transposition transformation processing and parameter quantization processing on the activation parameter matrix of the layer structure according to the identified abnormal channels of the layer structure, and replacing the fusion calculation between different parameter matrices with a fusion calculation between integer matrices corresponding to different parameter matrices, and a fusion calculation between abnormal channel matrices corresponding to different parameter matrices; wherein the transposition transformation processing includes switching the position of the parameters corresponding to the abnormal channels in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and the parameter quantization processing is used to perform parameter quantization processing on the non-abnormal channel matrix to obtain the integer type corresponding to the non-abnormal channel matrix.

[0008] In a fourth aspect, an embodiment of the present application provides a computing device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above method when executing the computer program.

[0009] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, wherein the computer program product includes a computer program, and the computer program implements the above method when executed by a processor.

[0011] According to an embodiment of the present application, when the model parameters are quantized, the abnormal channels corresponding to one or more layer structures included in the target model are identified. For the parameter matrix corresponding to each layer structure, according to the abnormal channels determined previously, the positions of the parameters corresponding to the abnormal channels in the parameter matrix are switched to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, thereby distinguishing the abnormal parameters and non-abnormal parameters in the parameter matrix. When the model parameters are quantized, the non-abnormal channel matrix is parameter quantized to obtain an integer matrix corresponding to the non-abnormal channel matrix. The abnormal channel matrix still corresponds to a floating-point matrix. In the model calculation link, the fusion calculation between different parameter matrices is replaced by the fusion calculation between the integer matrices corresponding to different parameter matrices. Combined calculations and fusion calculations between abnormal channel matrices corresponding to different parameter matrices are performed, so that when the computing unit of the computing device is called to perform model calculations, the integer computing unit in the computing hardware can be used to process the fusion calculations between integer matrices corresponding to different parameter matrices, and the floating-point computing unit can be used to process the fusion calculations between floating-point matrices corresponding to different parameter matrices. By separating the model parameters corresponding to the abnormal channels and the model parameters of the non-abnormal channels for calculation processing, and performing integer calculations on the non-abnormal parameters, the more efficient integer computing units in the computing hardware are fully utilized as much as possible to accelerate reasoning, thereby improving the efficiency of model reasoning, and at the same time avoiding the influence of abnormal parameters on the overall calculation accuracy in the operation of low-bit integer matrices, thereby ensuring the accuracy of the calculation results.

[0012] The above description is only an overview of the technical solution of this application. In order to more clearly understand the technical means of this application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of this application more obvious and easy to understand, the specific implementation methods of this application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0014] Figure 1 A schematic diagram of identifying abnormal channels in an embodiment of the present application is shown;

[0015] Figure 2 A schematic diagram showing parameter position adjustment of a weight parameter matrix in an embodiment of the present application is shown;

[0016] Figure 3 A schematic diagram showing parameter adjustment of an activation parameter matrix in an embodiment of the present application is shown;

[0017] Figure 4 A flow chart showing a method for quantifying model parameters according to an embodiment of the present application is shown;

[0018] Figure 5 A flow chart showing a model operation method according to an embodiment of the present application is shown;

[0019] Figure 6 A flow chart showing a method for quantifying model parameters according to an embodiment of the present application is shown;

[0020] Figure 7 A structural block diagram of a device for quantizing model parameters according to an embodiment of the present application is shown;

[0021] Figure 8 A structural block diagram of a model operation device according to an embodiment of the present application is shown;

[0022] Figure 9 A structural block diagram of a device for quantizing model parameters according to an embodiment of the present application is shown;

[0023] Figure 10 A block diagram of a computing device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] Hereinafter, only certain exemplary embodiments are briefly described. As will be appreciated by those skilled in the art, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and description are to be regarded as illustrative in nature and not restrictive.

[0025] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and all of them fall within the scope of protection of the embodiments of the present application.

[0026] Model quantization is a technique that reduces model size and computational requirements by reducing the precision of model parameters. This effectively compresses weight parameters, reduces bandwidth, and accelerates model inference efficiency. For example, model parameters stored as 32-bit floating-point numbers (FP32) are converted to the low-precision 8-bit integer format (INT8) through quantization, saving storage space, improving computational efficiency, and reducing memory usage and power consumption.

[0027] From a quantization perspective, mainstream quantization algorithms can be divided into two categories, depending on whether activations are quantized. One type of quantization algorithm quantizes only weights, such as GPTQ (Generative Pre-trained Transformer-Q, a natural language processing technique based on the Transformer model) and AWQ (Activation-aware Weight Quantization, activation-based weight quantization). These algorithms can quantize large model weights to low-bit integer data types like int4 without significantly degrading accuracy. While this approach reduces storage requirements, in practice, the quantized weights must be re-quantized to floating-point numbers. Therefore, they cannot directly leverage the hardware's integer computing units to accelerate inference, hindering hardware-level acceleration and failing to leverage the hardware's computational advantages. Consequently, the improvement in inference efficiency falls far short of expectations. The other type of quantization algorithm quantizes both activations and weights, such as Smoothquant. This algorithm smoothes outliers in activations by selecting a smoothing parameter and simultaneously inverses the corresponding weight parameters to ensure consistent results. Because the smoothed activation values and weights can be quantized simultaneously, computation is accelerated using the hardware's int8 computing units, preventing a significant decrease in model accuracy. However, due to outliers in the model parameters, this algorithm can fail when applied to hardware that accepts low-bit integer data, such as int4, resulting in a significant decrease in model accuracy.

[0028] To solve one or more of the above problems, the embodiment of the present application provides a new model quantization scheme, which takes the model to be parameter quantized as the target model. The model is usually composed of a plurality of layer structures (Layer) with different functions. Each layer structure is responsible for a specific computing task, and the complex processing tasks are completed together through the connection between the layer structures. Each layer structure can be further divided into a sub-layer structure. Taking a large model as an example, its layer structure includes basic layer structures such as input embedding layer, encoder, decoder, and output layer, as well as auxiliary layer structures such as residual connection layer and normalization layer. The encoder layer can further include a multi-head self-attention layer, a feedforward neural network layer, a residual connection layer, and the like.

[0029] Channels are used in the model to describe feature dimensions, representing the different features extracted by the model in a certain layer of structure. For example, in the multi-head attention mechanism, the channel dimension represents the feature dimension of each attention head. In the convolutional neural network, the channel dimension represents the number of input or output feature maps. When the behavior or characteristics of certain channels deviate, resulting in a decrease in model performance or abnormal output, such channels can be recorded as abnormal channels. By identifying these abnormal channels, other channels are recorded as non-abnormal channels. Further, in the parameter matrices corresponding to each layer of the model structure, the parameters corresponding to the abnormal channels and the parameters corresponding to the non-abnormal channels are distinguished. After the distinction, the parameters corresponding to the abnormal channels are aggregated into the abnormal channel matrix, and the parameters corresponding to the non-abnormal channels are aggregated into the non-abnormal channel matrix, thereby dividing the original parameter matrix into two matrices.

[0030] In the parameter quantization link, only the non-abnormal channel matrix corresponding to the non-abnormal channel is further parameter quantized to obtain the integer matrix corresponding to the non-abnormal channel. The abnormal channel matrix corresponding to the abnormal channel is not parameter quantized. The subsequent fusion operation of the model for different parameter matrices is further modified, and the original direct fusion operation is replaced by two separate fusion calculations. One is the fusion calculation between integer matrices corresponding to different parameter matrices, and the other is the fusion calculation between abnormal channel matrices as floating-point matrices corresponding to different parameter matrices. In this way, when the subsequent model performs fusion calculations between parameter matrices, different calculation units are used to process matrices of different data types. Specifically, for the fusion calculation between integer matrices, the integer calculation unit can be directly called for processing, and for the floating-point abnormal channel matrix that has not been quantized, the floating-point calculation unit can be called for processing. On the one hand, the non-abnormal parameters are calculated as integers, and the more efficient integer calculation units in the computing hardware are fully utilized to accelerate reasoning and improve the efficiency of model reasoning. On the other hand, by dividing the abnormal parameters and non-abnormal parameters into two matrices and distinguishing different calculation methods, the influence of abnormal parameters on the overall calculation accuracy in the operation of low-bit integer matrices is avoided, thereby ensuring the accuracy of the calculation results. After specific application tests, the large model quantized by the parameter quantization method of the embodiment of the present application has increased the reasoning speed several times compared to the int8 level.

[0031] Among them, the fusion calculation between different parameter matrices can be understood as the multiplication calculation between different matrices.

[0032] Abnormal channels in the model layer structure will lead to a decline in model performance. For example, some channels fail to accurately extract features, channel output values are incorrect, etc. By detecting the output value distribution of each channel, or detecting whether there are abnormal statistical values, gradient values, or comparing whether the final output of the model is abnormal, it can be used to determine whether the channel is abnormal. In one implementation of the embodiment of the present application, the abnormal channel is identified by the L2 norm of the parameter matrix. Generally speaking, the L2 norm is also called the L2 norm, which is the square root of the sum of the squares of all elements in the matrix. It is used to evaluate the size of the matrix. The larger the L2 norm, the larger the overall element values in the matrix. The smaller the L2 norm, the smaller the overall element values in the matrix. One manifestation of an abnormal matrix is that the element values in the matrix are abnormally large, which may be due to various reasons such as data anomalies, numerical overflow, or gradient explosion. The embodiment of the present application innovatively uses the L2 norm to evaluate the abnormality or non-abnormality of the channels in the layer structure. Taking the activation matrix as an example, the input data is selected, and the L2 norm of the activation matrix in the channel dimension is calculated. For each channel, it is sorted from large to small according to the L2 norm, and multiple channels with the highest sorting are selected as abnormal channels, that is, multiple channels with larger L2 norms are selected as abnormal channels. The input data selected here can be part of the sample data selected from the sample data used to train the model, such as sample data selected according to a set ratio (first ratio), and the selected abnormal channels can be the N channels with the highest sorting from large to small or the channels with a set ratio (second ratio) as abnormal channels. In addition to using the L2 norm to define abnormal channels in the activation matrix, other functions can also be selected according to actual needs to define abnormal channels in the activation matrix.

[0033] like Figure 1 A schematic diagram of identifying abnormal channels in an embodiment of the present application is shown. The L2 normal form of the activation matrix Ai in the channel dimension is calculated, and columns 3 and 6 are selected as two abnormal channels according to the L2 normal form.

[0034] According to the identification of abnormal channels, the parameter matrix is divided into an abnormal channel matrix and a non-abnormal channel matrix, that is, the rows or columns corresponding to the abnormal channels in the parameter matrix are clustered, and the rows or columns corresponding to the non-abnormal channels are also clustered, so that it can be directly split into two corresponding matrices. In a specific implementation, the areas where the two channels are clustered can be defined in the parameter matrix, and the positions of the rows or columns corresponding to the two channels are adjusted to the corresponding areas. For example, the left side or upper side of the parameter matrix is defined as the area where the parameters corresponding to the non-abnormal channels are clustered, and the right side or lower side of the parameter matrix is defined as the area where the parameters corresponding to the abnormal channels are clustered. When adjusting the parameter positions, only the parameter positions corresponding to the abnormal channels can be adjusted, or only the parameter positions corresponding to the non-abnormal channels can be adjusted, or both can be adjusted at the same time. In one embodiment, the parameter positions corresponding to the abnormal channels can be adjusted so that the parameters corresponding to the lower abnormal channels and the non-abnormal channels are clustered respectively, and the parameter matrix is divided into an abnormal channel matrix and a non-abnormal channel matrix.

[0035] like Figure 2 A schematic diagram of adjusting the parameter position of the weight parameter matrix in an embodiment of the present application is shown. Figure 1 For example, the left side shows that the abnormal channels are input channels 3 and 6. The parameters corresponding to rows 3 and 6 are adjusted to the positions of the last two rows. The result after adjustment is shown on the right. Weight parameter matrix W i After transposing, we get the matrix W i ', W i 'Can be divided into two parts, the quantifiable part W' i1 That is, the non-abnormal channel matrix and the abnormal channel matrix W′ i2 .

[0036] like Figure 3 A schematic diagram of adjusting the parameters of the activation parameter matrix in an embodiment of the present application is shown. Figure 1 For example, the left side shows that the abnormal channels are channels 3 and 6. The parameters corresponding to columns 3 and 6 are adjusted to the last two columns. The result after adjustment is shown on the right. i After transposing, we get the matrix A i ', A i 'Can be divided into two parts, the quantifiable part A' i1 That is, the non-abnormal channel matrix and the abnormal channel matrix A′ i2 .

[0037] Taking the fusion calculation as the multiplication of two parameter matrices as an example, according to the principle of matrix block multiplication, in order to replace the fusion calculation of the original parameter matrix with the multiplication calculation of the block matrix, the first parameter matrix with the multiplication calculation first is divided into blocks according to the column dimension, and the first parameter matrix is divided into two left and right matrices. The second parameter matrix with the multiplication calculation later is divided into blocks according to the row dimension, and the second parameter matrix is divided into two upper and lower matrices. Therefore, after the matrix multiplication of the score blocks, the abnormal parameter matrices corresponding to the abnormal channels are multiplied, and the non-abnormal parameter matrices corresponding to the non-abnormal channels are multiplied. Taking the multiplication of the weight parameter matrix by the activation parameter matrix as an example, when adjusting the parameter positions corresponding to the abnormal channels, for the weight parameter matrix, the weight parameters corresponding to each abnormal channel in the input channel dimension are sequentially swapped to the target position of the matrix. For the activation parameter matrix, the activation parameters corresponding to each abnormal channel in the channel dimension are sequentially swapped to the target position of the matrix.

[0038] Of course, it is understandable that the first parameter matrix can also be split according to the row dimension, and the second parameter matrix can be split according to the column dimension, and the split matrices can be replaced in the order of the multiplication calculation.

[0039] In the next step, when performing parameter quantization processing on the non-abnormal channel matrix in the parameter matrix, the quantization parameter corresponding to the non-abnormal channel matrix can be first determined, and then the non-abnormal channel matrix can be quantized into an integer matrix according to the quantization parameter. In one implementation of the calculation of the quantization parameter, the non-abnormal channel matrix can be quantized using a block quantization method, where the matrix is divided into multiple small blocks, and each small block is quantized independently. Compared with using the same quantization parameter for the entire matrix, block quantization allows each small block to use a different quantization parameter, thereby improving the accuracy and flexibility of quantization. Specifically, the non-abnormal channel matrix cut out from the parameter matrix can be divided into multiple block matrices according to the set matrix parameters, and then the maximum absolute value of each block matrix is calculated. The quantization parameter for quantizing the block matrix is determined according to the maximum absolute value corresponding to the block matrix, and then block quantization is performed on each block matrix using the quantization parameter corresponding to the block matrix.

[0040] Different types of model parameters can use different block matrix division methods. For the weight parameter matrix, the non-abnormal channel matrix cut out from the weight parameter matrix can be divided into multiple block matrices according to the output channel dimension. For the activation parameter matrix, the non-abnormal channel matrix cut out from the activation parameter matrix can be divided into multiple block matrices according to the token dimension.

[0041] Among them, for the fusion calculation between different parameter matrices, when replacing the fusion calculation with integer matrices and the fusion calculation between floating-point matrices, the calculation of quantization parameters can also be added, that is, specifically replacing the fusion calculation with integer matrices and quantization parameters. Taking the weight parameter matrix and the activation parameter matrix as an example, the quantization parameter of the integer matrix corresponding to the weight parameter matrix after transposition and quantization is recorded as the first quantization parameter (actually the set of quantization parameters corresponding to each integer block matrix), and the quantization parameter of the integer matrix corresponding to the activation parameter matrix after transposition and quantization is recorded as the second quantization parameter. Then the fusion calculation between different parameter matrices can be replaced with two parts, the first part is the fusion calculation of the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the integer matrix of the activation parameter matrix and the corresponding second quantization parameter, and the second part is the fusion calculation between the abnormal channel matrix of the weight parameter matrix and the abnormal channel matrix of the activation parameter matrix.

[0042] Accordingly, during the model operation, the model that has completed the above-mentioned quantization process can be called, and then the integer calculation unit in the computing hardware can be called to process the fusion calculation between the integer matrices corresponding to different parameter matrices. More specifically, the integer calculation unit is called to process the fusion calculation between the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the integer matrix of the activation parameter matrix and the corresponding second quantization parameter. For the abnormal channel matrix, the floating-point calculation unit in the computing hardware can be called to process the fusion calculation between the abnormal channel matrices corresponding to different parameter matrices.

[0043] The quantization of model parameters can be divided into static quantization and dynamic quantization. Static quantization refers to precalculating the quantization parameters before model inference and using the precalculated quantization parameters for quantization during the model inference process. Dynamic quantization refers to dynamically calculating the quantization parameters during the model inference process. The above-mentioned process of quantizing model parameters can be divided into transposition transformation processing and parameter quantization processing. The transposition transformation processing is used to switch the position of the parameters corresponding to the abnormal channel in the parameter matrix to split the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix. The parameter quantization processing is used to perform parameter quantization processing on the non-abnormal channel matrix to obtain the integer type corresponding to the non-abnormal channel matrix. Static quantization processing is used for the weight parameter matrix, that is, the weight parameter matrix can be transposed and quantized before the model is run; dynamic quantization processing is used for the activation parameter matrix, that is, during the model operation phase, the activation parameter matrix is transposed and quantized, and then the fusion calculation between different parameter matrices is replaced by the fusion calculation between the integer matrices corresponding to different parameter matrices and the fusion calculation between the abnormal channel matrices corresponding to different parameter matrices. Of course, other methods can also be used in specific implementations, such as static quantization of the activation parameter matrix, or static quantization of both the activation parameter matrix and the weight parameter matrix. This application does not impose any restrictions on this.

[0044] Combine as follows Figure 2 and Figure 3 The example below illustrates the quantization process after the parameter matrix is transposed:

[0045] The weight parameter matrix is W i , after transposition, it is converted to W′ i , split into the non-abnormal parameter matrix W′ corresponding to the non-abnormal channel i1 And the abnormal parameter matrix W′ corresponding to the abnormal channel i2 Assume W′ i1 It is a floating-point matrix of size N×M. The matrix parameter of the block matrix is set to the block size block-size (for example, 128). W′ is converted to the output channel dimension. i1 Divide into N×M / block-size block matrices, calculate the maximum absolute value max-value of the block matrix, and the first quantization parameter S corresponding to the block matrix of the weight parameter matrix can be calculated according to the following formula 1 w , according to formula 2, the quantized weight parameter can be obtained

[0046] S w =max-value / (2 bits-1 -1), bits=4, formula 1

[0047]

[0048] The activation parameter matrix is A i , after transposition, it is converted to A′ i , split into the non-abnormal parameter matrix A′ corresponding to the non-abnormal channel i1 And the abnormal parameter matrix A′ corresponding to the abnormal channel i2 A′ i1 It is a floating point matrix of size K×M (the activation matrix and the weight matrix have the same dimension 0), and A′ is divided into i1 The quantizable part A′ of the activation parameter matrix after the transposition is divided into K×M / block-size block matrices i1 Divide into blocks according to word unit dimensions and dynamically quantize Calculate the maximum absolute value max-value of the block matrix, and the second quantization parameter S corresponding to the block matrix of the activation parameter matrix can be calculated according to the following formula 3 A , according to formula 4, the quantized activation parameters can be obtained

[0049] S A =max-value / (2 bits-1 -1), bits=4, formula 3

[0050]

[0051] For the weight parameter matrix W i And the activation parameter matrix A i When performing multiplication calculations, according to the invariance principle of matrix permutation, it can be concluded that the multiplication results before and after transposition are the same. i W i =A′ i W′ i , further according to the matrix block multiplication property, we can get A′ i W i ′=A′ i1 W′ i1 +A′ i2 W′ i2 , thereby replacing the fusion calculation corresponding to the weight parameter matrix and the activation parameter matrix with the fusion calculation between the integer matrices corresponding to the weight parameter matrix and the activation parameter matrix, and the fusion calculation between the abnormal channel matrices corresponding to the weight parameter matrix and the activation parameter matrix. Further expansion, replacing the integer matrix of the weight parameter matrix corresponding to the non-abnormal channel and the first quantization parameter, and the integer matrix of the activation parameter matrix corresponding to the non-abnormal channel and the second quantization parameter fusion calculation, that is, Where * represents the matrix dot multiplication operation.

[0052] The execution subject of the embodiment of the present application can be an application, service, instance, functional module in software form, virtual machine (VM), container or cloud server, etc., or a hardware device with data processing function (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU), etc. The device for implementing service provision can be deployed on the computing device of the application party that provides the corresponding service or on a cloud computing platform that provides computing power, storage and network resources. The mode in which the cloud computing platform provides services to the outside world can be IaaS (Infrastructure as a Service), PaaS (Platform as a Service), SaaS (Software as a Service) or DaaS (Data as a Service). Taking the platform providing SaaS software as a service (Software as a Service) as an example, the cloud computing platform can use its own computing resources to provide the execution of processes such as model training and model parameter quantization, and the specific application architecture can be built according to service requirements.

[0053] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0054] The following describes in detail the technical solution of this application and how it solves the aforementioned technical problems using specific embodiments. The specific embodiments listed may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments. The following describes the embodiments of this application in detail with reference to the accompanying drawings.

[0055] The present application embodiment provides a method for quantifying model parameters, such as Figure 4 FIG. 4 is a flow chart of a method 400 for quantifying model parameters according to an embodiment of the present application. The method 400 may include:

[0056] In step S401, an abnormal channel of at least one layer structure in the target model is identified; in step S402, for the parameter matrix corresponding to each layer structure, the position of the parameters corresponding to the abnormal channel in the parameter matrix is switched to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and parameter quantization processing is performed on the non-abnormal channel matrix to obtain an integer matrix corresponding to the non-abnormal channel matrix; in step S403, the fusion calculation between different parameter matrices is replaced by a fusion calculation between integer matrices corresponding to different parameter matrices, and a fusion calculation between abnormal channel matrices corresponding to different parameter matrices.

[0057] According to an embodiment of the present application, when the model parameters are quantized, the abnormal channels corresponding to one or more layer structures included in the target model are identified. For the parameter matrix corresponding to each layer structure, according to the abnormal channels determined previously, the positions of the parameters corresponding to the abnormal channels in the parameter matrix are switched to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, thereby distinguishing the abnormal parameters and non-abnormal parameters in the parameter matrix. When the model parameters are quantized, the non-abnormal channel matrix is parameter quantized to obtain an integer matrix corresponding to the non-abnormal channel matrix. The abnormal channel matrix still corresponds to a floating-point matrix. In the model calculation link, the fusion calculation between different parameter matrices is replaced by the fusion calculation between the integer matrices corresponding to different parameter matrices. Combined calculations and fusion calculations between abnormal channel matrices corresponding to different parameter matrices are performed, so that when the computing unit of the computing device is called to perform model calculations, the integer computing unit in the computing hardware can be used to process the fusion calculations between integer matrices corresponding to different parameter matrices, and the floating-point computing unit can be used to process the fusion calculations between floating-point matrices corresponding to different parameter matrices. By separating the model parameters corresponding to the abnormal channels and the model parameters of the non-abnormal channels for calculation processing, and performing integer calculations on the non-abnormal parameters, the more efficient integer computing units in the computing hardware are fully utilized as much as possible to accelerate reasoning, thereby improving the efficiency of model reasoning, and at the same time avoiding the influence of abnormal parameters on the overall calculation accuracy in the operation of low-bit integer matrices, thereby ensuring the accuracy of the calculation results.

[0058] In one embodiment, the identifying of abnormal channels of at least one layer structure in the target model includes: selecting sample data to input into the target model, obtaining the L2 normal form in the channel dimension for the activation matrix of each layer structure of the target model; sorting from large to small according to the L2 normal form, and selecting multiple channels with the highest order as abnormal channels.

[0059] In one embodiment, the switching of the positions of the parameters corresponding to the abnormal channels in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix includes: identifying the parameter positions corresponding to the abnormal channels in the parameter matrix; adjusting the parameter positions corresponding to the abnormal channels so that the parameters corresponding to the abnormal channels and the non-abnormal channels are respectively clustered, and dividing the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix.

[0060] In one embodiment, when the parameter matrix includes a weight parameter matrix, adjusting the parameter positions corresponding to the abnormal channels includes: sequentially swapping the weight parameters corresponding to each abnormal channel in the input channel dimension to the target position of the matrix; when the parameter matrix includes an activation parameter matrix, adjusting the parameter positions corresponding to the abnormal channels includes: sequentially swapping the activation parameters corresponding to each abnormal channel in the channel dimension to the target position of the matrix.

[0061] In one embodiment, performing parameter quantization processing on the non-abnormal channel matrix to obtain an integer matrix corresponding to the non-abnormal channel matrix includes: determining a quantization parameter corresponding to the non-abnormal channel matrix; and quantizing the non-abnormal channel matrix into an integer matrix according to the quantization parameter.

[0062] In one embodiment, determining the quantization parameter corresponding to the non-abnormal channel matrix includes: dividing the non-abnormal channel matrix cut out from the parameter matrix into multiple block matrices according to the set matrix parameters; calculating and comparing the absolute values corresponding to the multiple block matrices, and determining the quantization parameter corresponding to the non-abnormal channel matrix according to the maximum absolute value corresponding to the multiple block matrices.

[0063] In one embodiment, when the parameter matrix includes a weight parameter matrix, dividing the non-abnormal channel matrix cut out from the parameter matrix into multiple block matrices includes: dividing the non-abnormal channel matrix cut out from the weight parameter matrix into multiple block matrices according to the output channel dimension; when the parameter matrix includes an activation parameter matrix, dividing the non-abnormal channel matrix cut out from the parameter matrix into multiple block matrices includes: dividing the non-abnormal channel matrix cut out from the activation parameter matrix into multiple block matrices according to the word dimension.

[0064] In one embodiment, the fusion calculation between different parameter matrices is replaced by a fusion calculation between integer matrices corresponding to different parameter matrices, and the fusion calculation between floating-point matrices corresponding to different parameter matrices includes: replacing the fusion calculation between the weight parameter matrix and the activation parameter matrix by a fusion calculation between the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the integer matrix of the activation parameter matrix and the corresponding second quantization parameter, and a fusion calculation between the abnormal channel matrix of the weight parameter matrix and the abnormal channel matrix of the activation parameter matrix.

[0065] The present application embodiment provides a model operation method, such as Figure 5 FIG. 5 is a flow chart of a model operation method 500 according to an embodiment of the present application. The method 500 may include:

[0066] In step 501, a target model is obtained, wherein the model parameters of the target model are quantized, and the fusion calculation between different parameter matrices is replaced by the fusion calculation between integer matrices corresponding to different parameter matrices, and the fusion calculation between abnormal channel matrices corresponding to different parameter matrices. The integer matrix corresponding to the non-abnormal channel matrix is obtained by performing parameter quantization processing on the non-abnormal channel matrix, and the non-abnormal channel matrix and the abnormal channel matrix are obtained by switching the positions of the parameters corresponding to the abnormal channels in the parameter matrix. In step 502, the integer calculation unit in the computing hardware is called to process the fusion calculation between the integer matrices corresponding to different parameter matrices.

[0067] In one embodiment, the calling of the integer computing unit in the computing hardware to process the fusion calculation between the integer matrices corresponding to different parameter matrices includes: calling the integer computing unit to process the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the fusion calculation with the integer matrix of the activation parameter matrix and the corresponding second quantization parameter.

[0068] In one embodiment, the method further includes: calling a floating-point computing unit in computing hardware to process fusion calculations between abnormal channel matrices corresponding to different parameter matrices.

[0069] The present application embodiment provides a method for quantifying model parameters, such as Figure 6 FIG. 6 is a flow chart of a method 600 for quantifying model parameters according to an embodiment of the present application. The method 600 may include:

[0070] In step 601, before the model is run, an abnormal channel of at least one layer structure in the target model is identified; in step 602, based on the abnormal channel of the identified layer structure, a transposition transformation process and a parameter quantization process are performed on the weight parameter matrix of the layer structure; in step 603, during the model running stage, based on the abnormal channel of the identified layer structure, a transposition transformation process and a parameter quantization process are performed on the activation parameter matrix of the layer structure, and the fusion calculation between different parameter matrices is replaced by a fusion calculation between integer matrices corresponding to different parameter matrices, and a fusion calculation between abnormal channel matrices corresponding to different parameter matrices; wherein, the transposition transformation process includes switching the position of the parameters corresponding to the abnormal channel in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and the parameter quantization process is used to perform parameter quantization on the non-abnormal channel matrix to obtain the integer type corresponding to the non-abnormal channel matrix.

[0071] Corresponding to the above examples and method embodiments provided in the embodiments of the present application, the embodiments of the present application also provide a device for quantifying model parameters. Figure 7 FIG. 7 is a block diagram of a model parameter quantization device 700 according to an embodiment of the present application. The device 700 may include:

[0072] The channel identification module 701 is used to identify abnormal channels of at least one layer structure in the target model; the matrix segmentation module 702 is used to switch the positions of the parameters corresponding to the abnormal channels in the parameter matrix corresponding to each layer structure to segment the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix; the parameter quantization processing module 703 is used to perform parameter quantization processing on the non-abnormal channel matrix to obtain an integer matrix corresponding to the non-abnormal channel matrix; the matrix fusion module 704 is used to replace the fusion calculation between different parameter matrices with the fusion calculation between the integer matrices corresponding to different parameter matrices, and the fusion calculation between the abnormal channel matrices corresponding to different parameter matrices.

[0073] In one embodiment, the channel identification module is specifically used to select sample data and input it into the target model, and obtain the L2 normal form in the channel dimension for the activation matrix of each layer structure of the target model; sort the L2 normal form from large to small, and select multiple channels with the highest order as abnormal channels.

[0074] In one embodiment, the matrix switching module includes: a parameter position identification submodule for identifying the parameter position corresponding to the abnormal channel in the parameter matrix; a parameter position adjustment submodule for adjusting the parameter position corresponding to the abnormal channel so that the parameters corresponding to the abnormal channel and the non-abnormal channel are respectively aggregated, and the parameter matrix is divided into an abnormal channel matrix and a non-abnormal channel matrix.

[0075] In one embodiment, when the parameter matrix includes a weight parameter matrix, the parameter position adjustment submodule is specifically used to sequentially exchange the weight parameters corresponding to each abnormal channel in the input channel dimension to the target position of the matrix; when the parameter matrix includes an activation parameter matrix, the parameter position adjustment submodule is specifically used to sequentially exchange the activation parameters corresponding to each abnormal channel in the channel dimension to the target position of the matrix.

[0076] In one embodiment, the parameter quantization processing module includes: a quantization parameter determination submodule, configured to determine a quantization parameter corresponding to the non-abnormal channel matrix; and a matrix quantization submodule, configured to quantize the non-abnormal channel matrix into an integer matrix according to the quantization parameter.

[0077] In one embodiment, the quantization parameter determination submodule is specifically used to divide the non-abnormal channel matrix cut out from the parameter matrix into multiple block matrices according to the set matrix parameters; the matrix quantization submodule is specifically used to quantize the corresponding block matrix according to the corresponding quantization parameter.

[0078] In one embodiment, when the parameter matrix includes a weight parameter matrix, the matrix division subunit is specifically used to divide the non-abnormal channel matrix cut out from the weight parameter matrix into multiple block matrices according to the output channel dimension; when the parameter matrix includes an activation parameter matrix, the matrix division subunit is specifically used to divide the non-abnormal channel matrix cut out from the activation parameter matrix into multiple block matrices according to the word dimension.

[0079] In one embodiment, the matrix fusion module is specifically used to replace the fusion calculation between the weight parameter matrix and the activation parameter matrix with the fusion calculation of the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the integer matrix of the activation parameter matrix and the corresponding second quantization parameter, as well as the fusion calculation between the abnormal channel matrix of the weight parameter matrix and the abnormal channel matrix of the activation parameter matrix.

[0080] According to an embodiment of the present application, when the model parameters are quantized, the abnormal channels corresponding to one or more layer structures included in the target model are identified. For the parameter matrix corresponding to each layer structure, according to the abnormal channels determined previously, the position of the parameters corresponding to the abnormal channels in the parameter matrix is switched to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, thereby distinguishing the abnormal parameters and the non-abnormal parameters in the parameter matrix. When the model parameters are quantized, the non-abnormal channel matrix is parameter quantized to obtain an integer matrix corresponding to the non-abnormal channel matrix. The abnormal channel matrix still corresponds to a floating-point matrix. In the model calculation link, the fusion calculation between different parameter matrices is replaced by the fusion calculation between the integer matrices corresponding to different parameter matrices. Fusion calculation, fusion calculation between abnormal channel matrices corresponding to different parameter matrices, so that when the computing unit of the computing device is called to perform model calculation, the integer computing unit in the computing hardware can be used to process the fusion calculation between integer matrices corresponding to different parameter matrices, and the floating-point computing unit can be used to process the fusion calculation between floating-point matrices corresponding to different parameter matrices. By separating the model parameters corresponding to the abnormal channel and the model parameters of the non-abnormal channel for calculation processing, and performing integer calculation on the non-abnormal parameters, the more efficient integer computing units in the computing hardware are fully utilized as much as possible to accelerate reasoning, improve the efficiency of model reasoning, and at the same time avoid the influence of abnormal parameters on the overall calculation accuracy in the operation of low-bit integer matrices, thereby ensuring the accuracy of the calculation results.

[0081] Corresponding to the above examples and method embodiments provided in the embodiments of the present application, the embodiments of the present application also provide a model operation device. Figure 8 FIG. 8 is a block diagram of a model operation device 800 according to an embodiment of the present application. The device 800 may include:

[0082] The module acquisition module 801 is used to obtain the target model. The model parameters of the target model are quantized, and the fusion calculation between different parameter matrices has been replaced by the fusion calculation between the integer matrices corresponding to the different parameter matrices, and the fusion calculation between the abnormal channel matrices corresponding to the different parameter matrices. The integer matrix corresponding to the non-abnormal channel matrix is obtained by performing parameter quantization processing on the non-abnormal channel matrix, and the non-abnormal channel matrix and the abnormal channel matrix are obtained by switching the positions of the parameters corresponding to the abnormal channel in the parameter matrix; the first calculation module 802 is used to call the integer calculation unit in the computing hardware to process the fusion calculation between the integer matrices corresponding to different parameter matrices.

[0083] In one embodiment, the first calculation module is specifically used to call the integer calculation unit to process the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the fusion calculation of the integer matrix of the activation parameter matrix and the corresponding second quantization parameter.

[0084] In one embodiment, the apparatus further includes a second computing module configured to call a floating-point computing unit in computing hardware to process fusion calculations between abnormal channel matrices corresponding to different parameter matrices.

[0085] Corresponding to the above examples and method embodiments provided in the embodiments of the present application, the embodiments of the present application also provide a device for quantifying model parameters. Figure 9 FIG. 1 is a block diagram of a model parameter quantization device 900 according to an embodiment of the present application. The device 900 may include:

[0086] The abnormal channel identification module 901 is used to identify the abnormal channels of at least one layer structure in the target model before the model is run; the weight parameter matrix processing module 902 is used to perform transposition transformation and parameter quantization on the weight parameter matrix of the layer structure according to the abnormal channels of the identified layer structure; the activation parameter matrix processing module 903 is used to perform transposition transformation and parameter quantization on the activation parameter matrix of the layer structure according to the abnormal channels of the identified layer structure during the model operation stage, and replace the fusion calculation between different parameter matrices with the fusion calculation between the integer matrices corresponding to different parameter matrices and the fusion calculation between the abnormal channel matrices corresponding to different parameter matrices; wherein the transposition transformation processing includes switching the position of the parameters corresponding to the abnormal channels in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and the parameter quantization processing is used to perform parameter quantization on the non-abnormal channel matrix to obtain the integer type corresponding to the non-abnormal channel matrix.

[0087] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0088] Figure 10 FIG. 1 is a block diagram of a computing device used to implement an embodiment of the present application. Figure 10 As shown, the computing device includes: a memory 1001 and a processor 1002. The memory 1001 stores a computer program that can be executed on the processor 1002. When the processor 1002 executes the computer program, the method in the above embodiment is implemented. The number of the memory 1001 and the processor 1002 can be one or more.

[0089] The computing device also includes:

[0090] The communication interface 1003 is used to communicate with external devices and perform data exchange transmission.

[0091] If the memory 1001, processor 1002, and communication interface 1003 are implemented independently, the memory 1001, processor 1002, and communication interface 1003 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 10 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0092] Optionally, in a specific implementation, if the memory 1001, the processor 1002 and the communication interface 1003 are integrated on a chip, the memory 1001, the processor 1002 and the communication interface 1003 can communicate with each other through an internal interface.

[0093] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0094] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the method provided in any embodiment of the present application.

[0095] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided in the embodiment of the present application.

[0096] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0097] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0098] Furthermore, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct memory bus random access memory (DR RAM).

[0099] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the present application are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0100] In the description of this specification, the reference terms "one embodiment," "some embodiments," "example," "specific example," or "some examples" mean that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials, or characteristics described may be combined in any appropriate manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification, as well as features of different embodiments or examples, unless they are mutually inconsistent.

[0101] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0102] Any process or method described in the flowchart or otherwise described herein can be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process. The scope of the preferred embodiments of the present application includes other implementations in which the functions may be performed in a different order than shown or discussed, including performing the functions substantially simultaneously or in reverse order depending on the functions involved.

[0103] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with such instruction execution systems, apparatuses or devices.

[0104] It should be understood that various parts of the present application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0105] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the aforementioned integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. The storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0106] The above is merely an exemplary embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various modifications or substitutions within the technical scope described in this application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for quantifying model parameters, comprising: identifying an abnormal channel of at least one layer structure in the target model; For the parameter matrix corresponding to each layer structure, the position of the parameters corresponding to the abnormal channel in the parameter matrix is switched to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and the parameters of the non-abnormal channel matrix are quantized to obtain an integer matrix corresponding to the non-abnormal channel matrix; The fusion calculation between different parameter matrices is replaced by the fusion calculation between integer matrices corresponding to different parameter matrices, and the fusion calculation between abnormal channel matrices corresponding to different parameter matrices.

2. The display method according to claim 1, wherein: The identifying of an abnormal channel of at least one layer structure in the target model comprises: Select sample data and input it into the target model, and obtain the L2 normal form of the activation matrix of each layer structure of the target model in the channel dimension; Sort the channels from large to small according to the L2 paradigm, and select the top channels as abnormal channels.

3. The display method according to claim 1, wherein: The position of the parameter corresponding to the switching abnormal channel in the parameter matrix divides the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, including: Identify the parameter positions corresponding to the abnormal channels in the parameter matrix; The parameter positions corresponding to the abnormal channels are adjusted so that the parameters corresponding to the abnormal channels and non-abnormal channels are respectively aggregated, and the parameter matrix is divided into an abnormal channel matrix and a non-abnormal channel matrix.

4. The display method according to claim 3, wherein: When the parameter matrix includes a weight parameter matrix, adjusting the parameter position corresponding to the abnormal channel includes: Sequentially swap the weight parameters corresponding to each abnormal channel in the input channel dimension to the target position of the matrix; When the parameter matrix includes an activation parameter matrix, adjusting the parameter position corresponding to the abnormal channel includes: Sequentially swap the activation parameters corresponding to each abnormal channel in the channel dimension to the target position of the matrix.

5. The display method according to claim 1, wherein: The performing parameter quantization processing on the non-abnormal channel matrix to obtain an integer matrix corresponding to the non-abnormal channel matrix includes: Determining a quantization parameter corresponding to the non-abnormal channel matrix; The non-anomalous channel matrix is quantized into an integer matrix according to the quantization parameter. The display method according to claim 5 , wherein: Determining the quantization parameter corresponding to the non-exceptional channel matrix includes: Dividing the non-abnormal channel matrix cut out from the parameter matrix into a plurality of block matrices according to the set matrix parameters; Calculating the maximum absolute value of each block matrix, and determining a quantization parameter for quantizing the block matrix according to the maximum absolute value corresponding to the block matrix; Quantizing the non-abnormal channel matrix into an integer matrix according to the quantization parameter includes: The corresponding block matrix is quantized according to the corresponding quantization parameter.

7. The display method according to claim 6, wherein: When the parameter matrix includes a weight parameter matrix, dividing the non-abnormal channel matrix cut out from the parameter matrix into a plurality of block matrices includes: Dividing the non-abnormal channel matrix cut out from the weight parameter matrix into a plurality of block matrices according to the output channel dimension; When the parameter matrix includes an activation parameter matrix, dividing the non-abnormal channel matrix cut out from the parameter matrix into a plurality of block matrices includes: The non-abnormal channel matrix cut out from the activation parameter matrix is divided into a plurality of block matrices according to word unit dimensions.

8. The display method according to claim 1, wherein: The fusion calculation between different parameter matrices is replaced by the fusion calculation between integer matrices corresponding to different parameter matrices, and the fusion calculation between abnormal channel matrices corresponding to different parameter matrices includes: The fusion calculation between the weight parameter matrix and the activation parameter matrix is replaced by a fusion calculation of the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the integer matrix of the activation parameter matrix and the corresponding second quantization parameter, as well as a fusion calculation between the abnormal channel matrix of the weight parameter matrix and the abnormal channel matrix of the activation parameter matrix.

9. A model operation method comprising: Obtaining a target model, wherein model parameters of the target model are quantized, and fusion calculations between different parameter matrices are replaced by fusion calculations between integer matrices corresponding to different parameter matrices, and fusion calculations between abnormal channel matrices corresponding to different parameter matrices. The integer matrices corresponding to the non-abnormal channel matrices are obtained by performing parameter quantization on the non-abnormal channel matrices, and the non-abnormal channel matrices and the abnormal channel matrices are obtained by switching the positions of the parameters corresponding to the abnormal channels in the parameter matrices. The integer computing unit in the computing hardware is called to process the fusion calculation between the integer matrices corresponding to different parameter matrices.

10. The display method according to claim 9, wherein: The calling of the integer computing unit in the computing hardware to process the fusion calculation between the integer matrices corresponding to different parameter matrices includes: The integer calculation unit is called to process the integer matrix of the weight parameter matrix and the corresponding first quantization parameter, and the fusion calculation of the integer matrix of the activation parameter matrix and the corresponding second quantization parameter.

11. The display method according to claim 9, wherein: The method further comprises: The floating-point computing unit in the computing hardware is called to process the fusion calculation between the abnormal channel matrices corresponding to different parameter matrices.

12. A method for quantifying model parameters, comprising: Before the model is run, an abnormal channel of at least one layer structure in the target model is identified; According to the identified abnormal channels of the layer structure, a transposition transformation process and a parameter quantization process are performed on the weight parameter matrix of the layer structure; During the model operation phase, according to the identified abnormal channels of the layer structure, the activation parameter matrix of the layer structure is subjected to transposition transformation and parameter quantization, and the fusion calculation between different parameter matrices is replaced by the fusion calculation between the integer matrices corresponding to the different parameter matrices, and the fusion calculation between the abnormal channel matrices corresponding to the different parameter matrices; Among them, the transposition transformation processing includes switching the positions of the parameters corresponding to the abnormal channels in the parameter matrix to divide the parameter matrix into an abnormal channel matrix and a non-abnormal channel matrix, and the parameter quantization processing is used to perform parameter quantization processing on the non-abnormal channel matrix to obtain the integer type corresponding to the non-abnormal channel matrix.

13. A computing device comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

15. A computer program product, wherein The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images