Network model quantification method and device, storage medium and program product

Abstract

The invention discloses a network model quantification method and device, a storage medium and a program product, and relates to the technical field of data processing, and the method comprises the steps: carrying out the partitioning of a to-be-quantized matrix in a to-be-quantized module, carrying out the subdivision, obtaining a partitioning matrix, calculating the partitioning importance and a quantification threshold value corresponding to each partitioning matrix, and carrying out the segmentation of the to-be-quantized matrix in the to-be-quantized module; the block importance reflects the importance of the respective corresponding block matrix in the corresponding to-be-quantized module, and the quantization threshold is calculated based on the block importance, so that different to-be-quantized modules correspond to different quantization thresholds, the flexibility of the quantization threshold is further improved, and different points of the to-be-quantized modules are considered in the application; according to the method, each block matrix is quantized on the basis of each block importance and the quantization threshold, so that each block matrix is quantized from the block matrix instead of taking the matrix to be quantized as a unit, the quantization is more refined during quantization, and the obtained quantized model is more accurate.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a network model quantization method, device, storage medium and program product. Background Art

[0002] With the rise of artificial intelligence, it has become a trend to install network models in edge devices and use edge devices to calculate network models. However, edge devices are not sufficient in terms of capacity and computing power, which requires quantification of network models.

[0003] In the related art, the model parameters in the module that needs to be quantized in the network model can be converted from floating point to fixed point, and after the conversion, the model parameters can be represented by a fixed point with a smaller number of bits. However, this type of model quantization method lacks flexibility, resulting in poor performance of the quantized model generated based on it and low accuracy when applied. Summary of the invention

[0004] The present application provides a network model quantization method, device, storage medium and program product to at least solve the problem that the quantization model parameters in the related art lack flexibility, thereby resulting in poor performance and accuracy of the quantized model.

[0005] In a first aspect, the present application provides a network model quantization method, comprising:

[0006] Divide the matrix to be quantized contained in the module to be quantized into blocks to obtain a plurality of block matrices; the module to be quantized is derived from the model to be quantized; there is at least one module to be quantized;

[0007] For each module to be quantized, the importance of each block matrix corresponding to each block matrix is ​​calculated based on a training data set, and the quantization threshold corresponding to the module to be quantized is calculated based on each block importance, and the training data set includes at least one training data;

[0008] For each module to be quantized, quantize each block matrix based on the importance of each block and the quantization threshold to obtain a block quantization matrix corresponding to each block matrix;

[0009] Generate a quantization weight matrix after quantization of the corresponding module to be quantized based on each block quantization matrix;

[0010] For each module to be quantized, the matrix to be quantized in the module to be quantized is replaced with the corresponding quantization weight matrix to obtain a quantized module corresponding to each module to be quantized, and a quantized model is generated based on each quantized module.

[0011] In a second aspect, the present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the steps of any of the above-mentioned network model quantization methods when executing the computer program.

[0012] In a third aspect, the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein when the computer program is executed by a processor, the steps of any of the above-mentioned network model quantization methods are implemented.

[0013] In a fourth aspect, the present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned network model quantization methods when executed by a processor.

[0014] The present application provides a network model quantization method, device, storage medium and program product. In the present application, for each module to be quantized, the matrix to be quantized in the module to be quantized is divided into blocks to obtain a block matrix, and the block importance and quantization threshold corresponding to each block matrix are calculated. In the present application, the block importance reflects the importance of each corresponding block matrix in the corresponding module to be quantized. In addition, the quantization threshold is calculated based on the block importance, so the quantization threshold corresponding to different modules to be quantized is different, thereby improving the flexibility of the quantization threshold, and different quantization thresholds reflect the characteristics of each module to be quantized, so the present application takes into account the differences between the modules to be quantized and is more flexible; in the present application, , each block matrix is ​​quantized based on the importance of each block and the quantization threshold. Therefore, when quantizing each block matrix, the quantization is done on the block matrix, rather than quantizing on the basis of the matrix to be quantized. Therefore, the present application is more detailed during quantization, so that the obtained block quantization matrix is ​​more accurate. Then, the quantization weight matrix of the corresponding module to be quantized is generated based on each block quantization matrix, so that the quantization weight matrix is ​​more accurate and more in line with the module to be quantized itself. Therefore, in the present application, the matrix to be quantized in the module to be quantized is replaced with the corresponding quantization weight matrix, so that the obtained quantized module is more accurate and more in line with its own characteristics. Therefore, the post-quantization model in the present application is used to make the result more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0016] Figure 1 A schematic diagram of an application scenario provided for this application;

[0017] Figure 2 A schematic diagram of a network model quantization method flow chart provided in Example 1;

[0018] Figure 3 A schematic diagram of a network model quantization method flow chart provided in Example 6;

[0019] Figure 4 A schematic diagram of a flow chart of a network model quantization method provided in Example 7;

[0020] Figure 5 A schematic diagram of a flow chart of a network model quantization method provided in Example 8;

[0021] Figure 6 A schematic diagram of network model quantization and application provided in Example 12;

[0022] Figure 7 A schematic diagram of the structure of a network model quantization device provided in Example 13;

[0023] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0026] In the related art, the model parameters in the module that needs to be quantized in the network model can be converted from floating point to fixed point, and after the conversion, the model parameters with a smaller number of bits can be obtained. However, this method lacks flexibility and is inconsistent with the characteristics of the module that needs to be quantized, resulting in poor performance of the quantized model generated based on this method and low accuracy when applied.

[0027] In order to solve the defects in the related art, the inventor of the present application has proposed a network model quantization method after a series of improvements. In order to solve the problem of lack of flexibility in model quantization in the related art, resulting in poor performance and low accuracy of the quantized model, in this scheme, for each module to be quantized, the matrix to be quantized contained in the module to be quantized is first divided into blocks, so that subdivision is achieved and each block matrix is ​​obtained. Further, the block importance corresponding to each block matrix is ​​calculated, and then the quantization threshold corresponding to each module to be quantized is calculated. The quantization threshold in this scheme is generated based on different modules to be quantized, so different modules to be quantized can have their own corresponding quantization thresholds, so that the quantization threshold is more in line with the corresponding module to be quantized, so for each module to be quantized, the block matrix is ​​quantized based on the corresponding block importance and quantization threshold, so the flexibility of quantization is improved; in addition, in this scheme, quantization is performed in units of block matrices, rather than in units of matrices to be quantized, so the quantization method of this scheme is more specific, so the final quantization weight matrix is ​​more accurate, and thus the quantized module and the quantized model are also more accurate.

[0028] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0029] In conjunction with the specific application environment architecture or specific hardware architecture on which the execution of the network model quantization method depends, the specific application environment architecture or specific hardware architecture is described herein.

[0030] Figure 1 A schematic diagram of an application scenario provided by this application. Figure 1 As shown, it includes: an electronic device 101.

[0031] The electronic device 101 may be a computer or a server, which is not limited here. It should be noted that the electronic device 101 may be an edge device.

[0032] Specifically, the user inputs a query text based on the electronic device 101 , and the electronic device 101 inputs the query text into the text encoding module 102 , which then outputs text features.

[0033] Furthermore, the electronic device 101 searches the image feature database based on the text feature to obtain the corresponding target image.

[0034] In this application scenario, the text encoding module 102 includes at least one post-quantization module in this application.

[0035] Embodiment 1

[0036] The execution subject of Embodiments 1 to 12 of the present application is a network model quantization device, which is located in an electronic device.

[0037] Figure 2 A schematic diagram of a network model quantization method provided in Example 1. Figure 2 As shown, including:

[0038] S201, dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain a plurality of block matrices; the module to be quantized is derived from the model to be quantized; there is at least one module to be quantized.

[0039] Among them, the matrix to be quantized is express, ,Right now yes OK, Matrix of columns.

[0040] in, Indicates A block matrix, , that is OK, A matrix of columns, where , ≥1.

[0041] For example, it is assumed that for module 1 to be quantized, the matrix to be quantized is ,right Block, get , which means that three block matrices are obtained.

[0042] In this embodiment, the module to be quantized is derived from the model to be quantized, and the module to be quantized in the model to be quantized is quantized to obtain a quantized model. Specifically, the matrix to be quantized in the module to be quantized is divided into blocks to obtain a block matrix, and then the block matrix is ​​quantized to obtain a quantized block matrix, and then the quantization weight matrix corresponding to the matrix to be quantized is obtained, so that the quantization weight matrix replaces the corresponding matrix to be quantized, and then the module to be quantized is quantized into a quantized module.

[0043] The quantized model refers to a model generated after quantizing the modules to be quantized included therein. The quantized model is a low-bit model that can adapt to devices with limited resources.

[0044] The post-quantization module refers to a module generated after the module to be quantized is quantized.

[0045] The module to be quantized refers to a module in which the weight matrix included therein needs to be quantized.

[0046] The matrix to be quantized refers to a weight matrix that needs to be quantized.

[0047] The quantization weight matrix is ​​obtained after the matrix to be quantized is quantized, and is generated by multiple block quantization matrices.

[0048] The block quantization matrix is ​​a matrix obtained by quantizing the block matrix.

[0049] The block matrix is ​​a matrix obtained by dividing the matrix to be quantized into blocks. It should be noted that a matrix to be quantized includes multiple block matrices.

[0050] It should be noted that a model to be quantized includes at least one module to be quantized and at least one non-quantized module, wherein the non-quantized module refers to a module that does not need to be quantized.

[0051] It should be noted that there is at least one module to be quantified, and the module to be quantified may include at least one type of module, for example, a query embedding generation module, a key embedding generation module, a value embedding generation module, and at least one type of module in a feedforward neural network layer.

[0052] It should be noted that, in this embodiment, the type of the module to be quantized remains unchanged during the quantization process.

[0053] S202, for each module to be quantized, calculating the block importance corresponding to each block matrix based on a training data set, and calculating the quantization threshold corresponding to the module to be quantized based on each block importance, the training data set includes at least one training data.

[0054] The block importance refers to the importance corresponding to the block matrix. It should be noted that one block matrix corresponds to one block importance.

[0055] The quantization threshold refers to the threshold corresponding to the module to be quantized. It should be noted that one module to be quantized corresponds to one quantization threshold. The training data set refers to the data set used when quantizing the module to be quantized.

[0056] In one method, the training data set is input into the model to be quantized, and the intermediate data is used to calculate the block importance corresponding to each block matrix.

[0057] The training data may be text data, and the language type of the text data is not limited.

[0058] It should be noted that one module to be quantized corresponds to one matrix to be quantized, and one matrix to be quantized can be divided into multiple block matrices, each block matrix corresponds to its own block importance, and one module to be quantized corresponds to one quantization threshold, so that different modules to be quantized may correspond to different quantization thresholds. Therefore, for different modules to be quantized, due to the increased flexibility of the quantization threshold, each module to be quantized can quantize its own quantization weight matrix under more appropriate circumstances.

[0059] It should be noted that each block matrix has its own corresponding block importance, which further reflects the importance of each block matrix to the quantization module, that is, reflects its influence on the performance of the quantization module.

[0060] S203 , for each module to be quantized, quantize each block matrix based on the importance of each block and the quantization threshold, so as to obtain a block quantization matrix corresponding to each block matrix.

[0061] The quantization threshold is express.

[0062] For example, assume that there are three block matrices, namely , as well as , the corresponding block importance is , as well as . Indicates The importance of the blocks corresponding to the block matrices.

[0063] For example, based on as well as right Quantify to get The corresponding block quantization matrix is ​​expressed as , indicating the The block quantization matrix corresponding to the block matrix is ​​obtained. For the remaining block matrices, the corresponding block quantization matrix is ​​obtained in the same way.

[0064] S204, generating a quantization weight matrix after quantization of the corresponding module to be quantized based on each block quantization matrix.

[0065] Among them, the quantization weight matrix is ​​used express.

[0066] Exemplarily, according to the above steps, three block quantization matrices are finally obtained, which are , as well as .

[0067] Furthermore, the three block quantization matrices are combined in rows and columns to generate a quantization weight matrix corresponding to the module to be quantized 1, that is, .

[0068] It should be noted that, according to the above exemplary embodiment, the corresponding quantization weight matrices are obtained for the remaining modules to be quantized in the same manner, which will not be elaborated here.

[0069] S205 , for each module to be quantized, replacing the matrix to be quantized in the module to be quantized with the corresponding quantization weight matrix to obtain a quantized module corresponding to each module to be quantized, and generating a quantized model based on each quantized module.

[0070] According to the above exemplary embodiment, the matrix to be quantized in the module to be quantized 1 Replace with , and then obtain the quantized module 1.

[0071] Furthermore, in the same manner as above, the matrices to be quantized in the remaining modules to be quantized are replaced with corresponding quantization weight matrices, thereby obtaining the remaining quantized modules.

[0072] Furthermore, a model composed of various quantized modules is a quantized model.

[0073] The present embodiment provides a network model quantization method. In the present embodiment, for each module to be quantized, the matrix to be quantized in the module to be quantized is divided into blocks to obtain a block matrix, and the block importance and quantization threshold corresponding to each block matrix are calculated. In the present embodiment, the block importance reflects the importance of each corresponding block matrix in the corresponding module to be quantized. In addition, the quantization threshold is calculated based on the block importance, so the quantization threshold corresponding to different modules to be quantized is different, thereby improving the flexibility of the quantization threshold, and different quantization thresholds reflect the differences between the modules to be quantized, so the present embodiment takes into account the differences between the modules to be quantized, which is more flexible; quantization is performed on each block matrix, because the quantization is performed on the block matrix instead of the matrix to be quantized as a unit, so the present embodiment is more refined, so that the obtained block quantization matrix is ​​more accurate and more in line with the characteristics of the corresponding module to be quantized.

[0074] Embodiment 2

[0075] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the importance of each block matrix based on the training data set, specifically including:

[0076] Step 1: Obtain the input data corresponding to each module to be quantized based on the training data set.

[0077] Specifically, the training data set is input into the model to be quantized to obtain input data.

[0078] Step 2: Calculate the block importance corresponding to each block matrix based on the input data.

[0079] This embodiment provides a network model quantization method. In this embodiment, the input data corresponding to each module to be quantized can be obtained based on the training data set. It can be seen that the input data is different for different modules to be quantized, so the input data is more in line with the respective data to be quantized, and ultimately the obtained block importance is more accurate.

[0080] Embodiment 3

[0081] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional way to obtain input data corresponding to each module to be quantized based on a training data set, specifically including:

[0082] Step 1: Input the training data set into the model to be quantized, and propagate forward through the neural network to obtain the intermediate data corresponding to each module to be quantized; the amount of intermediate data is equal to the amount of training data.

[0083] The intermediate data refers to the data generated by the training data set in the module to be quantized.

[0084] Exemplarily, if the model to be quantified includes multiple network layers, each network layer includes at least one module to be quantified, and the training data set is input into the model to be quantified, propagates forward through the neural network, propagates to each network layer, and passes through the module to be quantified in each network layer, and then outputs a set of data from the module to be quantified, which is the intermediate data. Exemplarily, the first network layer includes a module to be quantified A, then after the training data set is input into the first network layer, it passes through the module to be quantified A, and outputs a set of data A, which is the intermediate data of the module to be quantified A, and then the data continues to propagate, and is output from the first network layer to the second network layer, and the second network layer includes a module to be quantified B, and the data is transmitted to the module to be quantified B, and a set of data B is output, which is the intermediate data of the module to be quantified B.

[0085] It should be noted that if the training data set includes 512 training data, the number of intermediate data is 512.

[0086] Step 2: Construct the input data corresponding to each module to be quantized based on each intermediate data.

[0087] Specifically, the embedding dimension of the model to be quantized is determined, and the embedding dimension is determined as the number of rows of the input data. For example, if the embedding dimension is 5, the input data is 5 rows.

[0088] Further, the number of intermediate data is determined as the number of columns of input data. For example, if the number of intermediate data is 512, then the number of columns of input data is 512.

[0089] Specifically, assume that the embedding dimension of the model to be quantized is , the number of intermediate data is , then the input data of the module to be quantified is constructed based on the intermediate data, which is , expressed as OK, Matrix of columns.

[0090] This embodiment provides a network model quantization method. In this embodiment, corresponding intermediate data are obtained for different modules to be quantized, thereby constructing corresponding input data, and then quantization is performed based on the appropriate input data, so that the quantization result is more accurate.

[0091] Embodiment 4

[0092] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, which specifically includes:

[0093] If the block division method is free block division, a row of elements in the matrix to be quantized is determined as a block matrix to obtain multiple block matrices; the number of rows of the matrix to be quantized is equal to the number of block matrices.

[0094] Exemplarily, for the module to be quantized 1, the matrix to be quantized is , , that is, the total When free block division is performed, a row of elements is determined as a block matrix. Blocked into A block matrix.

[0095] This embodiment provides a network model quantization method. In this embodiment, a block matrix corresponding to a matrix to be quantized can be quickly obtained according to free block division.

[0096] Embodiment 5

[0097] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, which specifically includes:

[0098] Step 1: If the block division method is regular block division, obtain the preset number of blocks.

[0099] The preset number of blocks is the preset number of block matrices to be obtained.

[0100] Step 2: obtain multiple block matrices based on the number of rows of the matrix to be quantized and the preset number of blocks; the number of rows of the other block matrices except the last block matrix is ​​the first number of rows, and the number of rows of the last block matrix is ​​the second number of rows; the first number of rows is the integer of the ratio of the number of rows of the matrix to be quantized to the preset number of blocks; the second number of rows is less than or equal to the first number of rows.

[0101] Specifically, determine the number of rows of the matrix to be quantized. According to the above description, it is assumed that The preset number of blocks is .

[0102] Further, the ratio of the number of rows of the matrix to be quantized to the number of preset blocks is calculated and rounded up to an integer, which is expressed as ,in, Indicates rounding up to an integer.

[0103] like is an integer, then the number of rows of each block matrix is ​​the same, that is, the second row number is equal to the first row number. For example, assume is 8, is 4, then the first row number is =2, that is, the number of rows of each block matrix is ​​2. Specifically, according to the order of row elements in the matrix to be quantized, every two rows can be determined as a block matrix to obtain multiple block matrices.

[0104] like If is not an integer, the other block matrices except the last block matrix are the first row numbers. is 7, is 4, then the first row number is =2, that is, the number of rows of all block matrices except the last block matrix is ​​2, and the number of rows of the last block matrix is , the second row number is calculated to be 1, that is, the second row number of the last block matrix is ​​1.

[0105] This embodiment provides a network model quantization method. In this embodiment, the number of rows included in each block matrix is ​​determined in a targeted manner according to a preset number of blocks.

[0106] Embodiment 6

[0107] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the importance of the blocks corresponding to each block matrix based on the input data.

[0108] Figure 3 A schematic diagram of a network model quantization method provided in Example 6. Figure 3 As shown, including:

[0109] S301, inputting input data into a preset second-order derivative algorithm to output a second-order matrix.

[0110] The preset second-order derivative algorithm is a preset second-order algorithm, as shown in (1):

[0111] (1)

[0112] in, Represents input data, represents the transpose of the input data, where represents a second-order matrix.

[0113] S302, for each block matrix, calculate the average importance corresponding to each row element of the block matrix based on the second-order matrix; the average importance is the average value of the importance of the corresponding row elements in the block matrix.

[0114] It should be noted that each row in the block matrix has its own elements, and the average importance is the average of the importance of the corresponding row elements in the block matrix. For example, assuming that there are two rows in the block matrix, the average importance corresponding to each row element is calculated based on the second-order matrix, that is, two average importances are obtained, which respectively represent the average of the importance of the corresponding row elements.

[0115] S303: For each block matrix, the block importance corresponding to the block matrix is ​​calculated based on the corresponding average importance.

[0116] This embodiment provides a network model quantization method. In this embodiment, the average importance corresponding to each block matrix is ​​calculated. Since the average importance represents the average value of its corresponding row elements, the average importance can reflect the importance of each row element.

[0117] Embodiment 7

[0118] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the average importance corresponding to each row element of the block matrix based on the second-order matrix.

[0119] Figure 4 A schematic diagram of a network model quantization method provided in Example 7. Figure 4 As shown, including:

[0120] S401, inputting the inverse matrix of the second-order matrix into a preset element importance algorithm to output the element importance corresponding to each element in the block matrix.

[0121] The preset element importance algorithm is a preset algorithm for the importance of each element, as shown in (2):

[0122] (2)

[0123] in, Represents a second-order matrix The inverse matrix of Represents the elements on the diagonal, representing sensitivity; Indicates that in the block matrix Line Elements of a column; Indicates absolute value; Indicates that in the block matrix Line The element importance of the column elements.

[0124] S402, determining the number of elements included in each row of the block matrix.

[0125] S403, calculating the average importance corresponding to each row element of the block matrix based on the number of elements, the importance of each element and a preset average importance algorithm.

[0126] The preset average importance algorithm refers to a preset algorithm for the average importance of elements in each row, as shown in (3):

[0127] (3)

[0128] Among them, in this formula, represents the number of elements in each row of the block matrix (i.e., the number of columns in the block matrix). It should be noted that the number of columns in the block matrix is ​​consistent with the number of elements; Indicates the first The average importance of row elements, where ≥1.

[0129] This embodiment provides a network model quantization method. In this embodiment, the average importance corresponding to the block matrix averages the importance of elements included in each row to obtain the average importance to represent the overall importance.

[0130] Embodiment 8

[0131] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the block importance corresponding to the block matrix based on the corresponding average importance.

[0132] Figure 5 A schematic diagram of a network model quantization method provided in Example 8. Figure 5 As shown, including:

[0133] S501, calculating the average value of each row element in the block matrix.

[0134] In one embodiment, a preset average value algorithm is used to calculate the average value, wherein the preset average value algorithm is shown in formula (4):

[0135] (4)

[0136] in, Indicates the first The average value of the row elements.

[0137] Specifically, according to the number of elements included in each row of elements in the block matrix, the average value of each row of elements is calculated.

[0138] S502, calculating the variance importance corresponding to each row element in the block matrix based on the average value.

[0139] In one approach, a preset variance importance algorithm is used to calculate the variance importance, wherein the preset variance importance algorithm is shown in (5):

[0140] (5)

[0141] in, Indicates the first Variance importance of row elements.

[0142] Specifically, the Each element in the row element and Input into (5), and output Variance importance of row elements.

[0143] S503, inputting the importance of each variance and the average importance into a preset comprehensive importance algorithm to output the comprehensive importance corresponding to each row element in the block matrix.

[0144] The preset comprehensive importance algorithm is a preset algorithm for calculating the comprehensive importance corresponding to each row of elements.

[0145] In one approach, the preset comprehensive importance algorithm is shown in (6):

[0146] (6)

[0147] in, is the preset parameter, ; Indicates the first The overall importance of the row elements.

[0148] S504, calculating the block importance according to the number of rows included in the block matrix and each comprehensive importance.

[0149] Specifically, for any block matrix, the number of rows included in the block matrix is ​​determined, expressed as OK;

[0150] In one approach, the block importance is calculated using a preset block importance algorithm, where When it is greater than 1, the preset block importance algorithm is shown in formula (7):

[0151] (7)

[0152] in, is the block importance corresponding to the block matrix; where, Indicates the total number of rows of the block matrix; Indicates row elements, .

[0153] when =1, the block importance , is the comprehensive importance of the elements in the first row.

[0154] The present embodiment provides a network model quantization method. In the present embodiment, the average value of each row element is first calculated, and then the variance importance of each row element is calculated based on the average value, and then the comprehensive importance of each row element is calculated, so as to calculate the block importance based on the number of rows included in the block matrix. It can be seen that the block importance takes into account the comprehensive importance included in the block matrix, thereby making the block importance more accurate.

[0155] Embodiment 9

[0156] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the quantization threshold corresponding to the module to be quantized based on the importance of each block, and specifically includes:

[0157] Step 1: Determine the maximum block importance from the importance of each block.

[0158] For example, assume that there are two block matrices, namely and , the corresponding block importance is and , from which a maximum block importance is determined, .

[0159] Step 2: Calculate the product of the maximum block importance and the preset multiple, and determine the product as the quantization threshold corresponding to the module to be quantized.

[0160] The preset multiple is a pre-set multiple, which may be 0.6 or other multiples, and is not limited here.

[0161] For example, for the module 1 to be quantized, it corresponds to the two block matrices and block importance in the above step 1. Assume is the maximum block importance, then the quantization threshold of module 1 to be quantized is calculated as = *Preset multiples.

[0162] This embodiment provides a network model quantization method. In this embodiment, the quantization threshold corresponding to the module to be quantized can be calculated based on the maximum block importance. In this embodiment, different modules to be quantized may obtain different maximum block importances, which in turn makes their respective quantization thresholds different, thereby improving the flexibility for different modules to be quantized.

[0163] Embodiment 10

[0164] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of quantizing each block matrix based on the importance of each block and the quantization threshold to obtain a block quantization matrix corresponding to each block matrix, specifically including:

[0165] Method 1: For each block matrix, in response to the block importance of the block matrix being less than or equal to the quantization threshold, a preset first quantization algorithm is used to block-quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix.

[0166] The quantization degree of the preset first quantization algorithm is the first quantization degree.

[0167] In one embodiment, the preset first quantization algorithm includes a preset first quantization parameter algorithm and a preset first quantization matrix algorithm.

[0168] The preset first quantization parameter algorithm is a preset algorithm for calculating the quantization parameter generated at the first quantization level. The quantization parameter is used for the quantization matrix.

[0169] The preset first quantization matrix algorithm is a preset algorithm for performing a first quantization degree on the block matrix.

[0170] Among them, taking the module to be quantized 1, any block matrix in it as an example, the block importance of the block matrix is ​​less than or equal to the quantization threshold, that is, ≤ , then execute the following method to quantize the block matrix at the first quantization degree.

[0171] In one embodiment, a block matrix is ​​block-quantized by using a preset first quantization algorithm to obtain a block quantization matrix corresponding to the block matrix, including:

[0172] The block matrix is ​​input into a preset first quantization parameter algorithm to obtain a first quantization parameter; the first quantization parameter is input into a preset first quantization matrix algorithm to obtain a block quantization matrix.

[0173] The preset first quantization parameter algorithm is shown in (8):

[0174] (8)

[0175] in, Indicates A block matrix, is the scaling factor before quantization; is the binary matrix before quantization, ,in, express Elements in is the quantized scaling factor at the first quantization level; is the binary matrix after quantization at the first quantization level; F represents the norm.

[0176] The first quantization parameter refers to a quantized scaling factor and a quantized binary matrix at a first quantization level.

[0177] Specifically, according to the above (8), the quantized scaling factor and the quantized binary matrix at the first quantization level are calculated, which are respectively , then Input into the preset first quantization matrix algorithm to obtain the block quantization matrix corresponding to each block matrix.

[0178] The preset first quantization matrix algorithm is shown in (9):

[0179] (9) =

[0180] in, Indicates The block quantization matrix after the block matrix is ​​quantized.

[0181] Exemplarily, for each block matrix, the corresponding quantized scaling factor and quantized binary matrix at the first quantization level are calculated, and then the corresponding block quantization matrix is ​​calculated based on the quantized scaling factor and quantized binary matrix at the first quantization level.

[0182] It should be noted that, in this method, the first quantization degree is primary quantization, that is, the corresponding number of quantization times is 1.

[0183] Method 2: For each block matrix, in response to the block importance of the block matrix being greater than the quantization threshold, a preset second quantization algorithm and a preset quantization number are used to block quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix; the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm.

[0184] The quantization degree of the first quantization algorithm is preset to be the second quantization degree, so the first quantization degree is smaller than the second quantization degree.

[0185] The preset second quantization algorithm includes a preset second quantization parameter algorithm and a preset second quantization matrix algorithm.

[0186] The preset second quantization parameter algorithm is a preset algorithm for calculating the quantization parameter generated at the second quantization level. The quantization parameter is used for the quantization matrix.

[0187] The preset second quantization matrix algorithm is a preset algorithm for quantizing the block matrix at a second quantization degree.

[0188] Among them, taking the module to be quantized 1, any block matrix in it as an example, the block importance of the block matrix is ​​greater than the quantization threshold, that is, ≤ , then execute the following method to quantize the block matrix at the second quantization level.

[0189] In one embodiment, a block matrix is ​​block-quantized using a preset second quantization algorithm and a preset quantization number to obtain a block quantization matrix corresponding to the block matrix, including:

[0190] The block matrix is ​​input into a preset second quantization parameter algorithm to obtain a second quantization parameter corresponding to a preset quantization number; the second quantization parameter is input into a preset second quantization matrix algorithm to obtain a block quantization matrix.

[0191] The second quantization parameter refers to the quantized scaling factor and the quantized binary matrix at the second quantization level.

[0192] The preset number of quantization times refers to the number of quantization times at the second quantization level. ≥2.

[0193] It should be noted that the second quantization algorithm preset in the present application performs multiple quantizations based on the preset quantization times, and combines the quantization parameters generated after each quantization to calculate the block quantization matrix at the second quantization degree. Since the preset quantization times are ≥2, the calculated block quantization matrix has more bits and is more accurate.

[0194] The preset second quantization parameter algorithm is shown in (10):

[0195] (10)

[0196] in, For the The corresponding scaling factor before sub-quantization, For the The corresponding scaling factor after subquantization is For the The corresponding binary matrix after sub-quantization is No. The corresponding binary matrix before sub-quantization is ;In this formula, Indicates the preset quantization times; is the quantized scaling factor at the second quantization level, i.e. The corresponding scaling factor after sub-quantization; is the binary matrix after quantization under the second quantization degree, The binary matrix corresponding to the sub-quantization.

[0197] Specifically, according to the above (10), we can calculate ,in, , that is, the second quantization parameter corresponding to the preset quantization number is obtained.

[0198] Furthermore, a second quantization parameter corresponding to a preset number of quantization times is input into a preset second quantization matrix algorithm to obtain a block quantization matrix.

[0199] In one embodiment, the second quantization matrix algorithm is preset as shown in (11):

[0200] (11)

[0201] in, Indicates The block quantization matrix after the block matrix is ​​quantized.

[0202] For example, assuming that the preset quantization times , then the block quantization matrix is ​​calculated based on the following algorithm. Specifically, based on formula (8) , then input it into formula (12), and we get .

[0203] (12)

[0204] in, , is the corresponding scaling factor before the second quantization.

[0205] Further, , as well as Input into formula (13) to obtain the block quantization matrix.

[0206] (13)

[0207] It should be noted that, according to the above algorithm, the block quantization matrix after the quantization of each block matrix can be calculated.

[0208] It should be noted that in this application, the matrix to be quantized is converted from floating-point values ​​to discrete values.

[0209] It should be noted that the preset number of quantization times can be determined in advance according to the importance of the block matrix in the matrix to be quantized. For a module to be quantized, the preset number of quantization times can be one, or can be preset according to each block matrix in the module to be quantized, and there is no limitation here. In this embodiment, the preset number of quantization times is preset according to the block matrix, and the importance of each block matrix can be taken into account, so as to be more flexible.

[0210] According to the above embodiments, in this application, each module to be quantized is quantized separately, in order to obtain the corresponding quantization threshold for each module to be quantized, different quantization methods are implemented based on the quantization threshold, so that each block matrix can be quantized by a suitable method; the quantization method in this application uses different quantization degrees to distinguish, if the block importance is greater than the quantization threshold, the preset second quantization algorithm of the second quantization degree will be used to calculate the block quantization matrix. In this application, for the matrix to be quantized in each module to be quantized, in order to accurately obtain the quantization weight matrix, the matrix to be quantized is divided into blocks, the region is divided, and then the block importance corresponding to each block matrix is ​​calculated respectively, the block importance is used to characterize the importance of the corresponding block matrix in the matrix to be quantized, and then the quantization threshold is calculated based on the block importance, so that the quantization threshold changes with the module to be quantized, so it is more flexible and more in line with the module to be quantized itself; in addition, the block matrices are quantized separately to obtain the block quantization matrix, which is quantized from a local perspective, and the matrix to be quantized is subdivided, so the block quantization matrix is ​​more accurate, and finally the quantized module is more accurate. The quantized model in this application not only ensures that the model has certain performance, but also reduces the computing requirements due to the implementation of precise quantization, while also improving the accuracy and efficiency of image retrieval, providing strong support for the operation of such quantized models in edge devices.

[0211] It should be noted that the block importance can reflect the influence of the corresponding block matrix on the performance of the model to be quantized; if the influence is greater, a quantization method with a greater degree of quantization is used to quantize the block matrix.

[0212] The present embodiment provides a network model quantization method. In the present embodiment, different quantization algorithms are determined in a targeted manner according to the size of the block importance of the block matrix. If the block importance is less than or equal to the quantization threshold, a preset first quantization algorithm is used to perform block quantization; if the block importance is greater than the quantization threshold, a preset second quantization algorithm and a preset quantization number are used to quantize the block matrix blocks. In the present embodiment, the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm, so that the preset second quantization algorithm is more accurate; in addition, in the present embodiment, the corresponding quantization algorithm is selected more flexibly according to the size of the block importance.

[0213] Embodiment 11

[0214] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method based on each post-quantization module generating a post-quantization model, including:

[0215] Step 1: Input the query text into the text encoding module to output the text features corresponding to the query text; the text encoding module is derived from the quantized model; the text encoding module includes at least one quantized module.

[0216] The text encoding module is an encoding module that receives a query text and extracts features included in the query text.

[0217] The query text is the text to be queried. The network model quantization method in the present application retrieves a target image matching the query text from an image feature database based on the query text.

[0218] Step 2: Search the image feature database based on text features to obtain the target image corresponding to the query text.

[0219] The image feature database includes image features corresponding to at least one query image.

[0220] The target image refers to an image that matches the query text. It should be noted that the target image belongs to the query image, and there is at least one target image.

[0221] In one embodiment, searching in an image feature database based on text features to obtain a target image corresponding to the query text specifically includes:

[0222] Step 1: Calculate the similarity between the text feature and at least one image feature in the image feature database.

[0223] Step 2: Determine the similarity that meets the similarity threshold as the target similarity.

[0224] Step 3: Retrieve the query image corresponding to the target similarity in the image feature database, and determine the retrieved query image as the target image corresponding to the query text.

[0225] The similarity threshold is a pre-set threshold for determining the similarity between text features and image features, expressed as , It can be ≥0.8, or other data, and there is no restriction here.

[0226] The image feature database includes N image features, where N is at least one, and the image feature database is expressed as ,in, refers to the first image feature, that is, the image feature corresponding to the first query image; It refers to the Nth image feature, that is, the image feature corresponding to the Nth query image.

[0227] Specifically, the text encoding module extracts features from the query text and obtains the corresponding text features. express.

[0228] Furthermore, if the image feature database includes N image features, the network model quantization device calculates the similarity between the text feature and the N image features respectively, that is, obtains N similarities, and the similarity is expressed as express, . It refers to the text features and The similarity of the image features, that is, similarities. Among them, It refers to image features.

[0229] in, .

[0230] in, represent , represent , represent The transposed vector of Represents the vector modulus.

[0231] Specifically, The similarity of is determined as the target similarity, and the target similarity is expressed as .

[0232] Furthermore, the image feature database is retrieved The corresponding query image is: The target similarity is , which refers to the text features and The similarity of the image features, that is, the corresponding query image is The query image retrieved is query images, where .

[0233] Furthermore, The query image is determined as the target image corresponding to the query text.

[0234] It should be noted that the target similarity can be at least one.

[0235] It should be noted that the language type of the query text is consistent with the language type of the training data.

[0236] It should be noted that the text data can be in one language type or in multiple language types; if it is in multiple language types, the query text can also be in the corresponding consistent language type, so that the quantified model in this application can be applied in multiple language types, thereby improving adaptability.

[0237] The present embodiment provides a network model quantization method. In the present embodiment, the text encoding module includes at least one post-quantization module, and the post-quantization module is generated by replacing the corresponding matrix to be quantized with the corresponding quantization weight matrix, wherein the quantization weight matrix is ​​generated based on multiple block quantization matrices, and each block quantization matrix is ​​obtained by quantizing the block matrix based on the corresponding block importance and the quantization threshold of the corresponding module to be quantized, so that the matrix to be quantized is quantized according to the local area, and is also quantized based on the corresponding quantization threshold during the quantization process, so that the quantization method is more flexible; in the present embodiment, the quantization threshold may be different based on different modules to be quantized, so the quantization threshold in the present embodiment for different modules to be quantized is also flexible, so that it is more suitable for the modules to be quantized. Module, the quantization threshold is also flexible and not fixed. It is determined based on the importance of each corresponding block in the module to be quantized, so that the quantized quantization weight matrix is ​​more suitable for itself, so that the performance of the module after quantization is better; in this embodiment, each block matrix is ​​obtained by dividing the matrix to be quantized, so this embodiment subdivides the matrix to be quantized to obtain the block matrix, and then obtains the block quantization matrix of each block matrix respectively, so as to further refine the matrix to be quantized, further refine the quantization method, make the quantization weight matrix more accurate, and then make the text encoding module including at least one post-quantization module more accurate, so with the support of the post-quantization module, the query text is input into the text encoding module to obtain more accurate text features, and then the target image is retrieved more accurately.

[0238] Embodiment 12

[0239] This embodiment is a further refinement of the above embodiment, and includes:

[0240] At least one query image in the to-be-queried image database is input into an image encoding module to generate an image feature database; the image encoding module is derived from a quantized model; and the image encoding module includes at least one quantized module.

[0241] The image encoding module is an encoding module that receives a query image and extracts features included in the query image.

[0242] It should be noted that the post-quantization module in this embodiment is as described in Embodiment 1 and will not be described in detail here.

[0243] The database to be queried includes at least one query image, which can be expressed as including N query images. The query image is an unlabeled image, that is, the content or object type of the query image is unknown.

[0244] The image feature database includes image features corresponding to N query images, that is, includes N image features.

[0245] This embodiment also includes: inputting at least one query image in the to-be-queried image database into an image encoding module to generate an image feature database, specifically:

[0246] Step 1: Input at least one query image into an image encoding module to output an output feature embedding vector corresponding to each query image; the output feature embedding vector includes at least one feature vector.

[0247] Among them, the output feature embedding vector is expressed as , which is the The output feature embedding vector corresponding to the query image.

[0248] in, includes at least one eigenvector, which can be expressed as .

[0249] Step 2: Determine the maximum feature vector from the output feature embedding vector as the image feature corresponding to the query image.

[0250] Among them, the maximum eigenvector can also be the optimal eigenvector.

[0251] Specifically, The maximum eigenvector is determined to be the image feature corresponding to the query image.

[0252] Step three: gather the image features corresponding to each query image to generate an image feature database; the image feature database includes the feature characteristics corresponding to each query image; the image feature database includes at least one image feature.

[0253] The to-be-queried image database includes at least one query image, wherein the query image has no label, that is, the content in the query image is unknown.

[0254] This embodiment provides a network model quantization method. In this embodiment, the image encoding module includes at least one post-quantization module. Since the performance of the post-quantization module is better, the image feature database generated based on the image encoding module is more accurate.

[0255] Figure 6 This is an overall schematic diagram of network model quantization and application provided in Example 12.

[0256] like Figure 6 , including a model to be quantized 601, a quantized model 602, a text encoding module 603 and an image encoding module 604.

[0257] The model to be quantized 601 includes a module to be quantized 605 , and the model after quantization 602 includes a module after quantization 606 .

[0258] Among them, after the module to be quantized 605 is quantized, a quantized module 606 is obtained, thereby realizing the quantization of the model to be quantized 601 into the quantized model 602.

[0259] exist Figure 6 In the figure, the model to be quantized 601 is quantized to obtain a quantized model 602, which includes a text encoding module 603 and an image encoding module 604, wherein the text encoding module 603 and the image encoding module 604 include at least one quantized module 606.

[0260] Furthermore, the query text is input into the text encoding module 603 to obtain text features, and then the text features are searched in the image feature database, so as to output a target image corresponding to the query text.

[0261] The image database to be queried is input into the image encoding module 604, and the image feature database is output.

[0262] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0263] Embodiment 13

[0264] The following is an embodiment of the device of the present application. Figure 7 This is a schematic diagram of the structure of a network model quantization device provided in Example 13. The network model quantization device 700 includes the following modules:

[0265] The block module 701 is used to divide the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices; the module to be quantized comes from the model to be quantized; there is at least one module to be quantized;

[0266] A calculation module 702 is used to calculate the block importance corresponding to each block matrix for each module to be quantized based on a training data set. The calculation module 702 is also used to calculate the quantization threshold corresponding to the module to be quantized based on each block importance. The training data set includes at least one training data.

[0267] A quantization module 703 is used to quantize each block matrix based on the importance of each block and the quantization threshold for each module to be quantized, so as to obtain a block quantization matrix corresponding to each block matrix;

[0268] A generating module 704, configured to generate a quantization weight matrix after quantization of the corresponding module to be quantized based on each block quantization matrix;

[0269] The replacement module 705 is used to replace the quantization matrix in each module to be quantized with the corresponding quantization weight matrix to obtain a quantized module corresponding to each module to be quantized. The generation module 704 is also used to generate a quantized model based on each quantized module.

[0270] The calculation module 702 is specifically used to calculate the block importance corresponding to each block matrix based on the training data set:

[0271] Obtain input data corresponding to each module to be quantized based on the training data set;

[0272] Calculate the block importance corresponding to each block matrix based on the input data.

[0273] The calculation module 702, when obtaining the input data corresponding to each module to be quantized based on the training data set, is specifically used to:

[0274] The training data set is input into the model to be quantized, and propagated forward through the neural network to obtain the intermediate data corresponding to each module to be quantized; the amount of intermediate data is equal to the amount of training data;

[0275] Construct the input data corresponding to each module to be quantized based on each intermediate data.

[0276] The block module 701 is used to divide the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices:

[0277] If the block division method is free block division, a row of elements in the matrix to be quantized is determined as a block matrix to obtain multiple block matrices; the number of rows of the matrix to be quantized is equal to the number of block matrices.

[0278] The block module 701 is used to divide the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices:

[0279] If the block division method is regular block division, obtain the preset number of blocks;

[0280] Based on the number of rows of the matrix to be quantized and the preset number of blocks, multiple block matrices are obtained; the number of rows of the block matrices other than the last block matrix is ​​the first number of rows, and the number of rows of the last block matrix is ​​the second number of rows; the first number of rows is the upward integer of the ratio of the number of rows of the matrix to be quantized to the preset number of blocks; the second number of rows is less than or equal to the first number of rows.

[0281] The calculation module 702 is specifically used to calculate the block importance corresponding to each block matrix based on the input data:

[0282] Inputting the input data into a preset second-order derivative algorithm to output a second-order matrix;

[0283] For each block matrix, the average importance corresponding to each row element of the block matrix is ​​calculated based on the second-order matrix; the average importance is the average value of the importance of the corresponding row elements in the block matrix;

[0284] For each block matrix, the block importance corresponding to the block matrix is ​​calculated based on the corresponding average importance.

[0285] The calculation module 702, when calculating the average importance corresponding to each row element of the block matrix based on the second-order matrix, is specifically used to:

[0286] Input the inverse matrix of the second-order matrix into the preset element importance algorithm to output the element importance corresponding to each element in the block matrix;

[0287] Determine the number of elements included in each row of the block matrix;

[0288] The average importance of each row of the block matrix is ​​calculated based on the number of elements, the importance of each element and the preset average importance algorithm.

[0289] The calculation module 702, when calculating the block importance corresponding to the block matrix based on the corresponding average importance, is specifically used to:

[0290] Calculate the average value of each row element in the block matrix;

[0291] Calculate the variance importance corresponding to each row element in the block matrix based on the average value;

[0292] Input each variance importance and average importance into the preset comprehensive importance algorithm to output the comprehensive importance corresponding to each row element in the block matrix;

[0293] The block importance is calculated based on the number of rows in the block matrix and the comprehensive importance.

[0294] The calculation module 702 is specifically used to calculate the quantization threshold corresponding to the module to be quantized based on the importance of each block:

[0295] Determine the maximum block importance from the importance of each block;

[0296] The product of the maximum block importance and the preset multiple is calculated, and the product is determined as the quantization threshold corresponding to the module to be quantized.

[0297] The quantization module 703 is specifically used to quantize each block matrix based on the importance of each block and the quantization threshold to obtain a block quantization matrix corresponding to each block matrix:

[0298] For each block matrix, in response to the block importance of the block matrix being less than or equal to the quantization threshold, a preset first quantization algorithm is used to block-quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix;

[0299] For each block matrix, in response to the block importance of the block matrix being greater than a quantization threshold, a preset second quantization algorithm and a preset quantization number are used to block quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix; the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm.

[0300] This embodiment provides a network model quantization device, which also includes: an output module and a retrieval module;

[0301] After generating the quantized model based on each quantized module, the output module is used to input the query text into the text encoding module to output the text features corresponding to the query text; the text encoding module is derived from the quantized model; the text encoding module includes at least one quantized module;

[0302] The retrieval module is used to search in the image feature database based on the text features to obtain the target image corresponding to the query text.

[0303] The generation module 704 is further used to: input at least one query image in the to-be-queried image database into the image encoding module to generate an image feature database; the image encoding module is derived from the quantized model; the image encoding module includes at least one quantized module.

[0304] The description of the features in the embodiment corresponding to the network model quantization device can be found in the relevant description of the embodiment corresponding to the network model quantization method, and will not be repeated here.

[0305] Figure 8This is a schematic diagram of the structure of the electronic device provided in this application. Figure 8 As shown, the electronic device 800 provided in this embodiment includes: at least one processor 801 and a memory 802. Optionally, the electronic device 800 also includes a communication component 803. The processor 801, the memory 802 and the communication component 803 are connected via a bus 804.

[0306] In the specific implementation process, at least one processor 801 executes the computer execution instructions stored in the memory 802, so that at least one processor 801 executes the above-mentioned network model quantization method embodiment.

[0307] The specific implementation process of the processor 801 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0308] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0309] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0310] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0311] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned network model quantization method embodiments when running.

[0312] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0313] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any of the above-mentioned network model quantization method embodiments are implemented.

[0314] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned network model quantization method embodiments are implemented.

[0315] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0316] The above is a detailed introduction to a network model quantization method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A network model quantization method, characterized in that: include: Divide the matrix to be quantized contained in the module to be quantized into blocks to obtain a plurality of block matrices; The module to be quantified is derived from the model to be quantified; The number of the module to be quantified is at least one; For each module to be quantized, the block importance corresponding to each block matrix is ​​calculated based on a training data set, and the quantization threshold corresponding to the module to be quantized is calculated based on each block importance, wherein the training data set includes at least one training data; For each module to be quantized, quantize each of the block matrices based on the importance of each block and the quantization threshold, so as to obtain a block quantization matrix corresponding to each of the block matrices; Generate a quantization weight matrix after quantization of the corresponding module to be quantized based on each of the block quantization matrices; For each module to be quantized, the matrix to be quantized in the module to be quantized is replaced with a corresponding quantization weight matrix to obtain a quantized module corresponding to each module to be quantized, and a quantized model is generated based on each quantized module.

2. The method according to claim 1, characterized in that The calculating the block importance corresponding to each block matrix based on the training data set includes: Obtain input data corresponding to each module to be quantized based on the training data set; The block importance corresponding to each of the block matrices is calculated based on the input data.

3. The method according to claim 1, characterized in that The step of obtaining input data corresponding to each module to be quantized based on the training data set includes: Input the training data set into the model to be quantized, and propagate forward through the neural network to obtain intermediate data corresponding to each module to be quantized; the amount of the intermediate data is equal to the amount of the training data; The input data corresponding to each module to be quantized is constructed based on each intermediate data.

4. The method according to claim 1, characterized in that: The step of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain a plurality of block matrices includes: If the block division method is free block division, a row of elements in the matrix to be quantized is determined as a block matrix to obtain multiple block matrices; the number of rows of the matrix to be quantized is equal to the number of the block matrices.

5. The method according to claim 1, characterized in that The step of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain a plurality of block matrices includes: If the block division method is regular block division, obtain the preset number of blocks; Based on the number of rows of the matrix to be quantized and the preset number of blocks, multiple block matrices are obtained; the number of rows of the other block matrices except the last block matrix is ​​the first number of rows, and the number of rows of the last block matrix is ​​the second number of rows; the first number of rows is the upward integer of the ratio of the number of rows of the matrix to be quantized and the preset number of blocks; the second number of rows is less than or equal to the first number of rows.

6. The method according to claim 2, characterized in that The calculating the block importance corresponding to each of the block matrices based on the input data includes: Inputting the input data into a preset second-order derivative algorithm to output a second-order matrix; For each block matrix, the average importance corresponding to each row element of the block matrix is ​​calculated based on the second-order matrix; the average importance is the average value of the importance of the corresponding row elements in the block matrix; For each block matrix, the block importance corresponding to the block matrix is ​​calculated based on the corresponding average importance.

7. The method according to claim 6, characterized in that The calculating, based on the second-order matrix, the average importance corresponding to each row element of the block matrix comprises: Inputting the inverse matrix of the second-order matrix into a preset element importance algorithm to output the element importance corresponding to each element in the block matrix; Determining the number of elements included in each row of the block matrix; The average importance corresponding to each row element of the block matrix is ​​calculated based on the number of elements, the importance of each element and a preset average importance algorithm.

8. The method according to claim 6, characterized in that The calculating the block importance corresponding to the block matrix based on the corresponding average importance includes: Calculate the average value of each row element in the block matrix; Calculate the variance importance corresponding to each row element in the block matrix based on the average value; Inputting each of the variance importances and the average importance into a preset comprehensive importance algorithm to output the comprehensive importance corresponding to each row element in the block matrix; The block importance is calculated according to the number of rows included in the block matrix and each of the comprehensive importances.

9. The method according to claim 1, characterized in that: The calculating the quantization threshold corresponding to the module to be quantized based on the importance of each block includes: Determining the maximum block importance from among the block importances; The product of the maximum block importance and a preset multiple is calculated, and the product is determined as a quantization threshold corresponding to the module to be quantized.

10. The method according to claim 1, characterized in that The quantizing each of the block matrices based on the importance of each block and the quantization threshold to obtain a block quantization matrix corresponding to each of the block matrices includes: For each block matrix, in response to the block importance of the block matrix being less than or equal to the quantization threshold, a preset first quantization algorithm is used to block-quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix; For each block matrix, in response to the block importance of the block matrix being greater than the quantization threshold, a preset second quantization algorithm and a preset quantization number are used to block quantize the block matrix to obtain a block quantization matrix corresponding to the block matrix; the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm.

11. The method according to claim 1, characterized in that: After generating the quantized model based on each of the quantized modules, the method further includes: Inputting the query text into a text encoding module to output text features corresponding to the query text; the text encoding module is derived from a quantized model; the text encoding module includes at least one quantized module; Based on the text features, a search is performed in an image feature database to obtain a target image corresponding to the query text.

12. The method according to claim 11, characterized in that The method further comprises: At least one query image in the to-be-queried image database is input into an image encoding module to generate an image feature database; the image encoding module is derived from the quantized model; the image encoding module includes at least one quantized module.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, used to implement the steps of the network model quantization method as described in any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the network model quantization method according to any one of claims 1 to 12.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the network model quantization method according to any one of claims 1 to 12 are implemented.

Images