Network model quantization method, device, storage medium and program product

Abstract

This application discloses a network model quantization method, device, storage medium, and program product, relating to the field of data processing technology. The method includes: dividing the matrix to be quantized in a module into smaller blocks to obtain a block matrix; calculating the block importance and quantization threshold corresponding to each block matrix; the block importance reflects the importance of each corresponding block matrix in the corresponding module to be quantized; and the quantization threshold is calculated based on the block importance, so different modules to be quantized have different quantization thresholds, thus improving the flexibility of the quantization threshold. This application considers the differences between modules to be quantized. Each block matrix is ​​quantized based on its block importance and quantization threshold. Therefore, the quantization of each block matrix is ​​performed on the block matrix itself, rather than on the matrix to be quantized as a unit. This application provides more detailed quantization, resulting in a more accurate quantized model.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to network model quantization methods, devices, storage media and program products. Background Technology

[0002] With the rise of artificial intelligence, mounting network models on edge devices and using these devices to compute them has become a trend. However, edge devices are insufficient in terms of capacity and computing power, necessitating the quantization of network models.

[0003] In related technologies, the model parameters in the modules requiring quantization within the network model can be converted from floating-point to fixed-point representation, resulting in model parameters with smaller fixed-point representations. However, this type of model quantization method lacks flexibility, leading to poor performance and low accuracy in applications. Summary of the Invention

[0004] This application provides a method, device, storage medium, and program product for quantizing network models, in order to at least solve the problem in the related art that the lack of flexibility in quantizing model parameters leads to poor performance and accuracy of the quantized model.

[0005] Firstly, this application provides a method for quantizing network models, including:

[0006] The quantization matrix contained in the module to be quantized is divided into blocks to obtain multiple block matrices; the module to be quantized comes from the model to be quantized; there is at least one module to be quantized.

[0007] For each module to be quantized, the importance of each block matrix is ​​calculated based on the training dataset, and the quantization threshold of the module to be quantized is calculated based on the importance of each block. The training dataset includes at least one training data.

[0008] For each module to be quantized, each block matrix is ​​quantized separately based on the importance of each block and the quantization threshold to obtain the block quantization matrix corresponding to each block matrix.

[0009] Generate the corresponding quantization weight matrix after quantization of the module to be quantized based on each block quantization matrix;

[0010] For each module to be quantized, the matrix to be quantized in the module is replaced with the corresponding quantization weight matrix to obtain the quantized module corresponding to each module, and a quantized model is generated based on each quantized module.

[0011] Secondly, this application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described network model quantization methods.

[0012] Thirdly, this application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described network model quantization methods.

[0013] Fourthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described network model quantization methods.

[0014] This application provides a network model quantization method, device, storage medium, and program product. In this application, for each module to be quantized, the matrix to be quantized in the module is divided into blocks to obtain block matrices. The importance of each block matrix and the quantization threshold are calculated. In this application, the block importance reflects the importance of each corresponding block matrix in the corresponding module to be quantized. Furthermore, the quantization threshold is calculated based on the block importance, so the quantization threshold is different for different modules to be quantized, thereby improving the flexibility of the quantization threshold. Moreover, different quantization thresholds reflect the characteristics of each module to be quantized. Therefore, this application considers the differences between each module to be quantized, making it more flexible. Based on the importance of each block and the quantization threshold, each block matrix is ​​quantized. Therefore, the quantization of each block matrix is ​​performed on the block matrix itself, rather than on the matrix to be quantized. Thus, the quantization process in this application is more refined, making the resulting block quantization matrix more accurate. Then, based on each block quantization matrix, a quantization weight matrix for the corresponding module to be quantized is generated, making the quantization weight matrix more accurate and more closely aligned with the module itself. Therefore, in this application, the matrix to be quantized in the module to be quantized is replaced with the corresponding quantization weight matrix, making the resulting quantized module more accurate and more in line with its own characteristics. Therefore, the quantized model in this application is used for application, resulting in more accurate results. Attached Figure Description

[0015] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This application provides an illustration of an application scenario.

[0017] Figure 2 This is a schematic diagram of a network model quantization method provided in Embodiment 1;

[0018] Figure 3 This is a schematic diagram of a network model quantization method provided in Example 6;

[0019] Figure 4 This is a schematic diagram of a network model quantization method provided in Example 7;

[0020] Figure 5 This is a schematic diagram of a network model quantization method provided in Example 8;

[0021] Figure 6 This is a schematic diagram of the overall quantization and application of a network model provided in Example Twelve;

[0022] Figure 7 A schematic diagram of the network model quantization device provided in Embodiment Thirteen;

[0023] Figure 8 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0025] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0026] In related technologies, the model parameters in the modules requiring quantization in the network model can be converted from floating-point to fixed-point representation, resulting in model parameters with smaller fixed-point representations. However, this method lacks flexibility and is incompatible with the characteristics of the modules requiring quantization, leading to poor performance and low accuracy in applications.

[0027] To address the shortcomings of related technologies, the inventors of this application have proposed a network model quantization method through a series of improvements. To solve the problem of insufficient flexibility in model quantization in related technologies, leading to poor performance and low accuracy of the quantized model, this solution first divides the matrix to be quantized into blocks for each module to be quantized, thus achieving subdivision and obtaining each block matrix. Further, the importance of each block matrix is ​​calculated, and then the quantization threshold for each module to be quantized is calculated. The quantization threshold in this solution is generated based on different modules to be quantized, so different modules can have their own corresponding quantization thresholds, making the quantization thresholds more closely aligned with the corresponding modules. Therefore, for each module to be quantized, the block matrix is ​​quantized separately based on its corresponding block importance and quantization threshold, thus improving the flexibility of quantization. Furthermore, this solution quantizes on a block matrix basis, rather than on the matrix to be quantized, making the quantization method more specific. Therefore, the final quantized weight matrix is ​​more accurate, resulting in more accurate quantized modules and models.

[0028] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] The specific application environment architecture or specific hardware architecture on which the execution of the network model quantization method depends is described here.

[0030] Figure 1 This is a schematic diagram illustrating an application scenario provided in this application. For example... Figure 1 As shown, it includes: electronic device 101.

[0031] The electronic device 101 can be a computer or a server; there are no restrictions here. It should be noted that the electronic device 101 can be an edge device.

[0032] Specifically, the user inputs query text into the electronic device 101, which then inputs the query text into the text encoding module 102 and outputs the text features.

[0033] Furthermore, the electronic device 101 retrieves the corresponding target image from the image feature database based on text features.

[0034] In this application scenario, the text encoding module 102 includes at least one quantized module as described in this application.

[0035] Example 1

[0036] The execution subject of Embodiments 1 to 12 of this application is a network model quantization device, which is located in an electronic device.

[0037] Figure 2 This is a schematic diagram of a network model quantization method provided in Embodiment 1. Figure 2 As shown, it includes:

[0038] S201, the matrix to be quantized contained in the module to be quantized is divided into blocks to obtain multiple block matrices; the module to be quantized comes from the model to be quantized; there is at least one module to be quantized.

[0039] Wherein, the matrix to be quantized is used express, ,Right now yes OK, A matrix of columns.

[0040] in, Indicates the first A block matrix, That is OK, A matrix of columns, where, , ≥1.

[0041] For example, suppose we are referring to module 1 to be quantized, where the matrix to be quantized is... ,right Divide into blocks, obtain This means obtaining three block matrices.

[0042] In this embodiment, the module to be quantized originates from the model to be quantized, and the module to be quantized in the model to be quantized is quantized to obtain the quantized model. Specifically, the matrix to be quantized in the module to be quantized is divided into blocks to obtain block matrices. Then, each block matrix is ​​quantized to obtain a quantized block matrix, which in turn yields the quantization weight matrix corresponding to the matrix to be quantized. The quantization weight matrix is ​​then used to replace the corresponding matrix to be quantized, thereby quantizing the module to be quantized into the quantized module.

[0043] The quantized model refers to the model generated after quantizing the modules to be quantized included in the model. The aforementioned quantized model is a low-bit model adaptable to devices with limited resources.

[0044] Among them, the quantized module refers to the module generated after the module to be quantized is quantized.

[0045] The module to be quantized refers to the module whose weight matrix needs to be quantized.

[0046] Here, the matrix to be quantized refers to the weight matrix that needs to be quantized.

[0047] The quantization weight matrix is ​​obtained by quantizing the matrix to be quantized, and it is generated from multiple block quantization matrices.

[0048] The block quantization matrix is ​​the matrix obtained by quantizing the block matrix.

[0049] The block matrix is ​​the matrix obtained by dividing the matrix to be quantized into blocks. It should be noted that a matrix to be quantized consists of multiple block matrices.

[0050] It should be noted that a model to be quantized includes at least one module to be quantized and at least one non-quantized module. The non-quantized module refers to the module that does not require quantization.

[0051] It should be noted that there is at least one module to be quantized, which may include at least one type of module, such as a query embedding generation module, a key embedding generation module, a value embedding generation module, and at least one type of module in a feedforward neural network layer.

[0052] It should be noted that the type of the module to be quantized remains unchanged during the quantization process in this embodiment.

[0053] S202, for each module to be quantized, calculate the block importance corresponding to each block matrix based on the training dataset, and calculate the quantization threshold corresponding to the module to be quantized based on each block importance. The training dataset includes at least one training data.

[0054] Here, block importance refers to the importance of the block matrix. It should be noted that one block matrix corresponds to one block importance.

[0055] Here, the quantization threshold refers to the threshold corresponding to the module to be quantized. It should be noted that one quantization threshold corresponds to one module. The training dataset refers to the dataset used when quantizing the module.

[0056] In one approach, the training dataset is input into the model to be quantized, and intermediate data is used to calculate the importance of each block matrix.

[0057] The training data can be text data, and there are no restrictions on the language type of the text data.

[0058] It should be noted that a module to be quantized corresponds to a matrix to be quantized, and a matrix to be quantized can be divided into multiple sub-matrices. Each sub-matrice corresponds to its own sub-matrix importance. A module to be quantized corresponds to a quantization threshold. Therefore, different modules to be quantized can have different quantization thresholds. Thus, for different modules to be quantized, due to the increased flexibility of the quantization threshold, each module to be quantized can be quantized into its own quantization weight matrix under more appropriate conditions.

[0059] It should be noted that each block matrix has its own corresponding block importance, which reflects the importance of each block matrix to the quantization module, that is, its impact on the performance of the quantization module.

[0060] S203, for each module to be quantized, quantize each block matrix separately based on the importance of each block and the quantization threshold to obtain the block quantization matrix corresponding to each block matrix.

[0061] Among them, the quantization threshold is used express.

[0062] For example, suppose there are three partitioned matrices, namely... , as well as The importance of each corresponding block is as follows: , as well as . Indicates the first The importance of each block matrix.

[0063] For example, based on as well as right Quantification is performed to obtain The corresponding block quantization matrix is ​​represented as follows: , indicating the first The block quantization matrix corresponding to each block matrix is ​​obtained. The same method is used to obtain the corresponding block quantization matrix for the remaining block matrices.

[0064] S204 generates the corresponding quantization weight matrix of the module to be quantized based on each block quantization matrix.

[0065] Among them, the quantization weight matrix is ​​used express.

[0066] For example, following the steps above, three block quantization matrices are finally obtained, namely: , as well as .

[0067] Furthermore, the three block quantization matrices mentioned above are combined in rows and columns to generate the quantization weight matrix corresponding to module 1 to be quantized, i.e. .

[0068] It should be noted that, based on the above example, the corresponding quantization weight matrix is ​​obtained in the same way for the other modules to be quantized, which will not be elaborated here.

[0069] S205, for each module to be quantized, replace the matrix to be quantized in the module with the corresponding quantization weight matrix to obtain the quantized module corresponding to each module to be quantized, and generate the quantized model based on each quantized module.

[0070] Based on the above example, the matrix to be quantized in module 1 is... Replace with This leads to the quantized module 1.

[0071] Furthermore, following the same method described above, the unquantized matrices in the remaining modules to be quantized are replaced with the corresponding quantization weight matrices, thereby obtaining the remaining quantized modules.

[0072] Furthermore, the model composed of each quantized module is called the quantized model.

[0073] This embodiment provides a network model quantization method. For each module to be quantized, the matrix to be quantized within that module is divided into blocks to obtain block matrices. The importance of each block matrix and its quantization threshold are calculated. In this embodiment, the block importance reflects the importance of each corresponding block matrix within its respective module to be quantized. Furthermore, the quantization threshold is calculated based on the block importance, so different quantization thresholds apply to different modules to be quantized, thus improving the flexibility of the quantization threshold. Different quantization thresholds also reflect the differences between modules to be quantized, making this embodiment more flexible as it considers the differences between modules. Quantization of each block matrix is ​​performed on the block matrix itself, rather than on the entire matrix to be quantized, making this embodiment more refined and resulting in more accurate block quantization matrices that better match the characteristics of the corresponding module to be quantized.

[0074] Example 2

[0075] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the block importance corresponding to each block matrix based on the training dataset, specifically including:

[0076] Step 1: Obtain the input data corresponding to each module to be quantized based on the training dataset.

[0077] Specifically, the training dataset is input into the model to be quantized to obtain the input data.

[0078] Step 2: Calculate the block importance corresponding to each block matrix based on the input data.

[0079] This embodiment provides a network model quantization method. In this embodiment, the input data corresponding to each module to be quantized can be obtained based on the training dataset. It can be seen that the input data is different for different modules to be quantized, so the input data is more in line with the respective modules to be quantized, and the resulting block importance is more accurate.

[0080] Example 3

[0081] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for obtaining the input data corresponding to each module to be quantized based on the training dataset, specifically including:

[0082] Step 1: Input the training dataset into the model to be quantized, and propagate it forward through the neural network to obtain intermediate data for each module to be quantized; the number of intermediate data is equal to the number of training data.

[0083] Intermediate data refers to the data generated in the module to be quantized from the training dataset.

[0084] For example, if the model to be quantized includes multiple network layers, and each network layer includes at least one module to be quantized, the training dataset is input into the model, propagates forward through the neural network to each network layer, and passes through the module to be quantized in each network layer, thus outputting a set of data from the module to be quantized, which is intermediate data. For example, if the first network layer includes a module to be quantized A, then after the training dataset is input into the first network layer, it passes through module to be quantized A, and outputs a set of data A, which is the intermediate data of module to be quantized A. Then the data continues to propagate, outputting from the first network layer to the second network layer, which includes a module to be quantized B. The data is transmitted to module to be quantized B, and outputs a set of data B, which is the intermediate data of module to be quantized B.

[0085] It should be noted that if the training dataset includes 512 training data points, then the number of intermediate data points is 512.

[0086] Step 2: Construct the input data for each module to be quantized based on the intermediate data.

[0087] Specifically, determine the embedding dimension of the model to be quantized, which is defined as the number of rows in the input data. For example, if the embedding dimension is 5, then the input data consists of 5 rows.

[0088] Furthermore, the number of intermediate data points is determined by the number of columns in the input data. For example, if the number of intermediate data points is 512, then the number of columns in the input data is 512.

[0089] Specifically, assume the embedding dimension of the model to be quantized is... The number of intermediate data is Then, based on the intermediate data, the input data of the module to be quantized is constructed, which is... , represented as OK, A matrix of columns.

[0090] This embodiment provides a network model quantization method. In this embodiment, intermediate data corresponding to different modules to be quantized are obtained to construct corresponding input data. Then, quantization is performed based on the appropriate input data to make the quantization results more accurate.

[0091] Example 4

[0092] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, specifically including:

[0093] If the partitioning method is free partitioning, then one row of elements in the matrix to be quantized is determined as a partition matrix, so as to obtain multiple partition matrices; the number of rows in the matrix to be quantized is equal to the number of partition matrices.

[0094] For example, for module 1 to be quantized, which includes the matrix to be quantized as... , That is, the total When performing free partitioning, if a row of elements is defined as a partitioning matrix, then... Divided into blocks A block matrix.

[0095] This embodiment provides a network model quantization method. In this embodiment, the block matrix corresponding to the matrix to be quantized can be quickly obtained based on free block partitioning.

[0096] Example 5

[0097] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, specifically including:

[0098] Step 1: If the block division method is regular block division, then obtain the preset number of blocks.

[0099] The preset number of blocks is the number of block matrices to be obtained that is set in advance.

[0100] Step 2: Obtain multiple block matrices based on the number of rows in the matrix to be quantized and the preset number of blocks; the number of rows in the block matrices other than the last block matrix is ​​the first row number, and the number of rows in the last block matrix is ​​the second row number; the first row number is the integer up of the ratio of the number of rows in the matrix to be quantized to the preset number of blocks; the second row number is less than or equal to the first row number.

[0101] Specifically, determine the number of rows in the matrix to be quantized. Based on the above description, assume it is... Okay, the preset number of blocks is .

[0102] Furthermore, the ratio of the number of rows in the matrix to be quantized to the preset number of blocks is rounded up to the nearest integer, which is expressed as: ,in, This indicates rounding up to the nearest integer.

[0103] like If the integer value is true, then the number of rows in each block matrix is ​​the same, meaning the number of rows in the second row equals the number of rows in the first row. For example, suppose... It is 8. If the value is 4, then the number in the first row is 4. =2, meaning each block matrix has 2 rows. Specifically, multiple block matrices can be obtained by determining each pair of rows as a block matrix according to the order of the row elements in the matrix to be quantized.

[0104] like If the value is not an integer, then the number of the first row is used for all block matrices except the last one. For example, suppose... It is 7. If the value is 4, then the number in the first row is 4. =2, meaning that all block matrices except the last one have 2 rows each. The last block matrix has 2 rows. The calculation shows that the number of the second row is 1, meaning that the number of the second row of the last block matrix is ​​1.

[0105] This embodiment provides a network model quantization method. In this embodiment, the number of rows included in each block matrix is ​​determined in a targeted manner according to the preset number of blocks.

[0106] Example 6

[0107] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the block importance corresponding to each block matrix based on the input data.

[0108] Figure 3 This is a schematic diagram of a network model quantization method provided in Example 6. Figure 3 As shown, it includes:

[0109] S301: Input the input data into the preset second derivative algorithm to output a second-order matrix.

[0110] Among them, the preset second derivative algorithm is a pre-set second-order algorithm, as shown in (1):

[0111] (1)

[0112] in, Indicates input data, This represents the transpose of the input data, where, This represents a second-order matrix.

[0113] S302, For each block matrix, calculate the average importance of each row element of the block matrix based on the second-order matrix; the average importance is the average value of the importance of the corresponding row elements in the block matrix.

[0114] It should be noted that each row in the block matrix has its own elements, and the average importance is the average of the importance of the corresponding row elements in the block matrix. For example, assuming the block matrix has two rows, the average importance of each row element is calculated based on the second-order matrix, resulting in two separate average importance values, each representing the average importance of the corresponding row elements.

[0115] S303, for each block matrix, calculate the block importance corresponding to the block matrix based on the corresponding average importance.

[0116] This embodiment provides a network model quantization method. In this embodiment, the average importance corresponding to each block matrix is ​​calculated. Since the average importance represents the average value of the corresponding row elements, the average importance can reflect the importance of each row element.

[0117] Example 7

[0118] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the average importance of each row element of the block matrix based on the second-order matrix.

[0119] Figure 4 This is a schematic diagram of a network model quantization method provided in Example 7. Figure 4 As shown, it includes:

[0120] S401: Input the inverse of the second-order matrix into the preset element importance algorithm to output the element importance of each element in the block matrix.

[0121] Among them, the preset element importance algorithm is a pre-set importance algorithm for each element, as shown in (2):

[0122] (2)

[0123] in, Represents a second-order matrix The inverse matrix, Represents the elements on the diagonal, representing Sensitivity; Indicates the th element in the block matrix Line 1 Column elements; Represents absolute value; Indicates the th element in the block matrix Line 1 The importance of each element in a column.

[0124] S402 determines the number of elements in each row of the block matrix.

[0125] S403 calculates the average importance of each row of elements in the block matrix based on the number of elements, the importance of each element, and a preset average importance algorithm.

[0126] Among them, the preset average importance algorithm refers to the algorithm for the average importance of each row of elements, as shown in (3):

[0127] (3)

[0128] In this formula, This represents the number of elements in each row of the partitioned matrix (i.e., the number of columns in the partitioned matrix). It's important to note that the number of columns in the partitioned matrix is ​​the same as the number of elements; where... Represents the first element in the block matrix. The average importance of row elements, where, ≥1.

[0129] This embodiment provides a network model quantization method. In this embodiment, the average importance corresponding to the block matrix is ​​averaged with the importance of the elements included in each row to obtain the average importance, which represents the overall importance.

[0130] Example 8

[0131] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the block importance corresponding to the block matrix based on the corresponding average importance.

[0132] Figure 5 This is a schematic diagram of a network model quantization method provided in Example 8. Figure 5 As shown, it includes:

[0133] S501 calculates the average value of each row element in the block matrix.

[0134] In one approach, a preset averaging algorithm is used to calculate the average value, as shown in equation (4):

[0135] (4)

[0136] in, Represents the first element in the block matrix. The average value of the row elements.

[0137] Specifically, the average value of each row is calculated based on the number of elements in each row of the block matrix.

[0138] S502 calculates the variance importance of each row element in the block matrix based on the average value.

[0139] In one approach, a preset variance importance algorithm is used to calculate the variance importance, wherein the preset variance importance algorithm is as shown in (5):

[0140] (5)

[0141] in, Represents the first element in the block matrix. The importance of the variance of row elements.

[0142] Specifically, the first Each element in the row and Input into equation (5), and output the first... The importance of the variance of row elements.

[0143] S503: Input the variance importance and average importance into the preset comprehensive importance algorithm to output the comprehensive importance of each row element in the block matrix.

[0144] The preset comprehensive importance algorithm is a pre-set algorithm that calculates the comprehensive importance of each row of elements.

[0145] In one approach, the pre-defined comprehensive importance algorithm is as shown in (6):

[0146] (6)

[0147] in, These are preset parameters. ; Represents the first element in the block matrix. Overall importance of row elements.

[0148] S504, calculate the importance of each block based on the number of rows in the block matrix and the overall importance of each block.

[0149] Specifically, for any given block matrix, the number of rows in that block matrix is ​​determined, denoted as: OK;

[0150] In one approach, the block importance is calculated using a preset block importance algorithm, where, when When the value is greater than 1, the preset block importance algorithm is as shown in equation (7):

[0151] (7)

[0152] in, The importance of a block is the value corresponding to the block matrix; where, This represents the total number of rows in the block matrix; Indicates the first row element, .

[0153] when When =1, the importance of the partition is... , This represents the overall importance of the elements in the first row.

[0154] This embodiment provides a network model quantization method. In this embodiment, the average value of each row element is first calculated, then the variance importance of each row element is calculated based on the average value, and then the comprehensive importance of each row element is calculated. Thus, the block importance is calculated based on the number of rows included in the block matrix. It can be seen that the block importance takes into account the comprehensive importance included in the block matrix, thereby making the block importance more accurate.

[0155] Example 9

[0156] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional method for calculating the quantization threshold corresponding to the module to be quantized based on the importance of each block, specifically including:

[0157] Step 1: Determine the highest importance of each block from the total importance of each block.

[0158] For example, suppose there are two block matrices, namely... and The corresponding block importance is as follows: and From this, the highest importance of a segment is determined, which is... .

[0159] Step 2: Calculate the product of the maximum block importance and the preset multiple, and determine the product as the quantization threshold corresponding to the module to be quantized.

[0160] The preset multiplier is a pre-set multiplier, which can be 0.6 or other multipliers; there are no restrictions here.

[0161] For example, for module 1 to be quantized, which corresponds to the two block matrices and block importance in step one above, assume... To maximize the importance of the block, the quantization threshold for module 1 to be quantized is calculated as follows: = *Preset multiplier.

[0162] This embodiment provides a network model quantization method. In this embodiment, the quantization threshold corresponding to the module to be quantized can be calculated based on the maximum block importance. In this embodiment, different modules to be quantized may obtain different maximum block importance, which in turn makes their respective quantization thresholds different, thus improving the flexibility for different modules to be quantized.

[0163] Example 10

[0164] This embodiment is a further refinement of any of the above embodiments. This embodiment provides an optional method for quantizing each block matrix based on the importance of each block and the quantization threshold to obtain the block quantization matrix corresponding to each block matrix. Specifically, it includes:

[0165] Method 1: For each block matrix, in response to the block importance of the block matrix being less than or equal to the quantization threshold, a preset first quantization algorithm is used to quantize the block matrix to obtain the block quantization matrix corresponding to the block matrix.

[0166] The quantization degree of the first quantization algorithm is preset to be the first quantization degree.

[0167] In one approach, the preset first quantization algorithm includes a preset first quantization parameter algorithm and a preset first quantization matrix algorithm.

[0168] The preset first quantization parameter algorithm is a pre-set algorithm for calculating the quantization parameters generated at the first quantization level. The quantization parameters are used in the quantization matrix.

[0169] The preset first quantization matrix algorithm is a pre-set algorithm for performing the first quantization level on the block matrix.

[0170] Taking module 1 to be quantized, and any block matrix within it as an example, the block importance of the block matrix is ​​less than or equal to the quantization threshold, i.e. ≤ Then, the following method is used to quantize the block matrix at the first quantization level.

[0171] In one approach, a preset first quantization algorithm is used to quantize the block matrix in blocks to obtain the block-quantized matrix corresponding to the block matrix, including:

[0172] The block matrix is ​​input into a preset first quantization parameter algorithm to obtain the first quantization parameter; the first quantization parameter is input into a preset first quantization matrix algorithm to obtain the block quantization matrix.

[0173] The algorithm for the first quantization parameter is shown in (8):

[0174] (8)

[0175] in, Indicates the first A block matrix, This is the scaling factor before quantization; The binary matrix before quantization. ,in, express Elements in; This is the scaling factor after quantization at the first quantization level; is the quantized binary matrix at the first quantization level; F represents the norm.

[0176] The first quantization parameter refers to the scaling factor and the binary matrix after quantization at the first quantization level.

[0177] Specifically, based on the above (8), the scaling factor and the binary matrix after quantization at the first quantization level are calculated as follows: Next The input is fed into the preset first quantization matrix algorithm to obtain the block quantization matrix corresponding to each block matrix.

[0178] The algorithm for the first quantization matrix is ​​shown in (9):

[0179] (9) =

[0180] in, Indicates the first The block quantized matrix is ​​obtained by quantizing the block matrix.

[0181] For example, for each block matrix, the corresponding scaling factor and quantized binary matrix under the first quantization level are calculated, and then the corresponding block quantization matrix is ​​calculated based on the scaling factor and quantized binary matrix under the first quantization level.

[0182] It should be noted that in this method, the first level of quantization is one quantization, that is, the corresponding number of quantizations is 1.

[0183] Method 2: For each block matrix, in response to the block importance of the block matrix being greater than the quantization threshold, the block matrix is ​​quantized by a preset second quantization algorithm and a preset number of quantizations to obtain the block quantization matrix corresponding to the block matrix; the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm.

[0184] In this case, the quantization degree of the first quantization algorithm is preset to be the second quantization degree, so the first quantization degree is less than the second quantization degree.

[0185] The preset second quantization algorithm includes a preset second quantization parameter algorithm and a preset second quantization matrix algorithm.

[0186] The preset second quantization parameter algorithm is a pre-set algorithm for calculating the quantization parameters generated at the second quantization level. The quantization parameters are used in the quantization matrix.

[0187] Among them, the preset second quantization matrix algorithm is a pre-set algorithm for quantizing the block matrix at the second quantization level.

[0188] Taking any block matrix of module 1 to be quantized as an example, if the block importance of this block matrix is ​​greater than the quantization threshold, then... ≤ Then, the following method is used to quantize the block matrix at the second quantization level.

[0189] In one approach, a preset second quantization algorithm and a preset number of quantization steps are used to quantize the block matrix in blocks to obtain the block-quantized matrix corresponding to the block matrix, including:

[0190] The block matrix is ​​input into the preset second quantization parameter algorithm to obtain the second quantization parameter corresponding to the preset number of quantizations; the second quantization parameter is input into the preset second quantization matrix algorithm to obtain the block quantization matrix.

[0191] The second quantization parameter refers to the scaling factor and the binary matrix after quantization at the second quantization level.

[0192] The preset quantization count refers to the number of times quantization is performed at the second quantization level. ≥2.

[0193] It should be noted that the second quantization algorithm in this application performs multiple quantizations based on a preset number of quantizations. The quantization parameters generated after each quantization are then combined to calculate the block quantization matrix at the second quantization level. Since the preset number of quantizations is ≥2, the calculated block quantization matrix has more bits and is more accurate.

[0194] The algorithm for the preset second quantization parameter is shown in (10):

[0195] (10)

[0196] in, For the first The scaling factor before sub-quantization. For the first The scaling factor corresponding to the next quantization. For the first The binary matrix corresponding to the quantization step No. The binary matrix before the next quantization ;In this formula, Indicates the preset number of quantizations; The scaling factor after quantization at the second quantization level, i.e., the... The scaling factor corresponding to the sub-quantization; For the second level of quantization, the quantized binary matrix is... The binary matrix corresponding to the next quantization.

[0197] Specifically, based on the above (10), the following can be calculated respectively: ,in, This yields the second quantization parameter corresponding to the preset number of quantizations.

[0198] Furthermore, the second quantization parameter corresponding to the preset number of quantizations is input into the preset second quantization matrix algorithm to obtain the block quantization matrix.

[0199] In one approach, the second quantization matrix algorithm is preset as shown in (11):

[0200] (11)

[0201] in, Indicates the first The block quantized matrix is ​​obtained by quantizing the block matrix.

[0202] For example, assume a preset number of quantizations. The block quantization matrix is ​​then calculated based on the following algorithm. Specifically, it is calculated based on formula (8). Then input it into equation (12) to obtain .

[0203] (12)

[0204] in, , It is the scaling factor corresponding to the second quantization.

[0205] Furthermore, , as well as Input into equation (13) to obtain the block quantization matrix.

[0206] (13)

[0207] It should be noted that, based on the above algorithm, the block quantization matrix after quantization of each block matrix can be calculated.

[0208] It should be noted that in this application, the matrix to be quantized is converted from floating-point values ​​to discrete values.

[0209] It should be noted that the preset number of quantization iterations can be determined in advance based on the importance of the block matrices in the matrix to be quantized. For a single module to be quantized, the preset number of quantization iterations can be one, or it can be preset separately for each block matrix in that module; there is no limitation here. In this embodiment, the preset number of quantization iterations is preset separately for each block matrix, which takes into account the importance of each block matrix, thus providing greater flexibility.

[0210] According to the above embodiments, each module to be quantized is quantized separately in this application. In order to obtain the corresponding quantization threshold for each module, different quantization methods are implemented based on the quantization threshold, so that each block matrix can be quantized using an appropriate method. The quantization method in this application uses different quantization degrees to distinguish them. If the block importance is greater than the quantization threshold, a preset second quantization algorithm with a second quantization degree will be used to calculate the block quantization matrix. In this application, for the matrix to be quantized in each module, in order to accurately obtain the quantization weight matrix, the matrix to be quantized is divided into blocks and regions. Then, the block importance corresponding to each block matrix is ​​calculated. The block importance is used to characterize the importance of each corresponding block matrix in the matrix to be quantized. Then, the quantization threshold is calculated based on the block importance, so that the quantization threshold changes with the module to be quantized, which is more flexible and more in line with the module to be quantized itself. In addition, quantizing the block matrix separately to obtain the block quantization matrix is ​​quantization from a local perspective, which subdivides the matrix to be quantized, so the block quantization matrix is ​​more accurate, and finally the quantized module is more accurate. The quantized model in this application not only ensures that the model has certain performance, but also reduces the computational requirements due to the precise quantization, while improving the accuracy and efficiency of image retrieval, thus providing strong support for the operation of such quantized models on edge devices.

[0211] It should be noted that the importance of a block reflects the degree of influence of the corresponding block matrix on the performance of the model to be quantized; if the degree of influence is greater, then the block matrix is ​​quantized using a quantization method with a higher degree of quantization.

[0212] This embodiment provides a network model quantization method. In this embodiment, different quantization algorithms are determined in a targeted manner according to the importance of the blocks in the block matrix. If the importance of the blocks is less than or equal to the quantization threshold, a preset first quantization algorithm is used for block quantization. If the importance of the blocks is greater than the quantization threshold, a preset second quantization algorithm and a preset number of quantizations are used to quantize the blocks in the block matrix. In this embodiment, the quantization degree of the preset first quantization algorithm is less than that of the preset second quantization algorithm, thereby making the preset second quantization algorithm more accurate. In addition, this embodiment allows for more flexible selection of the corresponding quantization algorithm based on the importance of the blocks.

[0213] Example 11

[0214] This embodiment is a further refinement of any of the above embodiments. This embodiment is an optional approach based on the generation of the quantized model by each quantized module, including:

[0215] Step 1: Input the query text into the text encoding module to output the text features corresponding to the query text; the text encoding module comes from the quantized model; the text encoding module includes at least one quantized module.

[0216] The text encoding module is responsible for receiving the query text and extracting the features contained in the query text.

[0217] Here, the query text is the text to be queried. The network model quantization method in this application retrieves target images that match the query text from an image feature database.

[0218] Step 2: Retrieve from the image feature database based on text features to obtain the target image corresponding to the query text.

[0219] The image feature database includes image features corresponding to at least one query image.

[0220] The target image refers to the image that matches the query text. It should be noted that the target image belongs to the query image set, and there must be at least one target image.

[0221] In one approach, a search is conducted in an image feature database based on text features to obtain a target image corresponding to the query text, specifically including:

[0222] Step 1: Calculate the similarity between the text features and at least one image feature in the image feature database.

[0223] Step 2: Determine the similarity that meets the similarity threshold as the target similarity.

[0224] Step 3: Retrieve the query image corresponding to the target similarity in the image feature database, and determine the retrieved query image as the target image corresponding to the query text.

[0225] Here, the similarity threshold is a pre-set threshold for determining the similarity between text features and image features, expressed as: , It can be ≥0.8, or other data; there are no restrictions here.

[0226] The image feature database includes N image features, where N is at least one, and the image feature database is represented as follows: ,in, This refers to the first image feature, that is, the image feature corresponding to the first query image; It refers to the Nth image feature, that is, the image feature corresponding to the Nth query image.

[0227] Specifically, the text encoding module extracts features from the query text to obtain the corresponding text features, which are then used... express.

[0228] Furthermore, if the image feature database contains N image features, the network model quantization device calculates the similarity between the text features and the N image features respectively, thus obtaining N similarity scores. These similarity scores are used in [the following context: ...]. express, . This refers to text features and the first The similarity of the image features, i.e., the similarity of the first image feature. 1 similarity. Among them, It refers to the first Image features.

[0229] in, .

[0230] in, represent column vectors, represent column vectors, represent The transpose of , Represents the magnitude of the vector.

[0231] Specifically, will The similarity between the two is defined as the target similarity, and the target similarity is expressed as... .

[0232] Furthermore, it was retrieved from the image feature database. The corresponding query image at that time is: target similarity is That is, text features and the first The similarity of the image features, i.e., the similarity of the corresponding query image to the i-th image feature. Zhang, thus the retrieved query image is the first one. Zhang Query Image, among which .

[0233] Furthermore, the first The image in question was determined to be the target image corresponding to the query text.

[0234] It should be noted that there can be at least one target similarity.

[0235] It should be noted that the language type of the query text is the same as that of the training data.

[0236] It should be noted that the text data can be in one language type or multiple language types; if it is in multiple language types, the query text can also be in the corresponding consistent language type, thus enabling the quantized model in this application to be applied to multiple language types, improving its adaptability.

[0237] This embodiment provides a network model quantization method. The text encoding module includes at least one quantized module, which is generated by replacing the corresponding matrix to be quantized with a corresponding quantization weight matrix. The quantization weight matrix is ​​generated based on multiple block quantization matrices. Each block quantization matrix is ​​obtained by quantizing the block matrix separately based on its corresponding block importance and the quantization threshold of the corresponding module to be quantized. This allows the matrix to be quantized according to local regions, and quantization is also performed based on the corresponding quantization threshold during the quantization process, making the quantization method more flexible. In this embodiment, the quantization threshold may differ depending on the module to be quantized, so the quantization threshold in this embodiment is also flexible for different modules to be quantized, making it more closely aligned with the target module. The quantization threshold is also flexible and not fixed. It is determined based on the importance of each block in the module to be quantized, so that the quantized weight matrix is ​​more closely matched to itself, resulting in better performance of the quantized module. In this embodiment, each block matrix is ​​obtained by dividing the matrix to be quantized. Therefore, this embodiment subdivides the matrix to be quantized to obtain block matrices, and then obtains the block quantization matrix of each block matrix, thereby further refining the matrix to be quantized and the quantization method, making the quantization weight matrix more accurate. This, in turn, makes the text encoding module, which includes at least one quantized module, more accurate. Therefore, with the support of the quantized module, inputting query text into the text encoding module can obtain more accurate text features, thereby retrieving the target image more accurately.

[0238] Example 12

[0239] This embodiment is a further refinement of the above embodiments, and includes the following:

[0240] At least one query image from the image database to be queried is input into the image encoding module to generate an image feature database; the image encoding module is derived from the quantized model; the image encoding module includes at least one quantized module.

[0241] The image encoding module is a module that receives the query image and extracts the features contained in the query image.

[0242] It should be noted that the quantization module in this embodiment is as described in Embodiment 1, and will not be repeated here.

[0243] The database to be queried includes at least one query image, which can be represented as including N query images. The query images are unlabeled images, meaning that the content or object of the query images is unknown.

[0244] The image feature database contains image features corresponding to N query images, that is, it includes N image features.

[0245] This embodiment also includes: inputting at least one query image from the image database to the image encoding module to generate an image feature database, specifically:

[0246] Step 1: Input at least one query image into the image encoding module to output the output feature embedding vector corresponding to each query image; the output feature embedding vector includes at least one feature vector.

[0247] Wherein, the output feature embedding vector is represented as That is, the first The output feature embedding vector corresponding to the query image.

[0248] in, It includes at least one feature vector, which can be represented as .

[0249] Step 2: Determine the largest feature vector from the output feature embedding vector as the image feature corresponding to the query image.

[0250] The largest eigenvector can also be the optimal eigenvector.

[0251] Specifically, will The largest feature vector is determined, which is the image feature corresponding to the query image.

[0252] Step 3: Generate an image feature database by combining the image feature sets corresponding to each query image; the image feature database includes the feature characteristics corresponding to each query image; the image feature database includes at least one image feature.

[0253] The image database to be queried includes at least one query image, which is unlabeled, meaning the content of the query image is unknown.

[0254] This embodiment provides a network model quantization method. In this embodiment, the image encoding module includes at least one quantized module. Since the quantized module has better performance, the image feature database generated based on the image encoding module is more accurate.

[0255] Figure 6 This is a schematic diagram of the overall quantization and application of a network model provided in Example Twelve.

[0256] like Figure 6 It includes a model to be quantized 601, a quantized model 602, a text encoding module 603, and an image encoding module 604.

[0257] The model to be quantized 601 includes a module to be quantized 605, and the quantized model 602 includes a quantized module 606.

[0258] In this process, the module to be quantized 605 is quantized to obtain the quantized module 606, which in turn quantizes the model to be quantized 601 into the quantized model 602.

[0259] exist Figure 6 In the process, the model to be quantized 601 is quantized to obtain the quantized model 602, which includes a text encoding module 603 and an image encoding module 604, wherein the text encoding module 603 and the image encoding module 604 include at least one quantized module 606.

[0260] Furthermore, the query text is input into the text encoding module 603 to obtain text features, and then the text features are retrieved in the image feature database to output the target image corresponding to the query text.

[0261] The image database to be queried is input into the image encoding module 604, which outputs an image feature database.

[0262] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0263] Example 13

[0264] The following is an embodiment of the device of this application. Figure 7 This is a schematic diagram of the network model quantization device provided in Embodiment Thirteen. The network model quantization device 700 includes the following modules:

[0265] Blocking module 701 is used to divide the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices; the module to be quantized comes from the model to be quantized; there is at least one module to be quantized;

[0266] The calculation module 702 is used to calculate the block importance corresponding to each block matrix based on the training dataset for each module to be quantized. The calculation module 702 is also used to calculate the quantization threshold corresponding to each module to be quantized based on the block importance. The training dataset includes at least one training data.

[0267] The quantization module 703 is used to quantize each block matrix based on the importance of each block and the quantization threshold for each block to be quantized, so as to obtain the block quantization matrix corresponding to each block matrix.

[0268] The generation module 704 is used to generate the quantization weight matrix of the corresponding module to be quantized based on each block quantization matrix.

[0269] The replacement module 705 is used to replace the quantization matrix in each quantization module with the corresponding quantization weight matrix to obtain the quantized module corresponding to each quantization module. The generation module 704 is also used to generate a quantized model based on each quantized module.

[0270] Calculation module 702, when calculating the block importance corresponding to each block matrix based on the training dataset, is specifically used for:

[0271] The input data for each module to be quantized is obtained based on the training dataset.

[0272] The importance of each block is calculated based on the input data.

[0273] The computation module 702, when obtaining the input data corresponding to each module to be quantized based on the training dataset, is specifically used for:

[0274] The training dataset is input into the model to be quantized, and then propagated forward through the neural network to obtain intermediate data for each module to be quantized; the number of intermediate data is equal to the number of training data.

[0275] The input data for each module to be quantized is constructed based on the intermediate data.

[0276] The block segmentation module 701, when dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, is specifically used for:

[0277] If the partitioning method is free partitioning, then one row of elements in the matrix to be quantized is determined as a partition matrix, so as to obtain multiple partition matrices; the number of rows in the matrix to be quantized is equal to the number of partition matrices.

[0278] The block segmentation module 701, when dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices, is specifically used for:

[0279] If the block partitioning method is regular block partitioning, then obtain the preset number of blocks;

[0280] Multiple block matrices are obtained based on the number of rows of the matrix to be quantized and the preset number of blocks; the number of rows of the block matrices other than the last block matrix is ​​the first row number, and the number of rows of the last block matrix is ​​the second row number; the first row number is the integer up of the ratio of the number of rows of the matrix to be quantized to the preset number of blocks; the second row number is less than or equal to the first row number.

[0281] Calculation module 702, when calculating the block importance corresponding to each block matrix based on the input data, is specifically used for:

[0282] Input data is fed into a preset second-order derivative algorithm to output a second-order matrix;

[0283] For each block matrix, the average importance of each row element in the block matrix is ​​calculated based on the second-order matrix; the average importance is the average of the importance of the corresponding row elements in the block matrix.

[0284] For each block matrix, the block importance is calculated based on the corresponding average importance.

[0285] Calculation module 702, when calculating the average importance of each row element of the block matrix based on the second-order matrix, is specifically used for:

[0286] The inverse of the second-order matrix is ​​input into the preset element importance algorithm to output the element importance of each element in the block matrix;

[0287] Determine the number of elements in each row of the block matrix;

[0288] The average importance of each element in the block matrix is ​​calculated based on the number of elements, the importance of each element, and a preset average importance algorithm.

[0289] Calculation module 702, when calculating the block importance corresponding to the block matrix based on the corresponding average importance, is specifically used for:

[0290] Calculate the average value of each row element in the block matrix;

[0291] The variance importance of each row element in the block matrix is ​​calculated based on the average numerical value.

[0292] Input the variance importance and average importance into the preset comprehensive importance algorithm to output the comprehensive importance of each row element in the block matrix;

[0293] The importance of a block is calculated based on the number of rows in the block matrix and the overall importance of each block.

[0294] Calculation module 702, when calculating the quantization threshold corresponding to the module to be quantized based on the importance of each block, is specifically used for:

[0295] Determine the most important block from among all block importance scores;

[0296] Calculate the product of the maximum block importance and a preset multiple, and determine the product as the quantization threshold corresponding to the module to be quantized.

[0297] Quantization module 703, when quantizing each block matrix based on the importance of each block and the quantization threshold to obtain the block quantization matrix corresponding to each block matrix, is specifically used for:

[0298] For each block matrix, in response to the block importance of the block matrix being less than or equal to the quantization threshold, a preset first quantization algorithm is used to quantize the block matrix to obtain the block quantization matrix corresponding to the block matrix.

[0299] For each block matrix, in response to the block importance of the block matrix being greater than the quantization threshold, a preset second quantization algorithm and a preset number of quantizations are used to quantize the block matrix to obtain the block quantization matrix corresponding to the block matrix; the quantization degree of the preset first quantization algorithm is less than the quantization degree of the preset second quantization algorithm.

[0300] This embodiment provides a network model quantization device, which further includes an output module and a retrieval module;

[0301] After generating the quantized model based on each quantized module, the output module is used to input the query text into the text encoding module to output the text features corresponding to the query text; the text encoding module comes from the quantized model; the text encoding module includes at least one quantized module.

[0302] The retrieval module is used to search the image feature database based on text features to obtain the target image corresponding to the query text.

[0303] The generation module 704 is also used to: input at least one query image from the image database to be queried into the image encoding module to generate an image feature database; the image encoding module is derived from the quantized model; the image encoding module includes at least one quantized module.

[0304] For a description of the features in the embodiment corresponding to the network model quantization device, please refer to the relevant description of the embodiment corresponding to the network model quantization method, which will not be repeated here.

[0305] Figure 8A schematic diagram of the structure of the electronic device provided in this application. Figure 8 As shown, the electronic device 800 provided in this embodiment includes at least one processor 801 and a memory 802. Optionally, the electronic device 800 further includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0306] In the specific implementation process, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to execute the above-described network model quantization method embodiment.

[0307] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0308] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0309] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0310] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0311] Embodiments of this application also provide a computer-readable storage medium storing a computer program configured to execute the steps in any of the network model quantization method embodiments described above at runtime.

[0312] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0313] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the network model quantization method embodiments described above.

[0314] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described network model quantization method embodiments.

[0315] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0316] The foregoing has provided a detailed description of a network model quantization method provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A method for quantizing network models, characterized in that, include: The matrix to be quantized in the module to be quantized is divided into blocks to obtain multiple block matrices; The module to be quantized originates from the model to be quantized; The module to be quantized is at least one; For each module to be quantized, the training dataset is input into the model to be quantized, and the data is propagated forward through the neural network to obtain intermediate data corresponding to each module to be quantized; the number of intermediate data is equal to the number of training data. Based on the intermediate data, construct the input data corresponding to each module to be quantized; The input data is fed into a preset second-order derivative algorithm to output a second-order matrix; For each block matrix, the average importance of each row element in the block matrix is ​​calculated based on the second-order matrix; the average importance is the average value of the importance of the corresponding row elements in the block matrix. For each block matrix, the block importance corresponding to each block matrix is ​​calculated based on the corresponding average importance. The importance of each block matrix represents the degree of influence of each block matrix on the performance of the module to be quantized. Based on the largest block importance among the block importance values, the quantization threshold corresponding to the module to be quantized is calculated, and the training dataset includes at least one training data. For each of the block matrices, if the block importance of the block matrix is ​​less than or equal to the quantization threshold, a preset first quantization algorithm is used to quantize the block matrix to obtain the block quantization matrix corresponding to the block matrix. In response to the block importance of the block matrix being greater than the quantization threshold, the block matrix is ​​quantized by a preset second quantization algorithm and a preset number of quantizations to obtain the block quantization matrix corresponding to the block matrix; The quantization degree of the preset first quantization algorithm is less than that of the preset second quantization algorithm; Based on each of the block quantization matrices, generate the corresponding quantization weight matrix after quantization of the module to be quantized. For each module to be quantized, the matrix to be quantized in the module to be quantized is replaced with the corresponding quantization weight matrix to obtain the quantized module corresponding to each module to be quantized, and a quantized model is generated based on each quantized module. The query text is input into the text encoding module to output the text features corresponding to the query text; the text encoding module is derived from the quantized model; the text encoding module includes at least one quantized module; Based on the text features, an image feature database is searched to obtain the target image corresponding to the query text.

2. The method according to claim 1, characterized in that, The step of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices includes: If the partitioning method is free partitioning, then one row of the matrix to be quantized is determined as a partitioning matrix to obtain multiple partitioning matrices; the number of rows of the matrix to be quantized is equal to the number of partitioning matrices.

3. The method according to claim 1, characterized in that, The step of dividing the matrix to be quantized contained in the module to be quantized into blocks to obtain multiple block matrices includes: If the block partitioning method is regular block partitioning, then obtain the preset number of blocks; Multiple block matrices are obtained based on the number of rows of the matrix to be quantized and the preset number of blocks; the number of rows of the other block matrices except the last block matrix is ​​the first row number, and the number of rows of the last block matrix is ​​the second row number; the first row number is the integer up of the ratio of the number of rows of the matrix to be quantized to the preset number of blocks; the second row number is less than or equal to the first row number.

4. The method according to claim 1, characterized in that, The calculation of the average importance of each row element of the block matrix based on the second-order matrix includes: The inverse of the second-order matrix is ​​input into a preset element importance algorithm to output the element importance of each element in the block matrix; Determine the number of elements in each row of the block matrix; The average importance of each row of the block matrix is ​​calculated based on the number of elements, the importance of each element, and a preset average importance algorithm.

5. The method according to claim 1, characterized in that, The step of calculating the block importance corresponding to the block matrix based on the corresponding average importance includes: Calculate the average value of each row element in the block matrix; Calculate the variance importance of each row element in the block matrix based on the average value; The variance importance and the average importance are input into a preset comprehensive importance algorithm to output the comprehensive importance corresponding to each row element in the block matrix; The importance of a block is calculated based on the number of rows in the block matrix and the overall importance of each block.

6. The method according to claim 1, characterized in that, The step of calculating the quantization threshold corresponding to the module to be quantized based on the largest block importance among all the block importances includes: Determine the most important block from among the aforementioned block importance values; Calculate the product of the maximum block importance and a preset multiple, and determine the product as the quantization threshold corresponding to the module to be quantized.

7. The method according to claim 1, characterized in that, The method further includes: At least one query image from the image database to be queried is input into the image encoding module to generate an image feature database; the image encoding module is derived from the quantized model; the image encoding module includes at least one quantized module.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the network model quantization method as described in any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the network model quantization method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the network model quantization method as described in any one of claims 1 to 7.

Images