Quantization method and quantization apparatus

Abstract

The present application provides a quantization method and a quantization apparatus. The method comprises: performing singular value decomposition on a first matrix composed of word embedding vectors, so as to obtain a maximum eigenvalue of each block matrix among M block matrices; on the basis of the maximum eigenvalue of each block matrix, determining the number of quantization bits for each block matrix; performing quantization on each block matrix to obtain quantized block matrices; and, merging the quantized block matrices to obtain a quantized first matrix. The maximum eigenvalue of each block matrix can to some extent characterize the amount of semantic information carried in each block matrix, and the number of quantization bits for each block matrix can be allocated on the basis of the amount of semantic information carried in each block matrix, thereby allowing different bit quantities to be allocated to block matrices carrying different semantic information, achieving differentiated bit quantity allocation and reducing loss of semantic information, and accordingly improving user experience.

Description

Quantization methods and quantization devices

[0001] This application claims priority to Chinese Patent Application No. 202410539750.X, filed with the State Intellectual Property Office of China on April 26, 2024, entitled "Quantization Method and Quantization Device", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of general electronics technology, and specifically to quantization methods and quantization devices in the field of electronics technology. Background Technology

[0003] With the development of Natural Language Processing (NLP) technology, an increasing number of intelligent services are using natural language as their medium. Specifically, users can issue commands via natural language, and upon receiving these commands, robots can understand the user's intent using NLP and execute the corresponding tasks. While robots can utilize artificial intelligence (AI) capabilities through NLP to understand user intent, deploying AI capabilities directly on the robot would result in high costs. Therefore, AI capabilities can be deployed on edge servers. Edge servers can process the word embedding vectors obtained from the NLP input to the robot and send the quantized word embedding vectors back to the robot. The robot then understands the user's intent using these word embedding vectors and executes the corresponding tasks. Current technologies, when quantizing word embedding vectors, assume that each bit is equally important and allocate quantization resources evenly. However, in reality, the semantic information distribution of word embedding vectors is not uniform, and existing quantization schemes result in significant semantic loss, ultimately affecting the accuracy of natural language processing and leading to a poor user experience. Summary of the Invention

[0004] This application provides a quantization method and quantization apparatus that can reduce the loss of quantization semantic information, improve quantization performance, and thus enhance user experience.

[0005] In a first aspect, a quantization method is provided, comprising: obtaining a first matrix composed of word embedding vectors of natural language; performing singular value decomposition on the first matrix to obtain the maximum eigenvalue of each of M feature value blocks, wherein the M feature value blocks correspond one-to-one with M block matrices of the first matrix; determining the number of quantization bits of each block matrix corresponding to each feature value block based on the maximum eigenvalue of each feature value block; quantizing each block matrix based on the number of quantization bits of each block matrix to obtain each quantized block matrix; obtaining a quantized first matrix based on each quantized block matrix; and sending the quantized first matrix; wherein M is a positive integer, and the number of quantization bits of each block matrix is ​​greater than or equal to 0.

[0006] In the above scheme, singular value decomposition can be performed on the first matrix composed of word embedding vectors to obtain the maximum eigenvalue of each of the M block matrices. The number of quantization bits for each block matrix can be determined based on the maximum eigenvalue of each block matrix. Each block matrix is ​​then quantized to obtain each quantized block matrix. Finally, the quantized block matrices are merged to obtain the first quantized matrix. In other words, the maximum eigenvalue of each block matrix can, to some extent, characterize the amount of semantic information carried in each block matrix. The number of quantization bits can be allocated to each block matrix based on the amount of semantic information carried, thereby allocating different numbers of bits to block matrices carrying different semantic information. This allows for differentiated bit allocation, reduces semantic information loss, avoids the problem of large semantic information loss caused by uniform bit allocation, improves quantization performance, and ultimately enhances the user experience.

[0007] Optionally, the magnitude of the eigenvalue can, to some extent, characterize the magnitude of the semantic information carried by the block matrix. The larger the eigenvalue, the more semantic information the block matrix carries; the smaller the eigenvalue, the less semantic information the block matrix carries. Therefore, electronic devices can allocate the number of quantization bits to each block matrix according to the magnitude of the maximum eigenvalue of each eigenvalue block.

[0008] Optionally, singular value decomposition is performed on the first matrix to obtain the maximum eigenvalue of each eigenvalue block in the M eigenvalue blocks, including: performing singular value decomposition on the first matrix to obtain a matrix composed of eigenvectors of the first matrix, a diagonal matrix composed of eigenvalues ​​of the first matrix, and a unitary matrix.

[0009] Optionally, the feature matrix composed of the eigenvectors of the first matrix can be divided into M eigenvalue blocks. The M eigenvalue blocks correspond to the M block matrices of the first matrix. In other words, the M block matrices of the first matrix can be determined based on the M eigenvalue blocks.

[0010] Optionally, the quantization method described above can be performed by an electronic device, for example, the electronic device can send the quantized first matrix to the other end.

[0011] In some possible implementations, the quantization method further includes: determining the quantization error of each block matrix corresponding to each feature value block; wherein, determining the number of quantization bits of each block matrix corresponding to each feature value block based on the maximum eigenvalue of each feature value block includes: determining the number of quantization bits of each block matrix based on the maximum eigenvalue of each feature value block and the quantization error of each block matrix.

[0012] In the above scheme, the number of quantization bits of each block matrix can be determined not only based on the maximum eigenvalue of each eigenvalue block, but also based on the quantization error of each block matrix. This can improve the accuracy of determining the number of quantization bits of each block matrix.

[0013] Optionally, the quantization error of each block matrix corresponding to each feature block is determined, including: determining the quantization error of each block matrix using each block matrix before quantization and each block matrix after quantization.

[0014] In some possible implementations, the quantization method further includes: obtaining the size of each block matrix and the total number of quantized bits;

[0015] The step of determining the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block and the quantization error of each block matrix includes:

[0016] The number of quantization bits for each block matrix is ​​determined based on the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, and the total number of quantization bits.

[0017] In the above scheme, when determining the number of quantization bits for each block matrix, the number of quantization bits for each block matrix can be determined by referring to at least one of the following: the quantization error of each block matrix, the total number of bits, the size of each block matrix, or the largest eigenvalue of each eigenvalue block. This makes the determined number of quantization bits for each block matrix more accurate.

[0018] Optionally, obtaining the size of each block matrix may include: based on the size of the first matrix and the total number of block matrices, for example, the size of the first matrix is ​​n×D, the total number of block matrices is M, and the number of columns occupied by each block matrix is ​​n×(D / M), where D / M=L. For example, the total number of block matrices may be a preset value.

[0019] In some possible implementations, determining the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, and the total number of quantization bits includes:

[0020] The number of quantization bits for the m-th block matrix is ​​determined according to formula (1).

[0021] Where M is the number of block matrices, M0 is the quantization boundary, B is the total number of bits, L is the number of columns in the m-th block matrix, and λ m c is the largest eigenvalue of the m-th eigenvalue block. m The scalar quantization error related value of the m-th block matrix, c k The value related to the scalar quantization error of the k-th block matrix. This is for floor function.

[0022] Optional, in, Let be the scalar quantization error of the m-th block matrix.

[0023] Optional, in, denoted as the scalar quantization error of the k-th block matrix.

[0024] In some possible implementations, quantizing each block matrix according to the number of quantization bits of each block matrix to obtain each quantized block matrix includes:

[0025] The number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix are determined based on the number of quantization bits for each block matrix.

[0026] Each block matrix is ​​scalar quantized according to the number of quantization bits of each block matrix, to obtain each block matrix after scalar quantization;

[0027] Each block matrix is ​​vector quantized according to the number of quantization bits of each block matrix, to obtain each block matrix after vector quantization;

[0028] Each quantized block matrix is ​​obtained based on each block matrix after scalar quantization and each block matrix after vector quantization.

[0029] Optionally, the sum of the number of quantized bits in scalar quantization of each block matrix and the number of quantized bits in vector quantization of each block is less than or equal to the number of quantized bits in each block matrix.

[0030] Optionally, determining the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix based on the number of quantization bits for each block matrix includes: determining the number of quantization bits for vector quantization and the number of quantization bits for scalar quantization of each block matrix based on the number of quantization bits for each block matrix, the quantization error for scalar quantization of each block matrix, and the quantization error for vector quantization of each block matrix.

[0031] In some possible implementations, determining the scalar quantization bit count and the vector quantization bit count of each block matrix based on the number of quantization bits of each block matrix includes:

[0032] Under the constraint that the sum of the number of quantized bits for scalar quantization and the number of quantized bits for vector quantization of each block matrix is ​​less than or equal to the number of quantized bits for each block matrix, the value at which the quantization error of each block matrix is ​​minimized after scalar quantization based on the number of quantized bits for scalar quantization and vector quantization based on the number of quantized bits for vector quantization of each block matrix is ​​determined as the number of quantized bits for scalar quantization and the number of quantized bits for vector quantization of each block matrix.

[0033] In the above scheme, when determining the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix, it is necessary to determine them under certain constraints. These constraints are that the number of quantization bits for vector quantization and the number of quantization bits for scalar quantization of each block matrix are less than or equal to the number of quantization bits for each block matrix. This is to avoid the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization determined for a certain block matrix exceeding the number of quantization bits for that block matrix. Under these constraints, the electronic device needs to find the minimum value of the sum of the errors of scalar quantization and vector quantization, and determine the minimum value of the sum of the errors of scalar quantization and vector quantization as the number of quantization bits for vector quantization and the number of quantization bits for scalar quantization.

[0034] In some possible implementations, quantizing each block matrix according to the number of quantization bits of each block matrix to obtain each quantized block matrix includes: quantizing each block matrix using a first codebook according to the number of quantization bits of each block matrix to obtain each quantized block matrix.

[0035] In the above scheme, each block matrix can be quantized using the first codebook to obtain each quantized block matrix.

[0036] Optionally, the first codebook can be a preset original codebook, or a codebook obtained based on the preset original codebook.

[0037] Optionally, the quantization codebooks of the M block matrices can all be the first codebook.

[0038] In some possible implementations, the codewords included in the first codebook are used more frequently than a preset frequency; the quantization method further includes: sending indication information, the indication information being used to indicate the first codebook.

[0039] In the above scheme, during the quantization process using the first codebook, it is necessary to indicate the first codebook used to the other end to avoid the other end being unable to determine which codebook to use to recover the matrix before quantization.

[0040] In some possible implementations, the first codebook is obtained based on a second codebook, which includes the first codebook, and the error between the first codebook and the second codebook is less than a preset value.

[0041] Optionally, the first codebook is an adaptive codebook, and the second codebook is the original codebook. The adaptive codebook can be obtained from the original codebook, so that the error between the obtained adaptive codebook and the original codebook is less than a preset value.

[0042] Secondly, a quantization device is provided, comprising a processing unit and a communication unit. The processing unit is configured to: acquire a first matrix composed of word embedding vectors of natural language; perform singular value decomposition on the first matrix to obtain the maximum eigenvalue of each of M eigenvalue blocks, wherein the M eigenvalue blocks correspond one-to-one with M block matrices of the first matrix; determine the number of quantization bits for each block matrix corresponding to each eigenvalue block based on the maximum eigenvalue of each eigenvalue block; quantize each block matrix based on the number of quantization bits for each block matrix to obtain each quantized block matrix; and obtain a quantized first matrix based on each quantized block matrix. The communication unit is configured to transmit the quantized first matrix.

[0043] Where M is a positive integer, and the number of quantization bits in each block matrix is ​​greater than or equal to 0.

[0044] In some possible implementations, the processing unit is further configured to: determine the quantization error of each block matrix corresponding to each feature value block; wherein, the processing unit is specifically configured to: determine the number of quantization bits of each block matrix based on the maximum eigenvalue of each feature value block and the quantization error of each block matrix.

[0045] In some possible implementations, the processing unit is further configured to obtain the size and total number of quantized bits of each block matrix; specifically, the processing unit is configured to: determine the number of quantized bits of each block matrix based on the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, and the total number of quantized bits.

[0046] In some possible implementations, the processing unit is specifically used to: determine the number of quantization bits of the m-th block matrix according to formula (1).

[0047] Where M is the number of block matrices, M0 is the quantization boundary, B is the total number of bits, L is the number of columns in the m-th block matrix, and λ m c is the largest eigenvalue of the m-th eigenvalue block. m The scalar quantization error related value of the m-th block matrix, c k The value related to the scalar quantization error of the k-th block matrix. This is for floor function.

[0048] In some possible implementations, the processing unit is specifically configured to: determine the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix based on the number of quantization bits for each block matrix; perform scalar quantization on each block matrix based on the number of quantization bits for scalar quantization of each block matrix to obtain each block matrix after scalar quantization; perform vector quantization on each block matrix based on the number of quantization bits for vector quantization of each block matrix to obtain each block matrix after vector quantization; and obtain each block matrix after quantization based on each block matrix after scalar quantization and each block matrix after vector quantization.

[0049] In some possible implementations, the processing unit is specifically configured to: under the constraint that the sum of the number of quantization bits of scalar quantization and the number of quantization bits of vector quantization of each block matrix is ​​less than or equal to the number of quantization bits of each block matrix, perform scalar quantization based on the number of quantization bits of scalar quantization of each block matrix, and perform vector quantization based on the number of quantization bits of vector quantization of each block matrix, determine the value at which the quantization error of each quantized block matrix is ​​minimized as the number of quantization bits of scalar quantization and the number of quantization bits of vector quantization of each block matrix.

[0050] In some possible implementations, the processing unit is specifically used to: quantize each block matrix using a first codebook according to the number of quantization bits of each block matrix, to obtain each quantized block matrix.

[0051] In some possible implementations, the codewords included in the first codebook are used more frequently than a preset frequency; the communication unit is further configured to: send indication information, the indication information being used to indicate the first codebook.

[0052] In some possible implementations, the first codebook is obtained based on a second codebook, which includes the first codebook, and the error between the first codebook and the second codebook is less than a preset value.

[0053] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor, wherein the memory is used to store a computer program; and the processor is used to cause the communication device to execute the quantization method described in any one of the first aspects when the computer program is invoked.

[0054] Fourthly, embodiments of this application provide a chip system including a processor coupled to a memory, wherein the processor executes a computer program stored in the memory to implement the quantization method described in any of the above aspects.

[0055] The chip system can be a single chip or a chip module composed of multiple chips.

[0056] Fifthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the quantization method described in any one of the first aspects above.

[0057] Sixthly, embodiments of this application provide a computer program product that, when run on a computer, causes the computer to execute the quantization method described in any one of the first aspects.

[0058] It is understood that the beneficial effects of the second to sixth aspects mentioned above can be found in the relevant descriptions of the above aspects, and will not be repeated here. Attached Figure Description

[0059] Figure 1 is a schematic diagram of the application scenario provided in the embodiments of this application.

[0060] Figure 2 is a schematic diagram of another application scenario provided by an embodiment of this application.

[0061] Figure 3 is a schematic diagram of quantization provided in an embodiment of this application.

[0062] Figure 4 is a schematic diagram of the architecture for sending quantized word embedding vectors provided in an embodiment of this application.

[0063] Figure 5 is a schematic diagram of the quantization method provided in the embodiments of this application.

[0064] Figure 6 is a schematic diagram illustrating the effect of using the first codebook provided in the embodiment of this application.

[0065] Figure 7 is a schematic diagram of the information structure provided in an embodiment of this application.

[0066] Figure 8 is a schematic diagram of the process of using the first codebook provided in an embodiment of this application.

[0067] Figures (a) and (b) in Figure 9 are schematic diagrams illustrating the effects of the quantization method provided in the embodiments of this application.

[0068] Figure 10 is a schematic diagram of the quantization process provided in an embodiment of this application.

[0069] Figure 11 is a schematic diagram of another quantization process provided in an embodiment of this application.

[0070] Figure 12 is a schematic block diagram of the quantization device provided in an embodiment of this application.

[0071] Figure 13 is a schematic block diagram of another quantization device provided in an embodiment of this application. Detailed Implementation

[0072] It should be understood that the methods, situations, categories, and classifications of embodiments in this application are for the convenience of description only and should not constitute a special limitation. Various methods, categories, situations, and features in embodiments can be combined without contradiction.

[0073] It should also be understood that the terms "first," "second," and "third" in the embodiments of this application are for distinction only and should not constitute any limitation on this application. It should also be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0074] In this application embodiment, the number of nouns, unless otherwise specified, refers to "singular nouns or plural nouns," that is, "one or more." "At least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, or B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the related objects before and after are in an "or" relationship. For example, A / B means: A or B. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c means: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c can be single or multiple.

[0075] Furthermore, the terms "comprising" and "having," and any variations thereof, used in the description of this application are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0076] It should be noted that in the embodiments of this application, the words "exemplary" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0077] The methods and apparatus provided in this application are based on the same or similar technical concepts. Since the methods and apparatus solve problems in similar ways, the implementation of the apparatus and methods can refer to each other, and repeated parts will not be described again.

[0078] Figure 1 is a schematic diagram of an application scenario applicable to an embodiment of this application. The application scenario in Figure 1 is to convert human natural language into a digital form that electronic devices can recognize. As shown in Figure 1, human natural language can be input into a tokenizer, which can output tokens. For example, it can output 's' words, namely token 1, token 2, ..., token s. The tokens can be input into an embedding model, which can output word embedding vectors. For example, it can output n word embedding vectors, namely embedding1, embedding 2, ..., embedding n, where s and n are positive integers. Electronic devices can recognize word embedding vectors and capture the meaning of human language based on the word embedding vectors, thereby performing corresponding tasks. For example, the electronic device can be a robot, and the robot can perform corresponding tasks. Typically, the word segmenter and embedding model described above require artificial intelligence (AI) models for implementation. However, due to the high inference cost of AI models, it's impossible to deploy these models directly on the robot performing the task. To reduce costs, in some scenarios, the word segmenter and embedding models can be distributed and deployed on edge servers. The edge server can obtain word embedding vectors from these models. As shown in Figure 2, the word segmenter and embedding model from Figure 1 can be deployed on an edge server. After obtaining the word embedding vectors, the edge server can send them to the robot. The robot can then recognize the meaning of human language based on the word embedding vectors and perform corresponding tasks. During the process of sending word embedding vectors to the robot via a communication network, the edge server can perform quantization, source coding, channel coding, modulation, and other processing on the word embedding vectors before sending them to the robot. The robot can then use AI models, such as classifiers, to understand the semantics contained in the word embedding vectors and execute corresponding commands. When edge servers quantize word embedding vectors, they can convert continuous analog signals into discrete digital signals based on the quantized codebook. During quantization, the amplitude of the analog signal is divided into a series of quantization levels, each corresponding to a quantization value. Quantization may introduce errors due to quantization precision or the dynamic range of the model signal. Therefore, quantization can reduce the amount of data transmitted, but it can also affect the transmission quality.In existing quantization methods, each quantization bit is equally important. When the semantic information of the word embedding vector is unevenly distributed, it can lead to the introduction of large noise. As shown in Figure 3, scalar quantization is used. As shown in Figure 3, values ​​in the range of -4.0 to 4.0 need to be quantized into 3 bits. For example, values ​​between -4.0 and -3.0 are quantized into 000, values ​​between -3.0 and -2.0 are quantized into 001, and values ​​between -2.0 and -1.0 are quantized into 010. Therefore, quantization error is introduced. In other words, scalar quantization divides the data across the entire dynamic range into uniformly spaced intervals, with each range corresponding to a bit string value. When the semantic information of the word embedding vector is unevenly distributed, for example, suppose the word embedding vector has only one value between -4.0 and 0, which falls between -2.0 and -1.0. Therefore, this value is quantized as 010. However, since there are no other values ​​between -4.0 and 0, but since 000 corresponds to -4.0 to -3.0, 001 corresponds to -3.0 to -2.0, and 011 corresponds to -1.0 to 0, these three bit strings 000, 001, and 011 are not used. There are many possible values ​​between 0 and 4.0, but due to the limitation of 3 bits, these values ​​between 0 and 4.0 can only be quantized to a maximum of 4 bits: 100, 101, 110, and 111. This results in a significant loss of semantic information in word embedding vectors within the range of 0 to 4.0. In other words, word embedding vectors carry less semantic information in the range of -4.0 to 0 and more semantic information in the range of 0 to 4.0. However, the number of bit strings allocated between -4.0 and 0 is the same as the number of bit strings allocated between 0 and 4.0. Since -4.0 to 4.0 occupies a total of 3 bits, this leads to a significant loss of semantic information in word embedding vectors that carry more semantic information.

[0079] To better quantize word embedding vectors, vector quantization is often used. Unlike scalar quantization, vector quantization quantizes word embedding vectors across multiple dimensions, also known as multidimensional quantization. Vector quantization simultaneously maps all elements of the word embedding vector (such as frequency and amplitude) to discrete values ​​in different dimensions. It can maintain a high compression ratio while preserving the relevance and continuity of the word embedding vector as much as possible. Furthermore, the more bits used in vector quantization, the smaller the data differences caused by quantization. Each quantization bit in vector quantization is equally important; uneven distribution of semantic information in the word embedding vector can lead to significant semantic loss, thus affecting the user experience.

[0080] It should be noted that, for ease of description, the above example illustrates an edge server sending a quantized word embedding vector to a robot. This application can be applied to any quantization method in scenarios involving the transmission of word embedding vectors. For example, it is applicable to any scenario where device A transmits a quantized word embedding vector to device B, as shown in Figure 4. Figure 4 illustrates a scenario where device A sends a quantized word embedding vector to device B. As shown in Figure 4, device A can be an encoder, and device B can be a decoder. Device A can quantize the word embedding vector according to the method provided in this application's embodiments. After quantization, the vector enters the source encoder for processing. After processing by the source encoder, it can be channel encoded by a channel encoder, then modulated by a modulator, amplified by an amplifier, and then transmitted to device B through the channel between device A and device B. Device B's radio frequency front end receives the vector, which then passes through a demodulator and a discrete channel decoder before entering the source decoder. After the decoder, the recovered word embedding vector can be obtained. The quantization process of device A can use the codebook and perform quantization according to the quantization method provided in the embodiments of this application. The source decoder of device B can recover the word embedding vector according to the codebook.

[0081] To reduce quantization errors, in this embodiment, singular value decomposition can be performed on the first matrix composed of word embedding vectors to obtain the maximum eigenvalue of each of the M block matrices. The number of quantization bits for each block matrix can be determined based on the maximum eigenvalue of each block matrix. Each block matrix is ​​then quantized to obtain each quantized block matrix. Finally, the quantized block matrices are merged to obtain the first quantized matrix. In other words, the maximum eigenvalue of each block matrix can, to some extent, characterize the amount of semantic information carried in each block matrix. The number of quantization bits can be allocated to each block matrix based on the amount of semantic information carried, thereby allocating different numbers of bits to block matrices carrying different semantic information. This allows for differentiated bit allocation, avoiding the problem of significant loss of semantic information caused by uniform bit allocation, improving quantization performance, and thus enhancing the user experience.

[0082] Existing quantization schemes measure quantization effectiveness using mean squared error (MSE). This method focuses on the difference between the transmitted and received word embedding vectors. However, there may be semantic relationships between the transmitted and received word embedding vectors. Using MSE to measure quantization effectiveness may lead to inaccurate results; in other words, MSE cannot accurately represent the quantization effect. Therefore, point-wise inner-product loss (PIP) can be used to measure the semantic loss caused by quantization of word embedding vectors. For example, the point-wise inner-product loss can be... Where is the first matrix composed of word embedding vectors. To recover the matrix from the first quantized matrix, PIP not only considers the quantization error before and after quantization, but also fully considers whether the relative relationships between words, i.e., semantic associations, have changed before and after quantization. Therefore, PIP can better measure the accuracy of quantization.

[0083] The quantization method 500 in this embodiment is described below with reference to Figure 5. The quantization method 500 can be executed by an electronic device, such as the edge server shown in Figure 2. The quantization method 500 includes:

[0084] S510, the electronic device acquires the first matrix composed of word embedding vectors of natural language.

[0085] Optionally, the electronic device can receive natural language input and use a tokenizer within the device to obtain multiple words (tokens). These multiple words are then input into an embedding model to obtain multiple word embedding vectors, which can be combined to form a first matrix. For example, n word embedding vectors can form the first matrix.

[0086] S520, the electronic device performs singular value decomposition on the first matrix to obtain the maximum eigenvalue of each eigenvalue block in the M eigenvalue blocks, and the M eigenvalue blocks correspond one-to-one with the M block matrices that make up the first matrix.

[0087] Specifically, S520 includes: the electronic device can perform singular value decomposition on the first matrix to obtain a matrix composed of eigenvectors of the first matrix, a diagonal matrix composed of eigenvalues ​​of the first matrix, and a unitary matrix. The electronic device can divide the diagonal matrix composed of eigenvalues ​​of the first matrix into blocks to obtain M eigenvalue blocks. The electronic device determines the largest eigenvalue in each of the M eigenvalue blocks, and one eigenvalue block can correspond to a block matrix of the first matrix. For example, the first matrix is ​​x, and the size of x is n×D, where n is the number of word embedding vectors constituting the first matrix, and D is the dimension of each word vector. After singular value decomposition of x, it becomes: x = USV T Let V be a D×D unitary matrix, U be an n×D matrix, and S be a D×D diagonal matrix composed of the eigenvalues ​​of the first matrix x. Therefore, matrix S can be divided into M eigenvalue blocks, and the largest eigenvalue in each block can be determined. Each eigenvalue block corresponds to a block matrix in the first matrix. Thus, the M eigenvalue blocks correspond to the M block matrices of the first matrix; that is, there is a one-to-one correspondence between the M eigenvalue blocks and the M block matrices. In other words, each eigenvalue block can be found in the original matrix block within the first matrix. Optionally, each of the M eigenvalue blocks has the same size, therefore, each of the M block matrices has the same size.

[0088] Optionally, the loss function of PIP, which measures the quantization effect of the first quantization matrix, has an upper bound. For example, the upper bound of the PIP loss function is related to the largest eigenvalue of each of the M eigenvalue blocks, or the upper bound of the PIP loss function is limited by the largest eigenvalue of each of the M eigenvalue blocks. For example... ΔX m Let λ be the quantization error of the m-th block matrix. m Let be the maximum eigenvalue of the m-th eigenvalue block of the m-th block matrix. In other words, inspired by the upper bound of the PIP loss function, the number of quantization bits can be determined based on the maximum eigenvalue of the m-th eigenvalue block of the m-th block matrix. Therefore, S530 can be executed.

[0089] S530, the electronic device determines the number of quantization bits for each block matrix corresponding to each eigenvalue block based on the maximum eigenvalue of each eigenvalue block.

[0090] Optionally, the magnitude of the eigenvalue can, to some extent, characterize the amount of semantic information carried by the block matrix. A larger eigenvalue indicates more semantic information, while a smaller eigenvalue indicates less semantic information. Therefore, the electronic device can allocate the number of quantization bits to each block matrix based on the magnitude of the largest eigenvalue of each eigenvalue block. Optionally, the electronic device can allocate a larger number of quantization bits to the block matrix corresponding to the eigenvalue block with the largest eigenvalue, a smaller number of quantization bits to the block matrix corresponding to the eigenvalue block with the largest eigenvalue, and a medium number of quantization bits to the block matrix corresponding to the eigenvalue block with the largest eigenvalue in the middle. For example, the first matrix corresponds to three eigenvalue blocks: eigenvalue block 1, eigenvalue block 2, and eigenvalue block 3. Eigenvalue block 1 corresponds to block matrix 1 of the first matrix, eigenvalue block 2 corresponds to block matrix 2 of the first matrix, and eigenvalue block 3 corresponds to block matrix 3 of the first matrix. Block matrices 1, 2, and 3 together form the first matrix. In this circuit, the feature value of feature block 1 is 4, the feature value of feature block 2 is 3, and the feature value of feature block 3 is 1. Assuming the total number of bits is 8, the electronic device can allocate 4 bits to block matrix 1 corresponding to feature block 1, 3 bits to block matrix 2 corresponding to feature block 2, and 1 bit to block matrix 3 corresponding to feature block 3.

[0091] Optionally, the quantization method 400 further includes: the electronic device determining the quantization error of each block matrix corresponding to each eigenvalue block. In this case, step S430 includes: estimating the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block and the quantization error of each block matrix. Optionally, the electronic device can quantize each block matrix corresponding to each eigenvalue block to obtain quantized block matrices, and determine the quantization error of each block matrix using the unquantized and quantized block matrices. The electronic device can also determine the number of quantization bits for each block matrix based on the quantization error of each block matrix and the maximum eigenvalue of each eigenvalue block.

[0092] Optionally, the quantization method 500 further includes: determining the size of each block matrix and the total number of quantization bits. The electronic device determines the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block and the quantization error of each block matrix, including: the electronic device determining the number of quantization bits for each block matrix based on at least one of the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, or the total number of quantization bits. Optionally, each of the M block matrices has the same size. Optionally, the size of each block matrix can be preset or configured by other devices on the electronic device; this embodiment does not limit this. Optionally, the total number of quantization bits can be preset, configured by other devices on the electronic device, or set by the user. For example, if the electronic device is a transmitter, the total number of quantization bits can be T configured by the receiver on the electronic device. When determining the number of quantization bits for each block matrix, electronic devices can refer to at least one of the following: the quantization error of each block matrix, the total number of bits, the size of each block matrix, or the largest eigenvalue of each eigenvalue block. This makes the determined number of quantization bits for each block matrix more accurate.

[0093] Optionally, the quantization method 500 further includes: determining quantization boundaries, wherein the electronic device determines the number of quantization bits for each block matrix based on at least one of the following: the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, or the total number of quantization bits. This includes: the electronic device determining the number of quantization bits for each block matrix based on at least one of the following: the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, the total number of quantization bits, or the quantization boundary. Optionally, the electronic device determining the quantization boundaries includes: the electronic device can determine M-M0 block matrices from the M block matrices that do not require quantization bit splitting, and then...

[0094] The number of quantization bits required to allocate to the M0 block matrices is determined. In other words, the M-M0 block matrices do not require quantization bits, or the M-M0 block matrices are allocated 0 quantization bits. Optionally, the M block matrices can be sorted, for example, by sorting them from largest to smallest according to the largest eigenvalue of the M eigenvalue blocks. Then, the first M0 block matrices need to be allocated quantization bits, and the last M-M0 block matrices do not require component quantization bits. The sum of the quantization bits allocated to the first M0 block matrices is the total number of quantization bits. For example, if the total number of bits is 8 and M is 4, the electronic device can determine that M0 is 3, meaning that quantization bits need to be allocated to 3 of the 4 block matrices, and 1 block matrix does not require quantization bits. The electronic device can arrange the four block matrices according to the largest eigenvalue of the eigenvalue blocks. For example, if the largest eigenvalue block of block matrix 1 is A1, the largest eigenvalue block of block matrix 2 is A2, the largest eigenvalue block of block matrix 3 is A3, and the largest eigenvalue block of block matrix 4 is A4, and A1>A2>A3>A4, then the electronic device can determine the number of quantization bits to allocate to block matrix 1, block matrix 2, and block matrix 3, but does not need to allocate the number of quantization bits to block matrix 4. In other words, the largest eigenvalue of the eigenvalue block of block matrix 4 is relatively small, indicating that block matrix 4 carries less semantic information. Therefore, it is not necessary to allocate the number of quantization bits to block matrix 4.

[0095] Optionally, the electronic device determines the number of quantization bits for each block matrix based on at least one of the following: the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, or the total number of quantization bits. Specifically, the electronic device can determine the number of quantization bits for the m-th block matrix according to formula (1).

[0096] Where M is the number of block matrices, M0 is the quantization boundary, B is the total number of bits, L is the number of columns in the m-th block matrix, the size of each block matrix is ​​n×L, the size of the first matrix is ​​n×D, the first matrix can be divided into M blocks, and the first matrix can be divided into M matrix blocks according to columns, M*L=D, λ m c is the largest eigenvalue of the m-th eigenvalue block. m It is a value related to the scalar quantization error of the m-th block matrix, for example, in, Let c be the scalar quantization error of the m-th block matrix. k The value related to the scalar quantization error of the k-th block matrix. in, The scalar quantization error of the k-th block matrix. This is the floor function. Where M0 is the quantization boundary calculated by the electronic device. Optionally, B / M0 can be understood as the number of quantization bits per block matrix when uniformly distributing quantization bits among the M0 block matrices. It can be a calibration value, that is, the calibration value is calibrated to the number of quantization bits evenly distributed in the m-th block matrix.

[0097] Optionally, the electronic device can calculate the quantization boundary M0 according to the following algorithm:

[0098] Input: B, L, c m , λ m

[0099] Initialization: M0 = M

[0100] when When M0 = M0-1.

[0101] In other words, the electronic device calculates the number of quantization bits for each block matrix according to formula (1). When it is determined that the number of quantization bits for a certain block matrix is ​​less than 0, it means that the block matrix does not need to be allocated quantization bits.

[0102] It is understandable that the above formula (1) is only an example to describe how to determine the number of quantization bits of the m-th block matrix. Any form of modification can be made based on formula (1), and the embodiments of this application do not limit this.

[0103] S540, the electronic device quantizes each block matrix according to the number of quantization bits of each block matrix to obtain each quantized block matrix.

[0104] Optionally, S540 includes: quantizing each block matrix using a first codebook based on the number of quantization bits of each block matrix to obtain each quantized block matrix.

[0105] Optionally, if the codewords included in the first codebook have a usage frequency greater than a preset frequency, the quantization method 540 may further include sending indication information. This indication information is used to indicate the first codebook. In other words, during quantization using the first codebook, the electronic device needs to indicate the first codebook being used to the other end to avoid the other end being unable to determine which codebook to use to recover the matrix before quantization. Optionally, the first codebook may be obtained based on a second codebook. Optionally, the first codebook being obtained based on a second codebook allows the electronic device to determine the codewords included in the first codebook from the codewords included in the second codebook. Optionally, the electronic device determining the codewords included in the first codebook from the codewords included in the second codebook includes: the electronic device determining codewords in the second codebook whose usage frequency is greater than a preset frequency as codewords in the first codebook; and codewords in the second codebook whose usage frequency is less than or equal to a preset frequency cannot be codewords in the first codebook. For example, an electronic device acts as the transmitter, and both the transmitter and receiver maintain a second codebook. However, some codewords in the second codebook have a usage frequency higher than a preset frequency, while others have a usage frequency lower than or equal to the preset frequency. Therefore, to reduce the number of codewords in the codebook used, the electronic device can determine a first codebook with a usage frequency higher than the preset frequency and can indicate the indicated first codebook to the other end. Optionally, after determining the first codebook, the electronic device can send the first codebook to the other end. That is, at this point, the other end maintains both the original second codebook and the first codebook from the electronic device. If the electronic device uses the first codebook during quantization, it needs to send indication information to the other end to indicate the first codebook used, thus preventing the other end from being unable to determine which codebook to use to recover the matrix.

[0106] Optionally, the second codebook can be the original codebook, and the first codebook can be an adaptive codebook. Optionally, the error between the first and second codebooks is less than a preset value. In other words, to avoid reducing the accuracy of quantization, the electronic device needs to ensure that the difference between the first and second codebooks is not too large when determining the first codebook. This way, the electronic device uses fewer quantization bits when using the first codebook compared to when using the second codebook, as shown in Figure 6. The horizontal axis of Figure 6 represents the standard deviation σ of the codewords in the first codebook and the standard deviation σ of the codewords in the second codebook. u The ratio σ / σ u The vertical axis represents the number of bits B required when using the first codebook. E The ratio of B to the number of bits B required when using the second codebook E / B, as shown in Figure 6, σ / σ u A smaller value indicates a more uniform distribution of codewords in the first codebook compared to the second codebook. The codeword distribution in the second codebook is uneven. In this case, B... EA smaller / B ratio indicates that the number of bits required when using the second codebook is greater than the number of bits required when using the first codebook. Optionally, the electronic device can send indication information simultaneously with the quantized first codebook. For example, as shown in Figure 7, the indication information can occupy a control bit. For instance, a value of 1 for this control bit indicates that the electronic device is using the first codebook, and a value of 0 indicates that the electronic device is using the second codebook. Alternatively, the electronic device can send the indication information before or after sending the quantized first codebook. For example, as shown in Figure 8, the electronic device can update the adaptive codebook (first codebook) based on the original codebook (second codebook), quantize the first matrix x using the first codebook, for example, by dividing the first matrix x into blocks and then quantizing it using the quantization method described in this application embodiment. Then, the quantized first matrix is ​​sent to the other end through the channel. The other end can recover the quantized first matrix x' based on the first codebook.

[0107] The following example illustrates the process by which an electronic device determines a first codebook from a second codebook by solving a convex optimization problem. If the electronic device needs to determine the first codebook, it executes the following process; otherwise, it does not execute it and continues quantization using the second codebook. The process by which the electronic device determines the first codebook includes: counting the number of times the codewords in the second codebook are used, u. n (t)=αu n (t-1)+1 and the frequency used Where α∈(0,1] is the forgetting factor, and the function of the forgetting factor is to reduce the number of characters in the second codebook at time t1. n The number of times the codeword is used at time t is related to the number of times the first codeword is used. n The impact of the number of times a codeword is used, u n (t-1) represents the first digit in the second codebook at time t-1. n The number of times each codeword is used, u n (t) represents the second codebook at time t. n The electronic device can determine the number of times a codeword is used based on the time t-1. n The number of times each codeword is used is calculated at time t. n The number of times each codeword is used. Electronic devices can solve convex optimization problems to determine the first codebook. For example, the convex optimization problem is:

[0108] Where σ is the variance of the statistical distribution of codewords in the second codebook, and σ can be related to P. n (t) is related, B is the second codebook. This is the first codebook.

[0109] It should be noted that the method for the electronic device to determine the first codebook can be an independent embodiment and is not limited to quantization method 500. For example, the electronic device can determine the first codebook based on the second codebook, where the second codebook is the original codebook used by the peer and the electronic device, and the first codebook is the codebook determined by the electronic device. The electronic device can send indication information to the peer to indicate the first codebook. The quantization method used by the electronic device can be quantization method 500 or not, and can be other quantization methods. This application embodiment does not limit this. That is to say, the way the electronic device indicates the indicated first codebook does not limit the quantization method used. The electronic device can use existing quantization methods to quantize using the first codebook, or it can use the quantization method 500 provided in this application embodiment to quantize.

[0110] Optionally, S540 includes: the electronic device determining the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix based on the number of quantization bits for each block matrix; the electronic device performing scalar quantization on each block matrix based on the number of quantization bits for scalar quantization of each block matrix to obtain each block matrix after scalar quantization; the electronic device performing vector quantization on each block matrix based on the number of quantization bits for vector quantization of each block matrix to obtain each block matrix after vector quantization; and the electronic device obtaining each quantized block matrix based on each block matrix after scalar quantization and each block matrix after vector quantization. Optionally, the electronic device has no restrictions on the order in which it performs scalar quantization and vector quantization on each block matrix. For example, for a block matrix, the electronic device can perform scalar quantization and vector quantization simultaneously, or it can perform vector quantization first and then scalar quantization on each block matrix, or it can perform scalar quantization first and then vector quantization on a portion of the block matrix while simultaneously performing vector quantization on the remaining portion, and then perform vector quantization on a portion of the block matrix while simultaneously performing scalar-vector quantization on the remaining portion. In other words, the electronic device can determine the quantization order according to its own implementation, and this embodiment does not impose any restrictions on this. Optionally, the first codebook can be a codebook composed of a vector quantization codebook and a scalar quantization codebook.

[0111] Optionally, the electronic device determines the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix based on the number of quantization bits of each block matrix. This includes: performing scalar quantization based on the number of quantization bits for scalar quantization of each block matrix under the constraint that the sum of the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix is ​​less than or equal to the number of quantization bits of each block matrix; and determining the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix based on the value at which the quantization error of each block matrix is ​​minimized after vector quantization based on the number of quantization bits for vector quantization of each block matrix. In other words, when determining the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix, the electronic device needs to do so under certain constraints. These constraints require that the number of quantization bits for vector quantization and the number of quantization bits for scalar quantization of each block matrix be less than or equal to the total number of quantization bits for that block matrix. This prevents the number of quantization bits for scalar quantization and the number of quantization bits for vector quantization determined for a particular block matrix from exceeding the total number of quantization bits for that block matrix. Under these constraints, the electronic device needs to find the minimum sum of the errors of scalar quantization and vector quantization, and determine this minimum sum as the number of quantization bits for vector quantization and scalar quantization. Optionally, the first codebook can be a codebook composed of a vector quantization codebook and a scalar quantization codebook.

[0112] For example, it can be transformed into the following problem:

[0113] in, Let be the number of quantization bits in the m-th block matrix. The number of bits quantized by the scalar of the m-th block matrix. The number of bits for vector quantization of the m-th block matrix, ΔX m This is the sum of the errors after quantizing the m-th block matrix using both vector quantization (based on the number of quantization bits in vector quantization) and scalar quantization (based on the number of quantization bits in scalar quantization). For ΔX m The expected value. ΔX m for and The function, Where M is the number of block matrices, s m The normalized vector before scalar quantization of the m-th block matrix. ρ is the normalized vector after scalar quantization of the m-th block matrix.(m) The modulus of the m-th block matrix before vector quantization is also called the length of the m-th block matrix before vector quantization. The modulus of the m-th block matrix after vector quantization is also called the length of the m-th block matrix after vector quantization. Solving the above minimization problem, we get:

[0114] in, To determine the number of quantization bits in the obtained vector quantization, The number of quantization bits L obtained by solving the scalar quantization is the number of columns included in the m-th block matrix.

[0115] Optionally, the electronic device obtains each quantized block matrix based on each block matrix after scalar quantization and each block matrix after vector quantization, including: the electronic device merges the scalar and vector of each block matrix after scalar quantization and each block matrix after vector quantization to obtain each quantized block matrix.

[0116] Optionally, the electronic device performs scalar quantization on each block matrix according to the number of quantization bits of each block matrix, to obtain each block matrix after scalar quantization. This includes: the electronic device performs scalar quantization on each block matrix using a scalar quantization codebook according to the number of quantization bits of each block matrix. Optionally, the scalar quantization codebook can be preset or configured by other devices for the electronic device. For example, if the electronic device is the transmitter, the scalar quantization codebook can be configured by the receiver for the electronic device. This makes it convenient for the receiver and transmitter to use the same scalar quantization codebook. The transmitter can perform scalar quantization on the block matrix according to the scalar quantization codebook, and the receiver can recover the block matrix before quantization according to the scalar quantization codebook. Optionally, the electronic device performs vector quantization on each block matrix according to the number of quantization bits of each block matrix, to obtain each block matrix after vector quantization. This includes: the electronic device performs vector quantization on each block matrix using a vector quantization codebook according to the number of quantization bits of each block matrix. Optionally, the vector quantization codebook can be preset or configured by other devices for the electronic device. For example, if the electronic device is the transmitter, the vector quantization codebook can be configured by the receiver for the electronic device. This makes it convenient for the receiver and transmitter to use the same vector quantization codebook. The transmitter can perform vector quantization on the block matrix according to the vector quantization codebook, and the receiver can recover the block matrix before quantization according to the vector quantization codebook.

[0117] In the S550, the electronic device obtains the first quantized matrix based on each quantized block matrix.

[0118] Optionally, S550 includes: the electronic device can merge each quantized block matrix to obtain a quantized first matrix. Optionally, the electronic device can merge each quantized block matrix according to the block order of the block matrix.

[0119] The S560 electronic device can send the quantized first matrix.

[0120] Optionally, S560 may not directly transmit the quantized first matrix. Instead, it may process the quantized first matrix before transmission. For example, the electronic device may perform source encoder encoding, channel encoder encoding, modulator modulation, and amplifier amplification on the quantized first matrix as shown in Figure 4 before transmission. Optionally, S560 may be an optional step, and the electronic device may not transmit the quantized first matrix.

[0121] Optionally, S510-S560 described above can be executed in the quantization module of device A shown in Figure 4. That is, the electronic device can be device A as described above.

[0122] In the aforementioned quantization method 500, the electronic device can determine the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block. The maximum eigenvalue of each eigenvalue block can, to some extent, represent the amount of semantic information carried by each block matrix. Therefore, by allocating the number of quantization bits for each block matrix in combination with the maximum eigenvalue of each eigenvalue block, the number of quantization bits allocated to each block matrix can be adapted to the distribution of semantic information, thereby improving the quantization accuracy, reducing quantization error, and avoiding the problem that when the electronic device allocates the same number of quantization bits to each block matrix, the semantic information distribution of the block matrix is ​​uneven, resulting in some block matrices having too many quantization bits and others having too few quantization bits, leading to a significant loss of semantic information. For example, referring to the example in Figure 3, suppose the word embedding vector has only one value between -4.0 and 0, which falls between -2.0 and -1.0. Therefore, the semantic information carried between -4.0 and 0 is relatively small, so no bits need to be allocated to this range. Since there are many values ​​between 0 and 4.0, more bits can be allocated to this range, such as 3 bits. In other words, when the semantic information carried between 0 and 4.0 is relatively large, more quantization bits can be allocated to this range, and when the semantic information carried between -4.0 and 0 is relatively small, no quantization bits need to be allocated to this range. This can reduce the loss of semantic information and help improve quantization performance. For example, as shown in Figure 9, a comparison diagram of the effects of the quantization method provided in this application embodiment and the uniform quantization method is shown. Figure 9(a) is a comparison diagram of the first test dataset, and Figure 9(b) is a comparison diagram of the second test dataset. For example, the first test dataset is WordSim353 Test, and the second test dataset is MTurk771 Test. As shown in Figure 9(a), when the number of quantized information bits is 300 bits, the quantization method provided in this application embodiment is closer to natural language than the uniform quantization method. As shown in Figure 9(b), when the number of quantized information bits is 400 bits, the quantization method provided in this application embodiment is closer to natural language than the uniform quantization iterative method. That is to say, compared with the uniform quantization method, the quantization method provided in this application embodiment has less loss of semantic information. Therefore, the quantization accuracy of the quantization method provided in this application embodiment is higher.

[0123] The above describes the process by which an electronic device quantizes the first matrix composed of word embedding vectors. To better illustrate the quantization method 500, the quantization process is described below with reference to Figure 10. However, the quantization process in Figure 10 is an example and should not be construed as limiting the embodiments of this application. As shown in Figure 10: The electronic device can divide the first matrix x composed of word embedding vectors into blocks. x can be divided into M block matrices, namely v1, v2, ..., v M Each block matrix contains L columns. For each block matrix, a modulo operation is first performed, followed by normalization. Taking v1 as an example, the norm of v1 is calculated, which is equivalent to performing a modulo operation on v1 to obtain ρ1 = ||v1||. Then, v1 is normalized to obtain... The number of quantization bits required for scalar quantization of v1 can be determined using the quantization method 400 described above. Number of quantization bits for vector quantization of v1 Therefore, the quantization process for v1 may include quantizing the codebook for ρ1 using scalar quantization. and Performing uniform quantization, also known as scalar quantization, yields... Using vector quantization codebook and the number of quantization bits Perform vector quantization on s1 to obtain Then to and Merge to obtain the quantified result. By analogy, the quantized block matrix can be obtained. Then merge The first matrix obtained after quantization is

[0124] To better illustrate the quantization process of the first matrix composed of word embedding vectors, the following describes the quantization process in detail with reference to Figure 11. As shown in Figure 11, given the first matrix composed of word embedding vectors of natural language... x Hierarchical quantization is used to transform the first matrix. x Divide into M block matrices, the m-th block matrix is ​​v m For the block matrix v m It is possible to calculate the v of the block matrix. m Norm ρ m =||v m ‖ and the normalized vector For norm ρ m Uniform quantization is used, and the number of quantized bits is: The quantized value is For the normalized vector s m Vector quantization is performed using a Grassmannian quantizer, with the following number of quantization bits: The quantized value is Merge all and Quantification completed.

[0125] Figure 12 is a schematic block diagram of a quantization device 1200 provided in an embodiment of this application. As shown in Figure 12, the quantization device 1200 may include a processing unit 1210 and a communication unit 1220. The communication unit 1220 can implement corresponding communication functions, which can be internal communication within the quantization device 1200 or communication between the quantization device 1200 and other devices; the processing unit 1210 can implement corresponding processing functions. The communication unit 1220 may also be referred to as a communication interface or a transceiver unit. Optionally, the quantization device 1200 may further include a storage unit, which can be used to store instructions and / or data. The processing unit 1210 can read the instructions and / or data in the storage unit to enable the device to implement the aforementioned method embodiment.

[0126] In one possible design, the quantization device 1200 can be the electronic device in the quantization method 500 described above, or it can be a module or chip applied to an electronic device. The quantization device 1200 can be used to perform the steps or processes executed by the electronic device in the above method embodiments.

[0127] For details regarding the steps or processes performed by each unit in the quantization device 1200, please refer to the embodiments of the method described above; they will not be detailed here.

[0128] It should be understood that the "unit" in the quantization device 1200 can be implemented in hardware, software, or by hardware executing corresponding software. For example, the "unit" can refer to an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor, etc.) and memory for executing one or more software or firmware programs, combined logic circuitry, and / or other suitable components supporting the described functions. As another example, the communication unit 1220 can be replaced by a transceiver circuit (e.g., it may include receiving and transmitting circuitry), and the processing unit 1210 can be replaced by a processor or processing circuitry.

[0129] Figure 13 shows a schematic block diagram of another quantization device 1300 provided in an embodiment of this application. The quantization device 1300 may be a terminal device, a network device, an O-RU, or an O-DU, or it may be a chip, chip system, or processor that supports the terminal device, network device, O-RU, or O-DU in implementing the above methods. This device can be used to implement the methods described in the above method embodiments; for details, please refer to the descriptions in the above method embodiments.

[0130] The quantization device 1300 may include one or more processors 1310, which can also be referred to as processing units, and can implement certain control functions. The processor 1310 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, while the central processing unit can be used to control the quantization device (e.g., base station, baseband chip, user chip, DU or CU, etc.), execute software programs, and process the data from the software programs.

[0131] In an alternative design, processor 1310 may also store instructions and / or data that can be executed by processor 1310 to cause quantization device 1300 to perform the method described in the above method embodiments. Optionally, processing unit 1210 in quantization device 1200 may be processor 1310.

[0132] In another alternative design, the quantization device 1300 may include a communication interface 920 for implementing receiving and transmitting functions. For example, the communication interface 920 may be a transceiver circuit, interface, interface circuit, or transceiver. The transceiver circuit, interface, interface circuit, or transceiver for implementing receiving and transmitting functions may be separate or integrated. The aforementioned transceiver circuit, interface, interface circuit, or transceiver may be used for reading and writing code / data, or it may be used for signal transmission or delivery. Optionally, the communication unit 1220 in the quantization device 1200 may be a communication interface 1320.

[0133] Optionally, the quantization device 1300 may include one or more memories 1330, which may store instructions that can be executed on the processor 1310, causing the quantization device 1300 to perform the methods described in the above method embodiments. Optionally, the memories 1330 may also store data. Optionally, the processor 1310 may also store instructions and / or data. The processor 1310 and the memories 1330 may be provided separately or integrated together.

[0134] Those skilled in the art will understand that, for ease of explanation, Figure 13 only shows one memory and processor. In actual quantization devices, multiple processors and memories may exist. Memory may also be referred to as storage medium or storage device, etc., and the embodiments of this application do not limit this.

[0135] For example, a processor may include a baseband processor and a central processing unit (CPU). The baseband processor is mainly used for processing communication protocols and communication data, while the CPU is mainly used for controlling the entire terminal device, executing software programs, and processing the data in the software programs. The processor in Figure 9 integrates the functions of both a baseband processor and a CPU. Those skilled in the art will understand that the baseband processor and CPU can also be independent processors interconnected via technologies such as buses. Those skilled in the art will understand that a terminal device may include multiple baseband processors to adapt to different network standards, and multiple CPUs to enhance its processing capabilities. The various components of the terminal device can be connected via various buses. The baseband processor can also be described as a baseband processing circuit or a baseband processing chip. The CPU can also be described as a central processing circuit or a central processing chip. The function of processing communication protocols and communication data can be built into the processor or stored in a storage unit as a software program, which is then executed by the processor to implement the baseband processing function.

[0136] It should be understood that, in one possible design, the steps in the method embodiments provided in this application can be implemented by integrated logic circuits in the processor's hardware or by instructions in software form. The steps of the methods disclosed in the embodiments of this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules in the processor. The software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are not provided here.

[0137] It should be noted that the processor in the embodiments of this application can be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method embodiments can be completed by the integrated logic circuitry in the processor's hardware or by instructions in software form. The processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above methods.

[0138] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0139] This application also provides a computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the various steps or processes performed by the electronic device in any of the above method embodiments.

[0140] This application also provides a computer-readable storage medium storing program code that, when run on a computer, causes the computer to perform the various steps or processes performed by the electronic device in any of the above method embodiments.

[0141] This application also provides a quantization device, including a processor and an interface for sending and / or receiving signals, such that the processor performs the various steps or processes performed by the electronic device in any of the above method embodiments.

[0142] The above-described device and method embodiments are completely corresponding, with corresponding modules or units performing corresponding steps. For example, a communication unit or communication interface performs the receiving or sending steps in the method embodiment, while other steps besides sending and receiving can be performed by a processing unit or processor.

[0143] In the embodiments of this application, the terms and English abbreviations are exemplary examples given for ease of description and should not be construed as limiting the application in any way. This application does not preclude the possibility of defining other terms that can achieve the same or similar functions in existing or future agreements.

[0144] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable storage media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0145] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0146] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be based on the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0147] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0148] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0149] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0150] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs), etc.

[0151] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A quantification method, characterized in that, The quantization method includes: Obtain the first matrix composed of word embedding vectors of natural language; Singular value decomposition is performed on the first matrix to obtain the maximum eigenvalue of each of the M eigenvalue blocks, and the M eigenvalue blocks correspond one-to-one with the M block matrices of the first matrix; The number of quantization bits for each block matrix corresponding to each feature block is determined based on the maximum eigenvalue of each feature block; Each block matrix is ​​quantized according to the number of quantization bits of each block matrix to obtain each quantized block matrix; Based on each quantized block matrix, the first quantized matrix is ​​obtained; Send the quantized first matrix; Where M is a positive integer, and the number of quantization bits in each block matrix is ​​greater than or equal to 0.

2. The quantization method according to claim 1, characterized in that, The quantization method further includes: Determine the quantization error of each block matrix corresponding to each feature value block; Wherein, determining the number of quantization bits of each block matrix corresponding to each feature value block based on the maximum eigenvalue of each feature value block includes: The number of quantization bits for each block matrix is ​​determined based on the maximum eigenvalue of each eigenvalue block and the quantization error of each block matrix.

3. The quantization method according to claim 2, characterized in that, The quantization method further includes: Obtain the size and total number of bits for each block matrix; The step of determining the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block and the quantization error of each block matrix includes: The number of quantization bits for each block matrix is ​​determined based on the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, and the total number of quantization bits.

4. The quantization method according to claim 3, characterized in that, The step of determining the number of quantization bits for each block matrix based on the maximum eigenvalue of each eigenvalue block, the quantization error of each block matrix, the size of each block matrix, and the total number of quantization bits includes: The number of quantization bits for the m-th block matrix is ​​determined according to formula (1). Where M is the number of block matrices, M0 is the quantization boundary, B is the total number of bits, L is the number of columns in the m-th block matrix, and λ m c is the largest eigenvalue of the m-th eigenvalue block. m The scalar quantization error related value of the m-th block matrix, c k The value related to the scalar quantization error of the k-th block matrix. This is for floor function.

5. The quantization method according to any one of claims 1 to 4, characterized in that, The step of quantizing each block matrix according to the number of quantization bits of each block matrix to obtain each quantized block matrix includes: The number of quantization bits for scalar quantization and the number of quantization bits for vector quantization of each block matrix are determined based on the number of quantization bits for each block matrix. Each block matrix is ​​scalar quantized according to the number of quantization bits of each block matrix, to obtain each block matrix after scalar quantization; Each block matrix is ​​vector quantized according to the number of quantization bits of each block matrix, to obtain each block matrix after vector quantization; Each quantized block matrix is ​​obtained based on each block matrix after scalar quantization and each block matrix after vector quantization.

6. The quantization method according to claim 5, characterized in that, The step of determining the scalar quantization bit count and the vector quantization bit count of each block matrix based on the number of quantization bits of each block matrix includes: Under the constraint that the sum of the number of quantized bits for scalar quantization and the number of quantized bits for vector quantization of each block matrix is ​​less than or equal to the number of quantized bits for each block matrix, the value at which the quantization error of each block matrix is ​​minimized after scalar quantization based on the number of quantized bits for scalar quantization and vector quantization based on the number of quantized bits for vector quantization of each block matrix is ​​determined as the number of quantized bits for scalar quantization and the number of quantized bits for vector quantization of each block matrix.

7. The quantization method according to any one of claims 1 to 6, characterized in that, The step of quantizing each block matrix according to the number of quantization bits of each block matrix to obtain each quantized block matrix includes: Based on the number of quantized bits in each block matrix, each block matrix is ​​quantized using a first codebook to obtain each quantized block matrix.

8. The quantization method according to claim 7, characterized in that, The codewords included in the first codebook have a usage frequency greater than a preset frequency; the quantization method further includes: Send an instruction message, which is used to instruct the first codebook.

9. The quantization method according to claim 7 or 8, characterized in that, The first codebook is obtained based on the second codebook, which includes the first codebook, and the error between the first codebook and the second codebook is less than a preset value.

10. A quantization device, characterized in that, Includes a unit that performs the quantization method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the quantization method as described in any one of claims 1 to 9.

12. An electronic device, characterized in that, include: A memory and a processor, the memory being used to store a computer program, and the processor being used to cause the electronic device to perform the quantization method as described in any one of claims 1 to 9 when the computer program is invoked.

13. A computer program product, characterized in that, When the computer program product is run on a computer, the computer performs the quantization method as described in any one of claims 1 to 9.

Images