Vector quantization method, device, electronic device and storage medium for language model alignment

Through the similarity search and accumulation operation between the input codebook and the output codebook, the problem of poor adaptability of output vector sequences in the traditional multimodal model is solved, vector quantization and decompression and decoding are realized, and the adaptability of large language models is improved. Matching ability and matching effect.

CN118708990BActive Publication Date: 2024-11-05ATHENAEYES CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411199228.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-29
Publication Date
2024-11-05
Estimated Expiration
2044-08-29

AI Technical Summary

Technical Problem

The vector sequence output by the traditional multimodal model training method only corresponds to a specific large language model and cannot be adapted well with other large language models, resulting in poor adaptability.

Method used

By initializing the input codebook and the output codebook, search the similarity level in the input codebook based on the input vector, obtain the preset number of input codebook vector subscripts, and obtain the corresponding output codebook vector subscript in the output codebook. , perform accumulation to obtain the output vector, which is used to input the target large language model, and realize vector quantization and decompression decoding.

Benefits of technology

It improves the adaptability of different large language models, reduces information loss during compression, and enhances the matching ability of input vectors to large language models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118708990B_ABST
    Figure CN118708990B_ABST
Patent Text Reader

Abstract

The present application discloses a vector quantization method, device, electronic device and storage medium for language model alignment, the method comprising: initializing an input codebook and an output codebook; performing a similarity search in a plurality of input codebook vectors based on an input vector, and obtaining a preset number of input codebook vector subscripts in descending order of similarity; obtaining a preset number of output codebook vector subscripts in the output codebook corresponding to a preset number of input codebook vector subscripts; accumulating a preset number of output codebook vectors corresponding to a preset number of output codebook vector subscripts to obtain an output vector, and the output vector is used to input a target large language model. In this way, two codebooks are used, the input vector is compressed by the input codebook, and then matched with a downstream large language model by the output codebook, then for different large language models, only the corresponding output codebook needs to be replaced to achieve matching, thereby improving the adaptability to the large language model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a vector quantization method for language model alignment. Background Art

[0002] The Transformer structure in the large language model requires the input to be in the form of a vector sequence. However, images are not in a vector sequence format. In order for the large language model to understand the image content, a multimodal model is usually used to convert the image data into a vector sequence that the large language model can recognize. This process is called language model alignment.

[0003] The traditional method of training a multimodal model is to directly use the output vector sequence as the input of a large language model. However, the vector sequence output by the trained multimodal model only corresponds to a specific large language model and cannot be well adapted to other large language models. Therefore, there is an urgent need for a method to improve the adaptability to large language models. Summary of the invention

[0004] The present application provides a vector quantization method, device, electronic device and storage medium for language model alignment, which can enhance the adaptability of output vectors to different large language models.

[0005] In a first aspect, the present application provides a vector quantization method for language model alignment, the method comprising:

[0006] Initialize an input codebook and an output codebook; the input codebook includes a plurality of input codebook vectors, and the output codebook includes a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; the subscripts of the input code vectors in the input code correspond to the subscripts of the output codebook vectors in the output codebook in one-to-one correspondence;

[0007] Performing a similarity search among the multiple input codebook vectors based on the input vector, obtaining a preset number of input codebook vector subscripts in descending order of similarity, wherein the preset number of input codebook vector subscripts are used to represent the input vector after compression coding;

[0008] Obtaining a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts;

[0009] Accumulating the preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, wherein the output vectors are used to represent vectors obtained by decompressing and decoding the compressed and encoded input vectors, and the output vectors are used to input a target large language model, wherein the target large language model is a large language model adapted to the output codebook.

[0010] Optionally, performing similarity search among the multiple input codebook vectors based on the input vector, and obtaining a preset number of input codebook vector subscripts in descending order of similarity, includes:

[0011] Calculating similarities between the input vector and the multiple input codebook vectors respectively;

[0012] Sorting the preset number of input codebook vectors in descending order of similarity;

[0013] Obtain the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

[0014] Optionally, respectively calculating similarities between the input vector and the multiple input codebook vectors includes:

[0015] Performing point multiplication operations on the input vector and the multiple input codebook vectors respectively to obtain multiple point multiplication operation results, wherein the point multiplication operation results are used to represent the similarity between the input vector and the input codebook vector.

[0016] Optionally, the obtaining the preset number of input codebook vectors by sorting in descending order of similarity includes:

[0017] The multiple point product operation results are sorted in descending order, and the preset number of input codebook vectors are obtained in sequence according to the results of the descending order.

[0018] Optionally, the input codebook is used to compress the input vector, and the output codebook is used to generate the output vector so as to adapt to the large language model.

[0019] Optionally, the preset number is an integer, the preset number is greater than 1 and the preset number is smaller than the dimension of the input vector.

[0020] In a second aspect, the present application further provides a vector quantization device for language model alignment, the device comprising:

[0021] an initialization unit, configured to initialize an input codebook and an output codebook; the input codebook includes a plurality of input codebook vectors, the output codebook includes a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; the subscripts of the input code vectors in the input code correspond one-to-one to the subscripts of the output codebook vectors in the output codebook;

[0022] A searching unit, configured to perform a similarity search among the plurality of input codebook vectors based on the input vector, and obtain a preset number of input codebook vector subscripts in descending order of similarity, wherein the preset number of input codebook vector subscripts are used to represent the input vector after compression coding;

[0023] An acquiring unit, configured to acquire a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts;

[0024] An accumulation unit is used to accumulate the preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, wherein the output vectors are used to represent the vectors obtained by decompressing and decoding the compressed and encoded input vectors, and the output vectors are used to input a target large language model, and the target large language model is a large language model adapted to the output codebook.

[0025] Optionally, the search unit includes a calculation subunit, a sorting subunit and an acquisition subunit;

[0026] The calculation subunit is used to calculate the similarity between the input vector and the multiple input codebook vectors respectively;

[0027] The sorting subunit is used to sort the input codebook vectors in descending order of similarity to obtain the preset number;

[0028] The acquisition subunit is configured to acquire the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

[0029] Optionally, the computing subunit is specifically used for:

[0030] Performing point multiplication operations on the input vector and the multiple input codebook vectors respectively to obtain multiple point multiplication operation results, wherein the point multiplication operation results are used to represent the similarity between the input vector and the input codebook vector.

[0031] Optionally, the sorting subunit is specifically used to:

[0032] The multiple point product operation results are sorted in descending order, and the preset number of input codebook vectors are obtained in sequence according to the results of the descending order.

[0033] Optionally, the preset number is an integer, the preset number is greater than 1 and the preset number is smaller than the dimension of the input vector.

[0034] In a third aspect, the present application further provides an electronic device, the electronic device comprising a processor and a memory:

[0035] The memory is used to store computer programs;

[0036] The processor is used to execute the vector quantization method for language model alignment provided in the first aspect according to the computer program.

[0037] In a fourth aspect, the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the vector quantization method for language model alignment provided in the first aspect above.

[0038] It can be seen that this application has the following beneficial effects:

[0039] The present application provides a vector quantization method, device, electronic device and storage medium for language model alignment, the method comprising: initializing an input codebook and an output codebook; the input codebook comprises a plurality of input codebook vectors, the output codebook comprises a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; the input coding vector subscripts in the input coding correspond to the output codebook vector subscripts in the output codebook in a one-to-one manner; performing a similarity search in a plurality of input codebook vectors based on the input vector, obtaining a preset number of input codebook vector subscripts in descending order of similarity, the preset number of input codebook vector subscripts being used to characterize the input vector after compression coding; obtaining a preset number of output codebook vector subscripts in the output codebook corresponding to the preset number of input codebook vector subscripts; accumulating a preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain an output vector, the output vector being used to characterize a vector obtained by decompressing and decoding the input vector after compression coding, the output vector being used to input a target large language model, the target large language model being a large language model adapted to the output codebook. Thus, compared with the prior art, two codebooks are used in the embodiment of the present application, and the input vector is compressed and encoded by the input codebook, and then matched with the downstream target large language model by the output codebook. Then, for different target large language models, only the corresponding output codebook needs to be replaced to achieve matching with the target large language model, that is, the method provided by the embodiment of the present application can improve the adaptability with the large language model. In addition, in the process of compressing and encoding the input vector by the input codebook, a preset number of input codebook vector subscripts corresponding to a preset number of input codebook vectors similar to the input vector are obtained, which can reduce information loss in the compression process and improve the matching ability of the subsequent input vector with the large language model. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0041] Figure 1 A schematic diagram of a language model alignment process in an embodiment of the present application;

[0042] Figure 2 A schematic diagram of a vector quantization process in an embodiment of the present application;

[0043] Figure 3 A schematic diagram of a flow chart of a vector quantization method for language model alignment in an embodiment of the present application;

[0044] Figure 4a A process diagram of a vector quantization method for language model alignment in an embodiment of the present application Figure 1 ;

[0045] Figure 4b A process diagram of a vector quantization method for language model alignment in an embodiment of the present application Figure 2 ;

[0046] Figure 5 A schematic diagram of the structure of a vector quantization device for language model alignment provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0049] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. In addition, it should be noted that, for ease of description, only the parts related to the present application are shown in the accompanying drawings, not all structures.

[0050] First, the terms used in this application are explained:

[0051] Large Multimodal Models (LMMs) are models that can process multiple different types of data, including images, audio, text, etc. Multimodal models can better capture the relationship between multiple data, integrate and understand different data formats, and provide more comprehensive and accurate information.

[0052] Large Language Models (LLMs) are large-scale machine learning models based on deep learning. They are usually used to process Natural Language Processing (NLP) tasks. They are mainly trained on large text corpora and learn to understand the structure, semantics, context, and pragmatics of a language.

[0053] It should be noted that the Transformer structure in the large language model requires the input to be in the form of a vector sequence. However, images are not in a vector sequence format. In order for the large language model to understand the image content, a multimodal model is usually used to convert the image data into a vector sequence that the large language model can recognize. This process is called language model alignment.

[0054] See also Figure 1 , is a schematic diagram of a language model alignment process in an embodiment of the present application, an input image (InputImage) is sent to an image encoder (Image Encoder), and a fully connected layer (Fully Connected) is used to align the features output by Q-Former (a lightweight Transformer) to the same dimension as LLM. The visual features extracted by Q-Former are used to constrain the generation of LLM, which can provide the most useful visual information for LLM. LLM outputs text (Output Text). The "*" snowflake mark in the image encoder indicates that the parameters of the image encoder are fixed, that is, the parameters are in a frozen state. Similarly, the "*" snowflake mark in the large language model decoder indicates that the parameters of the large language model decoder are fixed, that is, the parameters are in a frozen state, that is Figure 1 A process is demonstrated to convert images into vector sequences with the help of a multimodal model and input them into a large language model.

[0055] It should be noted that the traditional multimodal model training method is to directly use the output vector sequence as the input of the large language model. However, the vector sequence output by the trained multimodal model only corresponds to a specific large language model and cannot be well adapted to other large language models.

[0056] There are methods in the prior art to solve the problem of poor adaptability. In the process of language model alignment, the continuous vector sequence is first converted into discrete symbols (tokens), and then the discrete symbols are expanded to the vocabulary of other large language models, so that it can adapt to any large language model. It should be noted that in natural language processing, token refers to the smallest semantic unit in the text, and the process of converting vectors into discrete symbols is called vector quantization (tokenizer).

[0057] It should be noted that vector quantization has been widely used in image generation tasks. For example, in high-definition image generation tasks, a trainable codebook is designed, which presets a large number of codebook vectors. The core of vector quantization is to find the codebook vector closest to the input vector and use the ID corresponding to the closest codebook vector as the quantized symbol (token). Figure 2 As shown in Figure 1, a schematic diagram of the vector quantization process is provided, which shows how a region in an image is quantized into a discrete symbol (such as Figure 2 The codebook corresponding to the symbol 1 shown in the figure) is used directly as the input of the Transformer, where the formula in the Transformer The probability of the entire sequence s is the product of the conditional probabilities of all its elements based on the previous elements. s represents the sequence. is the first elements, Represents all elements before position i in the sequence.

[0058] However, the applicant has found through research that the core of the above vector quantization method is to use a single integer to replace a high-dimensional vector. For image generation tasks, the vectors involved usually represent low-level features. Therefore, compressing them into integers will not only not weaken the model effect, but can improve the generalization ability of the model, thereby improving the efficiency of image generation.

[0059] As for the language model alignment task, the vectors involved contain rich and abstract semantic features. If the vector quantization method mentioned above is used to compress the vector into an integer, the semantic information will be lost, which will affect the large language model's ability to understand the image content. In other words, when the amount of information carried by the high-dimensional vector is huge, using a single integer to replace the high-dimensional vector will inevitably cause information loss, which will affect the compatibility with the large language model.

[0060] Based on this, an embodiment of the present application provides a vector quantization method, device, electronic device and storage medium for language model alignment, the method comprising: initializing an input codebook and an output codebook; the input codebook includes multiple input codebook vectors, and the output codebook includes multiple output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook, and the input code vector subscript in the input code corresponds to the output codebook vector subscript in the output codebook one by one; based on the input vector, a similarity search is performed in multiple input codebook vectors, and a preset number of input codebook vector subscripts are obtained in descending order of similarity, and the preset number of input codebook vector subscripts are used to characterize the input vector after compression coding; a preset number of output codebook vector subscripts corresponding to the preset number of input codebook vector subscripts in the output codebook are obtained; a preset number of output codebook vector subscripts corresponding to the preset number of output codebook vector subscripts are accumulated to obtain an output vector, the output vector is used to characterize the vector obtained by decompressing and decoding the compressed input vector, and the output vector is used to input a target large language model, and the target large language model is a large language model adapted to the output codebook.

[0061] In this way, two codebooks are used in the embodiment of the present application, and the input vector is compressed and encoded by the input codebook, and then matched with the downstream target large language model by the output codebook. Then, for different target large language models, only the corresponding output codebook needs to be replaced to achieve matching with the target large language model, that is, the method provided by the embodiment of the present application can improve the adaptability with the target large language model. Among them, in the process of compressing and encoding the input vector by the input codebook, the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors similar to the input vector are obtained, which can reduce the information loss in the compression process and improve the matching ability of the subsequent input vector with the large language model.

[0062] To facilitate understanding of the specific implementation of the vector quantization method for language model alignment provided in the embodiment of the present application, it will be described below with reference to the accompanying drawings.

[0063] It should be noted that the subject implementing the vector quantization method for language model alignment may be a vector quantization device for language model alignment provided in an embodiment of the present application, and the vector quantization device for language model alignment may be carried in an electronic device or a functional module of an electronic device. The electronic device in the embodiment of the present application may be any device capable of implementing the vector quantization method for language model alignment in the embodiment of the present application, for example, an Internet of Things (IoT) device.

[0064] See also Figure 3, which is a flow chart of a vector quantization method for language model alignment in an embodiment of the present application. The method can be applied to a vector quantization device for language model alignment, and the vector quantization device for language model alignment can be as follows Figure 5 The vector quantization device 500 for language model alignment shown in FIG. 5 may also be integrated in Figure 6 Functional modules in the electronic device 600 are shown.

[0065] like Figure 3 As shown, the embodiments of the present application may include, for example:

[0066] S10: Initialize the input codebook and the output codebook.

[0067] It should be noted that the input codebook includes multiple input codebook vectors, the output codebook includes multiple output codebook vectors, the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook, and the input code vector subscripts in the input code correspond to the output code vector subscripts in the output codebook. The input codebook is used to compress the code input vector, and the output codebook is used to generate the output vector to adapt to the downstream large language model.

[0068] As an example, the input codebook includes n input codebook vectors, namely codebook vector 1, codebook vector i...codebook vector n, and the output codebook correspondingly includes n output codebook vectors, namely codebook vector 1, codebook vector i...codebook vector n.

[0069] In a possible implementation, initializing the input codebook and the output codebook may include determining the number of codebook vectors in the input codebook and the output codebook, randomly generating an initial value for each codebook vector, and determining a subscript of each codebook vector, where the subscript is also an index identifier (Identity document, ID). Of course, the codebook vector may also be selected from a pre-trained codebook vector set, which does not affect the implementation of the embodiments of the present application.

[0070] S20: performing a similarity search in a plurality of input codebook vectors based on the input vector, and obtaining a preset number of input codebook vector subscripts in descending order of similarity.

[0071] It should be noted that the input vector can be a vector extracted from the image, the preset number is an integer, the preset number is greater than 1 and the preset number is less than the dimension of the input vector, and there are differences between the preset number of input codebook vectors, that is, the preset number of input codebook vectors are used to characterize input codebook vectors that are similar to the input vector in a preset number of aspects. In this way, compared with finding the closest similarity in the prior art, the embodiment of the present application can reduce the information loss in the compression process of the input vector by obtaining the closest preset number of similarities and paying attention to the similarities of multiple different aspects, thereby improving the matching ability of the subsequent input vector with the large language model.

[0072] As an example, the dimension of the input vector is 2048, k represents a preset number, k is greater than 1 but much smaller than the dimension of the input vector, for example, k can be 3, which means searching the input codebook for 3 input codebook vectors that are most similar to the input vector, or, for example, k can be 64, which means searching the input codebook for 64 input codebook vectors that are most similar to the input vector.

[0073] In one possible implementation, the input codebook vector and the input vector may be normalized vectors, then S20 provided in the embodiment of the present application may include S201: respectively calculating the similarity between the input vector and multiple input codebook vectors; S202: obtaining a preset number of input codebook vectors by sorting in descending order of similarity; S203: obtaining a preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

[0074] In a possible implementation, the point product operation result of the input vector and the input codebook vector can represent the similarity between the input vector and the input codebook vector, then, S201 provided in the embodiment of the present application may include performing point product operations on the input vector and multiple input codebook vectors respectively to obtain multiple point product operation results, wherein the point product operation results are used to represent the similarity between the input vector and the input codebook vector. S202 may include sorting the multiple point product operation results in descending order, and obtaining a preset number of input codebook vectors in sequence according to the results of the descending order.

[0075] It should be noted that the embodiment of the present application may also use other methods to obtain the input codebook vector and the input codebook vector subscript that are most similar to the input vector, which does not affect the implementation of the embodiment of the present application.

[0076] S30: Obtain a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts.

[0077] As an example, based on the input vector, the three most similar input codebook vectors are searched in the input codebook, which are input codebook vector i, input codebook vector j and input codebook vector k, respectively. The output token combination (that is, the three input codebook vector subscripts) includes i, j and k; according to the token combination, the output codebook is searched to obtain the output codebook i, the output codebook vector j and the output codebook vector k.

[0078] S40: Accumulating a preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, where the output vectors are used to input into a target large language model.

[0079] It should be noted that the output vector is used to represent the vector obtained by decompressing and decoding the compressed and encoded input vector. The target large language model is a large language model that is adapted to the output codebook. There may be multiple large language models, and the multiple large language models correspond to multiple output codebooks respectively. The target large language model is determined from the multiple large language models, and then the output codebook adapted to the target large language model is obtained to make the output vector adapted to the target large language model.

[0080] It should be noted that in the embodiment of the present application, the output vector is not required to maintain similarity with the input vector, because when a new loss function is introduced, the weight cannot be determined. If the specific weight cannot be determined to merge different loss functions together, it will make model training difficult. Therefore, in the training process of the embodiment of the present application, there is no need to add additional loss functions. This can more flexibly adapt to different task requirements, so as to improve the adaptability of different large language models.

[0081] It should be noted that if Figure 2 The vector quantization process of the prior art shown uses a single codebook to generate discrete symbols and directly uses the discrete symbols for the input of the Transformer, while the embodiment of the present application uses two codebooks, first performs compression encoding on the input vector, then obtains the corresponding output codebook vector from the output vector for accumulation, and finally obtains the output vector, which is used to input the large language model, that is, the final input large language model is in vector form.

[0082] In this way, the embodiment of the present application adopts two codebooks, namely, an input codebook and an output codebook. The input vector is compressed by the input codebook, and then matched with the downstream large language model by the output codebook. For different large language models, it is only necessary to replace the corresponding output codebook to achieve matching with the large language model. That is, the method provided in the embodiment of the present application can improve the adaptability to the large language model.

[0083] In order to make the method provided in the embodiment of the present application clearer and easier to understand, Figure 4a and Figure 4b A specific embodiment of this method is described below. Figure 4a and Figure 4b , which is a process diagram of a vector quantization method for language model alignment provided in an embodiment of the present application.

[0084] Step 1: Initialize the input codebook and output codebook.

[0085] The input codebook and the output codebook both include multiple codebook vectors. The number of codebook vectors in the input codebook corresponds to the number of codebook vectors in the output codebook.

[0086] Step 2: When an input vector is input, the k codebook vectors closest to the input vector are searched in the input codebook, and a token combination is output according to the IDs of the k codebook vectors. The token combination is used to approximately represent the input vector.

[0087] Wherein, k is an integer greater than 1 but much smaller than the dimension of the input vector. As an example, the dimension of the input vector is 2048, k is 3, and the IDs of the three selected codebook vectors are used to approximately represent the original input vector.

[0088] As an example, the three codebook vectors most similar to the input vector x are obtained from the input codebook, namely codebook vector i, codebook vector j and codebook vector k, and three IDs are output as token combinations. The token combination is used to represent the input vector x. For the specific process, see Figure 4a Schematic diagram of the process of the vector quantization method for language model alignment shown.

[0089] Step 3: Convert the token combination back to vector form according to the output codebook, and obtain the output vector to adapt to the downstream language model alignment task.

[0090] As an example, according to the token combination obtained above, the corresponding codebook vector i, codebook vector j and codebook vector k are obtained in the output codebook and accumulated to obtain the output vector y. It should be noted that the specific process of step 2 can be found in Figure 4b Schematic diagram of the process of the vector quantization method for language model alignment shown.

[0091] In the specific comparative experiment, the COCO2014 dataset is used to train the language model alignment task. COCO2014 (Common Objects in Context) is a large-scale image recognition, segmentation, and annotation dataset. The purpose is to enable the open source large language model "facebook / opt-2.7b" to have the ability to understand pictures. It should be noted that because training a multimodal model from scratch usually requires a lot of time and training data, in the specific implementation process, the blip2 model parameters that have not been trained for the language model alignment task are loaded, rather than training the multimodal model from the initial state.

[0092] Subsequently, on 8 H800 graphics cards, the model with loaded parameters was trained together with the large language model opt-2.7b for the language model alignment task, and 10 epochs were executed, where one epoch is equal to the process of training once using all the samples in the training set.

[0093] Finally, the evaluation results of the three solutions, namely the traditional language model alignment method, the traditional vector quantization method and the vector quantization method for language model alignment provided in this application, are compared. For specific results, please see Table 1:

[0094]

[0095] It should be noted that in the above Table 1, CIDEr, SPICE and BLEU@4 are evaluation indicators for the image caption task. As can be seen from Table 1, the CIDEr, SPICE and BLEU@4 index scores of the traditional language model alignment method are 1.196, 0.209 and 0.361 respectively, the CIDEr, SPICE and BLEU@4 index scores of the traditional vector quantization method are 0.083, 0.029 and 0.055 respectively, and the CIDEr, SPICE and BLEU@4 index scores of the vector quantization method for semantic model alignment provided in this application are 1.239, 0.215 and 0.377 respectively, that is, the vector quantization method for semantic model alignment provided in the embodiment of this application can achieve good implementation results under various evaluation indicators.

[0096] Among them, the consensus-based Image Description Evaluation (CIDEr) is a consistency-based image description evaluation indicator, which is mainly used to evaluate the consistency between the generated image description and the reference description. Specifically, CIDEr calculates the n-gram matching scores between the candidate description and multiple reference descriptions, and then performs a weighted average of these scores. The higher the CIDEr score, the higher the semantic consistency between the candidate description and the reference description.

[0097] Semantic Propositional Image Caption Evaluation (SPICE) is a semantic propositional image description evaluation metric, which is mainly used to evaluate the semantic consistency between the generated image description and the reference description. The SPICE score ranges from 0 to 1. The higher the score, the higher the semantic consistency between the candidate description and the reference description.

[0098] The Bilingual Evaluation Understudy (BLEU) is an evaluation indicator for evaluating the quality of sentences in machine translation, and is also applicable to image description tasks. The BLEU indicator requires a reference text as a standard to calculate the degree of similarity between the score of the generated text and the reference text. The BLEU score is a number between 0 and 1. The higher the BLEU score, the closer the generated text is to the reference text and the greater the similarity.

[0099] See also Figure 5 The embodiment of the present application further provides a vector quantization device 500 for language model alignment, the device 500 comprising:

[0100] An initialization unit 501 is used to initialize an input codebook and an output codebook; the input codebook includes a plurality of input codebook vectors, and the output codebook includes a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; the subscripts of the input code vectors in the input code correspond to the subscripts of the output codebook vectors in the output codebook in a one-to-one correspondence;

[0101] A searching unit 502 is configured to perform a similarity search among the plurality of input codebook vectors based on the input vector, and obtain a preset number of input codebook vector subscripts in descending order of similarity, wherein the preset number of input codebook vector subscripts are used to represent the input vector after compression coding;

[0102] An acquiring unit 503 is configured to acquire a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts;

[0103] The accumulation unit 504 is used to accumulate the preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, wherein the output vectors are used to represent the vectors obtained by decompressing and decoding the compressed and encoded input vectors, and the output vectors are used to input a target large language model, and the target large language model is a large language model adapted to the output codebook.

[0104] Optionally, the search unit 502 includes a calculation subunit, a sorting subunit and an acquisition subunit;

[0105] The calculation subunit is used to calculate the similarity between the input vector and the multiple input codebook vectors respectively;

[0106] The sorting subunit is used to sort the input codebook vectors in descending order of similarity to obtain the preset number;

[0107] The acquisition subunit is configured to acquire the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

[0108] Optionally, the computing subunit is specifically used for:

[0109] Performing point multiplication operations on the input vector and the multiple input codebook vectors respectively to obtain multiple point multiplication operation results, wherein the point multiplication operation results are used to represent the similarity between the input vector and the input codebook vector.

[0110] Optionally, the sorting subunit is specifically used to:

[0111] The multiple point product operation results are sorted in descending order, and the preset number of input codebook vectors are obtained in sequence according to the results of the descending order.

[0112] Optionally, the preset number is an integer, the preset number is greater than 1 and the preset number is smaller than the dimension of the input vector.

[0113] It should be noted that the specific implementation method of the device 500 and the technical effects achieved can be found in Figure 3 The relevant description in the method shown.

[0114] In addition, the present application embodiment also provides an electronic device 600, such as Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 602:

[0115] The memory 602 is used to store computer programs;

[0116] The processor 601 is used to execute according to the computer program Figure 3The method provided.

[0117] In addition, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method provided by the embodiment of the present application.

[0118] Through the description of the above implementation methods, it can be known that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a general hardware platform. Based on such an understanding, the technical solution of the present application can be embodied in the form of a software product, and the computer software product can be stored in a storage medium, such as a read-only memory (ROM) / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network communication device such as a router) to execute the methods described in each embodiment of the present application or some parts of the embodiments.

[0119] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is only schematic, in which the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the goals of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0120] The above is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. It should be noted that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the present application, and these improvements and modifications should also be regarded as the protection scope of the present application.

Claims

1. A vector quantization method for language model alignment, characterized in that: The method comprises: Initialize an input codebook and an output codebook; the input codebook includes a plurality of input codebook vectors, and the output codebook includes a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; and the subscripts of the input codebook vectors in the input codebook correspond one-to-one to the subscripts of the output codebook vectors in the output codebook; Performing a similarity search among the multiple input codebook vectors based on the input vector, obtaining a preset number of input codebook vector subscripts in descending order of similarity, wherein the preset number of input codebook vector subscripts are used to represent the input vector after compression coding; Obtaining a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts; Accumulating the preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, wherein the output vectors are used to represent vectors obtained by decompressing and decoding the compressed and encoded input vectors, and the output vectors are used to input a target large language model, wherein the target large language model is a large language model adapted to the output codebook.

2. The method according to claim 1, characterized in that The performing similarity search among the plurality of input codebook vectors based on the input vector, and obtaining a preset number of input codebook vector subscripts in descending order of similarity, comprises: Calculating similarities between the input vector and the multiple input codebook vectors respectively; Sorting the preset number of input codebook vectors in descending order of similarity; Obtain the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

3. The method according to claim 2, characterized in that The respectively calculating the similarities between the input vector and the plurality of input codebook vectors comprises: Performing point multiplication operations on the input vector and the multiple input codebook vectors respectively to obtain multiple point multiplication operation results, wherein the point multiplication operation results are used to represent the similarity between the input vector and the input codebook vector.

4. The method according to claim 3, characterized in that: The obtaining of the preset number of input codebook vectors by sorting in descending order of similarity includes: The multiple point product operation results are sorted in descending order, and the preset number of input codebook vectors are obtained in sequence according to the results of the descending order.

5. The method according to claim 1, characterized in that The preset number is an integer, the preset number is greater than 1 and the preset number is smaller than the dimension of the input vector.

6. A vector quantization device for language model alignment, characterized in that: The device comprises: an initialization unit, configured to initialize an input codebook and an output codebook; the input codebook includes a plurality of input codebook vectors, and the output codebook includes a plurality of output codebook vectors; the number of input codebook vectors in the input codebook is the same as the number of output codebook vectors in the output codebook; and the subscripts of the input codebook vectors in the input codebook correspond one-to-one to the subscripts of the output codebook vectors in the output codebook; A search unit, configured to perform a similarity search among the plurality of input codebook vectors based on the input vector, and obtain a preset number of input codebook vector subscripts in descending order of similarity, wherein the preset number of input codebook vector subscripts are used to represent the input vector after compression coding; An acquiring unit, configured to acquire a preset number of output codebook vector subscripts in the output codebook according to the preset number of input codebook vector subscripts; An accumulation unit is used to accumulate the preset number of output codebook vectors corresponding to the preset number of output codebook vector subscripts to obtain output vectors, wherein the output vectors are used to represent the vectors obtained by decompressing and decoding the compressed and encoded input vectors, and the output vectors are used to input a target large language model, and the target large language model is a large language model adapted to the output codebook.

7. The device according to claim 6, characterized in that The search unit is specifically used for: Calculating similarities between the input vector and the multiple input codebook vectors respectively; Sorting the preset number of input codebook vectors in descending order of similarity; Obtain the preset number of input codebook vector subscripts corresponding to the preset number of input codebook vectors.

8. The device according to claim 6, characterized in that The preset number is an integer, the preset number is greater than 1 and the preset number is smaller than the dimension of the input vector.

9. An electronic device, characterized in that: The electronic device comprises a processor and a memory: The memory is used to store computer programs; The processor is configured to execute the method according to any one of claims 1 to 5 according to the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the method according to any one of claims 1 to 5.