Text recognition method and device for large language model and electronic equipment

Abstract

The embodiment of the invention provides a text recognition method and device of a large language model and electronic equipment, and relates to the technical field of natural language processing. The method comprises the steps that a token sequence of a text sequence is obtained; determining a dummy token for storing the attention score offset, inputting the sliding window part token into the large language model, and outputting a first prediction token; calculating an attention score of each sliding window part token relative to the first prediction token, and selecting a plurality of key tokens; selecting a newly added token; taking the dummy token, the key token and the newly added token as a token combination, inputting the token combination into the large language model, and outputting a second prediction token; updating the token combination until the token sequence is completely input; and outputting a recognition result of the text sequence. The text recognition accuracy can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a text recognition method, a device and an electronic device for a large language model. Background Art

[0002] A large language model is a large neural network model specifically used in deep learning to understand, generate, and process natural language. It usually contains billions or even hundreds of billions of parameters. By training on large-scale text data, it has mastered rich language knowledge and complex semantic relationships, enabling it to generate coherent text, answer questions, translate languages, etc. The attention mechanism is a core component in the large language model, which can help the model "selectively focus on" specific parts of the text according to the different importance of the context when processing the text.

[0003] The necessity of the attention mechanism for large language models lies in its ability to capture global context and long-distance dependencies, allowing the model to focus on the key content in the sequence when processing long texts. At the same time, the attention mechanism supports parallel computing, improves the efficiency of the model, and enhances its language understanding and generation performance in multiple tasks. This enables large language models to generate more coherent and semantically consistent outputs, and show superior performance in complex semantic tasks.

[0004] However, the text recognition process of existing large language models is limited by the limited memory capacity related to the length of the input text, which makes it impossible to make the length of the input text sequence very large. In this case, the initial token or word unit will be discarded. As the text recognition process progresses, the recognition results of the model will seriously deviate from the text, resulting in a large deviation in text recognition accuracy, which limits the recognition performance of the large language model. Summary of the invention

[0005] The embodiment of the present invention provides a text recognition method, an electronic device and a computer-readable storage medium for a large language model to solve the problem that the existing large language model text recognition process is limited by the limited video memory capacity related to the length of the input text, which makes it impossible to make the length of the input text sequence very large. In this case, the initial token, i.e., word unit, will be discarded. As the text recognition process proceeds, the recognition result of the model will seriously deviate from the text, resulting in a large deviation in the text recognition accuracy, which limits the recognition performance of the large language model.

[0006] The embodiment of the present invention also discloses a text recognition method of a large language model, the method comprising:

[0007] S1: Get the token sequence of the text sequence;

[0008] S2: Determine a dummy token for storing the attention score bias at the head of the token sequence, input the sliding window part of the tokens in the optimized token sequence with the dummy token into the large language model, and output the first predicted token;

[0009] S3: Calculate the attention score of each sliding window partial token relative to the first predicted token, and select a first preset number of key tokens whose attention scores are greater than a preset attention score;

[0010] S4: Selecting a second preset number of newly added tokens from the token sequence and inputting them into the large language model;

[0011] S5: Input the dummy token, each key token and each newly added token as a token combination sequence into the large language model, and output the second predicted token;

[0012] S6: Use the second predicted token as the first predicted token, and the token combination as the sliding window partial token, and return to step S3 until the tokens in the token sequence are input into the large language model;

[0013] S7: Based on the obtained first prediction token and second prediction token, output the recognition result of the text sequence.

[0014] The embodiment of the present invention also discloses a text recognition device for a large language model, the device comprising:

[0015] The acquisition module is used to obtain the token sequence of the text sequence;

[0016] A determination module is used to determine a dummy token for storing an attention score bias at the head of a token sequence, input a sliding window portion of tokens in an optimized token sequence having the dummy token into a large language model, and output a first predicted token;

[0017] A calculation module, used to calculate the attention score of each sliding window partial token relative to the first predicted token, and select a first preset number of key tokens whose attention scores are greater than a preset attention score;

[0018] A selection module, configured to select a second preset number of newly added tokens from the token sequence to be input into the large language model;

[0019] The first input module is used to input the dummy token, each key token and each newly added token as a token combination into the large language model, and output a second predicted token;

[0020] A second input module is used to use the second predicted token as the first predicted token and the token combination as the sliding window partial token, and return to step S3 until the tokens in the token sequence are input into the large language model;

[0021] The output module is used to output the recognition result of the text sequence based on the obtained first prediction token and the second prediction token.

[0022] The embodiment of the present invention further discloses an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0023] Memory, used to store computer programs;

[0024] The processor is used to implement the method of the embodiment of the present invention when executing the program stored in the memory.

[0025] The embodiments of the present invention include the following advantages:

[0026] In the present invention, first, a dummy token for storing the attention score bias is determined at the head of the token sequence of the text sequence to be recognized, and then, a first predicted token is output according to the sliding window partial token, and then the attention score of each sliding window partial token relative to the first predicted token is calculated, and the key token with a higher score is selected, and multiple newly added tokens of the newly input large language model are selected from the token sequence, and the sliding window partial token is replaced based on the obtained dummy token, key token and newly added token to generate a new second prediction result, and so on, until the sliding window traverses the entire text sequence. In this process, the length of the token input to the large language model can adapt to different sizes of video memory capacity, and the adaptability is stronger. Moreover, the input token retains the initial token of the text sequence, avoids the situation where the recognition result deviates from the text, and improves the accuracy of the text recognition result. Each input token is based on the key token that has a greater impact on the previous prediction result, which can retain the important semantic information of the text. Each input token also has a newly added token of the text sequence, maintaining the continuity of the recognition result. In summary, this solution can process text sequences of different lengths without changing the original video memory size, can easily obtain the global and local features of the text sequence, and can well maintain the accuracy of the recognition results of the text sequence. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of a text recognition method using a large language model provided in an embodiment of the present invention;

[0028] Figure 2 is a block diagram of a text recognition device of a large language model provided in an embodiment of the present invention;

[0029] Figure 3 is a block diagram of an electronic device provided in an embodiment of the present invention;

[0030] Figure 4 is a schematic diagram of a computer-readable medium provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0032] Reference Figure 1 , shows a step flow chart of a text recognition method for a large language model provided in an embodiment of the present invention.

[0033] The specific steps may include:

[0034] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0035] Methods include:

[0036] S1: Get the token sequence of the text sequence.

[0037] Among them, the text sequence refers to the original natural language text to be recognized or processed, in the form of sentences, paragraphs or entire texts. The token sequence refers to the sequence of smaller units obtained by word segmentation or decomposition of the text sequence. In large language models, text needs to be decomposed into tokens when processing it, because the model cannot directly understand and process continuous text strings. The token sequence allows the model to treat each token as a separate input unit for attention calculation, semantic analysis, and feature extraction.

[0038] S2: Determine a dummy token for storing the attention score bias at the head of the token sequence, input the sliding window part of the tokens in the optimized token sequence with the dummy token into the large language model, and output the first predicted token.

[0039] The attention score bias represents the offset value gradually accumulated due to multiple Softmax operations, causing the attention score to deviate from its initial ideal distribution. This bias will affect the model's attention effect on long sequence content. Specifically, the attention score bias represents the value of the attention score deviation from the distribution caused by the softmax operation during training or inference.

[0040] Among them, Dummy token is a special "placeholder" token, which is inserted at the head of the token sequence and is mainly used to save the attention score bias. This bias is due to the fact that after multiple Softmax operations, the attention score gradually accumulates and deviates from the ideal distribution, thus affecting the model's attention to important content. The existence of Dummy token can absorb and stabilize these biases, so that the model can maintain the accuracy of attention distribution when processing long text sequences. The optimized token sequence refers to the token sequence after adding dummy tokens and adjusting it. The sliding window partial token refers to a small part of the token extracted from the optimized token sequence, and the sliding window mechanism is used to control the length of the sequence. The sliding window mechanism allows the model to process only a small range of tokens in a long sequence at a time, reducing the burden on video memory. The sliding window will gradually slide along the entire sequence so that the model covers the entire text sequence during processing. The first predicted token is the first prediction result generated by the model after the sliding window partial token is input.

[0041] It should be noted that a dummy token is added to the head of the token sequence to absorb the attention score bias, and then the sliding window part of the optimized token sequence containing the dummy token is input into the model to obtain the first prediction result (the first predicted token). This process ensures the stability of attention distribution in long sequence processing, reduces the burden of video memory and improves prediction accuracy.

[0042] In a possible implementation, determining a dummy token for storing the attention score bias at the head of the token sequence in S2 specifically includes:

[0043] S201: Determine an operation process of a large language model, where the operation process includes a training process and an inference process.

[0044] The training process is the stage of model learning and optimization. During the training process, the model updates parameters through a large amount of labeled or unlabeled data (such as text corpora) to learn the structure and semantic relationships of the language. The inference process is the application stage of the model to generate output on new data, that is, the actual prediction stage. During the inference process, the model no longer adjusts parameters, but predicts or generates based on the trained knowledge.

[0045] S202: Determine a dummy token according to the running process.

[0046] Specifically, the setting of dummy token is determined by judging whether the current operation process of the large language model is training or inference, and the current operation state is identified so that the appropriate dummy token can be set according to different states. This process ensures that the role of dummy token can effectively absorb the attention score bias in both training and inference, thereby optimizing the attention distribution of the model at different operation stages.

[0047] In a possible implementation, S202 specifically includes:

[0048] S2021: When the running process is a training process, add a dummy token to the head of the token sequence.

[0049] S2022: When the running process is an inference process, starting from the start position of the token sequence, a third preset number of tokens are selected in sequence as dummy tokens.

[0050] Specifically, in S2021, if it is a training process, a dummy token is directly added to the head of the token sequence. In S2022, if it is an inference process, a certain number (third preset number) of tokens are selected from the beginning of the sequence as dummy tokens. This method flexibly sets dummy tokens according to different states to ensure that both training and inference stages can effectively absorb attention score bias.

[0051] It should be noted that those skilled in the art can set the size of the third preset number according to actual needs, and the present invention is not limited thereto.

[0052] In a possible implementation, the added dummy token is equivalent to a token.

[0053] In a possible implementation manner, the third preset number is less than or equal to 5.

[0054] S3: Calculate the attention score of each sliding window partial token relative to the first predicted token, and select a first preset number of key tokens whose attention scores are greater than a preset attention score.

[0055] Among them, the attention score is a numerical indicator used by the model to measure the correlation between each sliding window partial token and the first predicted token. The higher the score, the greater the influence of the token on the generation of the first predicted token. In large language models, attention scores are usually calculated through a self-attention mechanism, using a Softmax function to generate a probability distribution so that the model pays attention to high-scoring tokens. Key tokens are those whose attention scores are higher than the preset threshold. The model identifies these key tokens because they play an important role in understanding the current context or subsequent generation. By screening key tokens, the model can focus more on important information in the text and improve the accuracy of text recognition and generation.

[0056] It should be noted that the model calculates the attention score of each sliding window partial token relative to the first predicted token, and selects the key tokens with scores higher than the preset value. The selected key tokens help the model focus on important information in long sequence processing and improve the accuracy of prediction.

[0057] It should be noted that those skilled in the art can set the preset attention score and the size of the first preset number according to actual needs, and the present invention is not limited thereto.

[0058] S4: Selecting a second preset number of newly added tokens from the token sequence and inputting them into the large language model.

[0059] Among them, the newly added token refers to the latest token selected from the original token sequence that has not been input into the model. These newly added tokens are usually located at the end of the sequence and represent the part of the text that has not been processed before the current sliding window. The introduction of newly added tokens can provide the model with the latest context information, ensure that the model maintains semantic coherence during the long text recognition process, and effectively connect the processed content with the new content.

[0060] It should be noted that those skilled in the art can set the size of the second preset number according to actual needs, and the present invention is not limited thereto.

[0061] In a possible implementation, the total number of newly added tokens is the same as the number of tokens in the sliding window portion.

[0062] It should be noted that the total number of newly added tokens is set to be equal to the number of tokens in the sliding window, which means that each time the sliding window moves, the newly introduced tokens will completely replace the old tokens in the sliding window. This design ensures that the model processes the same number of tokens each time when processing long texts, maintains the stability of video memory usage, and ensures semantic coherence.

[0063] In a possible implementation manner, after S4, the method further includes:

[0064] When the total number of newly added tokens in the sliding window part input to the large language model is less than the sum of the number of tokens calculated from the number of tokens in the dummy token, the first preset number and the second preset number, the newly added tokens in the sliding window part are used as the token combination sequence.

[0065] It should be noted that when the number of newly added tokens in the sliding window is less than the sum of the dummy token, the first preset number of key tokens, and the second preset number of newly added tokens, these newly added tokens are input into the model as a complete token combination sequence. This setting ensures that the complete text sequence is recognized as much as possible, thereby improving the recognition accuracy of the text sequence.

[0066] S5: Input the dummy token, each key token and each newly added token as a token combination into the large language model, and output the second predicted token.

[0067] It should be noted that the model integrates dummy tokens, key tokens, and newly added tokens into a complete token combination and inputs them into the large language model. These tokens represent the initial bias control, important information in the current context, and the latest context, respectively. Through this combined input, the model can take into account the global information and the latest information of a long sequence when outputting the second predicted token, achieving stable and coherent text generation or recognition. Moreover, in this process, the length of the input text sequence can be adjusted at will. Through this token selection mechanism, the accuracy and reliability of the recognition results can be improved and the recognition bias can be reduced without exceeding the size of the video memory.

[0068] It is understandable that there are three parts: adding dummy at the text header, adding the token with the highest attention score in each iteration, and adding the newly input token. The first part avoids the error accumulation problem; the second part increases the accuracy of text recognition while getting rid of the text length constraint problem; the third part increases the continuity of the predicted text.

[0069] S6: Use the second predicted token as the first predicted token and the token combination as the sliding window partial token, and return to step S3 until the tokens in the token sequence are input into the large language model.

[0070] It should be noted that the model uses the second predicted token as the new first predicted token, and sets the current token combination as the new sliding window partial token, and re-enters step S3 for the next round of processing. By repeating this step continuously, the model can gradually input and recognize the entire text sequence until all tokens are processed, thus achieving continuous recognition of long texts.

[0071] S7: Based on the obtained first prediction token and second prediction token, output the recognition result of the text sequence.

[0072] It should be noted that the model generates the final text recognition result based on the output sequence of the accumulated first prediction token and the second prediction token. By gradually predicting and combining these tokens, the model can form a complete and coherent output that accurately reflects the semantic content of the entire input text and completes the text sequence recognition task.

[0073] In a possible implementation, S7 specifically includes:

[0074] S701: Arrange the obtained first prediction token and second prediction token in output order.

[0075] S702: Output the arrangement result as the recognition result of the text sequence.

[0076] It should be noted that the entire solution, whether in the training process or the inference process, can be summarized as follows: Select the first few tokens of the text sequence. Select the tokens with relatively high attention scores in the middle of the text sequence. Select the latest token added to the sequence. Generate subsequent tokens by adding the tokens of the three parts to the calculation at the same time.

[0077] During the training process, the input token sequence is first obtained, then the dummy token is added, the Top-K high attention tokens are selected, and the nearest token is obtained. After that, the attention calculation is performed to generate a new token, and then the sliding window is updated, and the dummy token, high attention token and the nearest token are re-determined. This cycle is repeated to generate multiple new tokens until the entire sequence is traversed and the result, that is, the generated token, is output.

[0078] During the reasoning process, the input token sequence is first obtained, and then the initial few tokens are selected as dummy tokens, the Top-K high attention tokens are selected, and the most recent token is obtained. After that, the attention calculation is performed to generate a new token, and then the sliding window is updated, and the dummy token, high attention token and the most recent token are re-determined. This cycle is repeated to generate multiple new tokens until the entire sequence is traversed and the result, that is, the generated token, is output.

[0079] Among them, the initial token of the text sequence is important in the entire sequence for the following reasons. In the process of speech model inference calculation, there are multiple softmax operations. The main function of this operation is to make the data distribution more deviated and to perform gradient descent better. However, since the calculation result of softmax makes each item greater than 0, this leads to the fact that the attention scores of some tokens should have been 0 or negative, but due to the softmax operation, their attention scores are positive, and as the number of softmax increases, the attention scores accumulate, which will cause the attention scores to deviate from the original distribution. This is also the reason why the window attention mechanism, once the initial tokens of the sequence slide out of the window, the confusion of the sequence generation will become higher. In addition, the reason why several tokens with higher attention scores in the middle of the sequence need to be retained is that no matter in the actual training or inference process, the key tokens will always get higher attention scores. These tokens have a great influence on the subsequent output. Usually, these tokens contain important semantic information in the sequence. The influence of the latest token added in the sequence on the generation of subsequent tokens is mainly because the semantics in the sequence is usually continuous, and this part of the tokens has a need for semantic coherence for the generation of subsequent tokens. By retaining the above three tokens for subsequent token generation, the language model does not retain all tokens, saving the memory capacity required for the process of calculating the attention mechanism. The initial token is retained to absorb the attention score distribution deviation caused by softmax, the tokens with high attention scores in the sequence are retained so that key semantic information is not lost, and the latest token in the sequence is retained to ensure semantic coherence. After this process, the long sequence adaptive attention mechanism of the large language model can be realized.

[0080] More specifically, during the training process, first, input the token sequence. Then, select the beginning of the text sequence and add a dummy token. This token is used to save the value of the attention score deviation caused by the subsequent softmax operation. Secondly, select the top-K tokens with high attention scores in the text sequence. After that, get the token of the latest sequence. Finally, the three tokens are used together in the attention calculation to predict the generation of the next token.

[0081] For example, if you want to train a large language model, first, add a dummy token at the beginning of the sequence. Second, retain K tokens with higher attention scores during the training process. Third, select the most recently added W tokens for subsequent calculations. At this time, the number of attention tokens is 1+K+W. When the input sequence length is less than or equal to 1+K+W, all tokens in the entire sequence are added to the calculation of the attention score. When the length of the sequence is greater than 1+K+W, the number of tokens involved in the calculation is 1+K+W tokens. This judgment process is repeated until all training sequences are input. When the training is completed, during the inference process of the large language model, as long as K and W are kept consistent with the training process and a dummy token is added to the beginning of the sequence, inference can be performed.

[0082] More specifically, during the inference process, first, input the token sequence. Secondly, extract the initial token from the beginning of the text sequence, which can be 1, 2, 3, usually no more than 5. The token is used to save the value of the attention score deviation distribution caused by the subsequent softmax operation. After that, get the top-K tokens with high attention scores from the subsequent sequence. Then, get the token of the latest sequence. The three tokens jointly participate in the attention calculation to predict the generation of the next token.

[0083] For example, when a large language model is not trained, the adaptive attention mechanism can also be used. The specific steps are as follows: First, select the initial N tokens as dummy tokens to absorb the deviation of the attention score. Second, retain K tokens with higher attention scores. Third, select the latest W tokens for subsequent calculations. At this time, the number of attention tokens is N+K+W. When the input sequence length is less than or equal to N+K+W, all tokens in the entire sequence are added to the calculation of the attention score. When the length of the sequence is greater than N+K+W, the tokens involved in the calculation are the tokens in N+K+W. This judgment process is repeated until the entire sequence is input.

[0084] It should be noted that since the training process of the language model is inconsistent with the process proposed in the present invention, the above N, K, and W are all hyperparameters of the model. Experiments are needed to determine the values ​​of N, K, and W. Different models may have different selections of these three values.

[0085] It is understandable that this solution can achieve long sequence output of large language models under certain video memory conditions. And it can handle ultra-long sequences. By introducing Dummy Token to stabilize attention, the model is no longer limited by the length of the pre-trained sequence and can handle inputs far exceeding this length. Secondly, only a few additional operations are required to significantly enhance the robustness and inference ability of the model, and the calculation is efficient. In addition, the global Dummy Token and the local high-attention Token are used at the same time, combining global and local information. If it is a pre-training process, dummy tokens can be added to absorb the part of the attention score deviation caused by the softmax operation, and the above process can be achieved without occupying the number of tokens. For a large language model that has been trained, the initial few tokens of the sequence can still be selected to achieve stable output of the model.

[0086] In actual application, accurate recognition of long texts is achieved by gradually processing the tokens of the text sequence. First, the token sequence is decomposed and a dummy token is inserted at the head to absorb the attention bias. Then the sliding window mechanism is used to input the model batch by batch, and the attention score of each token relative to the prediction result is calculated to screen out important key tokens. At the same time, by introducing new tokens, the model continues to obtain the latest context information. At each step, the dummy token, key token and new token are combined and input into the model to generate a coherent prediction result until the entire sequence is processed. Finally, the prediction results are arranged in order and the complete text recognition result is output. This process ensures stable processing of long sequences within limited video memory and improves recognition accuracy and continuity.

[0087] The embodiments of the present invention include the following advantages:

[0088] In the present invention, first, a dummy token for storing the attention score bias is determined at the head of the token sequence of the text sequence to be recognized, and then, a first predicted token is output according to the sliding window partial token, and then the attention score of each sliding window partial token relative to the first predicted token is calculated, and the key token with a higher score is selected, and multiple newly added tokens of the newly input large language model are selected from the token sequence, and the sliding window partial token is replaced based on the obtained dummy token, key token and newly added token to generate a new second prediction result, and so on, until the sliding window traverses the entire text sequence. In this process, the length of the token input to the large language model can adapt to different sizes of video memory capacity, and the adaptability is stronger. Moreover, the input token retains the initial token of the text sequence, avoids the situation where the recognition result deviates from the text, and improves the accuracy of the text recognition result. Each input token is based on the key token that has a greater impact on the previous prediction result, which can retain the important semantic information of the text. Each input token also has a newly added token of the text sequence, maintaining the continuity of the recognition result. In summary, this solution can process text sequences of different lengths without changing the original video memory size, can easily obtain the global and local features of the text sequence, and can well maintain the accuracy of the recognition results of the text sequence.

[0089] In addition, refer to Figure 2 , shows a block diagram of a text recognition device of a large language model provided in an embodiment of the present invention.

[0090] The embodiment of the present invention further discloses a text recognition device 20 for a large language model, comprising:

[0091] An acquisition module 201 is used to acquire a token sequence of a text sequence;

[0092] A determination module 202 is used to determine a dummy token for storing an attention score bias at the head of the token sequence, input a sliding window portion of tokens in the optimized token sequence having the dummy token into the large language model, and output a first predicted token;

[0093] A calculation module 203, configured to calculate an attention score of each sliding window partial token relative to the first predicted token, and select a first preset number of key tokens whose attention scores are greater than a preset attention score;

[0094] A selection module 204, configured to select a second preset number of newly added tokens from the token sequence to be input into the large language model;

[0095] A first input module 205 is used to input the dummy token, each key token and each newly added token as a token combination into the large language model, and output a second predicted token;

[0096] The second input module 206 is used to use the second predicted token as the first predicted token and the token combination as the sliding window partial token, and return to step S3 until the tokens in the token sequence are input into the large language model;

[0097] The output module 207 is used to output the recognition result of the text sequence based on the obtained first prediction token and the second prediction token.

[0098] In a possible implementation, the determination module 202 determines a dummy token for storing the attention score bias at the head of the token sequence, specifically:

[0099] Determine the operation process of the large language model, where the operation process includes a training process and an inference process;

[0100] Determine the dummy token based on the running process.

[0101] In a possible implementation, an operation process of a large language model is determined, wherein the operation process includes a training process and an inference process, specifically:

[0102] When the running process is the training process, add a dummy token to the head of the token sequence;

[0103] When the running process is an inference process, starting from the start position of the token sequence, a third preset number of tokens are selected in sequence as dummy tokens.

[0104] In a possible implementation, the added dummy token is equivalent to a token.

[0105] In a possible implementation manner, the third preset number is less than or equal to 5.

[0106] In a possible implementation, the total number of newly added tokens is the same as the number of tokens in the sliding window portion.

[0107] In a possible implementation, the device further includes:

[0108] When the total number of newly added tokens in the sliding window part input to the large language model is less than the sum of the number of tokens calculated from the number of tokens in the dummy token, the first preset number and the second preset number, the newly added tokens in the sliding window part are used as the token combination sequence.

[0109] In a possible implementation, the output module 207 is specifically used to:

[0110] Arrange the obtained first prediction token and second prediction token in output order;

[0111] The arrangement result is output as the recognition result of the text sequence.

[0112] The embodiments of the present invention include the following advantages:

[0113] In the present invention, first, a dummy token for storing the attention score bias is determined at the head of the token sequence of the text sequence to be recognized, and then, a first predicted token is output according to the sliding window partial token, and then the attention score of each sliding window partial token relative to the first predicted token is calculated, and the key token with a higher score is selected, and multiple newly added tokens of the newly input large language model are selected from the token sequence, and the sliding window partial token is replaced based on the obtained dummy token, key token and newly added token to generate a new second prediction result, and so on, until the sliding window traverses the entire text sequence. In this process, the length of the token input to the large language model can adapt to different sizes of video memory capacity, and the adaptability is stronger. Moreover, the input token retains the initial token of the text sequence, avoids the situation where the recognition result deviates from the text, and improves the accuracy of the text recognition result. Each input token is based on the key token that has a greater impact on the previous prediction result, which can retain the important semantic information of the text. Each input token also has a newly added token of the text sequence, maintaining the continuity of the recognition result. In summary, this solution can process text sequences of different lengths without changing the original video memory size, can easily obtain the global and local features of the text sequence, and can well maintain the accuracy of the recognition results of the text sequence.

[0114] In addition, an embodiment of the present invention further provides an electronic device, such as Figure 3 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303 and a communication bus 1304, wherein the processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0115] Memory 1303, used for storing computer programs;

[0116] The processor 1301 is used to implement the text recognition method of the large language model in the method embodiment when executing the program stored in the memory 1303.

[0117] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0118] The communication interface is used for communication between the above terminal and other devices.

[0119] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0120] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0121] In the present invention, first, a dummy token for storing the attention score bias is determined at the head of the token sequence of the text sequence to be recognized, and then, a first predicted token is output according to the sliding window partial token, and then the attention score of each sliding window partial token relative to the first predicted token is calculated, and the key token with a higher score is selected, and multiple newly added tokens of the newly input large language model are selected from the token sequence, and the sliding window partial token is replaced based on the obtained dummy token, key token and newly added token to generate a new second prediction result, and so on, until the sliding window traverses the entire text sequence. In this process, the length of the token input to the large language model can adapt to different sizes of video memory capacity, and the adaptability is stronger. Moreover, the input token retains the initial token of the text sequence, avoids the situation where the recognition result deviates from the text, and improves the accuracy of the text recognition result. Each input token is based on the key token that has a greater impact on the previous prediction result, which can retain the important semantic information of the text. Each input token also has a newly added token of the text sequence, maintaining the continuity of the recognition result. In summary, this solution can process text sequences of different lengths without changing the original video memory size, can easily obtain the global and local features of the text sequence, and can well maintain the accuracy of the recognition results of the text sequence.

[0122] like Figure 4 As shown, in another embodiment provided by the present invention, a computer-readable storage medium 1401 is also provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the text recognition method of the large language model described in the above embodiment.

[0123] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the text recognition method of the large language model described in the above embodiment.

[0124] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0125] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0126] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0127] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A text recognition method for a large language model, characterized in that the method include: S1: Get the token sequence of the text sequence; S2: Determine a dummy token for storing the attention score bias at the head of the token sequence, input the sliding window part of the tokens in the optimized token sequence with the dummy token into the large language model, and output a first predicted token; S3: calculating the attention score of each sliding window partial token relative to the first predicted token, and selecting a first preset number of key tokens whose attention scores are greater than a preset attention score; S4: selecting a second preset number of newly added tokens from the token sequence and inputting them into the large language model; S5: Input the dummy token, each key token and each newly added token as a token combination into the large language model, and output a second predicted token; S6: using the second predicted token as the first predicted token and the token combination as the sliding window partial token, and returning to step S3 until the tokens in the token sequence are completely input to the large language model; S7: Based on the obtained first prediction token and second prediction token, output the recognition result of the text sequence.

2. The text recognition method of a large language model according to claim 1, characterized in that: The step S2 determines a dummy token for storing the attention score bias at the head of the token sequence, specifically including: S201: Determine an operation process of the large language model, wherein the operation process includes a training process and an inference process; S202: Determine the dummy token according to the running process.

3. The text recognition method of a large language model according to claim 2, characterized in that: The S202 specifically includes: S2021: When the running process is a training process, add the dummy token to the head of the token sequence; S2022: When the running process is an inference process, starting from the start position of the token sequence, a third preset number of tokens are selected in sequence as the dummy tokens.

4. The text recognition method of a large language model according to claim 3, characterized in that: The added dummytoken is equivalent to a token.

5. The text recognition method of a large language model according to claim 3, characterized in that: The third preset number is less than or equal to 5.

6. The text recognition method of a large language model according to claim 1, characterized in that: The total number of newly added tokens is the same as the number of tokens in the sliding window part.

7. The text recognition method of a large language model according to claim 1, characterized in that: After S4, the method further includes: When the total number of newly added tokens in the sliding window part input to the large language model is less than the sum of the number of tokens calculated by the number of tokens in the dummy token, the first preset number and the second preset number, the newly added tokens in the sliding window part are used as the token combination sequence.

8. The text recognition method of a large language model according to claim 1, characterized in that: The S7 specifically includes: S701: Arrange the obtained first prediction token and second prediction token in output order; S702: Output the arrangement result as the recognition result of the text sequence.

9. A text recognition device for a large language model, characterized in that: The device comprises: The acquisition module is used to obtain the token sequence of the text sequence; A determination module, configured to determine a dummy token for storing an attention score bias at the head of the token sequence, input a sliding window portion of tokens in the optimized token sequence having the dummy token into the large language model, and output a first predicted token; A calculation module, used to calculate the attention score of each sliding window partial token relative to the first predicted token, and select a first preset number of key tokens whose attention scores are greater than a preset attention score; A selection module, configured to select a second preset number of newly added tokens from the token sequence to be input into the large language model; A first input module, used for inputting the dummy token, each key token and each newly added token as a token combination into the large language model, and outputting a second predicted token; A second input module is used to use the second predicted token as the first predicted token and the token combination as the sliding window partial token, and return to step S3 until the tokens in the token sequence are completely input to the large language model; The output module is used to output the recognition result of the text sequence based on the obtained first prediction token and the second prediction token.

10. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is used to store computer programs; The processor is used to implement the text recognition method of the large language model as described in any one of claims 1-8 when executing the program stored in the memory.

Images