Text data processing method and device, storage medium and electronic equipment

Abstract

Embodiments of the present application provide a text data processing method and device, a storage medium and an electronic device, wherein the method comprises: in response to the input text obtained, converting the input text into an input vector corresponding to the input text, wherein the input text is a request text input by a request end; using a large language model to perform inference in a loop according to the input vector, sequentially sampling words from the inference result output by the large language model, and sequentially decoding the word sequences obtained by sampling words to obtain a group of decoding results; sequentially matching the group of decoding results with text elements in a text element table, and outputting the text elements matched from the text element table to the request end, wherein the text element table contains preset text elements allowed to be output to the request end. Through the present application, the problem of garbled code in the process of displaying the model output in the related art text data processing method is solved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a method and apparatus for processing text data, a storage medium, and an electronic device. Background Technology

[0002] As a type of large model in Artificial Intelligence (AI), the Large Language Model (LLM) is a natural language processing model based on deep learning technology. It can output inference results in a streaming manner based on the input vector. That is, it performs inference in a loop, samples the obtained inference results, decodes the sampled results, and outputs them.

[0003] The sampling results obtained from word extraction are usually tokens. However, a single word or character in some large language models may contain several tokens. Decoding the sampling results may result in temporary garbled text. Therefore, text data processing methods in related technologies suffer from the problem of garbled text during the display of model output. Summary of the Invention

[0004] This application provides a method and apparatus for processing text data, a storage medium, and an electronic device to at least solve the problem of garbled characters in the display model output process of text data processing methods in related technologies.

[0005] According to one aspect of the embodiments of this application, a method for processing text data is provided, comprising: in response to acquired input text, converting the input text into an input vector corresponding to the input text, wherein the input text is a request text input by a requesting end; performing inference cyclically using a large language model based on the input vector, sequentially sampling and extracting words from the inference results output by the large language model, and sequentially decoding the word sequence obtained from the sampling and extraction to obtain a set of decoding results; sequentially matching the set of decoding results with text elements in a text element table, and outputting the text elements matched from the text element table to the requesting end, wherein the text element table contains preset text elements that are allowed to be output to the requesting end.

[0006] According to another aspect of the embodiments of this application, a text data processing apparatus is provided, comprising: a conversion unit, configured to convert the input text into an input vector corresponding to the input text in response to an acquired input text, wherein the input text is a request text input by a requesting end; a first execution unit, configured to perform inference cyclically using a large language model based on the input vector, sequentially sample and extract words from the inference results output by the large language model, and sequentially decode the word sequence obtained from the sampled words to obtain a set of decoding results; and a second execution unit, configured to sequentially match the set of decoding results with text elements in a text element table, and output the text elements matched from the text element table to the requesting end, wherein the text element table contains preset text elements that are allowed to be output to the requesting end.

[0007] According to yet another embodiment of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and the computer program is configured to perform the steps in any of the above method embodiments when it is run.

[0008] According to yet another embodiment of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0009] According to yet another embodiment of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in any of the above method embodiments.

[0010] This application addresses the issue of garbled text in response to acquired input text, where the input text is the request text input by the requesting end. Based on the input vector, a large language model is used to perform iterative inference. The inference results output by the large language model are sequentially sampled and word-by-word extracted, and the resulting word sequence is sequentially decoded to obtain a set of decoding results. These decoding results are then sequentially matched with text elements in a text element table, and the matched text elements are output to the requesting end. The text element table contains pre-defined text elements permitted to be output to the requesting end. Since the text elements in the text element table are pre-defined and permitted to be output to the requesting end, the output obtained by the requesting end consists of normal text elements. By outputting the text elements matched from the text element table to the requesting end, the problem of garbled characters appearing during the display of the model output in related technologies is resolved. Attached Figure Description

[0011] Figure 1This is a schematic diagram of the hardware environment for an optional text data processing method according to an embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating an optional text data processing method according to an embodiment of this application;

[0013] Figure 3 This is a schematic diagram of an optional text data processing method according to an embodiment of this application;

[0014] Figure 4 This is a schematic diagram of another optional text data processing method according to an embodiment of this application;

[0015] Figure 5 This is a schematic diagram of another optional text data processing method according to an embodiment of this application;

[0016] Figure 6 This is a structural block diagram of an optional text data processing apparatus according to an embodiment of this application;

[0017] Figure 7 This is a structural block diagram of a computer system for an optional electronic device according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] The methods and embodiments provided in this application can be executed on a server, mobile terminal, computer terminal, or similar computing device. Taking running on a server as an example, Figure 1 This is a schematic diagram of the hardware environment for a text data processing method according to an embodiment of this application. Figure 1 As shown, a server may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.) and a memory 104 for storing data are also shown. The server may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server described above. For example, the server may also include components that are more complex than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0021] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the text data processing method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0022] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the computer terminal. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0023] Optionally, the text data processing method in this embodiment can be executed by a server. Here, "server" refers to the entire server, including the relevant components within the server that need to execute the text data processing method, as well as the processor, etc.; the text data processing method in this embodiment can also be executed by the processor 102. In some examples of this embodiment, the text data processing method is described using the example of execution by the server.

[0024] The text data processing method in this embodiment can be applied to scenarios where large language models output inference results using streaming throughput. A large language model (LLM) is a natural language processing model based on deep learning technology. It can output inference results using streaming throughput based on the input vector; that is, it iteratively performs inference, samples the obtained inference results for word extraction, and decodes the sampled results for output. Here, large language models can include, but are not limited to, GPT (Generative pre-trained transformer) and BERT (Bidirectional Encoder Representations from Transformers). Streaming throughput refers to the ability of a system or server to process input data or requests continuously and efficiently and produce output results. This processing method is usually based on streaming, meaning that data or requests flow through the system, and the system can process and respond to them in real time.

[0025] In large language model processing, word sampling is a technique used to generate text. The model can randomly select the next word from the probability distribution of the current word. The sampling result obtained by word sampling is usually a token. However, a word or a character in some large language models may contain several tokens. Decoding the sampling result may result in temporary garbled text. For example, when a character contains a set of tokens, but the sampling result obtained by word sampling only includes some tokens from that set, decoding the sampling result will not produce a character, resulting in garbled text.

[0026] In this embodiment, taking into account the above situations, the text elements in the text element table that match the decoding result are output during the text data processing process. This solves the problem of garbled characters appearing in the display model output during text data processing in related technologies. See also Figure 2 , Figure 2 This is a flowchart illustrating an optional text data processing method according to an embodiment of this application, such as... Figure 2 As shown, the above method includes the following steps:

[0027] Step S202: In response to the obtained input text, the input text is converted into an input vector corresponding to the input text, where the input text is the request text input by the requesting end.

[0028] The input text is the request text entered by the requesting party. The requesting party inputs the request text into the server, and the server receives the input text. Here, the requesting party and the server communicate with each other. The requesting party is not limited to mobile phones, computers, etc.

[0029] After the server obtains the input text, it converts the input text into an input vector corresponding to the input text. Here, the conversion of the input text into an input vector can be achieved through a vector conversion model or a word segmenter, etc. In this embodiment, the specific method of converting text into vector is not limited.

[0030] As an optional approach, in response to the acquired input text, the input text is converted into an input vector corresponding to the input text, including:

[0031] In response to the acquired input text, the input text is parsed using a tokenizer through a specified program to convert the input text into an input vector corresponding to the input text.

[0032] Optionally, the input text can be parsed using a tokenizer by a specified program to convert the input text into an input vector corresponding to the input text. For example, the input text can be parsed using a tokenizer (a text processing tool) to convert the input text into an input vector corresponding to the input text.

[0033] Step S204: Based on the input vector, the large language model is used to perform inference in a loop. The inference results output by the large language model are sampled and extracted word by word in sequence, and the word sequence obtained by sampling and extracting word by word is decoded in sequence to obtain a set of decoding results.

[0034] The server preloads a large language model into its memory. The server inputs the input vector into the large language model, performs inference iteratively using the model, and outputs the inference results. The inference results are then sampled and word-wise, and the resulting token sequences are decoded sequentially to obtain a set of decoding results. The inference results are represented by vectors. Sampling and word-wise the inference results from the large language model yields a token sequence, where the tokens are arranged in a specific order to represent a particular meaning. Decoding the sampled token sequences, i.e., converting them into corresponding text, can be implemented by the server running a decoding model or by using a tokenizer (a text processing tool).

[0035] Step S206: Match a set of decoding results with the text elements in the text element table in sequence, and output the text elements matched from the text element table to the requesting end. The text element table contains preset text elements that are allowed to be output to the requesting end.

[0036] The text element table contains preset text elements that can be output to the requesting client. The text elements in the text element table can be words, which are the most basic units of language, composed of one or more characters, and are linguistic symbols with independent meaning, such as apple and sun; the text elements in the text element table can also be phrases, which are linguistic units with specific meanings, composed of two or more words, such as watching a movie, laughing happily, and red apple; it should be noted that all text elements in the text element table are complete text elements that can be directly output to the requesting client.

[0037] Optionally, the text elements in the text element table include at least one of the following: character, word, phrase, symbol.

[0038] A set of decoding results is matched sequentially with the text elements in the text element table. This can be done by comparing whether a set of decoding results is the same as the text elements in the text element table. For example, if the text elements in the text element table are phrases, when a set of decoding results includes "red apple" and the text element table includes the text element "red apple", "red apple" can be output to the requesting end. When a set of decoding results includes "red" but the text element table does not include the text element "red", no output is sent to the requesting end. In this case, the large language model can be used to continue looping and performing inference, and word extraction, decoding, and other operations can be performed based on the new inference results.

[0039] According to the embodiments provided in this application, in response to the acquired input text, the input text is converted into an input vector corresponding to the input text, wherein the input text is the request text input by the requesting end; based on the input vector, a large language model is used to perform inference in a loop, the inference results output by the large language model are sampled and extracted word by word in sequence, and the word sequence obtained by sampling and extracting word by word is decoded in sequence to obtain a set of decoding results; the set of decoding results is matched with the text elements in the text element table in sequence, and the text elements matched from the text element table are output to the requesting end, wherein the text element table contains preset text elements that are allowed to be output to the requesting end, which can solve the problem of garbled characters in the display model output process of the text data processing methods in related technologies, and improve the user experience.

[0040] As an optional approach, before converting the input text into an input vector corresponding to the input text, the method further includes:

[0041] S11, Start the specified program and load the text element table, wherein the specified program is a program used to control the execution of the following steps: convert the input text into the input vector, perform inference in a loop using the large language model, sample and extract words from the inference results output by the large language model in sequence, decode the word sequence obtained by sampling and extracting words in sequence, match the set of decoding results with the text elements in the text element table in sequence, and output the text elements matched from the text element table to the requesting end.

[0042] In this embodiment, the designated program is a program running on the server for executing the text data processing method. Before executing the text data processing method of the above embodiment, the designated program needs to be started. The designated program can preload the models needed in the text data processing method into the server's memory, such as a large language model, a decoding model, etc. The designated program will also preload the text element table into the server's memory.

[0043] Optionally, the step of starting the specified program and loading the text element table in this embodiment can correspond to the model inference prefill stage. Here, the prefill stage refers to the stage of prefilling the model before inference. In this stage, the model is loaded into memory and may undergo some preprocessing operations so that it can respond to requests more quickly in the actual inference stage.

[0044] Optionally, the reasoning performed cyclically by the large language model may include reasoning in the prefill phase and reasoning in the Decodeing phase. Here, the Decodeing phase refers to the process by which the large language model generates reasoning results based on the input and the learned knowledge. In the Decodeing phase, the reasoning process is performed cyclically.

[0045] The embodiments provided in this application demonstrate how loading a text element table using a specified program can improve the processing efficiency of text data processing methods.

[0046] As an optional approach, the text elements in the text element table may include at least one of the following: characters, words, phrases, and symbols. The type of text elements in the text element table can be configured as needed; this embodiment does not impose any limitations on it.

[0047] In this embodiment, the inference process using a large language model, the sampling and word extraction of the inference results output by the large language model, the decoding of the word sequence obtained from the sampling and word extraction, and the matching of the decoding results with the text elements in the text element table can be performed in multiple rounds. That is, each time the large language model is used to perform inference, one round of the above-mentioned sampling, word extraction, decoding, and matching steps is performed. In this way, the timeliness of outputting text elements to the requesting end can be guaranteed, and the correlation between multiple rounds of inference can also be guaranteed, thereby improving the coherence (i.e., content coherence) and readability of the text elements output to the user.

[0048] Correspondingly, based on the input vector, the large language model is used to perform inference iteratively. The inference results output by the large language model are sampled and extracted word by word in sequence, and the word sequence obtained by sampling and extracting word by word is decoded in sequence to obtain a set of decoding results, including:

[0049] S21, Based on the input vector, use the large language model to perform inference repeatedly to obtain the inference result output by the large language model after each inference.

[0050] S22, after the large language model completes each inference operation and outputs the inference result, the current inference result is used as the current inference result for the following processing operations:

[0051] The current inference result is sampled to obtain the current word sequence, wherein the current word sequence contains at least one word sampled from the current inference result;

[0052] If a historical word sequence exists, the current word sequence is merged into the historical word sequence to obtain the updated current word sequence.

[0053] The updated current word sequence is decoded sequentially to obtain the current decoding result corresponding to the updated current word sequence.

[0054] After obtaining the transformed input vector, the large language model can be used to perform inference in a loop. Similar to the previous embodiments, the inference performed by the large language model in a loop can include inference in the prefill stage and inference in the Decodeing stage, which has been explained before and will not be repeated here.

[0055] After each inference operation, the large language model outputs an inference result, which can be represented by an inference result vector. At this point, the current inference result can be used as the current inference result for sampling and decoding. For the current inference result, sampling can be performed. The resulting sampling is one or more tokens. These tokens have a sequential order; therefore, the result of sampling the current inference result is the current token sequence, which contains at least one token sampled from the current inference result. Throughout the entire text data processing, the token sequence obtained through sampling includes the current token sequence.

[0056] Before performing the current inference using the large language model, at least one inference may have been performed, or no inference may have been performed. If at least one inference was performed before the current inference using the large language model, this can be further divided into cases where all decoded results match the text elements in the text element table, or cases where some matches fail. To ensure the continuity of the matching process, historical lexical sequences can be saved. These historical lexical sequences can be those that failed to decode in the lexical sequences preceding the current lexical sequence, or lexical sequences preceding the current lexical sequence that, excluding those whose corresponding decoded results matched the text elements in the text element table.

[0057] For example, the text element table is a vocabulary, where the text elements are words. A word inferred by the large language model contains three characters: character 1, composed of lexical units 1 and 2; character 2, composed of lexical units 3 and 4; and character 3, composed of lexical units 5, 6, and 7. If, in the previous inference, the position inferred could be either lexical unit 4 or lexical unit 5, then... Figure 3 As shown.

[0058] For the case where the historical lexical is a lexical that failed to be decoded in the lexical sequence preceding the current lexical sequence, if lexical 4 has been deduced, since lexical 1 and lexical 2 can be decoded into character 1, and lexical 3 and lexical 4 can be decoded into character 2, the historical lexical sequence is empty; if lexical 5 has been deduced, since lexical 1 and lexical 2 can be decoded into character 1, and lexical 3 and lexical 4 can be decoded into character 2, the historical lexical sequence only contains lexical 5.

[0059] For historical lexical units that are lexical units in the lexical sequence preceding the current lexical sequence, excluding those whose corresponding decoding results successfully match the text elements in the text element table, if lexical unit 4 has been deduced, the historical lexical unit sequence contains lexical units 1, 2, 3, and 4; if lexical unit 5 has been deduced, the historical lexical unit sequence contains lexical units 1, 2, 3, 4, and 5.

[0060] If a historical word sequence exists (i.e., the historical word sequences are not empty), the current word sequence can be merged into the historical word sequence to obtain an updated current word sequence. Then, the updated current word sequence is decoded sequentially to obtain the current decoding result corresponding to the updated current word sequence. By merging the current word sequence and the historical word sequence as described above, the accuracy of data processing can be guaranteed. Compared with the scheme of storing inference results that failed to match, the number of sampling and word extraction operations required can be reduced, improving the rationality of resource utilization.

[0061] For example, if the historical word is a word in the word sequence before the current word sequence that failed to be decoded, and word 5 has been deduced, the historical word sequence contains word 5, the current word sequence contains word 6 and word 7, and the merged word sequence contains word 5, word 6 and word 7, then decoding it can decode word 3.

[0062] For historical lexical units that are lexical units in the lexical sequence preceding the current lexical sequence, excluding those whose corresponding decoding results match the text elements in the text element table, if lexical unit 4 has been deduced, the historical lexical sequence contains lexical units 1, 2, 3, and 4, the current lexical sequence contains lexical units 5, 6, and 7, and the merged lexical sequence contains lexical units 1, 2, 3, 4, 5, 6, and 7, decoding them can yield characters 1, 2, and 3.

[0063] For historical lexical units that are lexical units in the lexical sequence preceding the current lexical sequence, excluding those whose corresponding decoding results match the text elements in the text element table, if lexical unit 5 has been deduced, the historical lexical sequence contains lexical units 1, 2, 3, 4, and 5, the current lexical sequence contains lexical units 6 and 7, and the merged lexical sequence contains lexical units 1, 2, 3, 4, 5, 6, and 7. Decoding these lexical units will yield characters 1, 2, and 3.

[0064] The embodiments provided in this application, by performing inference, sampling, decoding and matching steps in multiple rounds, can ensure the timeliness of outputting text elements to the requesting end, and at the same time ensure the correlation between multiple rounds of inference, thereby improving the coherence and readability of the text elements output to the user; using historical word sequence to save historical words can reduce the number of sampling operations that need to be performed and improve the rationality of resource utilization.

[0065] As an optional approach, a set of decoding results is sequentially matched against the text elements in the text element table, and the matched text elements are output to the requesting client, including:

[0066] S31, if all current decoding results are successful, match the current decoding results with the text elements in the text element table, output the matched text elements from the text element table to the requesting end, and trigger the large language model to perform the next inference, during which the historical word sequence is cleared.

[0067] In this embodiment, considering that if there are decoding failures (i.e., some lexical units are not decoded into corresponding characters), then even if subsequent matching with text elements in the text element table is performed, the matching result will inevitably fail. Therefore, to improve the rationality of resource utilization, if the current decoding result contains decoding failures, the large language model can be directly triggered to perform the next inference. Furthermore, the aforementioned historical lexical unit sequence can be updated based on the decoding result to ensure that the historical lexical units therein are of the same or similar type to the historical lexical units stored in the set historical lexical unit sequence.

[0068] If all current decoding results are successful (the current decoding result may contain decoding results for multiple symbols, such as the aforementioned character 1, character 2, and character 3), then match the current decoding result with the text elements in the text element table. If a text element is matched from the text element table, then the current decoding result is considered a successful match. In this case, the historical word sequence can be cleared.

[0069] Optionally, see the relevant technologies. Figure 4The text data processing methods are explained. Figure 4 This is a schematic diagram illustrating another optional text data processing method according to an embodiment of this application. Figure 4 This describes the processing flow of text data processing methods in related technologies, specifically including the following steps.

[0070] Step 1: Load and initialize the large language model; the server loads the large language model into memory.

[0071] Step 2: Obtain user input and perform prompt processing. Prompt processing is a text processing technique commonly used in natural language processing and dialogue systems. Prompt processing helps the system generate appropriate answers based on user input or questions. Here, prompt processing refers to the collective term for subsequent steps. This step is used to obtain user input and begin executing subsequent processing steps.

[0072] Step 3: The Tokenizer converts the text into a vector; the Tokenizer converts the user-input text into a vector representation.

[0073] Step 4: Perform prefill inference. The server uses the language model to perform prefill inference, obtains the inference result, samples words based on the inference result, performs decoding operation based on the sampled words, and directly outputs the decoded result to the user without processing. Here, since a word or character in some large language models is composed of several tokens, the result of decoding the sampled words may not be able to form a complete character, and an unknown character may be displayed. The complete character can only be displayed after receiving subsequent tokens, which affects the user experience.

[0074] Step 5: Perform Decoding stage inference; use the large language model to perform inference in a loop, that is, use the result of this time as the input for the next time, and perform inference in a loop until the eos token id is inferred, and then end the inference.

[0075] To address the problems existing in the text data processing methods of the aforementioned related technologies, and in conjunction with Figure 5 The process from deploying a large language model to processing the received input text can include the following steps:

[0076] Step 1: The inference program starts, loads the model file of the large language model, initializes the large language model, and begins to receive text requests (i.e., input text) from the requesting end (corresponding to the user).

[0077] Step 2: After receiving the text request, the inference program can use a tokenizer to parse the request and convert it into an input vector that the program can recognize.

[0078] Step 3: Based on the input vector, perform the prefill stage inference using the large language model, calculate the first inference result, and generate a key-value cache, which is stored in memory.

[0079] Step 4: The matching program uses the first inference result to look up the vocabulary (the process of sampling and decoding is omitted here; the decoding result corresponding to the first inference result is the first decoding result) and determines whether the first decoding result needs to be returned to the requesting end.

[0080] Step 5: If the judgment result is not to return the first decoding result to the requesting end, then write the first decoding result into memory and inform the inference program to execute the next inference (execute the next inference using the large language model);

[0081] Step 6: If the judgment result is to return the first decoding result to the requesting end, the first decoding result can be returned to the requesting end, and the inference program can be informed to execute the next inference.

[0082] Step 7: The inference program performs the next inference based on the previous key-value cache and the previous inference result, and returns the inference result to the matching program.

[0083] Step 8: After receiving the reasoning result, the matching program, together with the historical reasoning results (which may be historical word sequences), searches the vocabulary to determine whether the decoding result needs to be returned to the requesting end.

[0084] Step 9: If the judgment result is to return the decoding result to the requesting end, the inference program returns the decoding result (corresponding to the current inference result and the historical inference result) to the requesting end as a whole, clears the historical inference result cache, and informs the inference program to execute the next inference.

[0085] Here, due to the addition of reasoning programs and vocabulary functions, the reasoning program optimizes the results after each reasoning process, and then returns the optimized results to the user, which can reduce the number of returns and improve the pronunciation experience.

[0086] The embodiments provided in this application simplify the processing flow and improve the rationality of resource utilization by determining whether to perform a matching operation based on the decoding result.

[0087] As an optional approach, after decoding the updated current word sequence sequentially to obtain the current decoding result corresponding to the updated current word sequence, the above method further includes:

[0088] S41, if there are historical decoding results, the current decoding result is merged into the historical decoding results to obtain the updated current decoding result. The historical decoding results include: decoding results obtained by decoding the word sequences before the current word sequence, and decoding results that were successfully decoded but did not match the text elements in the text element table.

[0089] S42, if the current decoding result contains decoding results that failed to decode, update the historical decoding result with the successfully decoded result in the updated current decoding result, and update the historical word sequence with the word corresponding to the decoding result that failed to decode in the current decoding result.

[0090] In this embodiment, when historical lexical sequences include lexical sequences preceding the current lexical sequence that failed to decode, historical decoding results can be saved. That is, among the successfully decoded decoding results obtained from decoding lexical sequences preceding the current lexical sequence, the decoding results that did not successfully match text elements in the text element table can be saved. For example, ... Figure 3 As shown, for cases where word 4 or 5 has been deduced, since word 1 and word 2 can be decoded into character 1, and word 3 and word 4 can be decoded into character 2, the historical decoding results include: character 1 and character 2.

[0091] Since the current decoding result is obtained by decoding the failed words in the current word sequence, or the word sequence preceding the current word sequence, and does not include historical decoding results, if historical decoding results exist, the current decoding result can be merged into the historical decoding results to obtain an updated current decoding result. Matching with text elements in the text element table can be performed on the updated current decoding result.

[0092] Optionally, if the current decoding result contains decoding results that failed, similar to the previous embodiments, the step of matching text elements in the text element table can be omitted. To ensure the accuracy of information storage, the historical decoding results can be updated with the successfully decoded results in the updated current decoding result, and the historical word sequence can be updated with the words corresponding to the decoding results that failed in the current decoding result.

[0093] Optionally, to reduce the amount of data that needs to be maintained, the historical word sequences can be word sequences from previous word sequences, excluding those whose corresponding decoding results successfully match the text elements in the text element table. In this case, the current decoding result can be used directly for subsequent processing steps. If the current decoding result contains decoding results that failed, the historical word sequence can be updated to the updated current word sequence, and the step of matching with the text elements in the text element table is not performed.

[0094] To ensure convenient information retrieval, historical word sequences and / or historical decoding results can both be stored in a cache. To accommodate the sequential order of different historical words in the historical word sequence and the sequential order of different decoding results in the historical decoding results, the data structure storing the historical word sequences and / or historical decoding results can be a queue, a linked list, or other data structure that can represent the sequential order of elements. To ensure data storage efficiency, the storage space used to store the historical word sequences and / or historical decoding results can be pre-allocated.

[0095] By using the embodiments provided in this application, the number of words that need to be decoded can be reduced and the efficiency of information processing can be improved by using historical decoding results to save the decoding results obtained from decoding the word sequences before the current word sequence, including the decoding results that were successfully decoded but did not match the text elements in the text element table.

[0096] As an alternative approach, after converting the input text into an input vector corresponding to the input text, the method also includes:

[0097] S51, if the large language model outputs the latest inference result indicating the end of inference, output the inference end symbol to the requesting end.

[0098] In this embodiment, when the large language model outputs the latest inference result as eos token id (End of Sentence token id), the inference ends and the inference end symbol is output to the requesting end; here, eos token id is an identifier that is the end symbol of the inference process.

[0099] The embodiments provided in this application enhance the completeness and reliability of the text data processing method of this application.

[0100] According to another aspect of the embodiments of this application, a text data processing apparatus is also provided. This apparatus is used to implement the text data processing method provided in the above embodiments, and details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0101] Figure 6 This is a structural block diagram of a text data processing apparatus according to an embodiment of this application, such as... Figure 6 As shown, the device includes:

[0102] In response to the acquired input text, the conversion unit 602 converts the input text into an input vector corresponding to the input text, wherein the input text is the request text input by the requesting end;

[0103] The first execution unit 604 is used to perform inference in a loop using a large language model based on the input vector, to sample and extract words from the inference results output by the large language model in sequence, and to decode the word sequence obtained by sampling and extracting words in sequence to obtain a set of decoding results;

[0104] The second execution unit 606 is used to match a set of decoding results sequentially with the text elements in the text element table, and output the text elements matched from the text element table to the requesting end. The text element table contains preset text elements that are allowed to be output to the requesting end.

[0105] It should be noted that the acquisition unit 602 in this embodiment can be used to execute the above step S202, the first extraction unit 604 in this embodiment can be used to execute the above step S204, and the first execution unit 606 in this embodiment can be used to execute the above step S206.

[0106] According to the embodiments provided in this application, in response to the acquired input text, the input text is converted into an input vector corresponding to the input text, wherein the input text is the request text input by the requesting end; based on the input vector, a large language model is used to perform inference in a loop, the inference results output by the large language model are sampled and extracted word by word in sequence, and the word sequence obtained by sampling and extracting word by word is decoded in sequence to obtain a set of decoding results; the set of decoding results is matched with the text elements in the text element table in sequence, and the text elements matched from the text element table are output to the requesting end, wherein the text element table contains preset text elements that are allowed to be output to the requesting end, which can solve the problem of garbled characters in the display model output process of the text data processing methods in related technologies, and improve the user experience.

[0107] Optionally, the device also includes:

[0108] The loading unit is used to start a specified program and load a text element table before converting the input text into an input vector corresponding to the input text. The specified program is a program used to control the execution of the following steps: converting the input text into an input vector, performing inference in a loop using a large language model, sampling words from the inference results output by the large language model in sequence, decoding the word sequence obtained from the sampling words in sequence, matching a set of decoding results with the text elements in the text element table in sequence, and outputting the text elements matched from the text element table to the requesting end.

[0109] Optionally, the conversion unit includes:

[0110] The conversion module is used to parse the input text using a tokenizer in response to the acquired input text, thereby converting the input text into an input vector corresponding to the input text.

[0111] Optionally, the text elements in the text element table include at least one of the following: character, word, phrase, symbol;

[0112] The first execution unit includes:

[0113] The inference module is used to perform inference in a loop using a large language model based on the input vector, and obtain the inference result output by the large language model after each inference.

[0114] The first execution module is used to perform the following processing operations on the current inference result after each inference output by the large language model:

[0115] The current inference result is sampled to obtain the current word sequence, wherein the current word sequence contains at least one word sampled from the current inference result;

[0116] If a historical word sequence exists, the current word sequence is merged into the historical word sequence to obtain the updated current word sequence.

[0117] The updated current word sequence is decoded sequentially to obtain the current decoding result corresponding to the updated current word sequence;

[0118] Among them, a set of decoding results includes the current decoding result, and the historical lexical sequences include one of the following: lexical sequences in the lexical sequences before the current lexical sequence that failed to be decoded; or lexical sequences in the lexical sequences before the current lexical sequence that, except for lexical sequences whose corresponding decoding results successfully match the text elements in the text element table.

[0119] Optionally, the second execution unit includes:

[0120] The second execution module is used to match the current decoding result with the text elements in the text element table when all current decoding results are successfully decoded. The matched text elements are then output to the requesting end, and the large language model is triggered to perform the next inference. In this process, the historical word sequence is cleared.

[0121] In cases where the current decoding result includes a decoding failure, the large language model is directly triggered to execute the next inference.

[0122] Optionally, the device also includes:

[0123] The first update unit is used to decode the updated current word sequence sequentially to obtain the current decoding result corresponding to the updated current word sequence. When the historical word sequence includes word sequences that failed to be decoded in the word sequence before the current word sequence, if there is a historical decoding result, the current decoding result is merged into the historical decoding result to obtain the updated current decoding result. The historical decoding result includes: decoding results that failed to match the text elements in the text element table from the decoding results obtained from decoding the word sequence before the current word sequence and the decoding results that were successfully decoded.

[0124] The second update unit is used to update the historical decoding results to the successfully decoded results in the updated current decoding results when the current decoding results contain decoding results that failed to decode, and to update the historical word sequence to the word corresponding to the decoding results that failed to decode in the current decoding results.

[0125] Optionally, the device also includes:

[0126] The output unit is used to output an inference termination symbol to the requesting client after converting the input text into an input vector corresponding to the input text, and when the large language model outputs the latest inference result indicating the end of inference.

[0127] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0128] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0129] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0130] In one exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor and the input / output device is connected to the processor.

[0131] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0132] According to another aspect of the embodiments of this application, a computer program product is provided, the computer program product including a computer program / instructions comprising program code for performing the method shown in the flowchart. In such an embodiment, reference is made to... Figure 7 The computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the central processing unit 701, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments of this application above are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0133] refer to Figure 7 , Figure 7 This is a structural block diagram of a computer system for an optional electronic device according to an embodiment of this application.

[0134] Figure 7 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output interface 705 (I / O interface) is also connected to the bus 704.

[0135] The following components are connected to the input / output interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a local area network card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0136] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit 701, it performs various functions defined in the system of this application.

[0137] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0138] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0139] The above are merely preferred embodiments of this application and are not intended to limit the embodiments of this application. For those skilled in the art, various modifications and variations can be made to the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made within the principles of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A method for processing text data, characterized in that, include: In response to the acquired input text, the input text is converted into an input vector corresponding to the input text, wherein the input text is the request text input by the requesting end; Based on the input vector, a large language model is used to perform inference iteratively. The inference results output by the large language model are sampled and extracted word by word in sequence, and the word sequence obtained by sampling and extracting word by word is decoded in sequence to obtain a set of decoding results, including: Based on the input vector, a large language model is used to perform inference iteratively to obtain the inference result output by the large language model after each inference. After the large language model outputs an inference result after each inference, the current inference result is used as the current inference result and the following processing operations are performed: the current inference result is sampled to obtain a current word sequence, wherein the current word sequence contains at least one word sampled from the current inference result; if a historical word sequence exists, the current word sequence is merged into the historical word sequence to obtain an updated current word sequence; the updated current word sequence is decoded sequentially to obtain a current decoding result corresponding to the updated current word sequence; wherein the set of decoding results includes the current decoding result, and the historical word sequences include one of the following: word sequences in the word sequence before the current word sequence that failed to be decoded; word sequences in the word sequence before the current word sequence other than word sequences whose corresponding decoding results successfully match the text elements in the text element table. The set of decoding results is matched sequentially with the text elements in the text element table, and the text elements matched from the text element table are output to the requesting end. The text element table contains preset text elements that are allowed to be output to the requesting end. The text elements in the text element table include at least one of the following: characters, words, phrases, and symbols.

2. The method according to claim 1, characterized in that, Before converting the input text into an input vector corresponding to the input text, the method further includes: The specified program is launched and the text element table is loaded. The specified program is a program used to control the execution of the following steps: converting the input text into the input vector, performing inference in a loop using the large language model, sampling and extracting words from the inference results output by the large language model in sequence, decoding the word sequence obtained from the sampled words in sequence, matching the set of decoding results with the text elements in the text element table in sequence, and outputting the text elements matched from the text element table to the requesting client.

3. The method according to claim 2, characterized in that, The step of converting the acquired input text into an input vector corresponding to the input text includes: In response to the acquired input text, the input text is parsed using a word segmenter through the specified program to convert the input text into the input vector corresponding to the input text.

4. The method according to claim 1, characterized in that, The step of sequentially matching the set of decoding results with text elements in the text element table and outputting the matched text elements to the requesting client includes: If all current decoding results are successful, the current decoding results are matched with the text elements in the text element table, the matched text elements are output to the requesting end, and the large language model is triggered to perform the next inference, wherein the historical word sequence is cleared; In cases where the current decoding result includes a decoding failure, the large language model is directly triggered to perform the next inference.

5. The method according to claim 1, characterized in that, After sequentially decoding the updated current word sequence to obtain the current decoding result corresponding to the updated current word sequence, the method further includes: When the historical lexical sequence includes lexical sequences that failed to be decoded in the lexical sequence preceding the current lexical sequence, if a historical decoding result exists, the current decoding result is merged into the historical decoding result to obtain an updated current decoding result. The historical decoding result includes: decoding results that failed to match text elements in the text element table from the successfully decoded decoding results obtained by decoding the lexical sequence preceding the current lexical sequence. If the current decoding result includes decoding results that failed, the historical decoding result is updated to the successfully decoded result in the updated current decoding result, and the historical word sequence is updated to the word corresponding to the decoding result that failed in the current decoding result.

6. The method according to any one of claims 1 to 5, characterized in that, After converting the input text into an input vector corresponding to the input text, the method further includes: If the large language model outputs the latest inference result indicating the end of inference, an inference termination symbol will be output to the requesting client.

7. A text data processing apparatus, characterized in that, include: A conversion unit is configured to convert the input text into an input vector corresponding to the input text in response to the acquired input text, wherein the input text is the request text input by the requesting end; The first execution unit is configured to perform iterative inference using a large language model based on the input vector, sequentially sample words from the inference results output by the large language model, and sequentially decode the resulting word sequence to obtain a set of decoding results. This includes: an inference module configured to perform iterative inference using a large language model based on the input vector, obtaining the inference result output by the large language model after each inference iteration; and a first execution module configured to, after each inference iteration by the large language model, use the current inference result as the current inference result and perform the following processing operation: sample words from the current inference result to obtain the current word sequence, wherein the current word sequence... The sequence contains at least one lexical element sampled from the current inference result; if a historical lexical element sequence exists, the current lexical element sequence is merged into the historical lexical element sequence to obtain an updated current lexical element sequence; the updated current lexical element sequence is decoded sequentially to obtain a current decoding result corresponding to the updated current lexical element sequence; wherein, the set of decoding results includes the current decoding result, and the historical lexical elements in the historical lexical element sequence include one of the following: lexical elements in the lexical element sequence before the current lexical element sequence that failed to be decoded; lexical elements in the lexical element sequence before the current lexical element sequence other than lexical elements whose corresponding decoding results successfully match the text elements in the text element table; The second execution unit is used to match the set of decoding results sequentially with the text elements in the text element table, and output the text elements matched from the text element table to the requesting end, wherein the text element table contains preset text elements that are allowed to be output to the requesting end; The text elements in the text element table include at least one of the following: characters, words, phrases, and symbols.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images