Large language model construction method and system supporting streaming input

Abstract

The invention provides a large language model construction method and system supporting streaming input, and the method comprises the steps: collecting and selecting data, carrying out the data preprocessing, employing a pre-training model as a basis, carrying out the fine tuning on the pre-training model, and training a large language model which optimizes the streaming input and autoregression output mode; the method comprises the following steps: presetting the size of a data block, partitioning input data of a user, and receiving the input data of the user in a streaming manner; and inputting the spliced data to the large language model, judging whether to reply the user or not according to the output of the large language model based on the current input data of the large language model, and adopting a corresponding action to reply the user. According to the method, the response speed of the large language model is greatly improved, the large language model supports responding to the input data instruction of the user in a streaming input and parallel output mode, the effect of'thinking while listening 'is achieved, the advantages are more obvious especially under the scene that the user instruction is long, and the user experience is effectively improved.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence, large language models, multimodal large language models, and human-computer interaction technology; in particular, to a large language model construction method and system supporting streaming input. Background Art

[0002] Large Language Models (LLMs) are an important branch of the field of artificial intelligence. Through deep learning and large amounts of data training, large language models can understand and generate human-like text. The core of large language models is the Transformer architecture, which uses a self-attention mechanism to process sequence data, thereby improving processing efficiency and effectiveness. The rapid development of large language model technology has brought humans one step closer to achieving artificial general intelligence (AGI).

[0003] At present, large language models are continuously expanding their application areas, from text processing to multimodal data such as speech, images and videos. These models have been able to process and understand various types of data by combining advanced neural network architectures and large amounts of data training.

[0004] In the field of images, large language models for images (such as the DALL-E model released by OpenAI) can generate images based on text descriptions, demonstrating cross-modal interaction capabilities. This enables large language models to understand and generate visual content, providing users with a richer interactive experience.

[0005] Applications in the video field are more complex and involve the understanding of dynamic visual content. By combining the visual encoder and the language model, the large language model can process video data and perform tasks such as video question and answer. For example, the ChatVideo system can parse videos into trajectories and annotate their attributes, store them in a database, and interact with users through a database manager.

[0006] In the field of speech, large language models can perform real-time voice interaction without relying on additional automatic speech recognition ASR or text-to-speech TTS systems, which not only improves the overall performance of the model but also improves the response speed of the model. For example, ChatGPT 4o released by OpenAI supports end-to-end real-time voice conversations and can achieve continuous voice-to-speech conversations, improving the naturalness and fluency of voice interaction.

[0007] In the application of human-computer interaction, the response speed of the large language model is a very important indicator. At present, the operating mechanism of the large language model is based on all the user's input data and outputs in an autoregressive manner. In other words, at this stage, the large language model cannot process user requests in a streaming manner. Only after seeing all the user's input data can the large language model start output. This greatly limits the optimization of the large language model in terms of response speed. Especially when the user's input instructions are more complex and contain multiple intentions, the user's waiting time will be greatly extended, reducing the user's product experience of human-computer interaction. Summary of the invention

[0008] In view of this, the purpose of the present invention is to develop a large language model construction method and system that supports streaming input, divide the user's input data into blocks, splice the large language model prompt words, user context information, historical data blocks and current data blocks, and stream input them to the large language model for processing. The large language model outputs and replies to the user based on autoregressive output. The response speed of the large language model is greatly improved, so that the large language model has an "interruption" function, that is, during the user's speaking process, the large language model can give a corresponding reply. The large language model supports responding to the user's input data instructions in a streaming input and parallel output manner, achieving the ability of "listening, thinking and outputting", and providing a better user experience.

[0009] The present invention provides a method for constructing a large language model supporting streaming input, comprising the following steps:

[0010] S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, fine-tune the pre-trained model to train and optimize the large language model with streaming input and autoregressive output.

[0011] S2. Preset the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner;

[0012] If the length of the current unprocessed user data is greater than or equal to the preset data block size, the large language model prompt words, user context data, historically processed data blocks, and the current unprocessed data blocks are concatenated in the order of the user input time. After the concatenation is completed, the concatenated data is input into the large language model for processing;

[0013] If the length of the currently unprocessed data block is less than the preset data block size, the user's input data will continue to be cached;

[0014] S3, input the spliced ​​data (preferably, according to the request ID of the large language model) to the large language model, based on the current input data of the large language model and the output of the large language model, determine whether to reply to the user, and take corresponding actions to reply to the user, for example: agent mode;

[0015] If the output of the large language model is empty, for example, " <eps>", it means that based on the current user input data, the large language model does not have enough information to call the tool or cannot reply to the user yet, and needs to perform input reasoning for the next data block;

[0016] If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intention and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculations and outputs all content in the form of autoregressive output to reply to the user.

[0017] Specifically, the output content includes: tool call information, terms for replying to users, etc.

[0018] "Autoregressive output" in large language models (LLMs) usually refers to a sequence generation method. Large language models (autoregressive models) with streaming input and autoregressive output predict the next element in the sequence based on the output of the previous text. In the field of natural language processing (NLP), autoregressive models can generate text sequences, one word after another, until a complete sentence or paragraph is generated.

[0019] The autoregressive model has the following technical characteristics:

[0020] (1) Sequential dependency: The autoregressive model assumes that each element in the sequence (such as a word or character) depends only on the element before it. This dependency enables the autoregressive model to gradually build the output in a certain order.

[0021] (2) Probabilistic modeling: Autoregressive models usually predict the next element in a sequence based on a probability distribution. The model calculates the probability of each possible next element and selects the element with the highest probability as the output.

[0022] (3) Parameterization: Autoregressive models are usually modeled in a parameterized manner, and these parameters can be learned through training data. The parameterized form can be a neural network, a recurrent neural network (RNN), a long short-term memory network (LSTM), or a Transformer, etc.

[0023] The process of autoregressive forward computation involves forward propagation: at each step of generating a sequence, the model predicts the next element based on the current input (which may be empty or contain the first few elements of the sequence). This process is repeated until the complete sequence is generated or a termination condition is reached.

[0024] Furthermore, the method for training and optimizing the large language model with streaming input and autoregressive output in step S1 comprises the following steps:

[0025] S11. Collect relevant data in vertical fields as training data (for example, human-computer interaction field), which covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure that each input data block has a correct output annotation; ensure the diversity and representativeness of the training data to cover different user input modes and language styles;

[0026] S12. Clean the collected training data to remove invalid or erroneous data; regularize the data format to ensure that the data meets the requirements of large language model input; and divide the training data into blocks;

[0027] S13. Use the pre-trained large language model as the basis for supervised fine tuning (SFT). The training data includes the original training data and the block-based training data. Use reinforcement learning algorithms, such as PPO (Proximal Policy Optimization), to further optimize the decision-making process of the large language model.

[0028] S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

[0029] Furthermore, in the step S3, when the output of the large language model is not empty, while generating the output result based on the autoregressive forward calculation method, the large language model request ID is added, and the large language model output identifier is added to the input data of the large language model, such as "<@>", which is used to indicate the input data before the identifier, and then the next data block is processed.

[0030] Exemplarily, while the large language model generates output, an "output identifier" is added to the end of the current input data of the large language model to indicate that the output corresponding to the part of the input data has been generated, and then the processing of the next data block continues;

[0031] By adding a large language model output identifier, for the partially generated output results, the large language model can avoid repeatedly outputting the same content in the subsequent steps. In this way, the large language model can support responding to user input data instructions in a streaming input and parallel output manner, achieving the ability of "listening, thinking and outputting". After the user data is input, the large language model has basically completed the parsing, so it can respond to the user quickly.

[0032] Furthermore, the method of replying to the user in step S3 includes any one of the following two methods:

[0033] The first is to wait for the user's input data instruction to complete, and then give feedback to the user in the order of the output of the large language model;

[0034] The second is to respond to the user immediately once the large language model generates output and is able to feed back the result to the user during the user's input data instruction. This response method has a shorter response time, but may cause certain interference to the user.

[0035] Furthermore, the process of training and optimizing the large language model with streaming input and autoregressive output in step S1 includes:

[0036] Select an applicable neural network architecture, including: RNN, LSTM or Transformer;

[0037] A loss function is defined to measure the difference between the sequence predicted by the model and the true sequence; preferably, the loss function adopts cross entropy loss; and the parameters of the large language model are updated using a gradient descent optimization algorithm to minimize the loss function;

[0038] Initialize the model input. At the beginning of sequence generation, give the autoregressive model an initial input, such as a special start tag.

[0039] The autoregressive model predicts the next element based on the current input, and then uses the predicted element as the new input to continue predicting the next element, and iterates the prediction and sequence generation through this prediction process;

[0040] The generation process ends when the autoregressive model predicts the end marker of the sequence or reaches a predetermined length.

[0041] Furthermore, the process of iterative prediction and sequence generation includes:

[0042] Use a beam search strategy to select the best prediction at each step to generate higher quality sequences;

[0043] The Temperature Tuning strategy is used to control the diversity of the sequences generated by the autoregressive model by adjusting the temperature parameter of the probability distribution.

[0044] The present invention also provides a large language model construction system supporting streaming input, which executes the large language model construction method supporting streaming input as described above, including:

[0045] Training model module: used to collect and select data, and perform data preprocessing. Based on the pre-trained model, the module fine-tunes the pre-trained model to train and optimize the large language model with streaming input and autoregressive output.

[0046] The module for processing user input data in blocks is used to pre-set the size of data blocks, divide the user input data into blocks, and accept the user input data in a streaming manner; if the length of the current unprocessed user data is greater than or equal to the pre-set data block size, the large language model prompt words, user context data, historically processed data blocks, and the current unprocessed data blocks are spliced ​​in the chronological order of user input. After the splicing is completed, the spliced ​​data is input to the large language model for processing; if the length of the current unprocessed data block is less than the pre-set data block size, the user input data continues to be cached;

[0047] Output result module: used to input the spliced ​​data into the large language model, determine whether to reply to the user based on the current input data of the large language model and the output of the large language model, and take corresponding actions to reply to the user; if the output of the large language model is empty, it means that according to the current user input data, the large language model has not obtained enough information to call the tool or cannot reply to the user, and it is necessary to perform input reasoning for the next data block; if the output of the large language model is not empty, it means that according to the current user input data, the large language model can detect the user's intention, and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculation, outputs all content in the autoregressive output mode, and replies to the user.

[0048] Furthermore, the training model module includes:

[0049] Data preparation unit: used to collect relevant data in vertical fields as training data, which covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure the diversity and representativeness of the training data to cover different user input modes and language styles;

[0050] Data preprocessing unit: used to clean the collected training data and remove invalid or erroneous data; to regularize the data format to ensure that the data meets the requirements of large language model input; to divide the training data into blocks;

[0051] Model training unit: used to use the pre-trained large language model as a basis for supervised fine-tuning. The training data includes the original training data and the divided training data. The reinforcement learning algorithm is used to further optimize the decision-making process of the large language model.

[0052] Model evaluation unit: used to evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

[0053] The present invention also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the steps of the method for constructing a large language model supporting streaming input as described above are implemented.

[0054] The present invention also provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the method for constructing a large language model supporting streaming input as described above are implemented.

[0055] Compared with the prior art, the present invention has the following beneficial effects:

[0056] The large language model construction method and system supporting streaming input provided by the present invention divide the user's input data into blocks, splice the large language model prompt words, user context information, historical data blocks and current data blocks, and stream them to the large language model. The user's input data instructions can be streamed, and the large language model outputs and replies to the user based on the autoregressive output method. The response speed of the large language model is greatly improved, so that the large language model has an "interruption" function, that is, during the user's speaking process, the large language model can give a corresponding reply. The large language model supports responding to the user's input data instructions in a streaming input and parallel output manner, achieving the effect of "listening, thinking and outputting", especially in the scenario where the user's instructions are long, the advantage is more obvious, and the user experience is effectively improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the following detailed description of the preferred embodiment.The drawings are only for the purpose of illustrating the preferred embodiments and are not to be construed as limiting the invention.

[0058] In the attached picture:

[0059] Figure 1 A logic diagram of a process for a large language model to process streaming input data according to an embodiment of the present invention;

[0060] Figure 2 An example diagram of a large language model processing streaming input according to an embodiment of the present invention;

[0061] Figure 3 It is an arrangement diagram of the input data of the large language model of the streaming input according to the embodiment of the present invention;

[0062] Figure 4 A diagram showing the steps of training a large language model with streaming input according to an embodiment of the present invention;

[0063] Figure 5 A flow chart of a method for constructing a large language model supporting streaming input according to the present invention;

[0064] Figure 6 A flow chart of the method for training and optimizing a large language model with streaming input and autoregressive output according to the present invention;

[0065] Figure 7 The figure is a schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure. Instead, they are merely examples of systems and products consistent with some aspects of the present disclosure as detailed in the appended claims.

[0067] The terms used in this disclosure are for the purpose of describing specific embodiments only and are not intended to limit the disclosure. The singular forms "a", "the", and "the" used in this disclosure and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0068] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0069] The embodiments of the present invention are described in further detail below.

[0070] The embodiment of the present invention provides a method for constructing a large language model that supports streaming input, see Figure 5 As shown, the following steps are included:

[0071] S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, fine-tune the pre-trained model to train and optimize the large language model with streaming input and autoregressive output.

[0072] The process of training and optimizing a large language model with streaming input and autoregressive output includes:

[0073] Select an applicable neural network architecture, including: RNN, LSTM or Transformer;

[0074] Define a loss function to measure the difference between the sequence predicted by the model and the true sequence; use a gradient descent optimization algorithm to update the parameters of the large language model to minimize the loss function; in this embodiment, the loss function uses a cross entropy loss;

[0075] Initialize the model input. At the beginning of sequence generation, give the autoregressive model an initial input. In this embodiment, a special start marker is input.

[0076] The autoregressive model predicts the next element based on the current input, and then uses the predicted element as the new input to continue predicting the next element, and iterates the prediction and sequence generation through this prediction process;

[0077] The generation process ends when the autoregressive model predicts the end marker of the sequence or reaches a predetermined length.

[0078] The process of iterative prediction and sequence generation includes:

[0079] Use a beam search strategy to select the best prediction at each step to generate higher quality sequences;

[0080] The Temperature Tuning strategy is used to control the diversity of the sequences generated by the autoregressive model by adjusting the temperature parameter of the probability distribution.

[0081] The method for training and optimizing a large language model with streaming input and autoregressive output includes the following steps (e.g. Figure 6 shown):

[0082] S11. Collect relevant data in vertical fields (human-computer interaction fields) as training data, where the training data covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure the diversity and representativeness of the training data to cover different user input modes and language styles;

[0083] S12. Clean the collected training data to remove invalid or erroneous data; regularize the data format to ensure that the data meets the requirements of large language model input; and divide the training data into blocks;

[0084] Specifically, according to Figure 2 Prepare the segmented data and corresponding annotations in the format of

[0085] S13. Use the pre-trained large language model as the basis for supervised fine-tuning. The training data includes the original training data and the block-based training data. Use the reinforcement learning algorithm to further optimize the decision-making process of the large language model.

[0086] S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

[0087] Figure 4 The basic steps of large language model training with streaming input in this embodiment are shown.

[0088] S2. Preset the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner;

[0089] If the length of the current unprocessed user data is greater than or equal to the preset data block size, the large language model prompt words, user context data, historically processed data blocks, and the current unprocessed data blocks are concatenated in the order of the user input time, such as Figure 2 , 3 As shown, after the concatenation is completed, the concatenated data is input into the large language model for processing;

[0090] If the length of the currently unprocessed data block is less than the preset data block size, the user's input data will continue to be cached;

[0091] S3, inputting the concatenated data into the large language model, judging whether to reply to the user based on the current input data of the large language model and the output of the large language model, and taking corresponding actions to reply to the user;

[0092] If the output of the large language model is empty, it means that based on the current user input data, the large language model does not have enough information to call the tool or cannot reply to the user yet, and needs to perform input inference for the next data block;

[0093] If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intention and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculations and outputs all content in the form of autoregressive output to reply to the user.

[0094] The output content includes: tool call information, terminology for replying to users, etc.

[0095] When the output of the large language model is not empty, while generating the output result based on the autoregressive forward calculation method, the large language model request ID is added, and the large language model output identifier is added to the input data of the large language model to indicate the input data before the identifier, and then the next data block is processed.

[0096] In this embodiment, when the large language model generates output, an output identifier "<@>" is added to the end of the current input data of the large language model (such as Figure 2 As shown in the figure, it is used to indicate that the output corresponding to this part of the input data has been generated, and then continue to process the next data block; for the output results that have been generated, it is to avoid the large language model from repeatedly outputting the same content in the subsequent steps. In this way, the large language model can support responding to user input instructions in a streaming input and parallel output manner, achieving the purpose of "listening, thinking and outputting". After the user data is input, the large language model has basically completed the analysis, so that it can respond to the user quickly.

[0097] There are two ways to reply to users:

[0098] The first is to wait for the user's input data instruction to complete, and then give feedback to the user in the order of the output of the large language model;

[0099] The second is to respond to the user immediately once the large language model generates output and is able to feed back the result to the user during the user's input data instruction. This response method has a shorter response time.

[0100] like Figure 2 As shown in the figure, the output of the large language model corresponding to the first and second data blocks is empty, indicating that the large language model has not detected the user's intention at this time; after the third data block is input, the output of the large language model is no longer empty, and it starts to output all content in an autoregressive manner to obtain the first output result; while the large language model is outputting, when processing the next data block (i.e., the fourth data block), the output identifier "<@>" is added to the input data of the large language model to identify the input data before the output identifier based on the large language model, that is, the first three data blocks have generated output, and the subsequent large language model will not generate the same content again. Next, the fifth and sixth data blocks are input in a streaming manner, and the output of the large language model is empty. When the seventh data block is input in a streaming manner, the large language model detects the second intention based on the current input data, and starts to output all content in an autoregressive manner, while continuing to process the next data block until the input data instruction issued by the user ends.

[0101] like Figure 2 As shown, the user can be replied to in the third data block and the seventh data block, or the output of the large language model of the third data block and the output of the large language model of the seventh data block can be replied in sequence after the user's input data instruction is completed.

[0102] Figure 1 The flowchart of the large language model in this embodiment for processing streaming input data is shown.

[0103] The embodiment of the present invention further provides a large language model construction system supporting streaming input, which executes the large language model construction method supporting streaming input as described above, including:

[0104] Training model module: used to collect and select data and perform data preprocessing. Based on the pre-trained model, the large language model with streaming input and autoregressive output is trained and optimized by fine-tuning the pre-trained model. The training model module includes:

[0105] Data preparation unit: used to collect relevant data in vertical fields as training data, which covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure the diversity and representativeness of the training data to cover different user input modes and language styles;

[0106] Data preprocessing unit: used to clean the collected training data and remove invalid or erroneous data; to regularize the data format to ensure that the data meets the requirements of large language model input; to divide the training data into blocks;

[0107] Model training unit: used to use the pre-trained large language model as a basis for supervised fine-tuning. The training data includes the original training data and the divided training data. The reinforcement learning algorithm is used to further optimize the decision-making process of the large language model.

[0108] Model evaluation unit: used to evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model;

[0109] The module for processing user input data in blocks is used to pre-set the size of data blocks, divide the user input data into blocks, and accept the user input data in a streaming manner; if the length of the current unprocessed user data is greater than or equal to the pre-set data block size, the large language model prompt words, user context data, historically processed data blocks, and the current unprocessed data blocks are spliced ​​in the chronological order of user input. After the splicing is completed, the spliced ​​data is input to the large language model for processing; if the length of the current unprocessed data block is less than the pre-set data block size, the user input data continues to be cached;

[0110] Output result module: used to input the spliced ​​data into the large language model, determine whether to reply to the user based on the current input data of the large language model and the output of the large language model, and take corresponding actions to reply to the user; if the output of the large language model is empty, it means that according to the current user input data, the large language model has not obtained enough information to call the tool or cannot reply to the user, and it is necessary to perform input reasoning for the next data block; if the output of the large language model is not empty, it means that according to the current user input data, the large language model can detect the user's intention, and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculation, outputs all content in the autoregressive output mode, and replies to the user.

[0111] An embodiment of the present invention further provides a computer device, Figure 7 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention; see the accompanying drawings Figure 7 As shown, the computer device includes: an input system 23, an output system 24, a memory 22 and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the large language model construction method supporting streaming input as provided in the above embodiment; wherein the input system 23, the output system 24, the memory 22 and the processor 21 can be connected by a bus or other means, Figure 7 The example of connecting through bus is taken in the following.

[0112] The memory 22 is a readable and writable storage medium of a computing device, which can be used to store software programs, computer executable programs, such as the program instructions corresponding to the method for building a large language model supporting streaming input described in the embodiment of the present invention; the memory 22 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the device, etc.; in addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device; in some instances, the memory 22 may further include a memory remotely arranged relative to the processor 21, and these remote memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0113] The input system 23 may be used to receive input digital or character information, and to generate key signal input related to user settings and function control of the device; the output system 24 may include display devices such as display screens.

[0114] The processor 21 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 22, that is, implements the above-mentioned method for building a large language model supporting streaming input.

[0115] The computer device provided above can be used to execute the large language model construction method supporting streaming input provided in the above embodiment, and has corresponding functions and beneficial effects.

[0116] The embodiment of the present invention also provides a storage medium containing computer executable instructions, which are used to execute the large language model construction method supporting streaming input as provided in the above embodiment when executed by a computer processor. The storage medium is any of various types of memory devices or storage devices, and the storage medium includes: installation media, such as CD-ROM, floppy disk or tape system; computer system memory or random access memory, such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc.; the storage medium may also include other types of memory or combinations thereof; in addition, the storage medium may be located in the first computer system in which the program is executed, or may be located in a different second computer system, which is connected to the first computer system via a network (such as the Internet); the second computer system may provide program instructions to the first computer for execution. The storage medium includes two or more storage media that can reside in different locations (for example, in different computer systems connected via a network). The storage medium can store program instructions (for example, specifically implemented as a computer program) that can be executed by one or more processors.

[0117] Of course, the storage medium containing computer executable instructions provided in an embodiment of the present invention is not limited to the large language model construction method supporting streaming input as described in the above embodiment, and can also execute related operations in the large language model construction method supporting streaming input provided in any embodiment of the present invention.

[0118] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0119] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.< / eps>

Claims

1. A method for constructing a large language model supporting streaming input, characterized in that: The following steps are involved: S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, fine-tune the pre-trained model to train and optimize the large language model with streaming input and autoregressive output. S2. Preset the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner; If the length of the current unprocessed user data is greater than or equal to the preset data block size, the large language model prompt words, user context data, historically processed data blocks, and the current unprocessed data blocks are concatenated in the order of the user input time. After the concatenation is completed, the concatenated data is input into the large language model for processing; If the length of the currently unprocessed data block is less than the preset data block size, the user's input data will continue to be cached; S3, inputting the concatenated data into the large language model, judging whether to reply to the user based on the current input data of the large language model and the output of the large language model, and taking corresponding actions to reply to the user; If the output of the large language model is empty, it means that based on the current user input data, the large language model does not have enough information to call the tool or cannot reply to the user yet, and needs to perform input inference for the next data block; If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intention and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculations and outputs all content in the form of autoregressive output to reply to the user.

2. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that: The method for training and optimizing a large language model with streaming input and autoregressive output in step S1 comprises the following steps: S11. Collect relevant data in vertical fields as training data, where the training data covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure the diversity and representativeness of the training data to cover different user input modes and language styles; S12. Clean the collected training data to remove invalid or erroneous data; regularize the data format to ensure that the data meets the requirements of large language model input; and divide the training data into blocks; S13. Use the pre-trained large language model as the basis for supervised fine-tuning. The training data includes the original training data and the block-based training data. Use the reinforcement learning algorithm to further optimize the decision-making process of the large language model. S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

3. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that: In the S3 step, when the output of the large language model is not empty, while generating the output result based on the autoregressive forward calculation method, the large language model request ID is added, and the large language model output identifier is added to the input data of the large language model to indicate the input data before the identifier, and then the next data block is processed.

4. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that: The method of replying to the user in step S3 includes any of the following two methods: The first is to wait for the user's input data instruction to complete, and then give feedback to the user in the order of the output of the large language model; The second is to respond to the user immediately once the large language model generates output and is able to feed back the result to the user when the user's input data command is spoken.

5. The method for constructing a large language model supporting streaming input according to claim 2, characterized in that: The process of training and optimizing the large language model with streaming input and autoregressive output in step S1 includes: Select an applicable neural network architecture, including: RNN, LSTM or Transformer; Define a loss function to measure the difference between the sequence predicted by the model and the true sequence; use a gradient descent optimization algorithm to update the parameters of the large language model to minimize the loss function; Initialize the model input and give the autoregressive model an initial input at the beginning of sequence generation; The autoregressive model predicts the next element based on the current input, and then uses the predicted element as the new input to continue predicting the next element, and iterates the prediction and sequence generation through this prediction process; The generation process ends when the autoregressive model predicts the end marker of the sequence or reaches a predetermined length.

6. The method for constructing a large language model supporting streaming input according to claim 5, characterized in that: The process of iterative prediction and sequence generation includes: Using a beam search strategy, the best prediction at each step is selected to generate higher quality sequences; The temperature adjustment strategy is used to control the diversity of the sequences generated by the autoregressive model by adjusting the temperature parameter of the probability distribution.

7. A system for constructing a large language model supporting streaming input, executing the method for constructing a large language model supporting streaming input as claimed in any one of claims 1 to 6, characterized in that: include: Training model module: used to collect and select data, and perform data preprocessing. Based on the pre-trained model, the module fine-tunes the pre-trained model to train and optimize the large language model with streaming input and autoregressive output. Block processing module for user input data: used to pre-set the size of data blocks, divide the user input data into blocks, and accept the user input data in a streaming manner; If the length of the current unprocessed user data is greater than or equal to the preset data block size, the large language model prompt words, user context data, historically processed data blocks and the current unprocessed data blocks are concatenated in the order of the user input time. After the concatenation is completed, the concatenated data is input to the large language model for processing; if the length of the current unprocessed data block is less than the preset data block size, the user input data continues to be cached; Output result module: used to input the concatenated data into the large language model, determine whether to reply to the user based on the current input data of the large language model and the output of the large language model, and take corresponding actions to reply to the user; If the output of the large language model is empty, it means that based on the current user input data, the large language model does not have enough information to call the tool or cannot reply to the user yet, and needs to perform input inference for the next data block; If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intention and then output relevant content to reply to the user. The large language model generates output results based on autoregressive forward calculations and outputs all content in the form of autoregressive output to reply to the user.

8. The system for constructing a large language model supporting streaming input according to claim 7, characterized in that: The training model module includes: Data preparation unit: used to collect relevant data in vertical fields as training data, which covers various scenarios and fields where the large language model will be applied; annotate the training data to ensure the diversity and representativeness of the training data to cover different user input modes and language styles; Data preprocessing unit: used to clean the collected training data and remove invalid or erroneous data; to regularize the data format to ensure that the data meets the requirements of large language model input; to divide the training data into blocks; Model training unit: used to use the pre-trained large language model as a basis for supervised fine-tuning. The training data includes the original training data and the divided training data. The reinforcement learning algorithm is used to further optimize the decision-making process of the large language model. Model evaluation unit: used to evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for building a large language model supporting streaming input described in any one of claims 1 to 6 are implemented.

10. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the method for building a large language model supporting streaming input are implemented as described in any one of claims 1 to 6.

Images