A large language model construction method and system supporting streaming input

Abstract

The application provides a large language model construction method and system supporting streaming input, which comprises the following steps: collecting and selecting data, and performing data preprocessing; using a pre-training model as a basis, fine-tuning the pre-training model, and training and optimizing a large language model in a streaming input and autoregressive output mode; presetting the size of a data block, blocking input data of a user, and accepting the input data of the user in a streaming mode; inputting spliced data to the large language model, judging whether to reply to the user based on the current input data of the large language model and the output of the large language model, and taking corresponding actions to reply to the user. The response speed of the large language model is greatly improved, the large language model supports responding to input data instructions of a user in a streaming input and parallel output mode, an effect of "listening, thinking and outputting simultaneously" is achieved, and the advantage is more obvious in a scenario where a user instruction is long, and the user experience is effectively improved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, large language models, multimodal large language models, and human-computer interaction; specifically, it relates to a method and system for constructing large language models that support streaming input. Background Technology

[0002] Large Language Models (LLMs) are an important branch of artificial intelligence. Through deep learning and training on massive amounts of data, LLMs can understand and generate human-like text. The core of LLMs is the Transformer architecture, which uses a self-attention mechanism to process sequential data, thereby improving processing efficiency and effectiveness. The rapid development of LLM technology brings humanity a significant step closer to achieving Artificial General Intelligence (AGI).

[0003] Currently, large language models are continuously expanding their application areas, from text processing to multimodal data such as speech, images, and videos. These models, through the combination of advanced neural network architectures and extensive data training, are now able to process and understand various types of data.

[0004] In the field of images, large image language models (such as the DALL-E model released by OpenAI) can generate images based on text descriptions, demonstrating cross-modal interaction capabilities. This enables large language models to understand and generate visual content, providing users with a richer interactive experience.

[0005] Applications in the video field are more complex, involving the understanding of dynamic visual content. Large language models, by combining visual encoders and language models, can process video data and perform tasks such as video question answering. For example, the ChatVideo system can parse video into a trajectory, annotate its attributes, store it in a database, and enable interaction with users through a database manager.

[0006] In the field of speech, large language models can perform real-time voice interaction without relying on additional automatic speech recognition (ASR) or text-to-speech (TTS) systems. This improves both the overall performance and the response speed of the model. For example, OpenAI's ChatGPT 4o supports end-to-end real-time voice dialogue, enabling continuous voice-to-speech conversations and improving the naturalness and fluency of voice interaction.

[0007] In human-computer interaction (HCI) applications, response speed is a crucial metric for large language models. Currently, large language models operate based on all user input data, using an autoregressive approach for output. This means that large language models cannot process user requests in a streaming manner; they can only begin outputting after seeing all user input data. This significantly limits the optimization of response speed, especially when user input is complex and contains multiple intents, greatly extending user wait times and negatively impacting the user experience. Summary of the Invention

[0008] Therefore, the purpose of this invention is to develop a method and system for constructing a large language model that supports streaming input. The method involves segmenting user input data, concatenating large language model prompts, user context information, historical data blocks, and current data blocks, and feeding this data into the large language model for processing. The large language model outputs data and responds to the user based on an autoregressive output method. This significantly improves the response speed of the large language model, enabling it to "interrupt" – that is, provide a corresponding response while the user is speaking. The large language model supports responding to user input data commands in a streaming input and parallel output manner, achieving the ability to "listen, think, and output simultaneously," thus providing a superior user experience.

[0009] This invention provides a method for constructing a large language model that supports streaming input, comprising the following steps:

[0010] S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, train and optimize the large language model with streaming input and autoregressive output by fine-tuning the pre-trained model.

[0011] S2. Pre-set the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner;

[0012] If the length of the currently unprocessed user data is greater than or equal to the pre-set data block size, then the large language model prompts, user context data, historically processed data blocks, and currently unprocessed data blocks will be concatenated according to the time sequence of user input. After concatenation, the concatenated data will be input into the large language model for processing.

[0013] If the length of the currently unprocessed data block is less than the preset data block size, then continue caching the user's input data;

[0014] S3. Input the concatenated data (preferably, according to the request ID of the large language model) into the large language model. Based on the current input data of the large language model and the output of the large language model, determine whether to reply to the user and take corresponding actions to reply to the user, such as: intelligent agent mode.

[0015] If the output of the large language model is empty, for example, " <eps>If the result is "", it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is not yet able to respond to the user, and the input reasoning for the next data block needs to be performed.

[0016] If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and then output relevant content to reply to the user. The large language model generates the output result based on the autoregressive forward computation method and outputs all content in the autoregressive output method to reply to the user.

[0017] Specifically, the output includes: tool call information, terminology used to reply to users, etc.

[0018] In Large Language Models (LLMs), "autoregressive output" typically refers to a sequence generation method. A LLM with streaming input and autoregressive output predicts the next element in the sequence based on the output of the preceding text. In Natural Language Processing (NLP), autoregressive models can generate text sequences, word by word, until a complete sentence or paragraph is produced.

[0019] Autoregressive models have the following technical characteristics:

[0020] (1) Sequence dependency: Autoregressive models assume that each element in a sequence (such as a word or character) depends only on the elements preceding it. This dependency allows autoregressive models to build the output step by step in a certain order.

[0021] (2) Probabilistic modeling: Autoregressive models typically predict the next element in a sequence based on probability distributions. The model calculates the probability of each possible next element and selects the element with the highest probability as the output.

[0022] (3) Parameterization: Autoregressive models are usually modeled through parameterization, and these parameters can be learned from training data. The form of parameterization can be neural networks, recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or Transformers, etc.

[0023] The autoregressive forward computation process includes forward propagation: at each step of generating the sequence, the model predicts the next element based on the current input (which may be empty or contain the first few elements of the sequence). This process is repeated until a complete sequence is generated or a certain termination condition is met.

[0024] Furthermore, the method for training and optimizing a large language model with streaming input and autoregressive output in step S1 includes the following steps:

[0025] S11. Collect relevant data from vertical fields as training data (e.g., human-computer interaction field), the training data covering various scenarios and fields where the large language model will be applied; label the training data to ensure that each input data block has the correct output label; ensure the diversity and representativeness of the training data to cover different user input patterns and language styles;

[0026] S12. Clean the collected training data, removing invalid or erroneous data; standardize the data format to ensure that the data meets the input requirements of the large language model; divide the training data into blocks.

[0027] S13. Using a pre-trained large language model as a foundation, supervised fine-tuning (SFT) is performed. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms, such as PPO (Proximal Policy Optimization), are used to further optimize the decision-making process of the large language model.

[0028] S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

[0029] Furthermore, in step S3, when the output of the large language model is not empty, while generating the output result based on the autoregressive forward computation method, the large language model request ID is added, and the large language model output identifier, such as "<@>", is added to the input data of the large language model to indicate the input data before the identifier, and then the next data block is processed.

[0030] For example, while the large language model is generating output, an "output identifier" is added to the end of the current input data of the large language model to indicate that the output corresponding to this part of the input data has been generated, and then the processing of the next data block continues;

[0031] By adding output identifiers to the large language model, the same content is avoided from being repeatedly output by the large language model in subsequent steps for the already generated output results. In this way, the large language model can support responding to user input data commands in a streaming input and parallel output manner, achieving the ability to "think and output while listening". After the user data input is completed, the large language model has also basically completed parsing, thus enabling it to respond to the user quickly.

[0032] Furthermore, the method for responding to the user in step S3 includes either of the following two:

[0033] The first approach is to wait for the user's input data instructions to finish, and then provide feedback to the user in the order of the output of the large language model.

[0034] The second approach is to respond to the user immediately once the large language model generates output and can provide feedback to the user during the user's input data command. This response method has a shorter response time, but it may cause some interference to the user.

[0035] Furthermore, the process of training and optimizing the large language model with streaming input and autoregressive output in step S1 includes:

[0036] Choose a suitable neural network architecture, including: RNN, LSTM or Transformer;

[0037] Define a loss function to measure the difference between the sequence predicted by the model and the true sequence; preferably, the loss function is cross-entropy loss; use the gradient descent optimization algorithm to update the parameters of the large language model to minimize the loss function;

[0038] Initialize the model input: At the beginning of sequence generation, give the autoregressive model an initial input, such as a special start marker;

[0039] The autoregressive model predicts the next element based on the current input, and then uses the predicted element as the new input to continue predicting the next element. This prediction process is repeated iteratively to generate the sequence.

[0040] The generation process ends when the autoregressive model predicts the end marker of the sequence or reaches the predetermined length.

[0041] Furthermore, the iterative prediction and sequence generation process includes:

[0042] The beam search strategy is used to select the best prediction at each step to generate higher quality sequences.

[0043] The temperature tuning strategy is used to control the diversity of sequences generated by the autoregressive model by adjusting the temperature parameter of the probability distribution.

[0044] This invention also provides a large language model construction system supporting streaming input, which executes the large language model construction method supporting streaming input as described above, including:

[0045] Training Model Module: Used to collect and select data, and perform data preprocessing. Based on the pre-trained model, it trains and optimizes the large language model with streaming input and autoregressive output by fine-tuning the pre-trained model.

[0046] The chunked user input data processing module is used to pre-set the data block size, divide the user input data into chunks, and accept the user input data in a streaming manner. If the length of the currently unprocessed user data is greater than or equal to the pre-set data block size, the large language model prompts, user context data, historically processed data blocks, and the currently unprocessed data blocks are concatenated according to the time sequence of user input. After concatenation, the concatenated data is input to the large language model for processing. If the length of the currently unprocessed data block is less than the pre-set data block size, the user input data continues to be cached.

[0047] Output Result Module: This module inputs the concatenated data into the large language model. Based on the current input data of the large language model, it determines whether to respond to the user based on the output of the large language model and takes corresponding actions to respond to the user. If the output of the large language model is empty, it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is unable to respond to the user, and it needs to perform input reasoning for the next data block. If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and can output relevant content to respond to the user. The large language model generates the output result based on autoregressive forward computation and outputs all content according to the autoregressive output method to respond to the user.

[0048] Furthermore, the training model module includes:

[0049] Data preparation unit: used to collect relevant data in the vertical field as training data, the training data covering various scenarios and fields in which the large language model will be applied; to label the training data to ensure the diversity and representativeness of the training data, so as to cover different user input patterns and language styles;

[0050] Data preprocessing unit: used to clean the collected training data, removing invalid or erroneous data; to standardize the data format to ensure that the data meets the input requirements of the large language model; and to divide the training data into blocks.

[0051] Model training unit: Used to perform supervised fine-tuning based on a pre-trained large language model. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms are used to further optimize the decision-making process of the large language model.

[0052] Model Evaluation Unit: Used to evaluate the performance of large language models on independent test sets, with a focus on the output accuracy of large language models.

[0053] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the steps of the method for constructing a large language model supporting streaming input as described above.

[0054] The present invention also provides a computer device, the computer device including a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the large language model construction method supporting streaming input as described above.

[0055] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0056] The method and system for constructing a large language model that supports streaming input provided by this invention divides user input data into blocks, concatenates the large language model prompts, user context information, historical data blocks, and current data blocks, and feeds them to the large language model as streaming input. This allows for streaming processing of user input data commands. The large language model outputs and responds to the user based on an autoregressive output method, significantly improving the response speed of the large language model. This enables the large language model to have an "interruption" function, meaning that the large language model can provide corresponding responses while the user is speaking. The large language model supports responding to user input data commands in a streaming input and parallel output manner, achieving the effect of "listening and thinking while outputting," which is especially advantageous in scenarios with long user commands, effectively improving the user experience. Attached Figure Description

[0057] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0058] In the attached diagram:

[0059] Figure 1 This is a flowchart illustrating the process of processing streaming input data using a large language model, as described in an embodiment of the present invention.

[0060] Figure 2 This is an example diagram illustrating the processing of streaming input using a large language model according to an embodiment of the present invention;

[0061] Figure 3 This is a diagram showing the arrangement of input data for a large language model with streaming input, as described in an embodiment of the present invention.

[0062] Figure 4 This is a diagram illustrating the training steps of a large language model with streaming input according to an embodiment of the present invention.

[0063] Figure 5 This is a flowchart of a method for constructing a large language model that supports streaming input according to the present invention;

[0064] Figure 6 This is a flowchart of the method for training and optimizing a large language model with streaming input and autoregressive output according to the present invention.

[0065] Figure 7 This is a schematic diagram of the configuration of a computer device according to an embodiment of the present invention. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of systems and products consistent with some aspects of this disclosure as detailed in the appended claims.

[0067] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms "a," "the," and "the" as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0068] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0069] The embodiments of the present invention will be described in further detail below.

[0070] This invention provides a method for constructing a large language model that supports streaming input. See [link to relevant documentation]. Figure 5 As shown, it includes the following steps:

[0071] S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, train and optimize the large language model with streaming input and autoregressive output by fine-tuning the pre-trained model.

[0072] The process of training and optimizing a large language model with streaming input and autoregressive output includes:

[0073] Choose a suitable neural network architecture, including: RNN, LSTM or Transformer;

[0074] Define a loss function to measure the difference between the sequence predicted by the model and the true sequence; use the gradient descent optimization algorithm to update the parameters of the large language model to minimize the loss function; in this embodiment, the loss function is cross-entropy loss.

[0075] Initialize the model input: At the beginning of sequence generation, give the autoregressive model an initial input. In this embodiment, a special starting marker is used as the input.

[0076] The autoregressive model predicts the next element based on the current input, and then uses the predicted element as the new input to continue predicting the next element. This prediction process is repeated iteratively to generate the sequence.

[0077] The generation process ends when the autoregressive model predicts the end marker of the sequence or reaches the predetermined length.

[0078] The iterative prediction and sequence generation process includes:

[0079] The beam search strategy is used to select the best prediction at each step to generate higher quality sequences.

[0080] The temperature tuning strategy is used to control the diversity of sequences generated by the autoregressive model by adjusting the temperature parameter of the probability distribution.

[0081] The method for training and optimizing large language models with streaming input and autoregressive output includes the following steps (e.g.) Figure 6 (as shown)

[0082] S11. Collect relevant data from the vertical domain (human-computer interaction domain) as training data. The training data covers various scenarios and domains in which the large language model will be applied. Label the training data to ensure its diversity and representativeness, so as to cover different user input patterns and language styles.

[0083] S12. Clean the collected training data, removing invalid or erroneous data; standardize the data format to ensure that the data meets the input requirements of the large language model; divide the training data into blocks.

[0084] Specifically, according to Figure 2 The format in the document is used to prepare the segmented data and corresponding annotations;

[0085] S13. Using a pre-trained large language model as a foundation, supervised fine-tuning is performed. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms are used to further optimize the decision-making process of the large language model.

[0086] S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

[0087] Figure 4 This embodiment illustrates the basic steps of training a large language model using streaming input.

[0088] S2. Pre-set the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner;

[0089] If the length of currently unprocessed user data is greater than or equal to the pre-defined data block size, then the large language model prompts, user context data, historically processed data blocks, and currently unprocessed data blocks will be concatenated according to the chronological order of user input. Figure 2 , 3 As shown, after the splicing is completed, the spliced ​​data is input into the large language model for processing;

[0090] If the length of the currently unprocessed data block is less than the preset data block size, then continue caching the user's input data;

[0091] S3. Input the concatenated data into the large language model. Based on the current input data of the large language model and the output of the large language model, determine whether to reply to the user and take corresponding actions to reply to the user.

[0092] If the output of the large language model is empty, it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is not yet able to respond to the user, and it needs to perform input reasoning for the next data block.

[0093] If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and then output relevant content to reply to the user. The large language model generates the output result based on the autoregressive forward computation method and outputs all content in the autoregressive output method to reply to the user.

[0094] The output includes: tool call information, terminology used to reply to users, etc.

[0095] When the output of the large language model is not empty, while generating the output result based on the autoregressive forward computation method, the large language model request ID is added, and the large language model output identifier is added to the input data of the large language model to indicate the input data before the identifier, and then the next data block is processed.

[0096] In this embodiment, while the large language model generates output, an output identifier "<@>" is added to the end of the current input data of the large language model (e.g., ...). Figure 2 As shown in the figure, it is used to indicate that the output corresponding to this part of the input data has been generated, and then the processing of the next data block continues; for the output results of the already generated part, to avoid the large language model from repeatedly outputting the same content in the later steps, the large language model can support responding to the user's input commands in the form of streaming input and parallel output, to achieve the purpose of "thinking and outputting while listening". After the user's data input is completed, the large language model has also basically completed parsing, so as to be able to respond to the user quickly.

[0097] There are two ways to reply to users:

[0098] The first approach is to wait for the user's input data instructions to finish, and then provide feedback to the user in the order of the output of the large language model.

[0099] The second approach is to respond to the user immediately once the large language model generates output and can provide feedback to the user during the user's input data command. This response method has a shorter response time.

[0100] like Figure 2 As shown, the output of the large language model (LLM) for the first and second data blocks is empty, indicating that the LLM has not detected the user's intent at this point. After the third data block is input, the LLM output is no longer empty and begins to output all content in an autoregressive manner, resulting in the first output result. While the LLM is outputting, when processing the next data block (i.e., the fourth data block), an output identifier "<@>" is added to the input data of the LLM to indicate that the LLM is based on the input data before the output identifier. That is, the first three data blocks have already produced output, and the LLM will not generate the same content again in subsequent data blocks. Next, the fifth and sixth data blocks are input in a streaming manner, and the LLM output is empty. When the seventh data block is input in a streaming manner, the LLM detects the second intent based on the current input data and begins to output all content in an autoregressive manner, while continuing to process the next data block until the user's input data instruction ends.

[0101] like Figure 2 As shown, the system can respond to the user in the third and seventh data blocks, or it can respond sequentially after the user's input data command is completed, by replying to the output of the large language model in the third and seventh data blocks.

[0102] Figure 1 The flow logic of the large language model in this embodiment for processing streaming input data is shown.

[0103] This invention also provides a large language model construction system supporting streaming input, which executes the large language model construction method supporting streaming input as described above, including:

[0104] Training model module: Used to collect and select data, and perform data preprocessing. Based on a pre-trained model, it trains and optimizes a large language model with streaming input and autoregressive output by fine-tuning the pre-trained model. The training model module includes:

[0105] Data preparation unit: used to collect relevant data in the vertical field as training data, the training data covering various scenarios and fields in which the large language model will be applied; to label the training data to ensure the diversity and representativeness of the training data, so as to cover different user input patterns and language styles;

[0106] Data preprocessing unit: used to clean the collected training data, removing invalid or erroneous data; to standardize the data format to ensure that the data meets the input requirements of the large language model; and to divide the training data into blocks.

[0107] Model training unit: Used to perform supervised fine-tuning based on a pre-trained large language model. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms are used to further optimize the decision-making process of the large language model.

[0108] Model Evaluation Unit: Used to evaluate the performance of large language models on independent test sets, with a focus on the output accuracy of large language models;

[0109] The chunked user input data processing module is used to pre-set the data block size, divide the user input data into chunks, and accept the user input data in a streaming manner. If the length of the currently unprocessed user data is greater than or equal to the pre-set data block size, the large language model prompts, user context data, historically processed data blocks, and the currently unprocessed data blocks are concatenated according to the time sequence of user input. After concatenation, the concatenated data is input to the large language model for processing. If the length of the currently unprocessed data block is less than the pre-set data block size, the user input data continues to be cached.

[0110] Output Result Module: This module inputs the concatenated data into the large language model. Based on the current input data of the large language model, it determines whether to respond to the user based on the output of the large language model and takes corresponding actions to respond to the user. If the output of the large language model is empty, it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is unable to respond to the user, and it needs to perform input reasoning for the next data block. If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and can output relevant content to respond to the user. The large language model generates the output result based on autoregressive forward computation and outputs all content according to the autoregressive output method to respond to the user.

[0111] This invention also provides a computer device. Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention; see the accompanying drawings. Figure 7 As shown, the computer device includes: an input system 23, an output system 24, a memory 22, and a processor 21; the memory 22 is used to store one or more programs; when the one or more programs are executed by the one or more processors 21, the one or more processors 21 implement the large language model construction method supporting streaming input as provided in the above embodiments; wherein the input system 23, the output system 24, the memory 22, and the processor 21 can be connected via a bus or other means. Figure 7 Taking the example of a connection between China and Israel via a bus.

[0112] The memory 22, as a read / write storage medium for a computing device, can be used to store software programs and computer-executable programs, such as the program instructions corresponding to the large language model construction method supporting streaming input described in this embodiment of the invention. The memory 22 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on the use of the device. Furthermore, the memory 22 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 22 may further include memory remotely located relative to the processor 21, and these remote memories can be connected to the device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0113] The input system 23 can be used to receive input digital or character information, and generate key signal inputs related to user settings and function control of the device; the output system 24 may include display devices such as a display screen.

[0114] The processor 21 executes various functional applications and data processing of the device by running software programs, instructions and modules stored in the memory 22, thereby realizing the above-mentioned method for constructing a large language model that supports streaming input.

[0115] The computer equipment provided above can be used to execute the large language model construction method supporting streaming input provided in the above embodiments, and has corresponding functions and beneficial effects.

[0116] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the large language model construction method supporting streaming input as provided in the above embodiments. The storage medium can be any type of memory device or storage device, including: mounting media such as CD-ROM, floppy disk, or magnetic tape systems; computer system memory or random access memory such as DRAM, DDRRAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements; the storage medium may also include other types of memory or combinations thereof; furthermore, the storage medium may reside in a first computer system in which the program is executed, or it may reside in a different second computer system connected to the first computer system via a network (such as the Internet); the second computer system can provide program instructions to the first computer for execution. The storage medium includes two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). The storage medium may store program instructions (e.g., specifically implemented as a computer program) executable by one or more processors.

[0117] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the large language model construction method supporting streaming input as described in the above embodiments, but can also execute related operations in the large language model construction method supporting streaming input provided in any embodiment of the present invention.

[0118] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0119] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.< / eps>

Claims

1. A method for constructing a large language model that supports streaming input, characterized in that, Includes the following steps: S1. Collect and select data, and perform data preprocessing. Based on the pre-trained model, train and optimize the large language model with streaming input and autoregressive output by fine-tuning the pre-trained model. S2. Pre-set the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner; If the length of the currently unprocessed user data is greater than or equal to the pre-set data block size, then the large language model prompts, user context data, historically processed data blocks, and currently unprocessed data blocks will be concatenated according to the time sequence of user input. After concatenation, the concatenated data will be input into the large language model for processing. If the length of the currently unprocessed data block is less than the preset data block size, then continue caching the user's input data; S3. Input the concatenated data into the large language model. Based on the current input data of the large language model and the output of the large language model, determine whether to reply to the user and take corresponding actions to reply to the user. If the output of the large language model is empty, it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is not yet able to respond to the user, and it needs to perform input reasoning for the next data block. If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and then output relevant content to reply to the user. The large language model generates the output result based on the autoregressive forward computation method and outputs all content in the autoregressive output method to reply to the user.

2. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that, The method for training and optimizing a large language model with streaming input and autoregressive output in step S1 includes the following steps: S11. Collect relevant data from vertical fields as training data, which covers various scenarios and fields in which the large language model will be applied; label the training data to ensure its diversity and representativeness, so as to cover different user input patterns and language styles. S12. Clean the collected training data, removing invalid or erroneous data; standardize the data format to ensure that the data meets the input requirements of the large language model; divide the training data into blocks. S13. Using a pre-trained large language model as a foundation, supervised fine-tuning is performed. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms are used to further optimize the decision-making process of the large language model. S14. Evaluate the performance of the large language model on an independent test set, focusing on the output accuracy of the large language model.

3. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that, In step S3, when the output of the large language model is not empty, while generating the output result based on the autoregressive forward computation method, a large language model request ID is added, and a large language model output identifier is added to the input data of the large language model to indicate the input data before the identifier, and then the next data block is processed.

4. The method for constructing a large language model supporting streaming input according to claim 1, characterized in that, The method for responding to the user in step S3 includes any one of the following two: The first approach is to wait for the user's input data instructions to finish, and then provide feedback to the user in the order of the output of the large language model. The second approach is to respond to the user immediately once the large language model generates output and can provide feedback to the user during the user's input data command.

5. A system for constructing a large language model supporting streaming input, comprising executing the method for constructing a large language model supporting streaming input as described in any one of claims 1-4, characterized in that, include: Training Model Module: Used to collect and select data, and perform data preprocessing. Based on the pre-trained model, it trains and optimizes the large language model with streaming input and autoregressive output by fine-tuning the pre-trained model. The block processing module for user input data is used to pre-define the size of the data block, divide the user's input data into blocks, and accept the user's input data in a streaming manner. If the length of the currently unprocessed user data is greater than or equal to the preset data block size, then the large language model prompts, user context data, historically processed data blocks, and the currently unprocessed data blocks are concatenated according to the time sequence of user input. After concatenation, the concatenated data is input to the large language model for processing. If the length of the currently unprocessed data block is less than the preset data block size, then the user input data continues to be cached. Output Result Module: Used to input the concatenated data into the large language model, and based on the current input data of the large language model, determine whether to reply to the user according to the output of the large language model, and take corresponding actions to reply to the user; If the output of the large language model is empty, it means that based on the current user input data, the large language model has not obtained enough information to call the tool or is not yet able to respond to the user, and it needs to perform input reasoning for the next data block. If the output of the large language model is not empty, it means that based on the current user input data, the large language model can detect the user's intent and then output relevant content to reply to the user. The large language model generates the output result based on the autoregressive forward computation method and outputs all content in the autoregressive output method to reply to the user.

6. The large language model construction system supporting streaming input according to claim 5, characterized in that, The training model module includes: Data preparation unit: used to collect relevant data in the vertical field as training data, the training data covering various scenarios and fields in which the large language model will be applied; to label the training data to ensure the diversity and representativeness of the training data, so as to cover different user input patterns and language styles; Data preprocessing unit: used to clean the collected training data, removing invalid or erroneous data; to standardize the data format to ensure that the data meets the input requirements of the large language model; and to divide the training data into blocks. Model training unit: Used to perform supervised fine-tuning based on a pre-trained large language model. The training data includes the original training data and the training data after being divided into blocks. Reinforcement learning algorithms are used to further optimize the decision-making process of the large language model. Model Evaluation Unit: Used to evaluate the performance of large language models on independent test sets, with a focus on the output accuracy of large language models.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method for constructing a large language model that supports streaming input as described in any one of claims 1-4.

8. A computer device, the computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method for constructing a large language model that supports streaming input as described in any one of claims 1-4.

Images