Inference optimization method and apparatus for language model, electronic device, and storage medium

Abstract

The present disclosure provides an inference optimization method and apparatus for a language model, an electronic device, and a storage medium. The inference optimization method comprises: in response to acquiring at least one piece of request information requesting inference of input information using a preset language model, dividing first request information into a plurality of segments of request information according to a preset length; using the language model to perform full inference and hybrid inference on the plurality of segments of request information and second request information, so as to obtain a plurality of first inference results, wherein the second request information is at least one piece of request information obtained simultaneously with the first request information or obtained during the process of performing full inference or hybrid inference on the plurality of segments of request information; and using the language model to perform incremental inference on the plurality of first inference results, so as to obtain an inference result output by the language model when performing inference on the input information. The method reduces video memory occupancy during language model inference and improves the inference efficiency of the language model.

Description

Inference optimization method and device of language model, electronic equipment and storage medium Cross-reference of related disclosures The present disclosure claims priority to the Chinese patent application No. 2024110762113, filed on August 7, 2024, and entitled "Inference optimization method of language model", the entire content of which is incorporated herein by reference. TECHNICAL FIELD The present disclosure relates to the technical field of large language models, and in particular to an inference optimization method of language model, device, electronic equipment and storage medium. BACKGROUND The basic structure of a large language model is generally the decoder part of a Transformer language model. The Transformer language model is an autoregressive language model that predicts the next text token based on the previous text in the prediction process, and uses the newly obtained text token as input to predict the next text token. This process is repeated until a token termination indicating the end of the representation is generated. Inference is divided into two stages. The first stage is the full inference (or first inference, pre-filling) stage for the previous prompt of the user input. The second stage is the incremental inference stage for the model prediction result. Currently, for a long text request with a super-long input, the language model may need several seconds for full inference. The super-long input cannot be incrementally inferred simultaneously with the short input, resulting in a long waiting time for incremental inference, which reduces the efficiency of the language model in computing and inference. In addition, in large model inference, storage resources have become an important competitive factor for high-performance computing. In the case of processing a super-long input, the occupation of video memory resources is very tight, limiting the progress of long text inference of the language model. SUMMARY Therefore, the purpose of the present disclosure is to provide an inference optimization method, device, electronic equipment and storage medium of language model. Based on the idea of continuous batch processing, the at least one request information is segmented for full inference and mixed inference, which realizes less occupation of video memory when the language model infers the long text information, improves the throughput of the language model inference service in practical applications, optimizes the video memory occupation and model inference performance in the inference process through zero redundancy processing for full inference and mixed inference, and improves the efficiency of the language model in computing and inference. The present disclosure provides an inference optimization method of language model, which comprises: In response to obtaining at least one request information requesting to infer input information using a preset language model, dividing a first request information into multiple segments of request information according to a preset length; perform full reasoning and mixed reasoning on the multiple pieces of request information and second request information by using the language model to obtain multiple first reasoning results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained in a process of full reasoning or mixed reasoning on the multiple pieces of request information; perform incremental reasoning on the multiple first reasoning results by using the language model to obtain a reasoning result output by the language model in the process of reasoning on the input information. Optionally, the performing full reasoning and mixed reasoning on the multiple pieces of request information and second request information by using the language model to obtain multiple first reasoning results comprises: performing full reasoning on first request piece information in the multiple pieces of request information by using the language model, or performing full reasoning and mixed reasoning to obtain historical result information output by the full reasoning or the full reasoning and the mixed reasoning; in response to obtaining the second request information requesting to perform reasoning on input information by using a preset language model, taking second request piece information in the multiple pieces of request information and the second request information as a same batch of requests to obtain first request batch information; performing mixed reasoning by using the language model based on the historical result information and the first request batch information to obtain multiple first reasoning results. Optionally, the performing full reasoning and mixed reasoning on the multiple pieces of request information and second request information by using the language model to obtain multiple first reasoning results further comprises: in response to obtaining third request information requesting to perform reasoning on input information by using a preset language model, taking the multiple first reasoning results and the third request information as a same batch of requests to obtain second request batch information; performing mixed reasoning by using the language model based on the second request batch information to obtain multiple first reasoning results. Optionally, the performing full reasoning and mixed reasoning on the multiple pieces of request information and second request information by using the language model to obtain multiple first reasoning results comprises: performing full reasoning on first request piece information in the multiple pieces of request information by using the language model, or performing full reasoning and mixed reasoning to obtain first historical result information output by the full reasoning or the full reasoning and the mixed reasoning; in response to obtaining the second request information requesting to perform reasoning on input information by using a preset language model, taking second request piece information in the multiple pieces of request information and the second request information as a same batch of requests to obtain first request batch information; performing hybrid inference based on the first historical result information and the first request batch information by using the language model to obtain at least one first inference result and second historical result information; In response to obtaining the third request information requesting to perform inference on the input information by using the preset language model, the first inference result, the third request segment information in the multiple request information, and the third request information are taken as a same batch request to obtain third request batch information; Performing hybrid inference based on the second historical result information and the third request batch information by using the language model to obtain multiple first inference results. Optionally, the full-amount inference and the hybrid inference on the multiple request information and the second request information by using the language model to obtain multiple first inference results comprise: The multiple request information and the second request information are batch-merged, and the full-amount inference and the hybrid inference on the multiple request information and the second request information by using the language model to obtain multiple first inference results are performed; Optionally, the batch-merging of the multiple request information and the second request information and the full-amount inference and the hybrid inference on the multiple request information and the second request information by using the language model to obtain multiple first inference results comprise: Performing full-amount inference or full-amount inference and hybrid inference on the first request segment information in the multiple request information by using the language model to obtain historical result information output by the full-amount inference or the full-amount inference and the hybrid inference; In response to obtaining the second request information requesting to perform inference on the input information by using the preset language model, the second request segment information in the multiple request information and the second request information are merged to obtain first merged information; Performing zero-redundancy hybrid inference on the historical result information and the first merged information by using the language model to obtain multiple first inference results. Optionally, the zero-redundancy hybrid inference on the historical result information and the first merged information by using the language model to obtain multiple first inference results comprises: Performing label embedding and layer normalization processing on the first merged information and the historical result information respectively to obtain first merged input information; Performing vector linear transformation on the first merged input information by using a self-attention layer of the language model to obtain first vector input information; Performing zero-redundancy full-amount operation on the first vector input information by using the self-attention layer to obtain a first output result; perform full-amount feedforward processing on the first output result by using a feedforward layer of the language model to obtain a first intermediate result; obtain a plurality of first inference results based on N first merged input information, first vector input information, first output result and first intermediate result, N being a natural number greater than 1. Optionally, the zero-redundancy full-amount operation based on the first vector input information and using the self-attention layer to obtain the first output result comprises: The first vector input information is dispersedly input into a plurality of independent operators in the self-attention layer according to a preset division strategy; The plurality of independent operators respectively perform full-amount operation on the dispersedly input first vector input information to obtain a plurality of first operation results, and the plurality of first operation results are merged and written into the preset memory corresponding to the language model; The plurality of first operation results merged and written into the preset memory are split to obtain the first output result. Optionally, the batch-merging of the plurality of pieces of request information and the second request information and the zero-redundancy full-amount inference and mixed inference of the plurality of pieces of request information and the second request information by using the language model to obtain a plurality of first inference results further comprises: In response to obtaining third request information requesting inference on input information by using a preset language model, the first inference result and the third request information are merged to obtain second merged information; Zero-redundancy mixed inference is performed based on the second merged information by using the language model to obtain a plurality of first inference results. Optionally, the batch-merging of the plurality of pieces of request information and the second request information and the zero-redundancy full-amount inference and mixed inference of the plurality of pieces of request information and the second request information by using the language model to obtain a plurality of first inference results comprises: Full-amount inference, or full-amount inference and mixed inference, of first request segment information in the plurality of pieces of request information by using the language model is performed to obtain first historical result information output by the full-amount inference, or the full-amount inference and the mixed inference; In response to obtaining the second request information requesting inference on input information by using a preset language model, second request segment information in the plurality of pieces of request information and the second request information are merged to obtain first merged information; Zero-redundancy mixed inference is performed based on the first historical result information and the first merged information by using the language model to obtain at least one first inference result and second historical result information; In response to obtaining third request information requesting to infer input information using a preset language model, the first inference result, third request segment information in the multiple segments of request information, and the third request information are merged to obtain third merged information; Based on the second historical result information and the third merged information, zero-redundancy hybrid inference is performed using the language model to obtain multiple first inference results. The embodiments of the present disclosure also provide an inference optimization device of a language model, and the inference optimization device comprises: The segmentation processing module is configured to divide first request information into multiple segments of request information according to a preset length in response to obtaining at least one request information requesting to infer input information using a preset language model. The hybrid inference module is configured to perform full-inference and hybrid inference on the multiple segments of request information and second request information using the language model to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained in a process of performing full-inference or hybrid inference on the multiple segments of request information. The incremental inference module is configured to perform incremental inference on the multiple first inference results using the language model to obtain an inference result output by the language model in the process of inferring the input information. The embodiments of the present disclosure also provide an electronic device, which comprises a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory through the bus. The machine-readable instructions are executed by the processor to perform the steps of the inference optimization method of the language model as described above. The embodiments of the present disclosure also provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the inference optimization method of the language model as described above are performed. The inference optimization method of the language model provided by the embodiments of the present disclosure comprises: in response to obtaining at least one request information requesting to infer input long text information using a preset language model, dividing first request information into multiple segments of request information according to a preset length; performing full-inference and hybrid inference on the multiple segments of request information and second request information using the language model to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained in a process of performing full-inference or hybrid inference on the multiple segments of request information; and performing incremental inference on the multiple first inference results using the language model to obtain an inference result output by the language model in the process of inferring the input information. Compared with the method of processing the super-long input of the long text request by the language model in the prior art, the segmented full-inference and mixed-inference are performed on at least one request information based on the idea of continuous batch processing, the memory of the language model is less occupied when the long text information is inferred, the throughput of the language model inference service is improved in actual application, and the memory occupation and model inference performance in the inference process are optimized through zero redundancy processing for the full-inference and mixed-inference, and the efficiency of the language model in the calculation inference is improved. In order to make the above object, features and advantages of the present disclosure more obvious and easy to understand, the following preferred embodiments are specifically described below, and the detailed description is made below by combining with the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present disclosure, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings. FIG. 1 is a layered abstract schematic diagram of an inference framework of a language model provided by an embodiment of the present disclosure; FIG. 2 is a flowchart of an inference optimization method of a language model provided by an embodiment of the present disclosure; FIG. 3 is a segmented schematic diagram of full-inference of a language model provided by an embodiment of the present disclosure; FIG. 4 is an inference schematic diagram of zero redundancy processing of a language model provided by an embodiment of the present disclosure; FIG. 5 is a structural schematic diagram of an inference optimization device of a language model provided by an embodiment of the present disclosure; FIG. 6 is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION In order to make the object, technical solutions and advantages of the embodiments of the present disclosure more clear, the following will combine the drawings in the embodiments of the present disclosure to make a clear and complete description of the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, not all the embodiments. The components of the embodiments of the present disclosure described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the claimed present disclosure, but only represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, every other embodiment obtained by those skilled in the art without creative labor belongs to the scope of protection of the present disclosure. Currently, the classic architecture of the Transformer model in the language model includes three main components: a self-attention (Masked Multi-Head Attention) part configured to establish the dependency between positions in the input sequence; a residual connection and normalization operation part (Add&Norm); and a feedforward neural network part (FeedForward) and a multilayer perceptron (MLP) part. Through the cooperation of these components, the Transformer model can effectively capture the key information in the input sequence and achieve strong sequence modeling capability. Research has found that the proportion of the intermediate memory in the inference process based on the Transformer model structure mainly includes the input and output parts of the model, the output part of the self-attention layer, and the temporary memory part required in the dimension increasing and then dimension decreasing process of the feedforward layer; among them, the memory proportion is in a linear relationship with the length of the token processed by the language model, and the size of the intermediate memory part of the language model in the inference process has an important influence on processing long text input. In large model inference, storage resources have become an important competitive factor for high-performance computing. In the case of processing super-long input, the occupation of memory resources is very tight, which limits the process of long text inference of the language model. Here, when the language model based on the Transformer model structure receives a super-long input of long text request, the language model may need a few seconds in full inference, and the super-long input cannot be incrementally inferred at the same time as the short input, which makes the waiting time of the incremental inference too long, reducing the efficiency of the language model in computing and inference. In addition, in large model inference, storage resources have become an important competitive factor for high-performance computing. In the case of processing super-long input, the occupation of memory resources is very tight, which limits the process of long text inference of the language model. To solve the problems of at least one of the above aspects, the disclosure embodiment provides a language model inference optimization method. In the disclosure embodiment, when the preset language model infers the input long text information, the overall service framework of the language model includes but is not limited to an application business layer, an inference service processor, and an inference engine, etc. Among them, the application business layer includes various application businesses, for example, realizing application businesses such as dialogue chat, intelligent assistant, and creation; the inference service processor is responsible for processing multiple concurrent requests, for example, initializing requests, calling inference engines, and sampling inference results; and the inference engine can be classified according to the characteristics of inference, for example, full inference, incremental inference, and hybrid inference. FIG. 1 is a layered abstraction schematic diagram of a language model inference framework provided by the disclosure embodiment. As shown in FIG. 1, three abstraction layers are included in this example, the first abstraction layer (the uppermost layer) refers to model abstraction, which includes but is not limited to full reasoning process of execution context, incremental reasoning process of execution autoregression, and mixed reasoning including full reasoning and incremental reasoning; the second abstraction layer (the middle layer) refers to model layer abstraction for performing operations of all the same layers, which mainly includes but is not limited to Self Attention (self-attention layer) and FeedForward (feedforward layer); the third abstraction layer (the bottom layer) refers to operator abstraction, which includes but is not limited to FMHA (multi-head self-attention), linear transformation (Linear), addition and normalization (Add&Norm), and softmax, etc. Firstly, the full reasoning, mixed reasoning, and incremental reasoning involved in the embodiments of the present disclosure are introduced: Full reasoning refers to that when processing input text data information, the language model will perform a comprehensive analysis and processing through the entire data set stored by the model to generate a complete output or result, wherein the language model will consider all information and context related to the input text data information in the processing process to ensure the accuracy and comprehensiveness of the output. As an example of full reasoning, when a user inputs a complex question to a chat robot based on a language model, the chat robot can perform full reasoning in the following way: first, the chat robot analyzes the grammatical structure of the question and understands the content the user wants to ask; then, the chat robot retrieves its knowledge base to find information and answers related to the question; finally, the chat robot integrates these information to generate a complete, coherent and accurate answer and presents it to the user. Mixed reasoning refers to a strategy of flexibly selecting and combining full reasoning and incremental reasoning according to different parts or stages of a complex problem. Incremental reasoning is suitable for scenarios where the input data changes little or the reasoning task needs to be repeatedly executed. When the input data only changes locally, only the changed part is re-reasoned, rather than repeatedly reasoning the entire data set or input. Incremental reasoning can significantly reduce redundant calculations, reduce computational load and power consumption, and thus improve the execution speed and efficiency of reasoning. As an example of incremental reasoning, in a video target detection task, there is a lot of similar information between consecutive frames of a video. At this time, only the areas with large inter-frame differences can be incrementally reasoned, and in this way, repeated processing of the entire frame image can be reduced. Please refer to FIG. 2, which is a flowchart of a reasoning optimization method of a language model provided by the embodiments of the present disclosure. As shown in FIG. 2, the inference optimization method provided by the embodiments of the present disclosure specifically includes the following steps. In step S100, in response to obtaining at least one request information requesting to use a preset language model to infer input information, the first request information is divided into multiple pieces of request information according to a preset length. In the embodiments of the present disclosure, when inputting information to a preset language model and requesting to infer the input information by the language model, at least one request information can be obtained. Here, the input information includes but is not limited to long text information, short text information, context information, metadata, feature representation information, etc. The at least one request information refers to one or more information determined based on the information input to the language model, and the request information is used to instruct the language model to infer the input information. In addition, the first request information is the request information that needs to be split among the at least one request information. The preset length can be a division length specifically set according to the performance of the preset language model and the input long text information. In this step, in the specific implementation, first, at least one request information is obtained from an external request to infer the input information input to the preset language model; then, in response to the at least one request information being obtained, a preset length corresponding to the external request is obtained; finally, the first request information in the at least one request information is divided into multiple pieces of request information according to the preset length. For example, assuming that the length of the first request information is 4000 and the preset length is 2000, at this time, the first request information with a length of 4000 can be divided into 2 pieces of request information according to the preset length of 2000, and the length of each piece of request information is 2000. In step S200, the language model is used to perform full-inference and mixed-inference on the multiple pieces of request information and the second request information to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained during the full-inference or mixed-inference on the multiple pieces of request information. Since the multiple pieces of request information and multiple batches of request information need to be inferred in the embodiments of the present disclosure, the processing mode of the multiple pieces of request information and the second request information can include two processing modes of batch parallel processing and combined processing. One way is to improve the processing efficiency of the language model on the input information, and the information can be processed by the batch parallel processing mode of the multiple pieces of request information and the second request information. Another way is to reduce the resource occupation effect on the basis of improving the processing efficiency of the language model on the input information, and the information can be processed by the combined processing mode of the multiple pieces of request information and the second request information. Optionally, considering that there are multiple division cases for the first request information divided into multiple pieces of request information, one case is that the multiple pieces of request information are divided into N, and another case is that the multiple pieces of request information are divided into N+1. N is a natural number greater than 1. Based on this, for each of the two processing modes of the above batch parallel processing or combined processing, one case of the multiple pieces of request information being divided into N and another case of the multiple pieces of request information being divided into N+1 are included. The following will be described in specific examples for the two cases under the two processing modes in step S200. (1) The processing mode is the batch parallel processing mode: In the first case of the batch parallel processing mode, for example, assuming that the multiple pieces of request information are divided into 2, the language model is used to perform full reasoning and mixed reasoning on the multiple pieces of request information and the second request information to obtain multiple first reasoning results, which can specifically include: In step S211, the language model is used to perform full reasoning or full reasoning and mixed reasoning on the first request segment information in the multiple pieces of request information to obtain historical result information output by the full reasoning or full reasoning and mixed reasoning. In the embodiments of the present disclosure, according to whether the request information contains historical information and the input length of the request information, the reasoning mode actually performed by the language model when reasoning the request information is determined. Specifically, when the request information does not contain historical information, full reasoning is performed on the request information; when the request information contains historical information and the input length of the request information is 1, incremental reasoning is performed on the request information; and when the request information contains historical information and the input length of the request information is not 1, mixed reasoning is performed on the request information. In this step, based on whether the first request segment information in the divided multiple pieces of request information contains historical result information and the input length of the first request segment information, full reasoning or full reasoning and mixed reasoning is determined for the first request segment; and then, historical result information output by the full reasoning or full reasoning and mixed reasoning is obtained. For example, assuming that the A request information does not contain historical information and the input length is 2000, full reasoning is performed on the A request information. For another example, assuming that the B request information and the C request information are obtained, the B request information contains historical information and the input length is 1, incremental reasoning is performed on the B request information; and the C request information contains historical information and the input length is not 1, mixed reasoning is performed on the C request information. Here, the first request segment information is not the first segment in the physical sense, but the information that needs to be reasoned before the first request information is segmented and the second request information is received. Specifically, during the execution of full reasoning and full reasoning with historical information, the fragment information of the full reasoning and the fragment information of the full reasoning with historical information (which can be called hybrid reasoning) are both determined as the first requested fragment information. As an example of performing full inference on the first request fragment information, the statement "model.run_gpt(input_tokens=split_1_2000)" is used to perform inference on the first request fragment information among multiple request fragment information, where the length of the first request information (tokens) is 2000. Optionally, the statement "input_lengths=

[2000] The full inference is performed on the first request fragment information, where the first request fragment information uses the resource with index 0 and the length of the history result information is 0. Figure 3 is a segmented schematic diagram of full-scale reasoning of a language model provided in an embodiment of this disclosure. As shown in Figure 3, when performing full inference on the request information using the language model, assuming the first request information with an input length of 8000 is divided into four request information messages with a length of 2000, and by adding full inference with historical information, performing one full inference with an input length of 8000 becomes performing four full inferences with historical information; where the length of the historical information in the first full inference is 0. In this way, since the size of the intermediate GPU memory is linearly related to the length of the processed text token, the segmented processing of full inference can optimize the peak GPU memory usage. In the example shown in Figure 3, full inference by segmenting information reduces the system's intermediate video memory usage by three-quarters. Compared to the common method of optimizing video memory by offloading host devices, full inference by segmenting avoids the overhead of data copying between devices and the problem of complex heterogeneous scheduling. Step S212: In response to receiving the second request information that uses a preset language model to reason about the input information, the second request fragment information and the second request information in the multiple request information are treated as the same batch of requests to obtain the first request batch information. In the embodiments of the present disclosure, in the process of full reasoning or full reasoning and mixed reasoning on the first request segment information in the first request information, the second request information is obtained, which requests to use the preset language model to reason on the input information. At this time, the second request segment information in the first request information and the second request information can be merged into the same batch for parallel processing. For example, it is assumed that in the process of full reasoning or full reasoning and mixed reasoning on D1 request segment information in D request information, E request information is obtained. At this time, D2 request segment information in D request information and E request information need to be merged into the same batch request for parallel processing of the same batch request. Here, it should be understood that when at least one request information is obtained, the first request information needs to be divided into multiple segment request information immediately, and the second request segment information in the multiple segment request information and the second request information can be optionally merged into the same request batch information for parallel processing of the request batch information. Among them, the above at least one request information is not necessarily obtained at one time, but can be the second request information obtained in the process of processing the first request information. Step S213, based on the historical result information and the first request batch information, using the language model to perform mixed reasoning to obtain a plurality of first reasoning results. In this step, in the specific implementation, the historical result information and the first request batch information are taken as the input of the language model. At this time, since the input contains the historical result information and the input length is not 1, the language model can be used to perform mixed reasoning on the input to obtain a plurality of first reasoning results. Optionally, in addition to the above step of using the language model to perform full reasoning and mixed reasoning on the multiple segment request information and the second request information to obtain a plurality of first reasoning results, steps S214 to S215 are further included. Specifically, steps S214 to S215 are used to illustrate the reasoning method after the obtained second request information is reasoned and the third request information is further obtained. Specifically, it includes: Step S214, in response to obtaining the third request information which requests to use the preset language model to reason on the input information, taking the plurality of first reasoning results and the third request information as the same batch request to obtain the second request batch information. In the embodiments of the present disclosure, after the second request information is obtained in the process of reasoning the first request segment information in the first request information using the language model, the third request information is further obtained. The first reasoning result and the third request information can be taken as the same batch request to obtain the second request batch information. For example, assuming that the D2 request segment information in the D request information is merged with the E request information into first request batch information, the historical result information inferred from the D1 request segment information in the D request information and the first request batch information are input into the language model to perform mixed inference to obtain a plurality of first inference results S1, at this time, the F request information is obtained, and the first inference result S1 and the F request information are taken as the same batch request to obtain second request batch information. Step S215, based on the second request batch information, mixed inference is performed by using the language model to obtain a plurality of first inference results. In this step, in the specific implementation, the second request batch information is taken as the input of the language model, at this time, since the input contains the historical result information and the input length is not 1, the mixed inference can be performed on the input by using the language model to obtain a plurality of first inference results corresponding to the processing of the first request information, the second request information and the third request information. As an example of the first case in the batch parallel processing mode, first, the D request information obtained first is segmented to obtain D1 request segment information and D2 request segment information; then, the D1 request segment information is inferred to obtain historical result information Z, and in the process, the E request information is obtained; thereafter, the D2 request segment information needs to be merged with the E request information into first request batch information, and the historical result information Z and the first request batch information are input into the language model to perform mixed inference to obtain a plurality of first inference results S1; at this time, the F request information is obtained again, the first inference result S1 and the F request information are taken as the same batch request to obtain second request batch information; finally, the second request batch information is input into the language model to perform mixed inference to obtain a plurality of first inference results S2. The second case in the batch parallel processing mode, for example, assuming that the multi-segment request information is divided into 3, the language model is used to perform full inference and mixed inference on the multi-segment request information and the second request information to obtain a plurality of first inference results, which can specifically include: Step S221, full inference or full inference and mixed inference are performed on the first request segment information in the multi-segment request information by using the language model to obtain first historical result information output by the full inference or the full inference and mixed inference. Step S222, in response to obtaining the second request information in which the input information is inferred by using the preset language model, the second request segment information in the multi-segment request information and the second request information are taken as the same batch request to obtain first request batch information; Step S223, based on the first historical result information and the first request batch information, mixed inference is performed by using the language model to obtain at least one first inference result and second historical result information; The description of steps S221 to S223 can refer to the description of steps S211 to S213, and achieve the same technical effects, which will not be described here. Step S224, in response to obtaining the third request information for reasoning the input information using the preset language model, taking the first reasoning result, the third request segment information in the multi-segment request information, and the third request information as the same batch request to obtain third request batch information. In the embodiments of the present disclosure, since the third request segment information is included in the multi-segment request information, for the divided third request segment information, the third request segment information, the first reasoning result, and the third request information are taken as the same batch request to obtain the third request batch information for mixed reasoning of the third request batch information. Step S225, based on the second historical result information and the third request batch information, performing mixed reasoning using the language model to obtain a plurality of first reasoning results. In this step, the second historical result information and the third request batch information are taken as the input of the language model, and the input is mixed reasoned using the language model to obtain a plurality of first reasoning results. As an example of the second case in the batch parallel processing mode, first, the D request information obtained first is segmented to obtain D1 request segment information, D2 request segment information, and D3 request segment information; then, the D1 request segment information is reasoned to obtain the first historical result information Z1, and in the process, the E request information is obtained, and the D2 request segment information and the E request information need to be merged into the first request batch information; then, the first historical result information Z1 and the first request batch information are input into the language model to perform mixed reasoning to obtain at least one first reasoning result S1 and the second historical result information Z2; then, the F request information is obtained, the first reasoning result S1, the D3 request segment information, and the F request information are taken as the same batch request to obtain the third request batch information; finally, the third request batch information and the second historical result information Z2 are input into the language model to perform mixed reasoning to obtain a plurality of first reasoning results S2. (2) The processing mode is a merging processing mode: The first case in the merging processing mode, for example, assuming that the multi-segment request information is divided into 2, the multi-segment request information and the second request information are reasoned using the language model to obtain a plurality of first reasoning results, which can include: Step S230, the multi-segment request information and the second request information are merged in batches, and the multi-segment request information and the second request information are zero-redundancy fully reasoned and mixed reasoned using the language model to obtain a plurality of first reasoning results. In an embodiment of the present disclosure, in specific implementation, step S230 can include: Step S231, using the language model to perform full reasoning on the first request segment information in the multi-segment request information, or full reasoning and mixed reasoning, to obtain historical result information output by the full reasoning or the full reasoning and mixed reasoning. The description of S231 can refer to the description of S211, and the same technical effects can be achieved, which will not be described here. Step S232, in response to obtaining the second request information requesting to use the preset language model to perform reasoning on the input information, merging the second request segment information in the multi-segment request information and the second request information to obtain first merged information. In the embodiments of the present disclosure, in the process of reasoning the first request segment information, the second request information is also obtained, and the second request segment information and the second request information can be merged and processed to reduce the resource occupation of language model reasoning in a zero redundancy reasoning manner. In an example of the present disclosure, the second request information is obtained through the statement "request_2=GetLastRequest()", and the second request segment information and the second request information are merged through the statement "input_tokens=Merge(split_2_2000,request_2)" to obtain the first merged information. For example, it is assumed that in the process of reasoning the G1 request segment information in the G request information, the H request information is obtained, and at this time, the G1 request segment information and the H request information need to be merged as the first merged information for zero redundancy reasoning on the first merged information. Step S233, based on the historical result information and the first merged information, using the language model to perform zero redundancy mixed reasoning to obtain a plurality of first reasoning results. In the embodiments of the present disclosure, the language model will run reasoning in N layers in the reasoning process, and the N layers include a plurality of hidden layers. The plurality of hidden layers is a multi-level abstraction of input features, so as to linearly divide the data of the input layer into different types of data. N is a natural number greater than 1. In an embodiment of the present disclosure, in specific implementation, the plurality of first reasoning results in step S233 are obtained by performing zero redundancy mixed reasoning on the vectors included in the hidden layer, specifically including: S2331, respectively performing token embedding and layer normalization processing on the first merged information and the historical result information to obtain first merged input information. In this step, in specific implementation, firstly, the first merged information and the historical result information are subjected to model token embedding; then, data distribution characteristics included in the model token embedding information are calculated, and the data distribution characteristics are adjusted to a specified range for normalization processing; finally, a preset scale factor and a shift factor are introduced into the normalized data to obtain the first merged input information. S2332, the first merged input information is subjected to vector linear transformation by using the self-attention layer of the language model to obtain the first vector input information. In this step, the first merged input information is transformed by using the self-attention layer of the language model through three different linear transformation modes. Specifically, the first merged input information is multiplied by three different weight matrices to obtain Q, K, and V vectors, and the three vectors are determined as the obtained first vector input information. The three different weight matrices can correspond to trainable parameter matrices WQ, WK, and WV, respectively. S2333, based on the first vector input information, the self-attention layer is used to perform zero-redundancy full-quantity operation to obtain the first output result. In an embodiment of the present disclosure, in specific implementation, step S2333 can include: Step S23331, the first vector input information is dispersedly input into multiple independent operators in the self-attention layer according to a preset division strategy. In this step, based on the number of information in the first vector input information, the first vector input information is dispersedly input into multiple independent operators in the self-attention layer in sequence. For example, assuming that the first vector input information includes information with lengths of 1000, 1, 1000, and 1, the first vector input information is input into four independent operators respectively for subsequent operator operation. Step S23332, the multiple independent operators are used to respectively perform full-quantity operation on the dispersedly input first vector input information to obtain multiple first operation results, and the multiple first operation results are merged and written into the preset memory corresponding to the language model. In the embodiment of the present disclosure, the language model generates multiple first operation results at a time after processing the entire input sequence, and merges and writes the multiple first operation results into the preset memory corresponding to the language model. Step S23333, the multiple first operation results merged and written into the preset memory are split to obtain the first output result. In this step, according to the request processed input token information, the multiple first operation results are split in the preset memory to obtain the first output result. Step S2334, full-amount feedforward processing of the first output result is performed by using the feedforward layer of the language model to obtain a first intermediate result. In the embodiments of the present disclosure, based on the ability of the feedforward layer in the language model to process data, full-amount feedforward processing of the first output result is performed to obtain a first intermediate result, so as to perform operation on the first intermediate result by other processing included in the language model. Step S2335, based on the N first merged input information, the first vector input information, the first output result and the first intermediate result, a plurality of first inference results are obtained, and N is a natural number greater than 1. In the embodiments of the present disclosure, the hidden layer vector includes but is not limited to the first merged input information, the first vector input information, the first output result and the first intermediate result. Here, it should be understood that after full-amount feedforward processing of the first output result by the feedforward layer, no inference result is obtained, so after all layers of the model are run, the language model obtains a plurality of first inference results by performing other processing on the hidden layer vector. FIG. 4 is an inference schematic diagram of a zero-redundancy processing of a language model according to an embodiment of the present disclosure. As an example of zero-redundancy inference, as shown in FIG. 4, it is assumed that the first merged information includes four input information, the lengths of which are 1000, 1, 1000 and 1 in turn, and the shape size of the first merged information input is 2002; after embedding and layer normalization processing of the first merged information are performed respectively, since the above processing is for text token marking of each input information, the obtained intermediate video memory shape size is [2002, H] (H represents the width of the hidden layer). Optionally, in the inference process of the self-attention layer, since the attn operator for calculating the attention score is for each input information in the first merged information, after vector linear transformation (QKVlinear) of the first merged input information is performed, the first vector input information is dispersed into four independent operators (attn1-4) for execution, so as to internally split and write the obtained plurality of first operation results into the same block of preset video memory; then, full-amount feedforward processing of the first output result is performed by using the feedforward layer of the language model to obtain a first intermediate result; finally, based on the first merged input information, the first vector input information, the first output result and the first intermediate result, a plurality of first inference results are obtained by processing of other layers of the model. Through the above zero-redundancy processing at the input end of the language model, the separated merging operation in the operator operation part is realized, and the occupation of the temporary video memory by the language model during inference is reduced. Optionally, in addition to the full reasoning and mixed reasoning of the multi-segment request information and the second request information using the language model to obtain the plurality of first reasoning results, steps S231 to S233 are further included. Specifically, steps S234 to S235 are used to illustrate the reasoning method after the second request information is obtained. Specifically, steps S234 to S235 include: Step S234, in response to obtaining the third request information requesting to use the preset language model to reason the input information, the first reasoning result and the third request information are merged to obtain second merged information. In the embodiments of the present disclosure, for the obtained third request information, the first reasoning result and the third request information can be merged to obtain the second merged information, and the zero-redundancy mixed reasoning is performed on the second merged information. Step S235, based on the second merged information, zero-redundancy mixed reasoning is performed using the language model to obtain a plurality of first reasoning results. The description of step S235 can refer to the description of steps S2331 to S2335, and the same technical effects can be achieved, and thus will not be described here. As an example of the first case in the merging processing mode, first, the G request information obtained first is segmented to obtain G1 request segment information and G2 request segment information; then, the G1 request segment information is reasoned to obtain historical result information Y, and the H request information is obtained in the process; thereafter, the G2 request segment information and the H request information need to be merged into first merged information, and the historical result information Y and the first merged information are input into the language model to perform zero-redundancy mixed reasoning to obtain a plurality of first reasoning results X1; at this time, the I request information is obtained, the first reasoning result X1 and the I request information are merged to obtain second merged information; finally, the second merged information is input into the language model to perform zero-redundancy mixed reasoning to obtain a plurality of first reasoning results X2. The second case in the merging processing mode, for example, assuming that the multi-segment request information is divided into 3, the full reasoning and mixed reasoning of the multi-segment request information and the second request information using the language model to obtain the plurality of first reasoning results, specifically can include: Step S241, using the language model to perform full reasoning on the first request segment information in the multi-segment request information, or full reasoning and mixed reasoning, to obtain the first historical result information output by the full reasoning, or the full reasoning and mixed reasoning. Step S242, in response to obtaining the second request information requesting to use the preset language model to reason the input information, the second request segment information in the multi-segment request information and the second request information are merged to obtain first merged information. Step S243, based on the first historical result information and the first merged information, zero-redundancy hybrid reasoning is performed using the language model to obtain at least one first reasoning result and second historical result information. The description of steps S241 to S243 can refer to the description of steps S231 to S233, and achieve the same technical effects, which will not be described here. Step S244, in response to obtaining third request information requesting to use the preset language model to reason the input information, the first reasoning result, the third request segment information in the multiple request information, and the third request information are merged to obtain third merged information. In the embodiments of the present disclosure, for the obtained third request information, the first reasoning result, the third request segment information, and the third request information can be merged to obtain third merged information, so as to perform zero-redundancy hybrid reasoning on the third merged information and the second historical result information. Step S245, based on the second historical result information and the third merged information, zero-redundancy hybrid reasoning is performed using the language model to obtain multiple first reasoning results. The description of step S245 can refer to the description of steps S2331 to S2335, and achieve the same technical effects, which will not be described here. As an example of the second case in the batch parallel processing mode, first, the G request information obtained first is segmented to obtain G1 request segment information, G2 request segment information, and G3 request segment information; then, the G1 request segment information is reasoned to obtain first historical result information Y1, and in the process, the H request information is obtained, and the G2 request segment information and the H request information need to be merged into first merged information; then, the first historical result information Y1 and the first merged information are input into the language model to perform zero-redundancy hybrid reasoning to obtain at least one first reasoning result X1 and second historical result information Y2; then, the I request information is obtained, and the first reasoning result X1, the D3 request segment information, and the I request information are merged into third merged information; finally, the second historical result information Y2 and the third merged information are input into the language model to perform zero-redundancy hybrid reasoning to obtain multiple first reasoning results X2. Referring back to FIG. 2, step 300, using the language model to perform incremental reasoning on the multiple first reasoning results to obtain a reasoning result output by the language model reasoning the input information. In this step, when the multiple request information and the second request information are processed by the batch parallel processing mode, the multiple first reasoning results are taken as the results of the same batch to obtain a first request batch result, and the language model is used to perform incremental reasoning on the first request batch result to obtain a reasoning result output by the language model reasoning the input information. When the multi-segment request information and the second request information are processed by merging, the step can include: merging the plurality of first inference results to obtain merged information, and performing zero-redundancy incremental inference on the merged information by using the language model to obtain a merged inference result output by the language model when the input information is inferred. As an example of incremental inference on merged information, the merged information is incrementally inferred by the statement "logits = model.run_gpt(input_tokens = input_tokens, input_lengths = [1, 1, 1], resource_indices = [0, 1, 2], infer_categories = [1, 1, 1], history_lengths = [4001, 2001, 1000], is_first_finished_list = [1, 1, 1])". Wherein, input_tokens represents a list of input text tokens (Token), with a shape of [total_length], which needs to be removed padding when used; input_lengths represents a list of input lengths, with a shape of [max_batch_size]; resource_indices represents a list of computing resource positions, with a shape of [max_batch_size], and the value range is [0, max_batch_size-1]; infer_categories represents a list of input inference categories, with a shape of [max_batch_size], and the parameter 0 represents full inference, and the parameter 1 represents incremental inference; history_lengths represents the number of tokens that have been inferred, with a shape of [max_batch_size]; is_first_finished_list represents whether the input has been completed in the current full inference, i.e., whether it is the last segment of input, with a shape of [max_batch_size], and the parameter -1 represents an invalid value (used for incremental inference); the parameter 0 represents that the inference has not ended; and the parameter 1 represents that the inference has ended. Here, incremental inference is a full inference with historical information, and the input length is 1, i.e., full inference with historical information naturally supports incremental inference; since the length of the input batch process is 1, the difficulty of inference is in the IO bandwidth, therefore, the short input with length 1 is merged with the long input for inference, and the inference delay caused by the short input can be ignored. In this way, through the mixed inference process of segmented batch processing, continuous requests can be efficiently processed, the advantages of batch processing are fully utilized while ensuring real-time performance, the batch processing size of incremental inference is dynamically increased, and the system can effectively utilize computing resources and improve inference throughput and efficiency. The inference optimization method of the language model provided in the embodiments of the present disclosure implements smaller memory occupation of the language model when the language model performs inference on long text information, improves the throughput of the language model inference service in actual application, and optimizes the memory occupation and model inference performance in the inference process through zero redundancy processing for full-inference and mixed-inference, thereby improving the efficiency of the language model when performing computation inference. Please refer to FIG. 5, which is a structural schematic diagram of an inference optimization device of a language model provided in the embodiments of the present disclosure. As shown in FIG. 5, the inference optimization device 500 includes: The segmented processing module 510 is configured to divide the first request information into multiple pieces of request information according to a preset length in response to obtaining at least one request information requesting to perform inference on input information by using a preset language model. The mixed inference module 520 is configured to perform full-inference and mixed-inference on the multiple pieces of request information and the second request information by using the language model to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained in the process of performing full-inference or mixed-inference on the multiple pieces of request information. The incremental inference module 530 is configured to perform incremental inference on the multiple first inference results by using the language model to obtain an inference result output by the language model when performing inference on the input information. Optionally, when the mixed inference module 520 is used to perform full-inference and mixed-inference on the multiple pieces of request information and the second request information by using the language model to obtain the multiple first inference results, the mixed inference module 520 is configured to: perform full-inference or full-inference and mixed-inference on the first request piece information in the multiple pieces of request information by using the language model to obtain historical result information output by the full-inference or the full-inference and mixed-inference; in response to obtaining the second request information requesting to perform inference on the input information by using the preset language model, take the second request piece information in the multiple pieces of request information and the second request information as a same batch request to obtain first request batch information; perform mixed-inference on the historical result information and the first request batch information by using the language model to obtain the multiple first inference results. Optionally, when the mixed inference module 520 is configured to perform full inference and mixed inference on the multi-segment request information and the second request information using the language model to obtain the plurality of first inference results, the mixed inference module 520 is further configured to: in response to obtaining the third request information requesting to perform inference on the input information using the preset language model, taking the plurality of first inference results and the third request information as the same batch of requests to obtain second request batch information; based on the second request batch information, performing mixed inference using the language model to obtain the plurality of first inference results. Optionally, when the mixed inference module 520 is configured to perform full inference and mixed inference on the multi-segment request information and the second request information using the language model to obtain the plurality of first inference results, the mixed inference module 520 is further configured to: performing full inference, or full inference and mixed inference on the first request segment information in the multi-segment request information using the language model to obtain first historical result information output by the full inference, or the full inference and mixed inference; in response to obtaining the second request information requesting to perform inference on the input information using the preset language model, taking the second request segment information in the multi-segment request information and the second request information as the same batch of requests to obtain first request batch information; based on the first historical result information and the first request batch information, performing mixed inference using the language model to obtain at least one first inference result and second historical result information; in response to obtaining the third request information requesting to perform inference on the input information using the preset language model, taking the first inference result, the third request segment information in the multi-segment request information, and the third request information as the same batch of requests to obtain third request batch information; based on the second historical result information and the third request batch information, performing mixed inference using the language model to obtain the plurality of first inference results. Optionally, when the mixed inference module 520 is configured to perform full inference and mixed inference on the multi-segment request information and the second request information using the language model to obtain the plurality of first inference results, the mixed inference module 520 is further configured to: performing batch merging on the multi-segment request information and the second request information, and performing zero-redundancy full inference and mixed inference on the multi-segment request information and the second request information using the language model to obtain the plurality of first inference results; Optionally, when the mixed inference module 520 is configured to perform batch merging on the multi-segment request information and the second request information, and perform zero-redundancy full inference and mixed inference on the multi-segment request information and the second request information using the language model to obtain the plurality of first inference results, the mixed inference module 520 is further configured to: The first request segment information in the multi-segment request information is inferred in full or in full and mixed by using the language model, to obtain historical result information output by full inference or full and mixed inference; In response to obtaining second request information requesting to infer the input information by using the preset language model, the second request segment information in the multi-segment request information and the second request information are merged to obtain first merged information; Based on the historical result information and the first merged information, zero-redundancy mixed inference is performed by using the language model to obtain a plurality of first inference results. Optionally, when the mixed inference module 520 is configured to perform zero-redundancy mixed inference by using the language model based on the historical result information and the first merged information to obtain a plurality of first inference results, the mixed inference module 520 is further configured to: The first merged information and the historical result information are respectively subjected to mark embedding and layer normalization processing to obtain first merged input information; The first merged input information is subjected to vector linear transformation by using a self-attention layer of the language model to obtain first vector input information; Based on the first vector input information, zero-redundancy full calculation is performed by using the self-attention layer to obtain a first output result; The first output result is subjected to full feedforward processing by using a feedforward layer of the language model to obtain a first intermediate result; Based on N first merged input information, the first vector input information, the first output result and the first intermediate result, a plurality of first inference results are obtained, N being a natural number greater than 1. Optionally, when the mixed inference module 520 is configured to perform zero-redundancy full calculation by using the self-attention layer based on the first vector input information to obtain the first output result, the mixed inference module 520 is further configured to: The first vector input information is dispersedly input into a plurality of independent operators in the self-attention layer according to a preset division strategy; The dispersedly input first vector input information is respectively subjected to full calculation by using the plurality of independent operators to obtain a plurality of first calculation results, and the plurality of first calculation results are merged and written into a preset video memory corresponding to the language model; The plurality of first calculation results merged and written into the preset video memory are split to obtain the first output result. Optionally, when the mixed inference module 520 is configured to perform zero-redundancy full inference and mixed inference on the multi-segment request information and the second request information by using the language model based on the batch-merging of the multi-segment request information and the second request information to obtain a plurality of first inference results, the mixed inference module 520 is further configured to: In response to obtaining the third request information requesting to infer the input information by using the preset language model, the first inference result and the third request information are merged to obtain second merged information; Based on the second merged information, zero-redundancy hybrid inference is performed by using the language model to obtain a plurality of first inference results. Optionally, when the hybrid inference module 520 is configured to perform batch merging on the plurality of request information and the second request information, and perform zero-redundancy full-inference and hybrid inference on the plurality of request information and the second request information by using the language model to obtain a plurality of first inference results, the hybrid inference module 520 is further configured to: perform full-inference or full-inference and hybrid inference on the first request segment information in the plurality of request information by using the language model to obtain first historical result information output by the full-inference or the full-inference and hybrid inference; In response to obtaining the second request information requesting to infer the input information by using the preset language model, the second request segment information in the plurality of request information and the second request information are merged to obtain first merged information; Based on the first historical result information and the first merged information, zero-redundancy hybrid inference is performed by using the language model to obtain at least one first inference result and second historical result information; In response to obtaining the third request information requesting to infer the input information by using the preset language model, the first inference result, the third request segment information in the plurality of request information and the third request information are merged to obtain third merged information; Based on the second historical result information and the third merged information, zero-redundancy hybrid inference is performed by using the language model to obtain a plurality of first inference results. The inference optimization apparatus of the language model provided by the embodiments of the present disclosure performs segmented full-inference and hybrid inference on at least one request information based on the idea of continuous batch processing, realizes smaller occupation of the video memory when the language model infers long text information, improves the throughput of the language model inference service in actual application, and optimizes the video memory occupation and model inference performance in the inference process through zero-redundancy processing for full-inference and hybrid inference, thereby improving the efficiency of the language model in computing inference. Please refer to FIG. 6, which is a structural schematic diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG. 6, the electronic device 600 includes a processor 610, a memory 620 and a bus 630. The memory 620 stores machine readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 and the memory 620 communicate through the bus 630. When the machine readable instructions are executed by the processor 610, the steps of the inference optimization method of the language model in the method embodiment shown in FIG. 2 can be performed. For details, refer to the method embodiment, which will not be repeated here. The disclosure also provides a computer readable storage medium, which stores a computer program. When the computer program is run by a processor, the steps of the inference optimization method of the language model in the method embodiment shown in FIG. 2 can be performed. For details, refer to the method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. In several embodiments provided by the present disclosure, it should be understood that the disclosed system, device and method can be implemented by other means. The device embodiments described above are only schematic, for example, the division of units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some communication interface, device or unit, which can be electrical, mechanical or other forms. The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. According to actual needs, some or all of the units can be selected to achieve the purpose of the embodiment. In addition, each functional unit in the various embodiments of the present disclosure can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a nonvolatile computer readable storage medium executable by a processor. Based on this understanding, the technical solutions of the present disclosure essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present disclosure. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes. Finally, it should be noted that the above embodiments are only specific implementations of the present disclosure, used to illustrate the technical solutions of the present disclosure, and are not limiting. The protection scope of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can make modifications or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present disclosure, or make equivalent replacements to some of the technical features. The modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be covered within the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be subject to the protection scope of the claims. Industrial applicability By adopting the above scheme, the occupation of the video memory by the language model when reasoning on the long text information input is reduced, the throughput of the language model reasoning service is improved in actual application, and the video memory occupation and model reasoning performance in the reasoning process are optimized through zero redundancy processing for full reasoning and mixed reasoning, and the efficiency of the language model when performing calculation reasoning is improved.

Claims

1. A method for inference optimization of a language model, characterized in that, The inference optimization method comprises: In response to obtaining at least one request information requesting to use a preset language model to perform inference on input information, dividing first request information into multiple pieces of request information according to a preset length; Performing full-inference and mixed-inference on the multiple pieces of request information and second request information by using the language model to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained during the process of performing full-inference or mixed-inference on the multiple pieces of request information; Performing incremental inference on the multiple first inference results by using the language model to obtain an inference result output by the language model when performing inference on the input information.

2. The method of claim 1, wherein, The performing full-inference and mixed-inference on the multiple pieces of request information and second request information by using the language model to obtain multiple first inference results comprises: Performing full-inference or full-inference and mixed-inference on first request piece information in the multiple pieces of request information by using the language model to obtain historical result information output by the full-inference or the full-inference and the mixed-inference; In response to obtaining the second request information requesting to use a preset language model to perform inference on input information, taking second request piece information in the multiple pieces of request information and the second request information as a same batch of requests to obtain first request batch information; Performing mixed-inference by using the language model based on the historical result information and the first request batch information to obtain multiple first inference results.

3. The method of claim 2, wherein, Before the performing full-inference or full-inference and mixed-inference on first request piece information in the multiple pieces of request information by using the language model to obtain historical result information output by the full-inference or the full-inference and the mixed-inference, the method comprises: Determining whether to perform full-inference or full-inference and mixed-inference on the first request piece according to whether the first request piece information contains historical result information and the input length of the first request piece information.

4. The method according to any one of claims 1 to 3, characterized in that, The performing full-inference and mixed-inference on the multiple pieces of request information and second request information by using the language model to obtain multiple first inference results further comprises: In response to obtaining third request information requesting to use a preset language model to perform inference on input information, taking the multiple first inference results and the third request information as a same batch of requests to obtain second request batch information; Performing mixed-inference by using the language model based on the second request batch information to obtain multiple first inference results.

5. The method according to any one of claims 1 to 4, characterized in that, The performing full-inference and mixed-inference on the multiple pieces of request information and second request information by using the language model to obtain multiple first inference results comprises: Performing full-inference or full-inference and mixed-inference on first request piece information in the multiple pieces of request information by using the language model to obtain first historical result information output by the full-inference or the full-inference and the mixed-inference; In response to the second request information requesting to use the preset language model to infer the input information, the second request segment information in the multiple request information and the second request information are combined to obtain first combined information; Based on the first historical result information and the first combined information, zero-redundancy mixed inference is performed on the first combined information and the historical result information by using the language model to obtain multiple first inference results. In response to the third request information requesting to use the preset language model to infer the input information, the first inference result, the third request segment information in the multiple request information and the third request information are combined to obtain third request batch information; Based on the second historical result information and the third request batch information, mixed inference is performed by using the language model to obtain multiple first inference results.

6. The method according to any one of claims 1 to 5, characterized in that, The full-inference and mixed-inference of the multiple request information and the second request information by using the language model to obtain multiple first inference results comprises: The multiple request information and the second request information are batch-merged, and zero-redundancy full-inference and mixed-inference of the multiple request information and the second request information by using the language model are performed to obtain multiple first inference results.

7. The method of claim 6, wherein, The batch-merging of the multiple request information and the second request information, and the zero-redundancy full-inference and mixed-inference of the multiple request information and the second request information by using the language model to obtain multiple first inference results comprises: Full-inference of the first request segment information in the multiple request information by using the language model, or full-inference and mixed-inference, to obtain historical result information output by the full-inference or the full-inference and the mixed-inference; In response to the second request information requesting to use the preset language model to infer the input information, the second request segment information in the multiple request information and the second request information are combined to obtain first combined information; Based on the historical result information and the first combined information, zero-redundancy mixed-inference is performed by using the language model to obtain multiple first inference results.

8. The method of claim 7, wherein, The zero-redundancy mixed-inference of the historical result information and the first combined information by using the language model to obtain multiple first inference results comprises: The first combined information and the historical result information are respectively marked and embedded and layer-normalized to obtain first combined input information; The first combined input information is linearly transformed by using a self-attention layer of the language model to obtain first vector input information; Based on the first vector input information, zero-redundancy full-inference is performed by using the self-attention layer to obtain first output result; Full-inference is performed on the first output result by using a feedforward layer of the language model to obtain first intermediate result; Based on N first combined input information, first vector input information, first output result and first intermediate result, multiple first inference results are obtained, and N is a natural number greater than 1.

9. The method of claim 8, wherein, The first merging input information is obtained by performing tag embedding and layer normalization processing on the first merging information and the historical result information, respectively, including: Model tagging and embedding are performed on the first merged information and the historical result information; The data distribution characteristics included in the information embedded by the model label are calculated, and the data distribution characteristics are adjusted to a specified range and normalized. A preset scaling factor and translation factor are introduced into the normalized data to obtain the first merged input information.

10. The method according to claim 8 or 9, characterized in that, The step of performing zero-redundancy full computation using the self-attention layer based on the first vector input information to obtain the first output result includes: The first vector input information is distributed into multiple independent operators in the self-attention layer according to a preset partitioning strategy; The multiple independent operators are used to perform full operations on the first vector input information that is input in a dispersed manner to obtain multiple first operation results, and the multiple first operation results are merged and written into the preset display memory corresponding to the language model; The multiple first operation results that are merged and written into the preset video memory are split to obtain the first output result.

11. The method of claim 10, wherein, The step of distributing the first vector input information into multiple independent operators in the self-attention layer according to a preset partitioning strategy includes: Based on the amount of information in the first vector input information, the first vector input information is sequentially and distributed into multiple independent operators in the self-attention layer.

12. The method according to any one of claims 6-11, characterized in that, The step of merging the multiple request information segments and the second request information in batches, and using the language model to perform zero-redundancy full-scale reasoning and hybrid reasoning on the multiple request information segments and the second request information to obtain multiple first reasoning results, also includes: In response to receiving a third request message requesting reasoning on the input information using a preset language model, the first reasoning result and the third request message are merged to obtain a second merged message. Based on the second merged information, zero-redundancy hybrid reasoning is performed using the language model to obtain multiple first reasoning results.

13. The method according to any one of claims 5-12, characterized in that, The process involves merging the multiple request segments and the second request segment in batches, and then using the language model to perform zero-redundancy full-scale reasoning and hybrid reasoning on the multiple request segments and the second request segment, resulting in multiple first reasoning results, including: The language model is used to perform full reasoning, or full reasoning and hybrid reasoning, on the first request fragment information in the multiple request information segments to obtain the first historical result information output by the full reasoning, or the full reasoning and hybrid reasoning. In response to receiving the second request information requesting reasoning of the input information using a preset language model, the second request fragment information and the second request information in the multiple request information segments are merged to obtain the first merged information; Based on the first historical result information and the first merged information, zero-redundancy hybrid reasoning is performed using the language model to obtain at least one first reasoning result and second historical result information. In response to obtaining a third request information requesting to infer the input information using the preset language model, the first inference result, the third request segment information in the multiple request information, and the third request information are merged to obtain third merged information; Based on the second historical result information and the third merged information, zero-redundancy mixed inference is performed using the language model to obtain multiple first inference results.

14. The method according to any one of claims 1 to 13, characterized in that, When the multiple request information and the second request information are processed in a batch parallel processing manner, the incrementally inferring the multiple first inference results using the language model to obtain an inference result output by the language model when inferring the input information includes: The multiple first inference results are taken as results of the same batch to obtain a first request batch result; The first request batch result is incrementally inferred using the language model to obtain an inference result output by the language model when inferring the input information.

15. The method according to any one of claims 1 to 14, characterized in that, When the multiple request information and the second request information are processed in a merging processing manner, the incrementally inferring the multiple first inference results using the language model to obtain an inference result output by the language model when inferring the input information includes: The multiple first inference results are merged to obtain merged information; Zero-redundancy incrementally inferring the merged information using the language model to obtain a merged inference result output by the language model when inferring the input information.

16. An apparatus for inference optimization of a language model, the apparatus comprising: The inference optimization apparatus includes: The segmentation processing module is configured to divide first request information into multiple request information according to a preset length in response to obtaining at least one request information requesting to infer input information using a preset language model; The mixed inference module is configured to perform full-inference and mixed-inference on the multiple request information and second request information using the language model to obtain multiple first inference results, wherein the second request information is at least one request information obtained simultaneously with the first request information or obtained during full-inference or mixed-inference on the multiple request information; The incrementally inferring module is configured to incrementally infer the multiple first inference results using the language model to obtain an inference result output by the language model when inferring the input information.

17. An electronic device, comprising: The apparatus includes: A processor, a storage medium, and a bus, the storage medium stores program instructions executable by the processor, when the electronic device is running, the processor and the storage medium communicate through the bus, and the machine readable instructions are executed by the processor to perform the inference optimization method of the language model according to any one of claims 1-15.

18. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to perform the inference optimization method of the language model according to any one of claims 1-15.

Images