Long text processing method and device based on large language model, equipment and storage medium

Abstract

The invention provides a long text processing method and device based on a large language model, equipment and a storage medium, and relates to the technical field of large language models and the like. The long text processing method comprises the following steps: in response to a target query request, dividing a long text into a plurality of document fragments; inputting the local feature representation of the (i + 1) th document segment and the local feature representation of the ith document segment into an encoder of the fine-tuned large language model to generate the local feature representation of the (i + 1) th document segment; updating the ith global feature representation according to the local feature representation of the (i + 1) th document fragment to generate an updated global feature representation; and inputting the query statement and the updated global feature representation into a decoder of the large language model to generate a target query result.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to the field of large language models, and specifically to a method, apparatus, device, and storage medium for processing long text based on a large language model. Background Art

[0002] The Generative Large Language Model (GLLM) has a large number of parameters and training data. By utilizing more parameters and larger datasets for training, its performance and sample efficiency on various downstream tasks are effectively improved.

[0003] Large language models face computational inefficiencies and excessive resource consumption when processing long text inputs. In particular, in tasks such as contextual learning, dialogue systems, and retrieval-enhanced generation, the computational complexity of the model increases quadratically with increasing input length, leading to slower inference speeds, increased memory usage, and increased deployment costs. Summary of the Invention

[0004] The present disclosure provides a long text processing method, apparatus, device and storage medium based on a large language model.

[0005] According to one aspect of the present disclosure, a long text processing method based on a large language model is provided, comprising:

[0006] In response to a target query request, a long text is divided into multiple document segments;

[0007] Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0008] Updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0009] The query statement and the updated global feature representation are input into the decoder of the large language model to generate a target query result.

[0010] According to another aspect of the present disclosure, a method for fine-tuning a large language model for long text processing is provided, comprising:

[0011] In response to a sample query request, dividing a long text into multiple document segments;

[0012] Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0013] Updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0014] Inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result;

[0015] The large language model is fine-tuned according to the predicted query results and the actual query results.

[0016] According to one aspect of the present disclosure, a long text processing apparatus based on a large language model is provided, comprising:

[0017] A long text segmentation module, configured to segment a long text into multiple document segments in response to a target query request;

[0018] A local feature module is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0019] A global updating module, configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0020] The query result module is used to input the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result.

[0021] According to another aspect of the present disclosure, a large language model fine-tuning device for long text processing is provided, comprising:

[0022] A long text segmentation module, configured to segment a long text into multiple document segments in response to a sample query request;

[0023] A local feature module is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0024] A global updating module, configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0025] A prediction result module, configured to input the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result;

[0026] A model fine-tuning module is used to fine-tune the large language model according to the predicted query results and the actual query results.

[0027] According to another aspect of the present disclosure, an electronic device is provided, the electronic device including:

[0028] at least one processor; and

[0029] a memory communicatively connected to the at least one processor; wherein,

[0030] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method provided by any embodiment of the present disclosure.

[0031] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method provided by any embodiment of the present disclosure.

[0032] According to another aspect of the present disclosure, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the computer program implements the method provided according to any embodiment of the present disclosure.

[0033] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1a This is a flowchart of a long text processing method based on a large language model provided according to an embodiment of the present disclosure;

[0035] Figure 1b This is a schematic diagram of the principle of a long text processing method based on a large language model provided according to an embodiment of the present disclosure;

[0036] Figure 2 is a flowchart of another long text processing method based on a large language model provided according to an embodiment of the present disclosure;

[0037] Figure 3 This is a flowchart of a method for fine-tuning a large language model for long text processing provided in accordance with an embodiment of the present disclosure;

[0038] Figure 4is a flowchart of another method for fine-tuning a large language model for long text processing provided in accordance with an embodiment of the present disclosure;

[0039] Figure 5 1 is a schematic structural diagram of a long text processing device based on a large language model according to an embodiment of the present disclosure;

[0040] Figure 6 1 is a schematic structural diagram of a large language model fine-tuning device for long text processing provided in accordance with an embodiment of the present disclosure;

[0041] Figure 7 It is a block diagram of an electronic device used to implement the long text processing method based on a large language model or the large language model fine-tuning method for long text processing according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0042] Related technologies for processing long texts mainly fall into the following categories: The first category, structural optimization methods, improves the model architecture to enhance its ability to model long sequences. For example, this can be achieved by improving attention mechanisms (such as sparse attention, sliding window attention) or expanding positional encoding to improve the model's ability to model long sequences. However, this first category of solutions requires redesigning the network architecture and training model parameters from scratch, making it difficult to directly migrate and adapt existing large-scale pre-trained language models, resulting in a significant increase in training costs.

[0043] The second category, simple segmented soft-hint compression methods, segment the original natural language input and then convert each into a latent vector representation in a low-dimensional continuous space, replacing traditional discrete text prompts as input for downstream tasks. However, the problem with this second category of methods is that they generally lack task awareness and fail to dynamically adjust information extraction strategies based on specific application scenarios, which can easily lead to the loss of key semantics or the retention of redundant information. Existing single-stage compression strategies do not fully utilize the previous compression results to guide subsequent information screening, making it difficult to maintain global semantic consistency when processing very long texts.

[0044] The disclosed embodiments are intended to solve the problems of low computational efficiency and excessive resource consumption faced by large language models when processing long text inputs. In particular, in tasks such as contextual learning, dialogue systems, and retrieval-enhanced generation, as the input length increases, the computational complexity of the model increases quadratically, leading to problems such as slower reasoning speed, high memory usage, and increased deployment costs. In particular, in Transformer-based models, the computational complexity of the self-attention mechanism increases quadratically with the input length, causing a sharp increase in hardware resource consumption when processing long texts, seriously affecting the feasibility of model deployment in edge devices or real-time scenarios. In addition, related technologies often ignore task characteristics and semantic coherence when compressing prompt information, resulting in the loss of key information and affecting model performance.

[0045] The disclosed embodiment designs a task-aware recursive latent space prompt compression method, which, without changing the original model structure, compresses the input context with soft prompts and maps it into a low-dimensional continuous space for efficient representation. This method can not only dynamically identify and retain key information according to specific tasks, but also introduces a recursive compression mechanism, which enables the model to refer to the compressed representation of the previous paragraph when processing the current paragraph, thereby realizing cross-segment information fusion and global semantic modeling. This technology can be widely used in different downstream tasks that require efficient processing of long text input, such as intelligent customer service, automatic summarization, multi-round dialogue systems, RAG (Retrieval-Augmented Generation) question-answering systems, etc.

[0046] Figure 1a This is a flow chart of a long text processing method based on a large language model according to an embodiment of the present disclosure. The method is applicable to the case where a long text input is processed by a large language model. The method can be executed by a long text processing device based on a large language model, which can be implemented in software and / or hardware and can be integrated into an electronic device. Figure 1a As shown, the long text processing method based on the large language model of this embodiment may include:

[0047] S101, in response to a target query request, dividing a long text into multiple document segments;

[0048] S102, inputting the local feature representation of the i+1th document segment and the i-th document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; wherein i is a positive integer;

[0049] S103, updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0050] S104: Input the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result.

[0051] During the model usage phase, the model receives a long text input and a query statement in response to a target query request. The long text is then divided into multiple document segments based on the number of tokens. Different document segments can have the same number of tokens. For example, an 8,000-word long text can be divided into 100 document segments.

[0052] In the disclosed embodiments, a recursive approach is used to recursively generate a local feature representation for the current document segment based on the local feature representation of the previous document segment. Furthermore, the local feature representation of the current document segment is used to fuse and update the previously generated global feature representation across document segments to generate a new global feature representation. The local feature representation can be referred to as segment memory, and the global feature representation can be referred to as global sequence memory or contextual sequence representation. The local feature representation is used to capture the local contextual semantics of the document segment, while the global feature representation implements long-range dependencies through iterative updates.

[0053] refer to Figure 1b , a memory buffer can be used to store global feature representations. For the i+1th document fragment, the i+1th document fragment and the pre-generated local feature representation of the i-th document fragment can be input into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document fragment; wherein, i is a positive integer, for example, i is 1-99. And, the i-th global feature representation stored in the memory buffer is updated using the local feature representation of the i+1th document fragment to obtain the i+1th global feature representation, and the i+1th global feature representation is used to replace the i-th global feature representation in the memory buffer. In other words, the memory buffer is used to store the latest global feature representation and the latest local feature representation. After processing each document fragment, the query statement and the global feature representation stored in the memory buffer can be input into the decoder of the large language model to generate the target query result.

[0054] By adopting a two-level memory mechanism including local feature representation and global feature representation, it is used for local context compression and information fusion across document fragments respectively; and through the recursive compression strategy and memory buffer design, it effectively solves the problems of context fragmentation, redundant information accumulation and excessive memory usage existing in traditional methods. Therefore, the embodiment of the present disclosure proposes a recursive latent space prompt compression method based on soft prompt compression, which is used to optimize the reasoning efficiency and semantic integrity of large language models when processing long text inputs. This method generates a learnable soft prompt vector as the input representation of subsequent tasks by mapping the original natural language input into a continuous latent space, thereby achieving efficient prompt compression and context modeling without changing the original model structure.

[0055] The technical solution provided by the disclosed embodiments divides a long text into multiple document segments; not only does it recursively generate local feature representations for the document segments, but it also uses these local feature representations to update the cached global feature representations. In other words, it implements a two-level processing mechanism, using local feature representations for fine-grained context compression to retain key segment information; combined with global feature representations to achieve cross-document segment information fusion and global semantic modeling. While maintaining semantic integrity, it significantly reduces the memory overhead and computational latency of long text reasoning, improving the processing efficiency of large language models for long texts.

[0056] In an optional embodiment, the i+1th document fragment and the local feature representation of the i-th document fragment are input into the encoder of a fine-tuned large language model to generate the local feature representation of the i+1th document fragment, including: inputting the i+1th document fragment and the compressed word into the embedding layer of the fine-tuned large language model to obtain the embedding vector of the i+1th document fragment; inputting the embedding vector of the i+1th document fragment and the local feature representation of the i-th document fragment into the encoder of the large language model to generate the local feature representation of the i+1th document fragment.

[0057] Exemplarily, the local feature representation of the i+1th document fragment can be generated by recursive compression. For the i+1th document fragment, the i+1th document fragment and the preset compressed word unit can be input into the embedding layer of the large language model to obtain the embedding vector of the i+1th document fragment; the embedding vector of the i+1th document fragment and the pre-generated local feature representation of the i-th document fragment are input into the encoder of the large language model to recursively generate the local feature representation of the i+1th document fragment. In other words, the local feature representation of the i+1th document fragment can be expressed by the following formula:

[0058]

[0059] in, and Seg_mem i+1 are the local feature representations of the i-th and i+1-th document fragments, respectively. is the i+1th document segment, COMP is a special compression word attached to the input sequence to guide the compression operation; M θ is a compression model function, which can be an encoder of a large language model with a parameter of θ.

[0060] By utilizing the embedding layer of a large language model to extract embedding vectors for document fragments, the quality of the embedding vectors can be improved. By introducing compressed words, the length of the embedding vectors can be reduced, thereby reducing the amount of computation while retaining key information. Furthermore, through recursive processing, the local feature representation of the current document fragment is generated based on the local feature representation of the previous document fragment and the embedding vector of the current document fragment. This recursive mechanism ensures that the large language model can gradually accumulate key semantic information when processing long texts without losing information due to local window limitations. It can also retain the semantic coherence between different document fragments, further improving the generation quality of local feature representations.

[0061] In an optional embodiment, the method further includes: inputting the first document fragment, the task description information in the target query request, the query statement and the compressed word into the embedding layer of the large language model to obtain the embedding vector of the first document fragment; inputting the embedding vector of the first document fragment into the encoder of the large language model to generate a local feature representation of the first document fragment.

[0062] Taking into account the differences in the focus of different tasks on key information in the context, the embodiment of the present disclosure introduces task description information and query statements as adjustment inputs in the process of generating local feature representations to dynamically adjust the information screening strategy. By introducing task description information such as "Please answer the following questions" or "Please summarize the following content" and the current query statement such as "Which is the most popular museum in a certain city?" into the local feature representation of the first document fragment, the first local feature representation integrates the task and query information; then, the local feature representations of other document fragments are determined recursively, so that each local feature representation contains task and query information.

[0063] refer to Figure 1b , the local feature representation of the first document fragment can be expressed by the following formula:

[0064]

[0065] Among them, Seg_mem1 is the local feature representation of the first document segment, x task , x q COMP is task description information, query statement, first document fragment and compressed word; M θ is a compression model function, which can be an encoder of a large language model with a parameter of θ.

[0066] Figure 2 This is a flowchart of another long text processing method based on a large language model provided according to an embodiment of the present disclosure. Figure 2 Based on the above embodiment, the long text processing method based on the large language model of this embodiment may include:

[0067] S201, in response to a target query request, dividing a long text into multiple document segments;

[0068] S202: Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0069] S203, inputting the local feature representation of the (i+1)th document segment and the (i)th global feature representation into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension;

[0070] S204: Input the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result.

[0071] For the i+1th document segment, the local feature representation of the i+1th document segment and the i-th global feature representation stored in the memory buffer can be input into the pre-trained feature editing network to obtain the i+1th global feature representation of a fixed dimension output by the feature editing network. The i+1th global feature representation is then used to replace the i-th global feature representation in the memory buffer. In other words, the i+1th global feature representation can be expressed by the following formula:

[0072] Seq_mem i+1 =E θ (Seq_mem i , Seg_mem i+1 );

[0073] Among them, Seq_mem i , Seq_mem i+1 They are the sequence stored in the memory buffer for the i-th global feature representation and the i+1-th global feature representation; Seg_mem i+1 is the i+1th local feature representation; E θ Edit the network for the feature.

[0074] By adopting a feature editing network, the attention weight between the i-th global feature representation and the new i+1-th local feature representation is calculated, and a new global feature representation of fixed dimension is output; that is, by introducing a lightweight feature editing network to control the size of the global feature representation, that is, to control the size of the memory buffer, the global feature representation recorded in the memory buffer is not affected by the increase in the number of document fragments, avoiding the quadratic increase in the computational complexity of the model with the increase in input length, resulting in slower inference speed, high memory usage, increased deployment costs and other problems.

[0075] In an optional embodiment, the fixed dimension is a first numerical multiple of the local feature representation; and the feature editing network includes a second numerical number of stacked deformer encoder blocks.

[0076] The first and second values ​​are both fixed; for example, if the length of the local feature representation is 128, both the first and second values ​​can be 3 or 4. The feature extraction network includes a small number of transformer encoder blocks, which can be a lightweight network composed of several transformer layers. The feature extraction network uses a self-attention mechanism to automatically identify and retain key information, discard redundant content, and output a new global feature representation of fixed dimension, thereby improving the accuracy of the global feature representation.

[0077] In an optional embodiment, the updating of the i-th global feature representation based on the local feature representation of the i+1-th document fragment also includes: if the length of the i-th global feature representation is less than the fixed dimension, splicing the local feature representation of the i+1-th document fragment and the i-th global feature representation to obtain the i+1-th global feature representation.

[0078] For the i+1th document fragment, if the length of the global feature representation stored in the memory buffer (i.e., the i-th global feature representation) is less than the fixed dimension, the local feature representation of the i+1th document fragment can be concatenated with the i-th global feature representation to obtain the i+1th global feature representation that is less than or equal to the fixed dimension, thereby further improving processing efficiency.

[0079] That is to say, in the initial stage, the global feature representation can be constructed by splicing:

[0080] Seq_mem i+1 =Concat(Seq_mem i ,Seg_mem i+1 );

[0081] Among them, Concat is a concatenation operation.

[0082] In an optional embodiment, the step of inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result includes: inputting the query statement into the embedding layer of the large language model to obtain a query embedding vector; and inputting the global feature representation of the last document fragment and the query embedding vector into the decoder of the large language model to generate a target query result.

[0083] When the i+1th document fragment is the last document fragment, the memory buffer stores the global feature representation of the last document fragment. The final global feature representation fuses information from multiple document fragments in the long text. The query statement is input into the embedding layer of the large language model to obtain a query embedding vector. The global feature representation of the last document fragment and the query embedding vector are input into the decoder of the large language model to generate the target query result. By extracting the query embedding vector using the embedding layer of the large language model, the quality of the query embedding vector is improved. Finally, by generating the target query result based on the query embedding vector and the final global feature representation that fuses multiple document fragments in the long text, the semantic match between the target query result and the query statement is enhanced, thereby further improving the accuracy of the target query result.

[0084] The technical solution provided by the embodiments of the present disclosure, by introducing a lightweight feature editing network, updates the global feature representation according to the generated new local feature representation, and can output a new global feature representation of fixed dimension, thereby effectively controlling the capacity of the memory buffer, avoiding the quadratic increase in computational complexity due to the increase in the number of document fragments, and solving the bottleneck problems faced by traditional models such as inference delay, excessive memory usage and rising deployment costs; and, extracting high-dimensional query embedding vectors through the large language model embedding layer, significantly enhancing the vector representation quality, and generating target query results based on the vectors and the global feature representation that integrates multi-document information, which not only ensures the semantic matching between the target query results and the query statements, but also achieves improved query accuracy while controlling computing resource consumption through the coordinated optimization of the feature editing network and the embedding layer.

[0085] In an optional implementation, the fixed dimension is determined based on the total memory, available video memory, and model parameter scale of the running device to which the large language model belongs.

[0086] To further enhance adaptability and resource efficiency across various hardware environments, an adaptive memory buffer size configuration mechanism is proposed, building upon the fixed-length memory buffer mechanism. For example, the maximum memory buffer size is dynamically adjusted based on the computing resources of the device running the large language model and the parameter size of the loaded model. Computing resources can include memory capacity, GPU memory, or parallel processing capabilities.

[0087] Specifically, the hardware detection module is called during the initialization phase Automatically collects the total memory M of the running device total , Available video memory V free , model parameter scale S model Key indicators such as α are used to comprehensively estimate the optimal cache configuration parameter α that the running device can bear. mem , used to regulate the feature editing network E θ The output dimensions are:

[0088]

[0089] in, It is an experience function or a lightweight scheduling network, which can be optimized and fine-tuned according to the deployment platform type, such as edge devices, servers, and consumer GPUs. mem Used to control the maximum capacity of the memory buffer and dynamically determine the number of entries in the global feature representation. Through this mechanism, it is possible to flexibly adjust according to different hardware constraints without sacrificing the ability to retain key semantic information, thereby effectively reducing memory usage and improving cross-platform deployment efficiency. This method is particularly suitable for large language model reasoning in resource-constrained environments, such as edge inference devices, multi-model parallel deployment platforms, etc., further enhancing the engineering practicality and system stability of the embodiments of the present disclosure.

[0090] Figure 3 This is a flow chart of a method for fine-tuning a large language model for long text processing according to an embodiment of the present disclosure. The method is applicable to the case of fine-tuning a large language model for long text scenarios. The method can be executed by a large language model fine-tuning device for long text processing, which can be implemented in software and / or hardware and can be integrated into electronic devices. Figure 3 As shown, the large language model fine-tuning method for long text processing in this embodiment may include:

[0091] S301, in response to a sample query request, dividing a long text into multiple document segments;

[0092] S302: Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0093] S303, updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0094] S304, inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result;

[0095] S305: Fine-tune the large language model according to the predicted query result and the actual query result.

[0096] During the model fine-tuning phase, in response to sample query requests, the input long text and query statement are obtained and divided into multiple document segments based on the number of tokens. A recursive approach is used to recursively generate a local feature representation for the current document segment based on the local feature representation of the previous document segment. Furthermore, the local feature representation of the current document segment is used to fuse and update the previously generated global feature representation across document segments to generate a new global feature representation.

[0097] Moreover, a memory buffer can be used to store global feature representations. For the i+1th document fragment, the i+1th document fragment and the pre-generated local feature representation of the i-th document fragment can be input into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document fragment. Also, the i-th global feature representation stored in the memory buffer is updated using the local feature representation of the i+1th document fragment to obtain the i+1th global feature representation, and the i+1th global feature representation is used to replace the i-th global feature representation in the memory buffer. After processing each document fragment, the query statement and the global feature representation stored in the memory buffer can be input into the decoder of the large language model to generate a predicted query result; a loss function can be generated based on the difference between the predicted query result and the actual query result of the sample query request, and the large language model can be fine-tuned based on the loss function. An exemplary loss function can use a cross entropy function, and the large language model is fine-tuned based on a loss function minimization strategy.

[0098] The technical solution provided by the embodiments of this disclosure proposes a recursive latent space hint compression method based on soft hint compression, which is used to optimize the inference efficiency and semantic integrity of large language models when processing long text inputs. This method maps the original natural language input into a continuous latent space, generating a learnable soft hint vector as the input representation for subsequent tasks. This method achieves efficient hint compression and context modeling without changing the original model structure.

[0099] In an optional embodiment, the method further includes: inputting the first document fragment, the task description information in the sample query request, the query statement and the compressed word into the embedding layer of the large language model to obtain the embedding vector of the first document fragment; inputting the embedding vector of the first document fragment into the encoder of the large language model to generate a local feature representation of the first document fragment.

[0100] Considering that different tasks differ in their focus on key contextual information, the disclosed embodiments incorporate task description information and query statements as adjustment inputs during the generation of local feature representations to dynamically adjust the information filtering strategy. By incorporating the task description information and query statement into the local feature representation of the first document fragment, the first local feature representation incorporates both task and query information. Furthermore, by recursively determining the local feature representations of subsequent document fragments, each local feature representation incorporates both task and query information.

[0101] The local feature representation of the first document fragment can be expressed by the following formula:

[0102]

[0103] Among them, Seg_mem1 is the local feature representation of the first document segment, x task , x q COMP is task description information, query statement, first document fragment and compressed word; M θ is a compression model function, which can be an encoder of a large language model with a parameter of θ.

[0104] In an optional embodiment, the step of inputting the i+1th document fragment and the local feature representation of the i-th document fragment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document fragment includes: inputting the i+1th document fragment and the compressed word into the embedding layer of the large language model to be fine-tuned to obtain the embedding vector of the i+1th document fragment; and inputting the embedding vector of the i+1th document fragment and the local feature representation of the i-th document fragment into the encoder of the large language model to generate the local feature representation of the i+1th document fragment.

[0105] Exemplarily, the local feature representation of the i+1th document fragment can be generated by recursive compression. For the i+1th document fragment, the i+1th document fragment and the preset compressed word unit can be input into the embedding layer of the large language model to obtain the embedding vector of the i+1th document fragment; the embedding vector of the i+1th document fragment and the pre-generated local feature representation of the i-th document fragment are input into the encoder of the large language model to recursively generate the local feature representation of the i+1th document fragment. In other words, the local feature representation of the i+1th document fragment can be expressed by the following formula:

[0106]

[0107] in, and Seg_mem i+1 are the local feature representations of the i-th and i+1-th document fragments, respectively. is the i+1th document segment, COMP is a special compression word attached to the input sequence to guide the compression operation; M θ is a compression model function, which can be an encoder of a large language model with a parameter of θ.

[0108] Figure 4 This is another flowchart of fine-tuning a large language model for long text processing provided in accordance with an embodiment of the present disclosure. Figure 4 Based on the above embodiment, the method for fine-tuning a large language model for long text processing in this embodiment may include:

[0109] S401, in response to a sample query request, dividing a long text into multiple document segments;

[0110] S402: Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer;

[0111] If the length of the i-th global feature representation is less than the fixed dimension, continue to execute S403; otherwise, jump to execute S404;

[0112] S403, concatenating the local feature representation of the (i+1)th document segment and the (i)th global feature representation to obtain the (i+1)th global feature representation;

[0113] S404: Inputting the local feature representation of the (i+1)th document segment and the (i)th global feature representation into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension;

[0114] S405, inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result;

[0115] S406: Fine-tune the large language model according to the predicted query result and the actual query result.

[0116] For the i+1th document fragment, if the length of the i-th global feature representation stored in the memory buffer is less than the fixed dimension, the local feature representation of the i+1th document fragment can be concatenated with the i-th global feature representation to obtain the i+1th global feature representation less than or equal to the fixed dimension, thereby further improving processing efficiency.

[0117] That is to say, in the initial stage, the global feature representation can be constructed by splicing:

[0118] Seq_mem i+1 =Concat(Seq_mem i ,Seg_memi+1 );

[0119] Among them, Concat is a concatenation operation.

[0120] If the length of the i-th global feature representation stored in the memory buffer is equal to or greater than the fixed dimension, the local feature representation of the i+1-th document segment and the i-th global feature representation stored in the memory buffer can be input into the pre-trained feature editing network to obtain the i+1-th global feature representation of the fixed dimension output by the feature editing network, and the i+1-th global feature representation is used to replace the i-th global feature representation in the memory buffer. In other words, the i+1-th global feature representation can be expressed by the following formula:

[0121] Seq_mem i+1 =E θ (Seq_mem i , Seg_mem i+1 );

[0122] Among them, Seq_mem i , Seq_mem i+1 They are the sequence stored in the memory buffer for the i-th global feature representation and the i+1-th global feature representation; Seg_mem i+1 is the i+1th local feature representation; E θ Edit the network for the feature.

[0123] In an optional embodiment, the fixed dimension is a multiple of a first value of the local feature representation; the feature editing network includes a second number of stacked deformer encoder blocks. The first value and the second value are both fixed values. The feature extraction network is a lightweight network composed of multiple deformer layers.

[0124] In an optional embodiment, the step of inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result includes: inputting the query statement into the embedding layer of the large language model to obtain a query embedding vector; and inputting the global feature representation of the last document fragment and the query embedding vector into the decoder of the large language model to generate a predicted query result.

[0125] When the i+1th document fragment is the last document fragment, the memory buffer stores the global feature representation of the last document fragment. The final global feature representation integrates the information of multiple document fragments in the long text. The query statement is input into the embedding layer of the large language model to obtain the query embedding vector. The global feature representation of the last document fragment and the query embedding vector are input into the decoder of the large language model to generate the predicted query result.

[0126] For example, in the fine-tuning phase, an end-to-end approach can be used to optimize model parameters. The generation of the prediction query result can be expressed by the following formula:

[0127]

[0128] Among them, Seq_mem K is the global feature representation of the last segment (i.e., the K-th document segment), x q For query statements, To predict the query results, f θ This can be the decoder of the large language model to be fine-tuned.

[0129] And, the loss function adopts the cross entropy loss minimization strategy, which is defined as follows:

[0130]

[0131] Among them, y n is the word at position n; P(y n ∣∣Seq memK , x q ) is the probability of the word appearing predicted by the model; N is the total number of word units in the predicted query results, L CE By minimizing the loss function, the model can continuously optimize the generation process of local feature representation and global feature representation, thereby improving the quality of the final output.

[0132] The technical solution provided by the embodiments of the present disclosure has brought significant improvements to large language models in multiple dimensions, especially in terms of improving reasoning efficiency, reducing resource consumption and enhancing task adaptability. By converting the input long text into a compact soft prompt vector, not only the model input length is reduced, but also the computational complexity and reasoning delay are reduced, which is particularly suitable for application scenarios that are sensitive to response speed, such as real-time dialogue systems and intelligent customer service. At the same time, the introduced memory buffer mechanism can maintain a global memory pool of a fixed size, avoiding the memory explosion problem caused by the increase in the number of document fragments in related technologies, so that the product can run stably on low-resource devices or edge computing environments. Therefore, it not only solves the problem of balancing compression efficiency and semantic integrity in long text processing, but also provides key technical support for building lightweight, efficient, task-aware AI products, and has broad engineering application prospects and market promotion value.

[0133] In an optional embodiment, the fixed dimension is determined based on the total memory, available video memory, and model parameter size of the device running the large language model. To further improve adaptability and resource utilization efficiency in various hardware environments, an adaptive memory buffer size configuration mechanism is proposed based on the fixed-length memory buffer mechanism. Exemplarily, the output dimension of the feature editing network is regulated based on the total memory, available video memory, and model parameter size of the device running the large language model.

[0134] Figure 5 This is a schematic diagram of the structure of a long text processing device based on a large language model, according to an embodiment of the present disclosure. This device is suitable for processing long text input using a large language model. This device can be implemented using software and / or hardware and can be integrated into electronic devices.

[0135] like Figure 5 As shown, the long text processing apparatus 500 based on the large language model of this embodiment may include:

[0136] A long text segmentation module 510 is configured to segment a long text into multiple document segments in response to a target query request;

[0137] A local feature module 520 is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the fine-tuned large language model to generate a local feature representation of the i+1th document segment; wherein i is a positive integer;

[0138] A global updating module 530 is configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0139] The query result module 540 is configured to input the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result.

[0140] In an optional implementation, the global update module 530 is specifically configured to:

[0141] The local feature representation of the (i+1)th document segment and the (i)th global feature representation are input into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension.

[0142] In an optional implementation manner, the global update module 530 is further configured to:

[0143] If the length of the i-th global feature representation is less than the fixed dimension, the local feature representation of the i+1-th document segment and the i-th global feature representation are concatenated to obtain the i+1-th global feature representation.

[0144] In an optional embodiment, the fixed dimension is a first numerical multiple of the local feature representation; and the feature editing network includes a second numerical number of stacked deformer encoder blocks.

[0145] In an optional implementation, the fixed dimension is determined based on the total memory, available video memory, and model parameter scale of the running device to which the large language model belongs.

[0146] In an optional implementation, the local feature module 520 includes a local recursive unit, which is specifically configured to:

[0147] Input the i+1th document segment and compressed word into the embedding layer of the fine-tuned large language model to obtain the embedding vector of the i+1th document segment;

[0148] The embedding vector of the i+1th document segment and the local feature representation of the i-th document segment are input into the encoder of the large language model to generate the local feature representation of the i+1th document segment.

[0149] In an optional implementation, the local feature module 520 further includes an initialization unit, which is specifically configured to:

[0150] Inputting the first document fragment, the task description information in the target query request, the query sentence, and the compressed word into the embedding layer of the large language model to obtain an embedding vector for the first document fragment;

[0151] The embedding vector of the first document segment is input into the encoder of the large language model to generate a local feature representation of the first document segment.

[0152] In an optional implementation, the query result module 540 includes:

[0153] A query embedding unit, configured to input the query statement into the embedding layer of the large language model to obtain a query embedding vector;

[0154] The query result unit is used to input the global feature representation of the last document fragment and the query embedding vector into the decoder of the large language model to generate a target query result.

[0155] The technical solution provided by the embodiments of the present disclosure provides a soft prompt compression framework that combines task awareness, recursive compression, and a fixed-length memory buffer mechanism. It can significantly reduce the input length while effectively retaining key semantic information, thereby improving the processing quality of large language models in complex tasks.

[0156] Figure 6This is a schematic diagram of a large language model fine-tuning device for long text processing according to an embodiment of the present disclosure. The device is suitable for fine-tuning a large language model for long text scenarios. The device can be implemented in software and / or hardware and can be integrated into electronic devices. Figure 6 As shown, the large language model fine-tuning apparatus 600 for long text processing of this embodiment may include:

[0157] The long text segmentation module 610 is configured to segment the long text into multiple document segments in response to a sample query request;

[0158] A local feature module 620 is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the large language model to be fine-tuned to generate a local feature representation of the i+1th document segment; wherein i is a positive integer;

[0159] A global updating module 630 is configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation;

[0160] A prediction result module 640 is configured to input the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result;

[0161] The model fine-tuning module 650 is configured to fine-tune the large language model according to the predicted query results and the actual query results.

[0162] In an optional implementation, the global update module 630 is specifically configured to:

[0163] The local feature representation of the (i+1)th document segment and the (i)th global feature representation are input into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension.

[0164] In an optional implementation manner, the global update module 630 is further configured to:

[0165] If the length of the i-th global feature representation is less than the fixed dimension, the local feature representation of the i+1-th document segment and the i-th global feature representation are concatenated to obtain the i+1-th global feature representation.

[0166] In an optional embodiment, the fixed dimension is a first numerical multiple of the local feature representation; and the feature editing network includes a second numerical number of stacked deformer encoder blocks.

[0167] In an optional implementation, the fixed dimension is determined based on the total memory, available video memory, and model parameter scale of the running device to which the large language model belongs.

[0168] In an optional implementation, the local feature module 620 includes a local recursive unit, which is specifically configured to:

[0169] Input the i+1th document segment and compressed word into the embedding layer of the large language model to be fine-tuned to obtain the embedding vector of the i+1th document segment;

[0170] The embedding vector of the i+1th document segment and the local feature representation of the i-th document segment are input into the encoder of the large language model to generate the local feature representation of the i+1th document segment.

[0171] In an optional implementation, the local feature module 620 further includes an initialization unit, which is specifically configured to:

[0172] Inputting the first document fragment, the task description information in the sample query request, the query sentence, and the compressed word into the embedding layer of the large language model to obtain an embedding vector for the first document fragment;

[0173] The embedding vector of the first document segment is input into the encoder of the large language model to generate a local feature representation of the first document segment.

[0174] In an optional implementation, the prediction result module 640 includes:

[0175] A query embedding unit, configured to input the query statement into the embedding layer of the large language model to obtain a query embedding vector;

[0176] The prediction result unit is used to input the global feature representation of the last document segment and the query embedding vector into the decoder of the large language model to generate a predicted query result.

[0177] The technical solution provided by the embodiments of the present disclosure has brought significant improvements to large language models in multiple dimensions, especially in terms of improving reasoning efficiency, reducing resource consumption and enhancing task adaptability. By converting the input long text into a compact soft prompt vector, not only the model input length is reduced, but also the computational complexity and reasoning delay are reduced, which is particularly suitable for application scenarios that are sensitive to response speed, such as real-time dialogue systems and intelligent customer service. At the same time, the introduced memory buffer mechanism can maintain a global memory pool of a fixed size, avoiding the memory explosion problem caused by the increase in the number of document fragments in related technologies, so that the product can run stably on low-resource devices or edge computing environments. Therefore, it not only solves the problem of balancing compression efficiency and semantic integrity in long text processing, but also provides key technical support for building lightweight, efficient, task-aware AI products, and has broad engineering application prospects and market promotion value.

[0178] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0180] Figure 7 It is a block diagram of an electronic device used to implement the long text processing method based on a large language model or the large language model fine-tuning method for long text processing according to an embodiment of the present disclosure. Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0181] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0182] Multiple components in the electronic device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 701 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a material delivery unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a long text processing method based on a large language model or a large language model fine-tuning method for long text processing. For example, in some embodiments, a long text processing method based on a large language model or a large language model fine-tuning method for long text processing can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the long text processing method based on a large language model or the large language model fine-tuning method for long text processing described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the long text processing method based on a large language model or the large language model fine-tuning method for long text processing by any other appropriate means (e.g., by means of firmware).

[0184] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0185] like Figure 7As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the electronic device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0186] Multiple components in the electronic device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the electronic device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0187] The computing unit 701 can be various general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a material delivery unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as a long text processing method based on a large language model or a large language model fine-tuning method for long text processing. For example, in some embodiments, a long text processing method based on a large language model or a large language model fine-tuning method for long text processing can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the long text processing method based on a large language model or the large language model fine-tuning method for long text processing described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to execute the long text processing method based on a large language model or the large language model fine-tuning method for long text processing by any other appropriate means (e.g., by means of firmware).

[0188] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0189] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0190] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0191] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, audio input, or tactile input).

[0192] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web player through which a user can interact with embodiments of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0193] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0194] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, audio recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0195] Cloud computing refers to a technology system that provides network access to elastically scalable shared pools of physical or virtual resources. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on-demand in a self-service manner. Cloud computing technology provides efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain, as well as for model training.

[0196] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0197] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A long text processing method based on a large language model, comprising: In response to a target query request, a long text is divided into multiple document segments; Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; where i is a positive integer; Updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation; The query statement and the updated global feature representation are input into the decoder of the large language model to generate a target query result.

2. The method according to claim 1, wherein The updating of the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation includes: The local feature representation of the (i+1)th document segment and the (i)th global feature representation are input into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension.

3. The method according to claim 2, wherein updating the i-th global feature representation based on the local feature representation of the i+1-th document segment further comprises: If the length of the i-th global feature representation is less than the fixed dimension, the local feature representation of the i+1-th document segment and the i-th global feature representation are concatenated to obtain the i+1-th global feature representation.

4. The method according to claim 2, wherein: The fixed dimension is a first numerical multiple of the local feature representation; and the feature editing network includes a second numerical number of stacked deformer encoder blocks.

5. The method according to claim 2, wherein: The fixed dimension is determined based on the total memory, available video memory, and model parameter scale of the running device to which the large language model belongs.

6. The method according to claim 1, wherein The step of inputting the local feature representations of the i+1th document segment and the ith document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment includes: Input the i+1th document segment and compressed word into the embedding layer of the fine-tuned large language model to obtain the embedding vector of the i+1th document segment; The embedding vector of the i+1th document segment and the local feature representation of the i-th document segment are input into the encoder of the large language model to generate the local feature representation of the i+1th document segment.

7. The method according to claim 1, further comprising: Inputting the first document fragment, the task description information in the target query request, the query sentence, and the compressed word into the embedding layer of the large language model to obtain an embedding vector for the first document fragment; The embedding vector of the first document segment is input into the encoder of the large language model to generate a local feature representation of the first document segment.

8. The method according to claim 1, wherein Inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result includes: Inputting the query statement into the embedding layer of the large language model to obtain a query embedding vector; The global feature representation of the last document segment and the query embedding vector are input into the decoder of the large language model to generate a target query result.

9. A method for fine-tuning a large language model for long text processing, comprising: In response to a sample query request, dividing a long text into multiple document segments; Input the i+1th document segment and the local feature representation of the i-th document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer; Updating the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation; Inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result; The large language model is fine-tuned according to the predicted query results and the actual query results.

10. The method according to claim 9, wherein: The updating of the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation includes: The local feature representation of the (i+1)th document segment and the (i)th global feature representation are input into a feature editing network to generate an (i+1)th global feature representation of a fixed dimension.

11. The method according to claim 10, wherein updating the i-th global feature representation based on the local feature representation of the i+1-th document segment further comprises: If the length of the i-th global feature representation is less than the fixed dimension, the local feature representation of the i+1-th document segment and the i-th global feature representation are concatenated to obtain the i+1-th global feature representation.

12. The method according to claim 10, wherein: The fixed dimension is a first numerical multiple of the local feature representation; and the feature editing network includes a second numerical number of stacked deformer encoder blocks.

13. The method according to claim 10, wherein: The fixed dimension is determined based on the total memory, available video memory, and model parameter scale of the running device to which the large language model belongs.

14. The method according to claim 9, wherein The step of inputting the local feature representation of the i+1th document segment and the i-th document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment includes: Input the i+1th document segment and compressed word into the embedding layer of the large language model to be fine-tuned to obtain the embedding vector of the i+1th document segment; The embedding vector of the i+1th document segment and the local feature representation of the i-th document segment are input into the encoder of the large language model to generate the local feature representation of the i+1th document segment.

15. The method according to claim 9, further comprising: Inputting the first document fragment, the task description information in the sample query request, the query sentence, and the compressed word into the embedding layer of the large language model to obtain an embedding vector for the first document fragment; The embedding vector of the first document segment is input into the encoder of the large language model to generate a local feature representation of the first document segment.

16. The method according to claim 9, wherein Inputting the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result includes: Inputting the query statement into the embedding layer of the large language model to obtain a query embedding vector; The global feature representation of the last document segment and the query embedding vector are input into the decoder of the large language model to generate a predicted query result.

17. A long text processing device based on a large language model, comprising: A long text segmentation module, configured to segment a long text into multiple document segments in response to a target query request; A local feature module is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the fine-tuned large language model to generate the local feature representation of the i+1th document segment; where i is a positive integer; A global updating module, configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation; The query result module is used to input the query statement and the updated global feature representation into the decoder of the large language model to generate a target query result.

18. A large language model fine-tuning device for long text processing, comprising: A long text segmentation module, configured to segment a long text into multiple document segments in response to a sample query request; A local feature module is configured to input the local feature representations of the i+1th document segment and the ith document segment into the encoder of the large language model to be fine-tuned to generate the local feature representation of the i+1th document segment; where i is a positive integer; A global updating module, configured to update the i-th global feature representation according to the local feature representation of the i+1-th document segment to generate an updated global feature representation; A prediction result module, configured to input the query statement and the updated global feature representation into the decoder of the large language model to generate a predicted query result; A model fine-tuning module is used to fine-tune the large language model according to the predicted query results and the actual query results.

19. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 16.

20. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause a computer to execute the method according to any one of claims 1-16.

21. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 16.

Images