Compression method and device of big language model prompt text, equipment and medium

Abstract

The embodiment of the invention discloses a compression method and device for a big language model prompt text, electronic equipment and a storage medium. The method comprises the steps that a first number of to-be-compressed documents are generated based on an input question; evaluating respective importance of the to-be-compressed documents, and screening based on the importance to obtain a second number of to-be-compressed documents; based on the respective importance of the second number of to-be-compressed documents, distributing corresponding compression proportions to the second number of to-be-compressed documents, and performing compression processing on the second number of to-be-compressed documents based on the distributed compression proportions to obtain the second number of compressed documents; and inputting the input question and the second number of compressed documents into a large language model, so that the large language model outputs an answer corresponding to the input question. Therefore, according to the embodiment, document compression can be effectively carried out, so that the processing burden of the large language model is reduced, and the processing cost and time delay are further reduced.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device and computer-readable storage medium for compressing large language model prompt text. Background Art

[0002] With the rapid development of AI technology, Large Language Model (LLM) has become one of the core technologies in the field of natural language processing. However, there are some challenges in the practical application of LLM, including large language model hallucination and untimely knowledge update.

[0003] In order to solve the problems of large language model hallucination and untimely knowledge update, researchers proposed a retrieval-augmented generation (RAG) technology. Before generating text, the retrieval-augmented generation technology is used to first search from a large amount of relevant information, and then form prompt texts based on the retrieved multiple documents and input them into the generation model, which effectively alleviates the hallucination problem of the large language model and improves the speed of knowledge update.

[0004] However, the introduction of excessively long prompt texts due to retrieval-augmented generation techniques will increase the processing burden of large language models, thereby increasing the cost and latency of inference. Summary of the invention

[0005] To solve the above technical problems, the embodiments of the present application provide a method, device, electronic device and computer-readable storage medium for compressing a large language model prompt text, which can effectively compress documents to reduce the processing burden of the large language model, thereby reducing processing costs and delays.

[0006] According to one aspect of an embodiment of the present application, a method for compressing a large language model prompt text is provided, comprising: generating a first number of documents to be compressed based on an input question; evaluating the importance of each of the documents to be compressed, and screening to obtain a second number of documents to be compressed based on the importance; assigning corresponding compression ratios to the second number of documents to be compressed based on the importance of each of the second number of documents to be compressed, and compressing the second number of documents to be compressed based on the assigned compression ratios to obtain the second number of compressed documents; inputting the input question and the second number of compressed documents into a large language model, so that the large language model outputs an answer corresponding to the input question.

[0007] In another exemplary embodiment, the evaluating the importance of each of the documents to be compressed and selecting a second number of documents to be compressed based on the importance includes: calculating the perplexity of the input question for each of the documents to be compressed; and evaluating the importance of each of the documents to be compressed based on the perplexity of each of the documents to be compressed; wherein the importance is positively correlated with the perplexity.

[0008] In another exemplary embodiment, calculating the perplexity of the input question for each of the documents to be compressed includes: selecting a keyword word from the words contained in the input question; and determining the perplexity of the document to be compressed based on the number of words in the input question and the probability of the keyword word appearing in the document to be compressed.

[0009] In another exemplary embodiment, after obtaining the second number of compressed documents, it also includes: concatenating the first number of documents to be compressed into a first embedding vector, and concatenating the second number of compressed documents into a second embedding vector; if the semantic loss of the compressed documents is serious, concatenating the first embedding vector and the second embedding vector to obtain a concatenated vector; determining a first semantic feature vector based on the first embedding vector and the concatenated vector, and determining a second semantic feature vector based on the second embedding vector and the concatenated vector; mapping the first semantic feature vector and the second semantic feature vector to a common feature space to analyze the similarity between the semantic features contained in each semantic feature vector in the common feature space, and semantically enhancing each semantic feature vector based on the similarity.

[0010] In another exemplary embodiment, the method further includes: calculating a difference vector between the first embedding vector and the second embedding vector, and a norm of the difference vector; and determining the semantic loss of the compressed document according to the norm of the difference vector.

[0011] In another exemplary embodiment, the inputting the input question and the second number of compressed documents into the large language model so that the large language model outputs an answer corresponding to the input question includes: based on the importance of each of the second number of documents to be compressed, sorting the second number of documents to be compressed in descending order according to importance to obtain a sequence of documents to be compressed; and inputting the input question and the sequence of documents to be compressed into the large language model.

[0012] In another exemplary embodiment, the first number of documents to be compressed are spliced ​​into a first sequence, the answers corresponding to the input questions are spliced ​​into a second sequence, and the second number of compressed documents are spliced ​​into a third sequence; the method further includes: obtaining the longest common subsequence between the first sequence and the second sequence; extracting a target document from the longest common subsequence, and using the target document as the compressed document to be input into the language model, so that the large language model outputs the answer; the target document does not exist in the third sequence.

[0013] According to one aspect of an embodiment of the present application, a compression device for a large language model prompt text is provided, comprising: a to-be-compressed document generation module, configured to generate a first number of to-be-compressed documents based on an input question; an importance evaluation module, configured to evaluate the importance of each of the to-be-compressed documents, and to obtain a second number of to-be-compressed documents based on the importance screening; a compression module, configured to assign corresponding compression ratios to the second number of to-be-compressed documents based on the importance of each of the second number of to-be-compressed documents, and to perform compression processing on the second number of to-be-compressed documents based on the assigned compression ratios, so as to obtain the second number of compressed documents; and an input module, configured to input the input question and the second number of compressed documents into a large language model, so that the large language model outputs an answer corresponding to the input question.

[0014] According to one aspect of an embodiment of the present application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the electronic device implements the method for compressing large language model prompt text as described above.

[0015] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer executes the method for compressing a large language model prompt text as described above.

[0016] In the technical solution provided in the embodiments of the present application, a first number of documents to be compressed are generated specifically around the input problem, the importance of each of the first number of documents to be compressed is evaluated, and a second number of documents to be compressed with relatively high importance are screened to reduce the data scale and subsequent processing burden; then, based on the importance of each of the second number of documents to be compressed, corresponding compression ratios are assigned to the second number of documents to be compressed, and compression processing is performed separately based on the assigned compression ratios to retain key information while minimizing irrelevant information in the document, making the compressed document more refined, thereby greatly reducing the data volume of the prompt text, effectively reducing the processing burden of the large language model, and reducing the processing cost and delay.

[0017] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present application, and together with the specification, are used to explain the principles of the present application. Obviously, the drawings described below are only some embodiments of the present application, and for those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0019] Figure 1 is a flowchart of a method for compressing a large language model prompt text shown in an exemplary embodiment of the present application;

[0020] Figure 2 is a flow chart of evaluating the importance of a document to be compressed, shown in an exemplary embodiment of the present application;

[0021] Figure 3 is a flow chart of calculating the perplexity of a document to be compressed, shown in an exemplary embodiment of the present application;

[0022] Figure 4 is a flowchart of a method for compressing a large language model prompt text shown in another exemplary embodiment of the present application;

[0023] Figure 5 is a flow chart showing a method of determining the semantic loss of a compressed document according to an exemplary embodiment of the present application;

[0024] Figure 6 is a flowchart of inputting an input question and a sequence of documents to be compressed into a large language model, shown in an exemplary embodiment of the present application;

[0025] Figure 7 is a flowchart of extracting a target document and inputting it into a large language model, shown in an exemplary embodiment of the present application;

[0026] Figure 8 is a flowchart of a method for compressing a large language model prompt text shown in another exemplary embodiment of the present application;

[0027] Fig. 9 It is a block diagram of a large language model prompt text compression device shown in an exemplary embodiment of the present application;

[0028] Fig.10 It is an overall block diagram of compressing a large language model prompt text shown in an exemplary embodiment of the present application;

[0029] Fig.11 A schematic diagram of the structure of a computer system suitable for implementing an electronic device of an embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments that are the same as the present application. Instead, they are only examples of devices and methods that are the same as some aspects of the present application as detailed in the attached claims.

[0031] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in the form of an application program, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0033] It should be noted that the "multiple" mentioned in this application refers to two or more than two. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship.

[0034] The following is an introduction and explanation of several terms and background technologies involved in this application:

[0035] LLM (Large Language Model): is an artificial intelligence model designed to understand and generate human language. By training on large amounts of text data, it can perform a wide range of tasks, including text summarization, translation, sentiment analysis, and more. They are characterized by their large scale, which helps them learn complex patterns in language data.

[0036] RAG (Retrieval-Augmented Generation): is a framework that combines information retrieval technology and large language models (LLM). It assists the model in generating answers by retrieving information from data sources, thereby improving the accuracy and relevance of the output.

[0037] With the rapid development of AI technology, Large Language Model (LLM) has become one of the core technologies in the field of natural language processing. However, there are some challenges in the practical application of LLM, including large language model hallucination and untimely knowledge update.

[0038] In order to solve the problems of large language model hallucination and untimely knowledge update, researchers proposed a retrieval-augmented generation (RAG) technology. Before generating text, the retrieval-augmented generation technology is used to first search from a large amount of relevant information, and then form prompt texts based on the retrieved multiple documents and input them into the generation model, which effectively alleviates the hallucination problem of the large language model and improves the speed of knowledge update.

[0039] However, the introduction of excessively long prompt texts due to retrieval-augmented generation techniques will increase the processing burden of large language models, thereby increasing the cost and latency of inference.

[0040] Based on this, the present application proposes a compression method, device, electronic device and computer-readable storage medium for large language model prompt text. The purpose is to generate a first number of documents to be compressed around the input problem, evaluate the importance of each of the first number of documents to be compressed, and screen out a second number of documents to be compressed with relatively high importance, so as to reduce the data scale and subsequent processing burden; then, based on the importance of each of the second number of documents to be compressed, the second number of documents to be compressed are respectively assigned corresponding compression ratios, and compression processing is performed based on the assigned compression ratios, so as to retain key information while minimizing irrelevant information in the document, so that the compressed document is more refined, further significantly reducing the amount of data, effectively reducing the processing burden of the large language model, and thus reducing the processing cost and delay.

[0041] See also Figure 1 , Figure 1 It is a flowchart of a method for compressing a large language model prompt text shown in an exemplary embodiment of the present application.

[0042] The following describes in detail the method for compressing a large language model prompt text proposed in an embodiment of the present application, using a computer as a specific execution subject.

[0043] like Figure 1 As shown, in an exemplary embodiment, the method for compressing a large language model prompt text includes at least S110 to S140, which are described in detail as follows:

[0044] S110: Generate a first number of documents to be compressed based on the input question.

[0045] S120, evaluating the importance of each of the documents to be compressed, and selecting a second number of documents to be compressed based on the importance.

[0046] S130, based on the importance of each of the second number of documents to be compressed, respectively assign corresponding compression ratios to the second number of documents to be compressed, and respectively compress the second number of documents to be compressed based on the assigned compression ratios to obtain a second number of compressed documents.

[0047] S140: Input the input question and the second number of compressed documents into the large language model, so that the large language model outputs an answer corresponding to the input question.

[0048] The above S110 - S140 are respectively described in detail below.

[0049] In S110, when an input question is received, documents related to the input question are collected from a large number of information sources to form an initial first number of documents to be compressed.

[0050] In some embodiments of the present application, the input question is searched through the search enhancement generation technology to extract a first number of documents to be compressed from a massive information source.

[0051] In other embodiments of the present application, a text matching algorithm in natural language processing is used to calculate the semantic similarity between massive information sources and the input question, and a first number of documents to be compressed whose semantic similarity reaches a certain threshold are screened out.

[0052] By focusing on documents related to the input question and removing a large amount of irrelevant information, a relatively concentrated scope is defined for subsequent processing, avoiding the large language model from wasting resources on processing irrelevant data.

[0053] In S120, different documents to be compressed contribute to accurately answering the input question to different degrees. Some documents contain key core knowledge, the latest research results or important data support, that is, documents with a higher degree of importance; while more documents may simply mention related concepts, that is, documents with a lower degree of importance.

[0054] In some embodiments of the present application, the second number is smaller than the first number.

[0055] In some embodiments of the present application, the frequency of citation of the document to be compressed by other relevant documents and materials is determined. If the document to be compressed is frequently cited by many subsequent studies and reports in the industry, it means that it has a certain influence and reference value in the industry and has a high importance; on the contrary, if the document to be compressed is cited less frequently in the industry, it means that it lacks a certain influence and reference value in the industry and has a low importance.

[0056] In other embodiments of the present application, please refer to Figure 2 , Figure 2 The flowchart of evaluating the importance of documents to be compressed is shown in an exemplary embodiment of the present application. The step S120 of evaluating the importance of each document to be compressed and selecting a second number of documents to be compressed based on the importance includes the following steps S210-S220:

[0057] S210, calculating the perplexity of the input question for each document to be compressed.

[0058] S220, evaluating the importance of each document to be compressed according to the perplexity of each document to be compressed; wherein the importance is positively correlated with the perplexity.

[0059] The above S210 - S220 are respectively described in detail below.

[0060] In S210, perplexity is an important indicator for evaluating the degree of association between the input question and the document to be compressed.

[0061] In some embodiments of this application, please refer to Figure 3 , Figure 3 1 is a flowchart of calculating the perplexity of a document to be compressed as shown in an exemplary embodiment of the present application. Step S210 of calculating the perplexity of the input question for each document to be compressed includes the following steps S310-S320:

[0062] S310: Select a keyword word from the word words included in the input question.

[0063] S320, determining the perplexity of the document to be compressed according to the number of word units in the input question and the probability of the keyword word units appearing in the document to be compressed.

[0064] The above S310 - S320 are respectively described in detail below.

[0065] In S310, the input question is often a natural language expression composed of multiple word units, and not all word units are equally important in measuring the degree of relevance between the word unit and the document to be compressed. The keyword unit is the word unit that is most representative in the input question, best reflects the core semantics, and is most critical in judging the relevance between the word unit and the document to be compressed.

[0066] When it comes to the selection of key words, content words such as nouns, verbs and adjectives are more critical in expressing the core semantics, while function words such as auxiliary words and prepositions often only play an auxiliary connecting role and have little impact on the semantic core.

[0067] In some embodiments of the present application, the input question is “how to improve the efficiency of solar photovoltaic power generation”, and thus the selected keywords are “improve”, “solar energy”, “photovoltaic power generation” and “efficiency”.

[0068] In other embodiments of the present application, the input question is "What factors affect the battery life of an electric vehicle?", and thus, the selected keywords are "electric vehicle," "battery life," and "factors."

[0069] In S320, in some embodiments of the present application, the calculation formula of the perplexity is as follows:

[0070]

[0071] In the above calculation formula, s k is represented by the perplexity of the kth document to be compressed; N is represented by the number of words in the input question; K is represented by the first number; x i Represented as the i-th word in the input question; x i It can also be expressed as a selected keyword element; Indicates that x appears in the kth document to be compressed i probability.

[0072] In S220, there is a positive correlation between perplexity and importance. The greater the perplexity, the higher the importance of the document to be compressed; the smaller the perplexity, the lower the importance of the document to be compressed.

[0073] In some embodiments of the present application, after obtaining the perplexity of each document to be compressed, the documents to be compressed are sorted according to the size of the perplexity.

[0074] In some embodiments of the present application, the documents to be compressed whose perplexity reaches a set perplexity threshold are screened and retained.

[0075] In other embodiments of the present application, a set number of documents to be compressed arranged from high to low perplexity are screened and retained.

[0076] In S130, after the second number of documents to be compressed are screened, some redundant information may still exist in the retained documents to be compressed, including repeated statements, background introductions that are not critical to answering questions, and the like.

[0077] The importance of the document to be compressed is inversely proportional to the corresponding compression ratio. For documents with higher importance to be compressed, a lower compression ratio is assigned; for documents with lower importance to be compressed, a higher compression ratio is assigned. The compression ratio can reflect the degree of compression of the document to be compressed, and the compression ratio is proportional to the degree of compression of the document to be compressed. The higher the compression ratio, the higher the degree of compression of the document to be compressed; the lower the compression ratio, the lower the degree of compression of the document to be compressed.

[0078] In some embodiments of the present application, the compression ratios of the second number of documents to be compressed are determined based on factors such as their importance, document length, and information density.

[0079] In some embodiments of the present application, for documents that are very important and have high information density but are long in length, a compression ratio of 20%-30% can be allocated, that is, about 70%-80% of relatively redundant content is deleted; and for documents that are slightly less important and have average information density, the compression ratio can be set at 50%-60%.

[0080] In some embodiments of the present application, the compression processing method can adopt text summary extraction, using the automatic summarization algorithm in natural language processing to extract the core sentences and key ideas of the document to form a compressed document in the form of a summary; a text deletion strategy can also be adopted to remove modifying words, examples and other content that have little impact on the core semantics based on grammatical and semantic information such as part of speech and semantic role, so as to achieve text compression.

[0081] After S130, please refer to Figure 4 , Figure 4 FIG. 4 is a flowchart of another exemplary embodiment of the present application showing a method for compressing a large language model prompt text. The method for compressing a large language model prompt text also includes the following S410-S440:

[0082] S410: Concatenate a first number of documents to be compressed and convert them into a first embedding vector, and concatenate a second number of compressed documents and convert them into a second embedding vector.

[0083] In S410, illustratively, the first embedding vector is

[0084] Wherein, K represents the first quantity; Indicates the first document to be compressed; Indicates the second document to be compressed; Represents the Kth document to be compressed.

[0085] The second embedding vector is

[0086] Among them, K ′ is expressed as a first quantity; Indicates the first document to be compressed; Indicates the second document to be compressed; Represents the Kth document to be compressed.

[0087] S420: If the semantic loss of the compressed document is serious, concatenate the first embedding vector and the second embedding vector to obtain a concatenated vector.

[0088] In S420, see Figure 5 , Figure 5 FIG. 1 is a flowchart of determining the semantic loss of a compressed document shown in an exemplary embodiment of the present application. In other embodiments of the present application, the step of determining the semantic loss of the compressed document further includes the following S510-S520:

[0089] S510, calculating a difference vector between the first embedding vector and the second embedding vector, and a norm of the difference vector;

[0090] S520: Determine the semantic loss of the compressed document according to the norm of the difference vector.

[0091] In S510 - S520 , illustratively, the first embedding vector is V1 , the second embedding vector is V2 , and the difference vector is ΔV.

[0092] Therefore, the difference vector is equal to the first embedded vector minus the second embedded vector, that is, ΔV=V1-V2; the norm of the difference vector is ||ΔV||.

[0093] The smaller the norm of the difference vector, the smaller the semantic difference between the compressed document and the document to be compressed, that is, the smaller the semantic loss of the compressed document, and the more complete the semantic information is retained. Similarly, the larger the norm of the difference vector, the greater the semantic difference between the compressed document and the document to be compressed, that is, the greater the semantic loss of the compressed document, and the more incomplete the semantic information is retained.

[0094] After determining the semantic loss of the compressed document, the norm of the difference vector is compared with the set threshold. If the norm of the difference vector is less than or equal to the set threshold, it is determined that the compressed document has good semantic consistency with the document to be compressed. If the norm of the difference vector is greater than the set threshold, it is determined that the semantic loss of the compressed document is serious.

[0095] S430: Determine a first semantic feature vector according to the first embedding vector and the concatenated vector, and determine a second semantic feature vector according to the second embedding vector and the concatenated vector.

[0096] In S430, after obtaining the first embedding vector and the concatenated vector, it is necessary to further extract therefrom a vector representation that is more representative and can better reflect the key semantic features, that is, the first semantic feature vector.

[0097] Similarly, after obtaining the second embedding vector and the concatenated vector, it is necessary to further extract a more representative vector representation that can better reflect the key semantic features, that is, the second semantic feature vector.

[0098] S440, mapping the first semantic feature vector and the second semantic feature vector to a common feature space, analyzing similarities between semantic features included in each semantic feature vector in the common feature space, and performing semantic enhancement on each semantic feature vector based on the similarities.

[0099] In S440, by mapping the first semantic feature vector and the second semantic feature vector to a common feature space, which is equivalent to considering them in a unified "semantic coordinate system", the similarity of the semantic features they contain can be measured more conveniently and accurately.

[0100] In some embodiments of the present application, a pre-trained language model BERT (Bid i rect ional Encoder Representation from Transformers) is used to calculate the similarity between semantic features.

[0101] The purpose of similarity analysis is to understand the degree of association and differences between the document to be compressed and the compressed document at the semantic level, and then to perform semantic enhancement on each semantic feature vector based on the similarity.

[0102] In some embodiments of the present application, the weights of semantic features can be adjusted dynamically to highlight the dimension with the greatest semantic similarity. For example, core semantic features such as entities and relationships can be emphasized while irrelevant noise can be weakened.

[0103] In S140, after the second number of compressed documents are obtained through screening and compression, the input question is input into the large language model together with the second number of compressed documents, so that the large language model can analyze and reason based on the compressed documents that are relatively refined and highly relevant to the question, thereby generating an accurate answer.

[0104] In some embodiments of this application, please refer to Figure 6 , Figure 6 is a flowchart of inputting an input question and a sequence of documents to be compressed into a large language model, shown in an exemplary embodiment of the present application. In other embodiments of the present application, step S140 of inputting the input question and a second number of compressed documents into the large language model so that the large language model outputs an answer corresponding to the input question includes the following S610-S620:

[0105] S610: Based on the importance of each of the second number of documents to be compressed, the second number of documents to be compressed are sorted in descending order according to their importance to obtain a sequence of documents to be compressed.

[0106] S620: Input the input question and the sequence of documents to be compressed into the large language model.

[0107] In S610-S620, in some embodiments of the present application, the second number of documents to be compressed is The sequence of documents to be compressed is

[0108] Among them, r1 represents the importance score; r2 represents the importance score; rK ′ Expressed as importance score; r1>r2>...>rK ′ .

[0109] By sorting the documents to be compressed in descending order of importance and inputting the input question and the sequence of documents to be compressed into the large language model, the accuracy of the answer to the input question output by the large language model can be significantly improved.

[0110] As can be seen from the above S110-S140, a first number of documents to be compressed are generated specifically around the input problem, the importance of each of the first number of documents to be compressed is evaluated, and a second number of documents to be compressed with relatively high importance are screened to reduce the data scale and subsequent processing burden; then, based on the importance of each of the second number of documents to be compressed, corresponding compression ratios are assigned to the second number of documents to be compressed, and compression processing is performed separately based on the assigned compression ratios to retain key information while minimizing irrelevant information in the document, so that the compressed document is more refined, thereby greatly reducing the data volume of the prompt text, effectively reducing the processing burden of the large language model, and reducing the processing cost and delay.

[0111] See also Figure 7 , Figure 7 is a flowchart of extracting target documents and inputting a large language model as shown in an exemplary embodiment of the present application. In some embodiments of the present application, a first number of documents to be compressed are spliced ​​to form a first sequence, answers corresponding to input questions are spliced ​​to form a second sequence, and a second number of compressed documents are spliced ​​to form a third sequence. The method for compressing a large language model prompt text also includes the following S710-S720:

[0112] S710: Obtain the longest common subsequence between the first sequence and the second sequence.

[0113] S720, extracting a target document from the longest common subsequence, and using the target document as a compressed document to be input into the language model, so that the large language model outputs an answer; there is no target document in the third sequence.

[0114] The above S710 - S720 are respectively described in detail below.

[0115] Before S710, the first number of documents to be compressed are spliced ​​to form a first sequence. The answers corresponding to the input questions are spliced ​​to form a second sequence. The second number of documents to be compressed are spliced ​​to form a third sequence.

[0116] In S710 to S720, the longest common subsequence is a subsequence with the longest length that is contained in both the first sequence and the second sequence.

[0117] The current document is the document that exists in the longest common subsequence and does not exist in the third sequence. The target document is the important but missed document.

[0118] In some embodiments of the present application, the target document is concatenated with the second number of compressed documents to form a third number of compensated documents to ensure that important documents are restored and can be input into the large language model together with the second number of compressed documents. The third number is greater than the second number. The third number is less than the first number.

[0119] In some embodiments of this application, please refer to Figure 8 , Figure 8 FIG. 1 is a flowchart of another exemplary embodiment of the present application showing a method for compressing a large language model prompt text. The method for compressing a large language model prompt text also includes the following S810-S870:

[0120] S810, input a long text prompt word.

[0121] In S810, the long text prompt word is the prompt text. The long text prompt word includes input questions, documents and instructions. The long text prompt word can be expressed as:

[0122] Among them, x RAG Represents a long text prompt word; x ins Represented as a command; All are documents; x represents the input question; represents the kth document retrieved by the retrieval enhancement generation technology, and L represents the total number of documents retrieved.

[0123] S820, evaluating the documents retrieved by the retrieval enhancement generation technology, and retaining the most relevant documents.

[0124] In S820, the importance of the total L documents retrieved is evaluated, and several documents with higher importance are retained to obtain new text prompt words. The new text prompt words can be expressed as follows:

[0125]

[0126] Among them, K ′ It means that after removing documents irrelevant to the user's question, there are still K ′ documents.

[0127] S830, calculate the relative perplexity to perform more detailed compression and retain the content closely related to the problem.

[0128] In S830, the new text prompt words are compressed more carefully, and the content closely related to the question is retained, so that the compression result contains more key information related to the question.

[0129] In some embodiments of the present application, relative perplexity (i.e., distribution change caused by question conditions) is calculated to represent the association between word units and questions. i Its importance score is r k It can be calculated as:

[0130] r k =perplexity(x i |x ＜i )-perplexity(x i |x,x ＜i ),

[0131] Among them, perplexity(·) represents the perplexity.

[0132] S840 ensures semantic coherence during compression to prevent information loss or inference errors.

[0133] In S840, in order to ensure that the semantic coherence is not broken during the compression process and focus on the internal connection between sentences, semantic enhancement can be performed through a semantic enhancement engine to prevent information loss or reasoning errors.

[0134] S850, based on the retrieval enhancement generation technology, the key information is rearranged to improve the reasoning accuracy.

[0135] In S850, the compressed documents are sorted in descending order according to their importance, so as to improve the inference accuracy of the large language model.

[0136] S860, restore key information that may be omitted or weakened during the compression process to ensure the integrity of the prompt word.

[0137] In S860, key information that may be omitted or weakened during the compression process is restored through a keyword recovery component, especially important keywords in the prompt word are restored to ensure that the core content of the final prompt word is complete.

[0138] S870: Output the compressed prompt word.

[0139] In S870, the compressed prompt word is finally output.

[0140] In summary, the above-mentioned large language model prompt text compression method includes but is not limited to the following beneficial effects:

[0141] First, extract key information from the input question and assign relevance scores to different parts of the prompt word based on the topic and intent of the question. This problem-oriented approach ensures a more reasonable priority allocation during the prompt word compression process, thereby giving priority to retaining core information closely related to the question in subsequent compression steps, avoiding accidental deletion or omission of key information.

[0142] Second, it can ensure that the compressed prompt words can accurately express user needs during the model reasoning process and handle potential compression errors, thereby ensuring the integrity of the prompt words and the accuracy of model reasoning.

[0143] Third, this patent design is particularly suitable for the application scenarios of retrieval enhancement generation technology. By efficiently compressing and optimizing the information retrieved from the external knowledge base, this method can significantly reduce the computational burden of large language models when processing long contexts and reduce the cost of inference.

[0144] Fig. 9 It is a block diagram of a large language model prompt text compression device shown in an exemplary embodiment of the present application, and the device includes a document generation module 910 to be compressed, an importance evaluation module 920, a compression module 930 and an input module 940.

[0145] The to-be-compressed document generating module 910 is configured to generate a first number of to-be-compressed documents based on the input question.

[0146] The importance evaluation module 920 is configured to evaluate the importance of each of the documents to be compressed, and obtain a second number of documents to be compressed based on the importance screening.

[0147] The compression module 930 is configured to assign corresponding compression ratios to the second number of documents to be compressed based on the importance of each of the second number of documents to be compressed, and compress the second number of documents to be compressed based on the assigned compression ratios to obtain a second number of compressed documents.

[0148] The input module 940 is configured to input the input question and the second number of compressed documents into the large language model, so that the large language model outputs an answer corresponding to the input question.

[0149] In another exemplary embodiment, the importance evaluation module 920 further includes a perplexity calculation unit and an importance evaluation unit. The perplexity calculation unit is configured to calculate the perplexity of the input question for each document to be compressed. The importance evaluation unit is configured to evaluate the importance of each document to be compressed according to the perplexity of each document to be compressed; wherein the importance is positively correlated with the perplexity.

[0150] In another exemplary embodiment, the perplexity calculation unit further includes a keyword-gram selection subunit and a perplexity determination subunit. The keyword-gram selection subunit is configured to select keyword-grams from the word-grams included in the input question. The perplexity determination subunit is configured to determine the perplexity of the document to be compressed based on the number of word-grams in the input question and the probability of the keyword-grams appearing in the document to be compressed.

[0151] In another exemplary embodiment, the compression device of the large language model prompt text also includes a document splicing module, a vector splicing module, a semantic feature vector determination module and a semantic enhancement module. The document splicing module is configured to convert a first number of documents to be compressed into a first embedding vector after splicing, and to convert a second number of compressed documents into a second embedding vector after splicing. The vector splicing module is configured to splice the first embedding vector and the second embedding vector to obtain a spliced ​​vector if the semantic loss of the compressed document is serious. The semantic feature vector determination module is configured to determine the first semantic feature vector based on the first embedding vector and the spliced ​​vector, and to determine the second semantic feature vector based on the second embedding vector and the spliced ​​vector. The semantic enhancement module is configured to map the first semantic feature vector and the second semantic feature vector to a common feature space, so as to analyze the similarity between the semantic features contained in each semantic feature vector in the common feature space, and to semantically enhance each semantic feature vector based on the similarity.

[0152] In another exemplary embodiment, the compression apparatus for large language model prompt text further includes a calculation module and a semantic loss determination module. The calculation module is configured to calculate a difference vector between the first embedding vector and the second embedding vector, and a norm of the difference vector. The semantic loss determination module is configured to determine the semantic loss of the compressed document according to the norm of the difference vector.

[0153] In another exemplary embodiment, the input module 940 further includes a sorting unit and an input unit. The sorting unit is configured to sort the second number of documents to be compressed in descending order of importance based on the importance of each of the second number of documents to be compressed to obtain a sequence of documents to be compressed. The input unit is configured to input the input question and the sequence of documents to be compressed into the large language model.

[0154] In another exemplary embodiment, the compression device of the large language model prompt text further includes a longest common subsequence acquisition module and a target document extraction input module. The longest common subsequence acquisition module is configured to obtain the longest common subsequence between the first sequence and the second sequence. The target document extraction input module is configured to extract the target document from the longest common subsequence and use the target document as the compressed document to be input into the language model, so that the large language model outputs an answer; the target document does not exist in the third sequence.

[0155] Fig.10 This is an overall block diagram of a large language model prompt text compression shown in an exemplary embodiment of the present application, which includes a question-aware compression module 1010 and a post-processing module 1020. The question-aware compression module 1010 includes a question-oriented high-level compression component, a question-oriented low-level compression component, an adaptive compression ratio calculator, and a semantic enhancement engine. The post-processing module 1020 includes a question reordering component and a keyword recovery component.

[0156] It should be noted that the compression device for the large language model prompt text provided in the above embodiment and the compression method for the large language model prompt text provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment and will not be repeated here. In actual application, the compression device for the large language model prompt text provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0157] An embodiment of the present application also provides an electronic device, comprising: one or more processors; a storage device for storing one or more programs, and when the one or more programs are executed by one or more processors, the electronic device implements the large language model prompt text compression method provided in the above-mentioned embodiments.

[0158] Fig.11 The structure diagram of the computer system suitable for implementing the electronic device of the embodiment of the present application is shown. It should be noted that: Fig.11 The computer system 1200 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0159] like Fig.11 As shown, the computer system 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1202 or the program loaded from the storage part 1208 to the random access memory (RAM) 1203, such as executing the method described in the above embodiment. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, the ROM 1202 and the RAM 1203 are connected to each other through the bus 1204. The input / output (I / O) interface 1205 is also connected to the bus 1204.

[0160] The following components are connected to the I / O interface 1205: an input section 1206 including a keyboard, a mouse, etc.; an output section 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1208 including a hard disk, etc.; and a communication section 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1209 performs communication processing via a network such as the Internet. A drive 1210 is also connected to the I / O interface 1205 as needed. A removable medium 1211, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1210 as needed so that a computer program read therefrom is installed into the storage section 1208 as needed.

[0161] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication section 1209, and / or installed from a removable medium 1211. When the computer program is executed by a central processing unit (CPU) 1201, various functions defined in the system of the present application are executed.

[0162] It should be noted that the computer-readable medium shown in the embodiment of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (Erasable Programmable Read On ly Memory, EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (Compact Disc Read-Only Memory, CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable computer program is carried. This propagated data signal can take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. A computer program contained on a computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0163] The flowchart and block diagram in the accompanying drawings illustrate the possible architecture, functions and operations of the system, method and computer program product according to various embodiments of the present application. Wherein, each box in the flowchart or block diagram can represent a module, a program segment, or a part of the code, and the above-mentioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0164] The units involved in the embodiments described in this application may be implemented by software or hardware, and the units described may also be set in a processor. The names of these units do not, in some cases, constitute limitations on the units themselves.

[0165] Another aspect of the present application also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the method for compressing the large language model prompt text as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiment, or may exist independently without being assembled into the electronic device.

[0166] Another aspect of the present application also provides a computer program product or a computer program, which includes a computer instruction stored in a computer-readable storage medium. A processor of a computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method for compressing a large language model prompt text provided in each of the above embodiments.

[0167] The above content is only a preferred exemplary embodiment of the present application and is not intended to limit the implementation scheme of the present application. A person skilled in the art can easily make corresponding changes or modifications based on the main concept and spirit of the present application. Therefore, the scope of protection of the present application shall be based on the scope of protection required by the claims.

Claims

1. A method for compressing a large language model prompt text, characterized in that: include: generating a first number of documents to be compressed based on the input question; evaluating the importance of each of the documents to be compressed, and selecting a second number of documents to be compressed based on the importance; Based on the importance of each of the second number of documents to be compressed, respectively assign corresponding compression ratios to the second number of documents to be compressed, and respectively perform compression processing on the second number of documents to be compressed based on the assigned compression ratios to obtain the second number of compressed documents; The input question and the second number of compressed documents are input into a large language model, so that the large language model outputs an answer corresponding to the input question.

2. The method according to claim 1, characterized in that The step of evaluating the importance of each of the documents to be compressed and selecting a second number of documents to be compressed based on the importance includes: Calculating the perplexity of the input question for each of the documents to be compressed; According to the perplexity of each of the documents to be compressed, the importance of each of the documents to be compressed is evaluated; wherein the importance is positively correlated with the perplexity.

3. The method according to claim 2, characterized in that The calculating the perplexity of the input question for each of the documents to be compressed includes: Selecting a keyword word from the word words included in the input question; The perplexity of the document to be compressed is determined according to the number of word units in the input question and the probability of the key word units appearing in the document to be compressed.

4. The method according to claim 1, characterized in that: After obtaining the second number of compressed documents, the method further includes: Concatenating the first number of documents to be compressed into a first embedding vector, and concatenating the second number of compressed documents into a second embedding vector; If the semantic loss of the compressed document is serious, concatenating the first embedding vector and the second embedding vector to obtain a concatenated vector; Determine a first semantic feature vector according to the first embedding vector and the concatenated vector, and determine a second semantic feature vector according to the second embedding vector and the concatenated vector; The first semantic feature vector and the second semantic feature vector are mapped to a common feature space, so as to analyze the similarity between the semantic features contained in each semantic feature vector in the common feature space, and semantic enhancement is performed on each semantic feature vector based on the similarity.

5. The method according to claim 4, characterized in that The method further comprises: Calculating a difference vector between the first embedding vector and the second embedding vector, and a norm of the difference vector; The semantic loss of the compressed document is determined according to the norm of the difference vector.

6. The method according to claim 1, characterized in that Inputting the input question and the second number of compressed documents into the large language model so that the large language model outputs an answer corresponding to the input question includes: Based on the importance of each of the second number of documents to be compressed, the second number of documents to be compressed are sorted in descending order according to their importance, so as to obtain a sequence of documents to be compressed; The input question and the to-be-compressed document sequence are input into the large language model.

7. The method according to claim 1, characterized in that The first number of documents to be compressed are spliced ​​to form a first sequence, the answers corresponding to the input questions are spliced ​​to form a second sequence, and the second number of compressed documents are spliced ​​to form a third sequence; the method further includes: Obtaining the longest common subsequence between the first sequence and the second sequence; A target document is extracted from the longest common subsequence, and the target document is used as a compressed document to be input into the language model, so that the large language model outputs the answer; the target document does not exist in the third sequence.

8. A compression device for large language model prompt text, characterized in that: include: A module for generating documents to be compressed, configured to generate a first number of documents to be compressed based on the input question; an importance evaluation module, configured to evaluate the importance of each of the documents to be compressed, and screen and obtain a second number of documents to be compressed based on the importance; a compression module configured to assign corresponding compression ratios to the second number of documents to be compressed based on the importance of each of the second number of documents to be compressed, and to perform compression processing on the second number of documents to be compressed based on the assigned compression ratios, so as to obtain the second number of compressed documents; The input module is configured to input the input question and the second number of compressed documents into the large language model, so that the large language model outputs an answer corresponding to the input question.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the method for compressing a large language model prompt text as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method for compressing a large language model prompt text according to any one of claims 1 to 7.

Images