Compression method and device of prompt information, electronic equipment and storage medium

Abstract

The invention provides a compression method and device of prompt information, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of natural language processing, deep learning and the like. According to the specific implementation scheme, original first prompt information of a large model is obtained; obtaining target length constraint information; compressing the first prompt information based on the target length constraint information to obtain second prompt information of the large model; wherein the first word element number of the first prompt information is larger than the second word element number of the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information. The first prompt information is compressed through the target length constraint information to generate the short second prompt information, so that the data volume of data analysis performed by the artificial intelligence model can be reduced, the processing cost and the data processing time can be reduced, and meanwhile, the calling charge of the black box application programming interface can be reduced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of natural language processing, deep learning and the like, and more particularly to a prompt information compression method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the wide application of large language models (LLM) in natural language processing tasks, they exhibit strong reasoning and understanding capabilities in long text processing scenarios. However, when the input context increases, LLM faces the challenges of increased computational complexity and performance decline, especially in the process of key information extraction and reasoning. In addition, since black box application programming interfaces (APIs) charge by the number of tokens, lengthy prompts can lead to significant cost increases. SUMMARY

[0003] The present disclosure provides a prompt information compression method, device, electronic device and storage medium.

[0004] According to a first aspect of the present disclosure, a prompt information compression method is provided, comprising: obtaining first prompt information of a large model original; obtaining target length constraint information; compressing the first prompt information based on the target length constraint information to obtain second prompt information of the large model; wherein the first token quantity of the first prompt information is greater than the second token quantity of the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

[0005] According to a second aspect of the present disclosure, a prompt information compression device is provided, comprising: a first obtaining module configured to obtain first prompt information of a large model original; a second obtaining module configured to obtain target length constraint information; and a compression module configured to compress the first prompt information based on the target length constraint information to obtain second prompt information of the large model; wherein the first token quantity of the first prompt information is greater than the second token quantity of the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the prompt information compression method of the above-mentioned first aspect embodiment.

[0007] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, and the computer instructions are stored on the computer-readable storage medium and used to cause the computer to perform the compression method of the prompt information according to the embodiment of the first aspect.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, and the computer program product comprises computer program instructions, and the computer program instructions are executed by a processor to implement the compression method of the prompt information according to the embodiment of the first aspect.

[0009] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

[0010] Beneficial effects: the first prompt information is compressed by the target length constraint information to generate shorter second prompt information, which can reduce the data amount of data analysis by the artificial intelligence model, reduce the processing cost and data processing time, and reduce the calling charge of the black box application programming interface. BRIEF DESCRIPTION OF DRAWINGS

[0011] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0012] Figure 1 A flowchart of a compression method of prompt information provided by an embodiment of the present disclosure is provided;

[0013] Figure 2 A flowchart of another compression method of prompt information provided by an embodiment of the present disclosure is provided;

[0014] Figure 3 A flowchart of another compression method of prompt information provided by an embodiment of the present disclosure is provided;

[0015] Figure 4 A flowchart of another compression method of prompt information provided by an embodiment of the present disclosure is provided;

[0016] Figure 5 A flowchart of another compression method of prompt information provided by an embodiment of the present disclosure is provided;

[0017] Figure 6 A schematic diagram of a supervised fine-tuning SFT of a pre-trained large language model is provided for an embodiment of the present disclosure;

[0018] Figure 7 A structural schematic diagram of a prompt information compression device provided by an embodiment of the present disclosure is provided;

[0019] Figure 8A block diagram of an electronic device for a compression method of prompt information according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0020] Exemplary embodiments of the disclosure are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the disclosure by a person of ordinary skill in the art, and should not be construed as limiting the scope of the disclosure. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the disclosure. Also, the descriptions herein of arrangements and devices have been simplified for purposes of clarity and brevity.

[0021] A compression method of prompt information, an apparatus, and an electronic device according to an embodiment of the disclosure are described below with reference to the accompanying drawings.

[0022] Artificial Intelligence (AI) is a discipline that studies enabling a computer to simulate certain thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of human life, and includes both hardware and software technologies. Artificial intelligence hardware technologies generally include computer vision technology, speech recognition technology, natural language processing technology, and learning / deep learning, big data processing technology, knowledge graph technology, etc.

[0023] Natural Language Processing (NLP) is an important direction in the field of computer science and artificial intelligence. It studies various theories and methods that enable effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Natural language processing is mainly applied to machine translation, public opinion monitoring, automatic abstract, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, etc.

[0024] Deep Learning (DL) is a new research direction in the field of Machine Learning (ML), which is introduced into machine learning to make it closer to the original goal-artificial intelligence. Deep learning is to learn the internal laws and representation levels of sample data, and the information obtained in this learning process is very helpful for the interpretation of data such as text, images, and sounds. The ultimate goal of deep learning is to enable machines to have analysis and learning capabilities like people, and to be able to recognize text, images, and sound data. Deep learning is a complex machine learning algorithm, and the results achieved in speech and image recognition far exceed previous related technologies.

[0025] To solve these problems, prompt compression techniques are proposed to expand the input range of LLMs by reducing the length of prompts while maintaining or improving the performance of downstream tasks. However, the current soft and hard prompt compression methods in the art have problems such as lack of explainability, poor generalization ability, and incoherent semantics, making it difficult to achieve efficient and high-quality prompt compression in practical applications.

[0026] Figure 1 A flowchart of a prompt information compression method provided by an embodiment of the present disclosure.

[0027] As Figure 1 indicated, the prompt information compression method can include:

[0028] S101, obtaining first prompt information of an original large model.

[0029] It should be noted that the prompt information compression method of the present application can be applied to the scene of text prompt information processing, and can also be applied to the scene of sentence prompt information processing, which is not limited herein. The execution subject of the prompt information compression of the present application can be a prompt information compression device of the present application, which can be set on an electronic device. Optionally, the execution subject can include a server, a user terminal and other intelligent devices. Optionally, the user terminal includes but is not limited to a mobile phone, a computer, a smart voice interactive device, etc. Optionally, the server includes but is not limited to a network server, an application server, and can also be a server of a distributed system, or a server combined with a blockchain, etc. The present disclosure is not limited specifically.

[0030] A large model refers to a deep learning model with a very large number of parameters (usually several hundred million to several trillion parameters) trained by a large amount of data. They can understand and generate natural language, and have certain reasoning, programming, logical judgment and other abilities. The large model in the present disclosure refers to an artificial intelligence model in particular.

[0031] It should be noted that the prompt information (Prompt) refers to a text instruction or question input by a user when using an artificial intelligence model, which is used to guide the model to generate a specific type of answer, perform a task or conduct reasoning.

[0032] The prompt information can be text prompt information, and can also be voice prompt information input by a requester through a voice input device, which is not limited herein.

[0033] S102, obtaining target length constraint information.

[0034] It should be noted that the target length constraint information is a limitation condition imposed on the length of the output result in some optimization, search or generation tasks. Such constraints are usually used to control the size of the output content to ensure that it meets the needs of specific application scenarios.

[0035] The target length constraint information can be designed in advance or set by a person according to actual use requirements, which is not limited here.

[0036] In another possible implementation manner, the target length constraint information can also be changed based on the field, scene, etc. to which the input prompt text belongs. For example, the target length constraint information corresponding to the prompt text for asking questions and the target length constraint information corresponding to the prompt text for generating data can be different.

[0037] In another possible implementation manner, the target length constraint information can also include a unified format to be obtained after compression. For example, when automatically generating an advertisement script, an abstract, a classification tag, etc., the respective formats of the advertisement script, the abstract, and the classification tag are constrained in the target length constraint information. In this way, the compressed prompt information is in a unified format and has a clear structure, which is more suitable for building an automated prompt generation system.

[0038] In S103, the first prompt information is compressed based on the target length constraint information to obtain second prompt information, wherein the first token quantity of the first prompt information is greater than the second token quantity of the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

[0039] It should be noted that the compression of the first prompt information to obtain the second prompt information means that a shorter and more concise prompt text is generated under the premise of maintaining semantic integrity, so as to improve the model response efficiency, reduce token consumption or adapt to specific input restrictions.

[0040] In a possible implementation manner, the first prompt information is compressed based on the target length constraint information to obtain the second prompt information, which can be filtered based on the target length constraint information to generate the second prompt information that meets the target length constraint information.

[0041] In another possible implementation manner, the first prompt information can also be sequentially filtered in sequence according to the order of the prompt text until the second prompt information that meets the target length constraint information is filtered out.

[0042] In another possible implementation manner, the compression process is not the operation of screening and deleting the first prompt information, but also can be converting long words or word groups into short words or word groups with the same meaning, so that a shorter and more concise prompt text can be generated under the premise of ensuring semantic integrity.

[0043] In the embodiments of the present disclosure, the original first prompt information is first acquired, then the target length constraint information is acquired, and finally the first prompt information is compressed based on the target length constraint information to obtain the second prompt information, wherein the first word quantity of the first prompt information is greater than the second word quantity of the second prompt information. By compressing the first prompt information based on the target length constraint information, the data amount for data analysis by the artificial intelligence model can be reduced, the processing cost and data processing time can be reduced, and the calling charge of the black box application programming interface can be reduced.

[0044] In the embodiments of the present disclosure, the first prompt information is compressed based on the target length constraint information to obtain the second prompt information, and the word quantity difference between the first word quantity and the second word quantity can be determined. In response to the word quantity difference being less than a first set value, the first prompt information is re-compressed until the word quantity difference corresponding to the second prompt information is greater than or equal to the first set value.

[0045] It should be noted that the re-compression refers to re-compressing the first prompt information to reduce the word quantity of the first prompt information and realize the simplification of the first prompt information. The re-compression is not only one round, but needs to be determined according to the result of the re-compression, for example, the current demand or end condition cannot be met after re-compression, and the re-compression needs to be performed again according to the current compression result.

[0046] In a possible implementation manner, the word group or sentence with a smaller word quantity can be replaced by the word group or sentence with the same semantic in the first prompt information through re-compression.

[0047] In another possible implementation manner, the words in the first prompt information that have little influence on instructions or semantics can also be deleted through re-compression.

[0048] In another possible implementation manner, in response to the word quantity difference being less than the first set value, a re-compression instruction is generated and displayed. By judging whether the word quantity difference before and after the compression of the prompt information reaches a preset threshold, it is determined whether to perform re-compression, so as to ensure that each compression has a significant effect, improve the model use efficiency, reduce the cost, and enhance the intelligence and stability of the prompt information generation system.

[0049] It should be noted that the re-compression indication is used to indicate whether the current compressed first prompt information should be re-compressed to further reduce the volume or adapt to the specific format requirement. For example, if the current first prompt information after re-compression meets the condition that the difference in the number of tokens is less than the first set value, the re-compression indication can be "no need to re-compress", or if the current first prompt information after re-compression still does not meet the condition that the difference in the number of tokens is less than the first set value, the re-compression indication can be "still need to re-compress", and the current compressed first prompt information is re-compressed until the condition that the difference in the number of tokens is less than the first set value is met.

[0050] It should be noted that the first set value is determined according to the target length constraint information, and the first set value can also be included in the target length constraint information.

[0051] In the above embodiment, the first prompt information is compressed based on the target length constraint information to obtain the second prompt information, and the second prompt information can also be obtained by Figure 2 Further explanation, the method comprises:

[0052] S201, determining a generated previous token sequence in the second prompt information.

[0053] S202, generating a next token in the second prompt information according to the first prompt information, the target length constraint information and the generated previous token sequence in the second prompt information, to obtain the second prompt information.

[0054] It should be noted that the token is the smallest semantic or grammatical unit used by the model when processing text. It should be noted that the token is not equal to the Chinese character, letter, word or punctuation itself, but is determined according to the tokenizer strategy of the model. For example, "Hello world!" can be divided into three tokens "Hello", "world" and "!", and "Transformer-based model" can be divided into four tokens "Trans", "former", "-based" and "model". It should be noted that different artificial intelligence models can have different results when dividing the prompt text. For example, model A divides "Transformer-based model" into four tokens "Trans", "former", "-based" and "model", and model B divides "Transformer-based model" into five tokens "Tran", "sfor", "mer-", "base", "dmod" and "el".

[0055] It can be understood that the i-th word unit in the second prompt information is obtained by sequentially compressing the first prompt information, and thus the sequence of the previously generated word units in the second prompt information in the present disclosure is a sequence of word units generated before the i-th word unit, where i is an integer, 2≤i≤M, and M is the number of second word units.

[0056] In the embodiments of the present disclosure, the method for generating word units based on the prompt information input into the artificial intelligence model can be various, and is not limited herein.

[0057] Optionally, the word boundaries of the prompt text can be divided by a preset division rule to generate the word units. The division rule can be changed according to actual design needs, and is not limited herein. For example, a regular expression can be used to match a specific pattern to obtain the word units from the prompt information.

[0058] Optionally, the prompt information can also be segmented by a preset predefined dictionary or vocabulary to generate the word units.

[0059] Optionally, the prompt information can also be processed by a preset division model to generate the word units. The division model can be a neural network model, for example, a bidirectional long short-term memory (BiLSTM) model, a Transformer model, etc.

[0060] In the embodiments of the present disclosure, after obtaining the sequence of the previously generated word units, the method for generating the i-th word unit in the second prompt information based on the first prompt information, the target length constraint information, and the sequence of the previously generated word units can be various.

[0061] In one possible implementation, since the prompt text is a text with semantics or some requirements, the second prompt information generated after compressing the first prompt information still needs to maintain the same semantics or requirements as the first prompt information. Therefore, when sequentially compressing the word units of the first prompt information, the subsequent word compression rules need to be adjusted according to the semantics of the sequence of the previously generated word units of the second prompt information, so as to ensure the integrity of the semantics of the second prompt information and the consistency between the second prompt information and the first prompt information. At the same time, the finally generated second prompt information also needs to be constrained according to the target length constraint information to meet the length requirements, format requirements, etc. in the target length constraint information.

[0062] In the embodiments of the present disclosure, the generated previous token sequence in the second prompt information is first determined, and then the next token in the second prompt information is generated according to the first prompt information, the target length constraint information and the generated previous token sequence in the second prompt information, to obtain the second prompt information. In this way, by using the first prompt information, the target length constraint information and the previous token sequence, the next token in the second prompt information is generated, which can dynamically generate high-quality prompt information meeting the length constraint without losing the semantic of the task, effectively improve the model usage efficiency, reduce the cost, and enhance the controllability and practicality of the prompt information generation.

[0063] In the above embodiments, the first prompt information is compressed based on the target length constraint information to obtain the second prompt information of the large model, and the second prompt information can also be obtained by Figure 3 Further explanation,

[0064] S301, input the first prompt information and the target length constraint information into the generative target compression model.

[0065] It should be noted that the target compression model in the embodiments of the present disclosure can be an autoregressive generative model. The autoregressive generative model (Autoregressive Generative Model) is a kind of deep learning model widely used in natural language processing, speech synthesis, image generation and other fields.

[0066] In the embodiments of the present disclosure, the next token is predicted step by step according to the existing first prompt information, so as to generate the complete second prompt information.

[0067] The target compression model in the embodiments of the present disclosure is trained in advance and can be stored in the storage space of the electronic device to facilitate retrieval and use when needed.

[0068] For example, given a first prompt information sequence x=(x1, x2,..., xn), where n represents the number of tokens of the original prompt, and the compressed second prompt information sequence is Where m represents the number of tokens of the compressed prompt, and m<<n. The compression rate τ is defined as m / n, and the value range is (0, 1].

[0069] The data expression of the target compression model is:

[0070]

[0071] Where: C θ represents a lightweight compressor model with parameters θ, L t represents the target length constraint, The generation of depends not only on the first prompt information x, but also on the previously generated token sequence.

[0072] As can be seen from the above expression, the target compression model in the present solution can ensure that the generated compressed prompt maintains semantic coherence. Compared with traditional methods, the generative framework has the following advantages: 1) allows new expression methods to be generated, rather than simply deleting tokens; 2) maintains the natural language characteristics of the generated text; and 3) supports dynamic length control.

[0073] In S302, the next token in the second prompt information is predicted by the target compression model according to the first prompt information, the target length constraint information, and the generated previous token sequence in the second prompt information.

[0074] In the embodiments of the present disclosure, the first prompt information and the target length constraint information are first input into the generative target compression model, and then the next token in the second prompt information is predicted by the target compression model according to the first prompt information, the target length constraint information, and the generated previous token sequence in the second prompt information. In this way, by processing the first prompt information and the target length constraint information through the target compression model to generate the second prompt information, it can be ensured that the generated compressed prompt maintains semantic coherence, and the compression efficiency of the first prompt information can be improved.

[0075] In the above embodiments, the target length constraint information can also be obtained by Figure 4 Further explanation, the method comprises:

[0076] In S401, a target downstream task associated with the first prompt information is determined.

[0077] In implementation, the requirements of different tasks for prompt information can be quite different, and the target length constraint information corresponding to different tasks can be different. Therefore, starting from the task, the target length constraint information is deduced, which can be adapted to different models or application scenarios.

[0078] The target downstream task can be various. For example, the target downstream task can include a classification task, an abstract generation, a programming assistance, etc.

[0079] In S402, the target length constraint information is determined according to the target downstream task.

[0080] In the embodiments of the present disclosure, the target length constraint information can be determined according to different target downstream tasks by setting the target length constraint information-task mapping rule in advance. For example, the target length constraint information range of the classification task can be 50-100 tokens, the target length constraint information range of the abstract generation can be 100-300 tokens, and the target length constraint information range of the programming assistance can be 100-400 tokens.

[0081] In a possible implementation manner, the model computing power information of the task model corresponding to the target downstream task can be determined first, then the task demand information of the target downstream task is determined, and finally the target length constraint information is determined according to at least one of the task demand information and the model computing power information.

[0082] It should be noted that the task demand information is a limitation condition of the task to be executed at present, for example, a minimum / ideal prompt information length required to complete the task, an output format requirement, a content complexity, and the like.

[0083] The model computing power information of the task model is the processing capability of the model, for example, a maximum supported token number, a response delay, whether to support parallel computing, and the like.

[0084] The above embodiment is a more refined and systematic prompt information length constraint generation method. It not only considers the task demand information of the downstream task, but also combines the computing power limitation of the model itself, for example, the inference speed, the maximum context support, and the like, so as to realize a target length constraint information determination mechanism based on the "task-model" double factors.

[0085] Optionally, the target length constraint information can be determined only according to the task demand information. For example, the task demand information indicates that the current task allows a relatively long response time, and a relatively large target length constraint information can be selected; the task demand information indicates that the current task allows a relatively short response time, and the target length constraint information can be appropriately increased.

[0086] Optionally, the target length constraint information can be determined only according to the model computing power information of the task model. For example, the available computing power of the task model is relatively large, and a relatively small target length constraint information can be selected; the available computing power of the task model is relatively small, and the target length constraint information can be appropriately reduced.

[0087] Optionally, the target length constraint information can be determined by comprehensively considering the task demand information and the model computing power information. In the embodiment of the present disclosure, the method for determining the target length constraint information by comprehensively considering the task demand information and the model computing power information can be various. For example, a weight value can be assigned to the task demand information and the model computing power information respectively, and then the target length constraint information is determined by comprehensively calculating based on the task demand information, the model computing power information and the respective weight values.

[0088] In the embodiment of the present disclosure, the target downstream task associated with the first prompt information is determined first, and then the target length constraint information is determined according to the target downstream task. In this way, by identifying the downstream task corresponding to the first prompt information and determining the target length constraint information accordingly, intelligent compression and optimization of the prompt information can be realized, so as to improve the model calling efficiency, reduce the cost, and enhance the system adaptability and intelligent level.

[0089] In the above embodiments, the obtaining process of the generative target compression model can also be performed by Figure 5 Further explanation, the method comprises:

[0090] S501, obtaining a training sample, wherein the training sample comprises sample prompt information and reference prompt information corresponding to the sample prompt information.

[0091] It should be noted that the sample prompt information is the original prompt information that needs to be input to the artificial intelligence model, and the reference prompt information corresponding to the sample prompt information is the reference prompt information output by the artificial intelligence model based on the sample prompt information under ideal conditions.

[0092] S502, determining sample constraint length information of the training sample.

[0093] In the embodiments of the present disclosure, the determination of the sample constraint length information of the training sample can refer to the content in the above embodiments, which will not be repeated here.

[0094] S503, performing supervised fine-tuning SFT on the pre-trained large language model according to the training sample and the sample constraint length information, to obtain a target compression model.

[0095] Supervised fine-tuning (Supervised Fine-Tuning, SFT) refers to further fine-tuning using a labeled high-quality data set based on a pre-trained language model. In this way, the model can better adapt to specific tasks or application scenarios, and improve the accuracy of the model.

[0096] In one possible implementation, supervised fine-tuning (Supervised Fine-Tuning, SFT) can use a labeled supervised data set to further train the model based on an existing large-scale pre-trained language model, so that it better adapts to a specific task (such as question and answer, summary, translation, code generation, etc.).

[0097] In the embodiments of the present disclosure, the supervised fine-tuning SFT is performed on the pre-trained large language model according to the training sample and the sample constraint length to obtain a generative target compression model. First, the training sample and the sample constraint length information are input into the large language model. The large language model outputs the predicted prompt information corresponding to the sample prompt information according to the training sample and the sample constraint length information. Then, cross-entropy loss calculation is performed on the predicted prompt information and the reference prompt information to determine the loss function of the large language model. Finally, the model parameters of the large language model are adjusted according to the loss function until the SFT ends, and the generative target compression model is obtained.

[0098] In the embodiments of the present disclosure, the next word in the predicted prompt information is generated by the large language model according to the sample prompt information, the sample length constraint information, and the previously uploaded sequence of reference word units in the reference prompt information, to obtain the predicted prompt information.

[0099] The cross-entropy loss function is used for optimization, which can ensure that the compressed prompt generated by the model is highly consistent with the reference prompt at the word unit level.

[0100] In the embodiments of the present disclosure, the process of supervised fine-tuning SFT of the pre-trained large language model can be as shown in Figure 6

[0101] Wherein, the data set D represents the data set used for supervised fine-tuning, which contains input and corresponding expected output.

[0102] Pre-trained LMC θ0 : represents inputting the sample (x) in the data set into a pre-trained large language model (Pre-trained Language Model, LM), denoted as C θ0 . The model has been preliminarily trained through a large amount of text data and has certain language understanding and generation ability.

[0103] SFT process: Next, the pre-trained model C θ0 is further optimized through supervised fine-tuning. In this process, the model parameters are updated from θ0 to θ1, and then to θ2, that is, C θ1 → C θ2 . This shows that the model is constantly learning the knowledge of specific tasks or application scenarios in the data set D.

[0104] Loss function (L SFT ): The supervised fine-tuning uses a loss function L SFT , which measures the difference between the output generated by the model and the expected output in the data set. By minimizing this loss function, the model can gradually adjust its parameters to better adapt to specific tasks.

[0105] SFT LM: After supervised fine-tuning, the final result is a language model optimized for specific tasks, called SFT LM. The performance of this model on specific tasks is usually better than that of the original pre-trained model.

[0106] For example, given the original prompt x and the reference compressed prompt x_ref = (y1, y2,..., y_m), the loss function is defined as follows:

[0107]

[0108] Where: I(y i ​(j) is an indicator function that takes value 1 when the i-th token of the reference prompt equals the j-th token in the vocabulary V, and 0 otherwise; C θ (j | x, y < i) represents the probability of the j-th token predicted by the model given the first i-1 tokens; |V| represents the size of the vocabulary.

[0109] The cross-entropy loss function has the following technical advantages: (1) accurate token-level optimization: directly minimizing the difference between the model prediction distribution and the one-hot distribution of the reference prompt ensures that each generated token is as close as possible to the reference standard. (2) stable gradient propagation: compared to sequence-level indicators such as ROUGE-L, cross-entropy loss provides more direct and stable gradient signals. (3) high training efficiency: perfectly matched with the autoregressive generation process, supporting parallelized computation.

[0110] Corresponding to the prompt information compression method provided by the above-mentioned several embodiments, an embodiment of the present disclosure also provides a prompt information compression device. Since the prompt information compression device provided by the embodiment of the present disclosure corresponds to the prompt information compression method provided by the above-mentioned several embodiments, the implementation of the above-mentioned prompt information compression method is also applicable to the prompt information compression device provided by the embodiment of the present disclosure, which will not be described in detail in the following embodiments.

[0111] Figure 7 A structural schematic diagram of a prompt information compression device provided by an embodiment of the present disclosure. As shown in the figure, the prompt information compression device comprises a first acquisition module 710, a second acquisition module 720 and a compression module 730. Figure 7

[0112] The first acquisition module 710 is configured to acquire the first prompt information of the large model.

[0113] The second acquisition module 720 acquires target length constraint information.

[0114] The compression module 730 is configured to compress the first prompt information based on the target length constraint information to obtain the second prompt information of the large model, wherein the first token quantity of the first prompt information is greater than the second token quantity of the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

[0115] In an embodiment of the present disclosure, the compression module 730 is further configured to generate the next token in the second prompt information according to the first prompt information, the target length constraint information and the generated previous token sequence in the second prompt information, to obtain the second prompt information.

[0116] ​In an embodiment of the disclosure, the compression module 730 is further configured to: input the first prompt information and the target length constraint information into a generative target compression model; and predict, by the generative target compression model, a next token in the second prompt information according to the first prompt information, the target length constraint information, and a generated prefix token sequence in the second prompt information.

[0117] In an embodiment of the disclosure, the second acquisition module 720 is further configured to: determine a target downstream task associated with the first prompt information; and determine the target length constraint information according to the target downstream task.

[0118] In an embodiment of the disclosure, the second acquisition module 720 is further configured to: determine model computing power information of a task model corresponding to the target downstream task; determine task demand information of the target downstream task; and determine the target length constraint information according to at least one of the task demand information and the model computing power information.

[0119] In an embodiment of the disclosure, the compression module 730 is further configured to: determine a token quantity difference between the first token quantity and the second token quantity; and in response to the token quantity difference being less than a first set value, re-compress the first prompt information until a token quantity difference corresponding to the second prompt information is greater than or equal to the first set value.

[0120] In an embodiment of the disclosure, the compression module 730 is further configured to: in response to the token quantity difference being less than the first set value, generate a re-compression indication and display the re-compression indication.

[0121] In an embodiment of the disclosure, a process of obtaining the generative target compression model includes: obtaining training samples, wherein the training samples include sample prompt information and reference prompt information corresponding to the sample prompt information; determining sample constraint length information of the training samples; and performing supervised fine-tuning SFT on a pre-trained large language model according to the training samples and the sample constraint length information to obtain the target compression model.

[0122] In an embodiment of the disclosure, performing the supervised fine-tuning SFT on the pre-trained large language model according to the training samples and the sample constraint length to obtain the generative target compression model includes: inputting the training samples and the sample constraint length information into the large language model, and outputting, by the large language model, predicted prompt information corresponding to the sample prompt information according to the training samples and the sample constraint length information; performing cross-entropy loss calculation on the predicted prompt information and the reference prompt information to determine a loss function of the large language model; and adjusting model parameters of the large language model according to the loss function until the SFT ends, to obtain the target compression model.

[0123] In one embodiment of the present disclosure, the device is further configured to generate, by the large language model, a next word element in the predicted prompt information according to the sample prompt information, the sample length constraint information, and the pre-sequence reference word element sequence uploaded in the reference prompt information, to obtain the predicted prompt information.

[0124] The first prompt information is compressed by the target length constraint information to generate the shorter second prompt information, which can reduce the data amount for data analysis by the artificial intelligence model, reduce the processing cost and data processing time, and reduce the calling charge of the black box application programming interface.

[0125] In the technical solution of the present disclosure, the acquisition, storage, and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0126] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0127] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0128] As shown in Figure 8 The device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to computer programs / instructions stored in a read-only memory (ROM) 802 or loaded into a random access memory (RAM) 803 from a storage unit 806. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0129] A plurality of components in the device 800 are connected to the I / O interface 805, including: an input unit 806 such as a keyboard, a mouse, etc.; an output unit 807 such as various types of displays, speakers, etc.; a storage unit 808 such as a magnetic disk, an optical disk, etc.; and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0130] The computing unit 801 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs various methods and processes described above, such as the compression method of prompt information. For example, in some embodiments, the compression method of prompt information can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 806. In some embodiments, part or all of the computer program / instructions can be loaded and / or installed onto the device 800 via the ROM 802 and / or the communication unit 809. When the computer program / instructions are loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the compression method of prompt information described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the compression method of prompt information by any other appropriate means, such as by means of firmware.

[0131] Various implementations of the systems and techniques described above herein can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs / instructions that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0132] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0133] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0135] The systems and techniques described herein can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0136] The computer system can include clients and servers. This description can be mainly described in the context of a client-server relationship. However, distribution of functionality between separate elements is also within the scope of the present disclosure. Thus, variations of these embodiments are within the scope of the present disclosure. The servers can be cloud servers, servers of a distributed system, or servers combined with a blockchain.

[0137] It should be understood that various forms of flow shown above can be used with orders of the steps re-ordered; added steps; or steps deleted, such that the skilled person will understand that the steps recited in the disclosure can be executed in parallel, executed in sequence, or in different orders, without limitation, as long as the desired results of the technology disclosed in the present disclosure are achieved.

[0138] The specific embodiments described above do not constitute an exhaustive list of all possible embodiments as modifications can be made by one skilled in the art, especially in light of the above teachings. Any and every modification, equivalent replacement, and improvement made within the spirit and principles of the present disclosure are to be encompassed within the scope of the present disclosure.

Claims

1. A method for compressing prompt information, wherein: The method comprises: Get the original first prompt information of the large model; Get target length constraint information; compressing the first prompt information based on the target length constraint information to obtain second prompt information of the large model; The number of first word units in the first prompt information is greater than the number of second word units in the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

2. The method according to claim 1, wherein The compressing the first prompt information based on the target length constraint information to obtain the second prompt information includes: The next word in the second prompt information is generated according to the first prompt information, the target length constraint information, and the preceding word sequence generated in the second prompt information to obtain the second prompt information.

3. The method according to claim 1, wherein The compressing the first prompt information based on the target length constraint information to obtain the second prompt information of the large model includes: Inputting the first prompt information and the target length constraint information into a generative target compression model; The target compression model predicts the next word in the second prompt information according to the first prompt information, the target length constraint information and the preceding word sequence generated in the second prompt information.

4. The method according to any one of claims 1 to 3, wherein The obtaining of target length constraint information includes: Determining a target downstream task associated with the first prompt information; The target length constraint information is determined according to the target downstream task.

5. The method according to claim 4, wherein The determining the target length constraint information according to the target downstream task includes: Determine model computing power information of the task model corresponding to the target downstream task; Determining task requirement information of the target downstream task; The target length constraint information is determined based on at least one of the task requirement information and the model computing power information.

6. The method according to any one of claims 1 to 3, wherein The method further comprises: determining a word-unit quantity difference between the first word-unit quantity and the second word-unit quantity; In response to the word unit quantity difference being less than a first set value, the first prompt information is recompressed until the word unit quantity difference corresponding to the second prompt information is greater than or equal to the first set value.

7. The method according to claim 6, characterized in that The method further comprises: In response to the word unit quantity difference being smaller than a first set value, a re-compression instruction is generated and displayed.

8. The method according to claim 3, wherein: The process of obtaining the generated target compression model includes: Acquire a training sample, wherein the training sample includes sample prompt information and reference prompt information corresponding to the sample prompt information; Determining sample constraint length information of the training sample; According to the training samples and the sample constraint length information, supervised fine-tuning SFT is performed on the pre-trained large language model to obtain the target compression model.

9. The method according to claim 8, wherein The step of performing supervised fine-tuning (SFT) on the pre-trained large language model according to the training sample and the sample constraint length to obtain the target compression model includes: Inputting the training sample and the sample constraint length information into the large language model, and having the large language model output prediction prompt information corresponding to the sample prompt information based on the training sample and the sample constraint length information; Performing cross entropy loss calculation on the predicted prompt information and the reference prompt information to determine a loss function of the large language model; The model parameters of the large language model are adjusted according to the loss function until the SFT is completed to obtain the target compression model.

10. The method according to claim 9, wherein: The method comprises: The large language model generates the next word in the predicted prompt information according to the sample prompt information, the sample length constraint information and the pre-order reference word sequence uploaded in the reference prompt information to obtain the predicted prompt information.

11. A device for compressing prompt information, comprising: A first acquisition module is used to obtain the original first prompt information of the large model; A second acquisition module is used to obtain target length constraint information; a compression module, configured to compress the first prompt information based on the target length constraint information to obtain second prompt information of the large model; The number of first word units in the first prompt information is greater than the number of second word units in the second prompt information, and the semantics of the second prompt information is related to the semantics of the first prompt information.

12. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

13. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the prompt information compression method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instruction is executed by a processor, the prompt information compression method according to any one of claims 1 to 10 is implemented.

Images