Method and device for training compression model and method and device for processing information based on compression model

Abstract

The embodiment of the invention discloses a method and device for training a compression model and processing information based on the compression model. A specific embodiment of the method comprises the following steps: retrieving a document set related to information to be processed; extracting an abstract statement set related to the to-be-processed information from the document set; and inputting the to-be-processed information and the abstract statement set into a pre-trained compression model, and outputting a target statement set after statement quantity compression. According to the embodiment, the abstract generation accuracy and continuity can be improved.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to methods and devices for training compression models and processing information based on the compression models. Background Art

[0002] Current Retrieval-Augmented Generation (RAG) systems demonstrate strong capabilities in summarization tasks, supplementing external knowledge by retrieving relevant documents, thereby improving the content completeness, factual consistency, and information coverage of summaries. However, the performance of RAG systems is highly dependent on the quality of the retrieved documents. When the retrieved content contains a large amount of noise or irrelevant information, the accuracy and coherence of the summary can be severely affected. Furthermore, processing redundant information significantly increases computational costs, slowing the inference process and limiting the practical application of RAG systems in summarization tasks. Summary of the Invention

[0003] The embodiments of the present disclosure provide methods and devices for training a compression model and processing information based on the compression model.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for processing information based on a compression model, comprising: retrieving a document collection related to the information to be processed; extracting a summary sentence collection related to the information to be processed from the document collection; inputting the information to be processed and the summary sentence collection into a pre-trained compression model, and outputting a target sentence collection with a compressed number of sentences.

[0005] In some embodiments, extracting a set of summary sentences related to the information to be processed from a document collection includes: inputting sentences in the document collection and the information to be processed into a pre-trained extraction model to obtain the relevance of each sentence to the information to be processed; and outputting sentences whose relevance is greater than a predetermined threshold as the set of summary sentences.

[0006] In some embodiments, the information to be processed and the set of summary sentences are input into a pre-trained compression model, and a target sentence set with a compressed number of sentences is output, including: inputting the information to be processed and the set of summary sentences into the pre-trained compression model to determine the number of compressed sentences; and screening out summary sentences with the highest number of relevant sentences from the set of summary sentences as the target sentence set and outputting them.

[0007] In some embodiments, the method further includes: inputting the information to be processed and the target sentence set into a dialogue generation model, and outputting suggestion information.

[0008] In a second aspect, an embodiment of the present disclosure provides a method for training a compression model, comprising: obtaining training data, wherein the training data includes sample information, a set of sample summary sentences, and a compression rate label; inputting the sample information and the set of sample summary sentences into the compression model to obtain a predicted compression rate; and adjusting the network parameters of the compression model based on the difference between the predicted compression rate and the compression rate label.

[0009] In some embodiments, obtaining training data includes: obtaining sample information, a set of sample summary sentences, and true answers to the sample information, wherein the set of sample summary sentences is obtained by extracting sentences related to the sample information from a set of documents related to the sample information; constructing multiple subsets based on the set of sample summary sentences, wherein the number of sentences in the multiple subsets is different; obtaining predicted answers corresponding to the multiple subsets through a dialogue generation model, and determining the number of sentences in the subsets in which the predicted answers successfully match the true answers; and determining a compression rate label based on the minimum number of sentences in the subsets that successfully match.

[0010] In some embodiments, the number of the plurality of subsets is N, where N is a natural number greater than 1; constructing the plurality of subsets according to the set of sample summary sentences comprises: selecting the first n sample summary sentences from the set of sample summary sentences in descending order of relevance to the sample information to generate the nth subset, and n [1, N]; and obtaining predicted answers corresponding to multiple subsets through a dialogue generation model, and determining the number of sentences in the subsets in which the predicted answers successfully match the true answers, including: setting n to N, and performing the following matching steps: inputting the information to be processed and the sample summary sentences in the nth subset into the dialogue generation model, obtaining the predicted answers of the nth subset, and matching the predicted answers of the nth subset with the true answers; in response to a successful match and n≠1, setting n to n-1, and continuing to perform the matching step; in response to a failed match and n≠N, determining that the minimum number of sentences in the successfully matched subset is n+1.

[0011] In a third aspect, an embodiment of the present disclosure provides a device for processing information based on a compression model, comprising: a retrieval unit configured to retrieve a document set related to the information to be processed; an extraction unit configured to extract a summary sentence set related to the information to be processed from the document set; and a compression unit configured to input the information to be processed and the summary sentence set into a pre-trained compression model, and output a target sentence set with the number of sentences compressed.

[0012] In some embodiments, the extraction unit is further configured to: input the sentences and information to be processed in the document collection into a pre-trained extraction model to obtain the relevance of each sentence with the information to be processed; and output the sentences whose relevance is greater than a predetermined threshold as a summary sentence set.

[0013] In some embodiments, the compression unit is further configured to: input the information to be processed and the summary sentence set into a pre-trained compression model to determine the number of compressed sentences; and filter out the summary sentences with the highest number of relevance from the summary sentence set as the target sentence set for output.

[0014] In some embodiments, the apparatus further includes a generation unit configured to: input the information to be processed and the target sentence set into the dialogue generation model, and output suggestion information.

[0015] In a fourth aspect, an embodiment of the present disclosure provides a device for training a compression model, comprising: an acquisition unit configured to acquire training data, wherein the training data includes sample information, a set of sample summary sentences, and a compression rate label; a prediction unit configured to input the sample information and the set of sample summary sentences into the compression model to obtain a predicted compression rate; and an adjustment unit configured to adjust the network parameters of the compression model according to the difference between the predicted compression rate and the compression rate label.

[0016] In some embodiments, the acquisition unit is further configured to: acquire sample information, a set of sample summary sentences, and true answers to the sample information, wherein the set of sample summary sentences is obtained by extracting sentences related to the sample information from a set of documents related to the sample information; construct multiple subsets based on the set of sample summary sentences, wherein the number of sentences in the multiple subsets is different; obtain predicted answers corresponding to the multiple subsets through a dialogue generation model, and determine the number of sentences in the subsets in which the predicted answers successfully match the true answers; and determine a compression rate label based on the minimum number of sentences in the subsets that successfully match.

[0017] In some embodiments, the number of the plurality of subsets is N, where N is a natural number greater than 1; the acquisition unit is further configured to: select the first n sample summary sentences from the sample summary sentence set in descending order of relevance to the sample information, and generate the nth subset, n [1,N]; set n to N and perform the following matching steps: input the information to be processed and the sample summary sentences in the nth subset into the dialogue generation model, obtain the predicted answer of the nth subset, and match the predicted answer of the nth subset with the true answer; in response to a successful match and n≠1, set n to n-1 and continue to perform the matching steps; in response to a failed match and n≠N, determine that the minimum number of sentences in the successfully matched subset is n+1.

[0018] In a fifth aspect, an embodiment of the present disclosure provides an electronic device comprising: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement a method as described in any one of the first aspect or the second aspect.

[0019] In a sixth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method as described in any one of the first aspect or the second aspect is implemented.

[0020] In a seventh aspect, an embodiment of the present disclosure provides a computer program product, comprising a computer program, which implements the method as described in any one of the first aspect or the second aspect when executed by a processor.

[0021] The training compression model and the method and apparatus for processing information based on the compression model provided by the embodiments of the present disclosure construct training data containing sample information, sample summary sentences extracted from retrieved documents, and their corresponding compression ratios by annotating them as training triples. Based on this training data, a compression model is trained. This compression model predicts the optimal summary sentence required by the RAG system based on the information to be processed and the summary sentences extracted from the retrieval results. This allows the RAG system to obtain sufficient contextual information, avoid the loss of key details due to over-compression, and improve the quality and completeness of the summary.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied; Figure 2 is a flow chart of an embodiment of a method for processing information based on a compression model according to the present disclosure; Figure 3 is a schematic diagram of an application scenario of the method for processing information based on a compression model according to the present disclosure; Figure 4 is a flowchart of one embodiment of a method for training a compression model according to the present disclosure; Figure 5 is a structural diagram of an embodiment of an apparatus for processing information based on a compression model according to the present disclosure; Figure 6 is a structural diagram of an embodiment of an apparatus for training a compression model according to the present disclosure; Figure 7It is a structural diagram of a computer system of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0024] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0025] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0026] Figure 1 An exemplary system architecture 100 is shown to which embodiments of the method of training a compression model and processing information based on the compression model, and the apparatus for training a compression model and processing information based on the compression model of the present disclosure can be applied.

[0027] like Figure 1 As shown, system architecture 100 may include a first terminal 101, a second terminal 102, a network 103, a database server 104, and a server 105. Network 103 is a medium for providing a communication link between first terminal 101, second terminal 102, database server 104, and server 105. Network 103 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0028] The first terminal 101 and the second terminal 102 interact with the server 105 via the network 103 to receive or send messages, etc. Various client applications can be installed on the first terminal 101 and the second terminal 102, such as model training applications, human-computer dialogue applications, shopping applications, payment applications, web browsers, and instant messaging tools.

[0029] The first terminal 101 and the second terminal 102 herein can be either hardware or software. When the first terminal 101 and the second terminal 102 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), laptop computers, and desktop computers. When the first terminal 101 and the second terminal 102 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software programs or software modules (for example, to provide distributed services) or as a single software program or software module. This is not specifically limited here.

[0030] Database server 104 can be a database server that provides various services. For example, the database server can store training data. The training data includes a large number of samples. The samples may include sample information, a set of sample summary sentences, and a compression rate label. Thus, user 110 can also select samples from the training data stored in database server 104 via first terminal 101 and second terminal 102.

[0031] Server 105 can also be a server that provides various services, such as a backend server that supports various applications displayed on first terminal 101 and second terminal 102. The backend server can use samples from the training data sent by first terminal 101 and second terminal 102 to train an initial model and can send the training results (e.g., the generated compression model) to first terminal 101 and second terminal 102. In this way, first terminal 101 and second terminal 102 can use the generated compression model to compress summary information. Server 105 can also be equipped with a search engine and a dialogue generation model. The search engine searches the information to be processed sent by first terminal 101 and second terminal 102 to obtain a collection of relevant documents. A collection of summary sentences related to the information to be processed is then extracted from the document collection. The collection of summary sentences is then compressed using the compression model. The information to be processed and the compressed target sentence set are then input into the dialogue generation model to generate recommendation information.

[0032] Database server 104 and server 105 can be either hardware or software. If they are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. If they are software, they can be implemented as multiple software programs or software modules (for example, to provide distributed services), or as a single software program or software module. This is not specifically limited here. Database server 104 and server 105 can also be servers in a distributed system, or servers integrated with blockchain. Database server 104 and server 105 can also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts equipped with artificial intelligence technology.

[0033] It should be noted that the method for training a compression model or the method for processing information based on a compression model provided in the embodiments of the present disclosure is generally executed by the server 105. Accordingly, the device for training a compression model or the device for processing information based on a compression model is generally also provided in the server 105.

[0034] It should be noted that, in the case where the server 105 can implement the relevant functions of the database server 104 , the database server 104 may not be provided in the system architecture 100 .

[0035] It should be understood that Figure 1 The number of the first terminal, the second terminal, the network, the database server, and the server is only illustrative. According to the implementation requirements, there can be any number of the first terminal, the second terminal, the network, the database server, and the server.

[0036] Continue to refer Figure 2 , shows a process 200 of an embodiment of a method for processing information based on a compression model according to the present disclosure. The method for processing information based on a compression model includes the following steps: Step 201: Retrieve a document collection related to the information to be processed.

[0037] In this embodiment, the execution subject of the method for processing information based on the compression model (for example Figure 1 The server (shown in FIG. 1 ) can receive information to be processed sent by a user via a terminal via a wired or wireless connection. The information to be processed can be a query. The execution entity can use a search engine to retrieve a collection of documents related to the information to be processed from the internet or a third-party database. Prior to retrieval, the information to be processed can be pre-processed using at least one of the following: data cleaning, data enhancement, data expansion, and data decomposition.

[0038] Optionally, clean the data before searching. Remove special characters, HTML tags, stop words, and so on. Correct spelling and grammatical errors, and remove duplicate and irrelevant information. Cleaned data can improve search hit rates.

[0039] Optionally, data enhancement can be performed on the information being processed before retrieval. This involves increasing the diversity and richness of the data through various means. For example, synonym substitution can be used, such as replacing "beautiful" with "pretty" or "charming." Text can also be paraphrased, using different expressions to explain the same content. Translations into other languages can also be included to further expand the boundaries of knowledge.

[0040] Optionally, before retrieval, data expansion can be performed on the information to be processed. For example, the information to be processed "how to make chocolate cake" can be expanded to include information containing keywords such as "chocolate melting method" and "batter mixing skills".

[0041] Optionally, before searching, the information to be processed can be broken down into its data. For example, for the query "Applications and Challenges of Quantum Computing in Finance," it can be broken down into subqueries such as "Principles of Quantum Computing," "Existing Computing Requirements in Finance," "How Quantum Computing Meets These Requirements," and "Technical Challenges Faced in Applications." Relevant information is then retrieved for each subquery, and the answers to each subquery are finally integrated to provide a comprehensive response.

[0042] Step 202: extract a set of summary sentences related to the information to be processed from the document set.

[0043] In this embodiment, the similarity (also known as relevance) between each sentence in each document in the document collection and the information to be processed is calculated, and the sentences are ranked based on the similarity. Ultimately, the top-ranked sentences are selected to form the summary sentence set. Similarity can be calculated using methods such as Euclidean distance, Jaccard similarity, and cosine similarity.

[0044] Optionally, an encoder-decoder model can be trained to generate a more concise set of summary sentences from the information to be processed and the retrieved document set.

[0045] Step 203: Input the information to be processed and the summary sentence set into a pre-trained compression model, and output a target sentence set with a compressed number of sentences.

[0046] In this embodiment, the compression model is used to predict the number of compressed sentences, K, based on the information to be processed and the summary sentence set. The K most relevant summary sentences in the summary sentence set are selected to form the target sentence set. The specific training process of the compression model is shown in process 400 below.

[0047] The compression model's compression ratio is dynamically adjusted based on the input content. That is, the number of sentences compressed by the compression model is not fixed but dynamically adjusted based on the complexity of the information being processed and the quality of the retrieved document set. If the complexity of the information being processed is high and the quality of the retrieved document set is low, there will still be a large number of target sentences after compression. If the complexity of the information being processed is low and the quality of the retrieved document set is high, there will be fewer target sentences after compression.

[0048] The method provided by the above-mentioned embodiment of the present disclosure solves the problem of over-compression through an adaptive compression mechanism, so that the RAG system can obtain sufficient contextual information, avoid the loss of key details due to over-compression, and improve the quality and completeness of the summary. The prediction mechanism is used to directly determine the compression rate, avoiding complex reinforcement learning and high computational overhead. Efficient context compression is achieved to ensure that the accuracy of information is guaranteed while reducing the consumption of computing resources when generating summaries. By dynamically adjusting the compression rate, various complex query scenarios can be flexibly responded to according to query characteristics and retrieval results. Through the adaptive compression mechanism, irrelevant or low-value documents can be effectively filtered, reducing the interference of noise on the generation of summaries, while ensuring the integrity and relevance of the summary content, significantly improving the generation quality and response speed.

[0049] In some optional implementations of this embodiment, extracting a set of summary sentences related to the information to be processed from a document collection includes: inputting sentences in the document collection and the information to be processed into a pre-trained extraction model to obtain the relevance of each sentence to the information to be processed; and outputting sentences whose relevance is greater than a predetermined threshold as the set of summary sentences.

[0050] The extraction model can be a neural network, such as an LLM (Large Language Model). The extraction model is trained to retain high-information-gain summaries relevant to the question as much as possible. The input of the extraction model is the retrieved document, and the output is the sentences closest to the benchmark answer, sorted by relevance. Sentences with low relevance are ignored, and sentences with relevance greater than a predetermined threshold are used as summary sentences. During training, through comparative learning, the model uses the benchmark answer as the target and learns how to extract the most relevant sentences from the retrieved documents that can form an accurate summary, ensuring the recall of key information. At the same time, to improve the robustness of the extraction model to noisy documents, a specific fine-tuning task is designed, using question relevance and information gain as optimization targets to avoid missing important clues.

[0051] In some optional implementations of this embodiment, the information to be processed and the set of summary sentences are input into a pre-trained compression model, and a target sentence set with a compressed number of sentences is output, including: inputting the information to be processed and the set of summary sentences into the pre-trained compression model to determine the number of compressed sentences; and screening out summary sentences with the highest number of relevant sentences from the set of summary sentences as the target sentence set and outputting them.

[0052] The compression model outputs the compression ratio (which can be defined as the number of sentences after compression divided by the number of sentences before compression, or as the minimum number of summary sentences required to answer a correct answer). The summary sentences with the highest number of relevant sentences are then filtered out from the summary sentence set based on the compression ratio and output as the target sentence set.

[0053] The compression ratio is adaptively determined by comprehensively considering the complexity of the query and the quality of the retrieval results. This approach ensures optimal filtering of the context content, retaining only the core information that supports the query answer, thereby compressing the context size while maintaining the integrity of key content.

[0054] In some optional implementations of this embodiment, the method further includes: inputting the information to be processed and the target sentence set into the dialogue generation model, and outputting suggestion information.

[0055] For each summary sentence set of the target information and its related documents, the compression model first filters out the most relevant target sentence set from the summary sentence set. The target information and target sentence set are then concatenated as input to the generative model, which outputs recommendation information, also known as dialogue information. The compression model is integrated with the generative model to facilitate reasoning in downstream tasks.

[0056] Experimental results on four different datasets show that by removing irrelevant noisy documents, the RAG system significantly improves its reasoning efficiency. Furthermore, thanks to the rationality of the compression method, the system generates answers with no significant performance loss compared to the full context.

[0057] Continue to see Figure 3 , Figure 3 This is a schematic diagram of an application scenario of the method for processing information based on a compression model according to this embodiment. Figure 3 In the application scenario, it is divided into four stages: 1. Retrieval: For a given information to be processed Through retrieval, a relevant document set D is obtained.

[0058] 2. Extraction: Perform extractive preliminary compression on the document set D, extract all possible sentence sets from each document and merge these sentence sets into a unified summary sentence set, and sort them in descending order of relevance to the information to be processed, such as {S1,…,S i ,S N}, a total of N summary sentences. Among them, S i Represents a summary statement, i represents the order in the summary statement set, the smaller the i value, the higher the S i The greater the relevance to the information to be processed, the greater the relevance of S1 to the information to be processed. N The least relevance to the information to be processed.

[0059] 3. Compression: The summary sentence set will be compressed. By combining the complexity of q and the quality of D, the summary sentence set is further truncated, and only the most relevant K summary sentences are retained to obtain the target sentence set. The target sentence set is sorted in descending order of relevance to the information to be processed, such as {S1,…,S K}, among which S1 has the greatest correlation with the information to be processed, S K Minimizes relevance to the information being processed. This provides the generative model with a simplified and representative context. By compressing and filtering, it significantly reduces redundant information in the context, ensuring that the generative model processes only the most relevant context, significantly improving the efficiency and quality of summary generation.

[0060] 4. Generation: Input the information to be processed and the target sentence set into the dialogue generation model and output the recommended information.

[0061] Further references Figure 4 , which shows a process 400 of another embodiment of a method for processing information based on a compression model. The process 400 of the method for processing information based on a compression model includes the following steps: Step 401: Obtain training data.

[0062] In this embodiment, the training data includes multiple training samples, each of which includes sample information, a set of sample summary sentences, and a compression rate label. The sample information can be a query. After searching the sample information, a set of related documents is obtained. Summary sentences are then extracted from the document set to obtain a set of sample summary sentences.

[0063] In order to allow the compression model to predict the compression rate, an annotation method based on the feedback of the real RAG system is adopted. For each set of sample information and sample summary sentences, the goal is to find the minimum number of sentences that the RAG system can answer correctly as the target sentence set.

[0064] The compression ratio label (i.e., the true compression ratio) can be the ratio of the minimum number of sentences that can be answered correctly to the number of sentences in the sample summary sentence set. Alternatively, the compression ratio label can be the minimum number of sentences that can be answered correctly. In other words, the compression ratio can be expressed as either an absolute value or a relative value. If it is a relative value, it must be converted to an absolute value when selecting target sentences.

[0065] Step 402: Input the sample information and the sample summary sentence set into the compression model to obtain a predicted compression rate.

[0066] In this embodiment, the compression model can predict the sentences required by the RAG system based on sample information and a set of sample summary sentences. This compression model can be fine-tuned based on the Llama2-7b model and trained using a dataset of triplets containing sample information, a set of sample summary sentences, and compression rate labels to minimize the loss between the predicted compression rate and the true compression rate.

[0067] Step 403: Adjust the network parameters of the compression model according to the difference between the predicted compression rate and the compression rate label.

[0068] In this embodiment, the training goal is to reduce the prediction compression rate The actual compression ratio The error between . The loss function can be a cross entropy loss function, for example, the loss function The definition is as follows:

[0069] Where M represents the number of training samples, is the true compression ratio in the i-th training sample , is the predicted compression ratio of the i-th training sample.

[0070] During fine-tuning, the Llama2-7b model learns to map sample information and a set of sample summary sentences to the true compression rate, and the optimization process updates the model parameters θ to minimize the classification loss:

[0071] Among them, the model parameters θ are updated by gradient descent, and η is the learning rate.

[0072] To evaluate the performance of the trained compression model, the predicted compression ratio is compared with the actual compression ratio. Metrics such as accuracy, precision, and recall are used to assess the effectiveness of the compression model in predicting the optimal number of sentences. This approach allows us to estimate the approximate number of documents required to generate a high-quality response based on the information being processed and the documents retrieved.

[0073] The training data is continuously changed to execute steps 401-403 until the loss value converges or the predetermined number of iterations is reached, and the compression model training is completed.

[0074] In some optional implementations of this embodiment, obtaining training data includes: obtaining sample information, a set of sample summary sentences, and true answers to the sample information, wherein the set of sample summary sentences is obtained by extracting sentences related to the sample information from a set of documents related to the sample information; constructing multiple subsets based on the set of sample summary sentences, wherein the number of sentences in the multiple subsets is different; obtaining predicted answers corresponding to the multiple subsets through a dialogue generation model, and determining the number of sentences in the subsets in which the predicted answers successfully match the true answers; and determining a compression rate label based on the minimum number of sentences in the successfully matched subsets.

[0075] The sample summary sentence set can be constructed into multiple subsets based on permutations and combinations, wherein the number of sentences in each subset varies. Each subset is input into the dialogue generation model to determine which subset can receive a correct answer. The compression rate label is determined based on the minimum number of sentences in the subset that can receive a correct answer. For example, if there are 10 summary sentences in total, and only 1-3 sentences cannot be answered correctly, then the minimum number of sentences is 4. The compression rate label can be 4 / 10=0.4. Alternatively, 4 can be directly used as the compression rate label. As long as the compression rate label and the predicted compression rate are both absolute values or relative values, it will be sufficient.

[0076] In some optional implementations of this embodiment, the number of multiple subsets is N, where N is a natural number greater than 1; constructing multiple subsets based on the sample summary sentence set includes: selecting the first n sample summary sentences from the sample summary sentence set in descending order of relevance to the sample information to generate the nth subset, where n∈[1,N]; and obtaining predicted answers corresponding to the multiple subsets through a dialogue generation model, and determining the number of sentences in the subsets in which the predicted answers successfully match the true answers, including: setting n to N and performing the following matching step: inputting the information to be processed and the sample summary sentences in the nth subset into the dialogue generation model, obtaining the predicted answer of the nth subset, and matching the predicted answer of the nth subset with the true answer; in response to a successful match and n≠1, setting n to n-1 and continuing to perform the matching step; in response to a failed match and n≠N, determining that the minimum number of sentences in the successfully matched subset is n+1.

[0077] From all sentences in the summary set, the top K most relevant to the information being processed are selected. Subsets are gradually constructed to verify whether the answer generated by the dialogue model is closest to the true answer. This approach starts with a larger subset and gradually reduces it until the smallest subset that meets the correctness criteria is found. This method ensures that the context is as compact and relevant as possible, thus balancing information content and computational efficiency.

[0078] For example, N summary sentences can generate N subsets: {S1}, {S1, S2}, {S1, S2, S3},…, {S1,S2,…,S N-1},{S1,S2,…,S N}.

[0079] Among them, the summary statement S1 has the greatest relevance to the information to be processed, and the summary statement S N The least relevance to the information to be processed.

[0080] First, we start from the subset with the largest number of statements {S1, S2, ..., S N}Start to verify whether the predicted answer matches the real answer, set the initial value of n to N. If the match is successful, verify the subset of statements with N-1 number {S1, S2, ..., S N-1 Whether the predicted answer matches the true answer. Until the nth subset that does not match is found, the minimum number of statements in the subset that matches successfully is n+1.

[0081] In response to the matching failure and n=N, it indicates that the training data is invalid, and the training data is replaced and the compression model is continued to be trained.

[0082] In response to a successful match and n=1, the minimum number of statements in the subset that is successfully matched is 1.

[0083] Further references Figure 5 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for processing information based on a compression model. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0084] like Figure 5 As shown, the apparatus 500 for processing information based on a compression model in this embodiment includes a retrieval unit 501, an extraction unit 502, and a compression unit 503. The retrieval unit 501 is configured to retrieve a document set related to the information to be processed; the extraction unit 502 is configured to extract a summary sentence set related to the information to be processed from the document set; and the compression unit 503 is configured to input the information to be processed and the summary sentence set into a pre-trained compression model and output a target sentence set with a reduced number of sentences.

[0085] In this embodiment, the specific processing of the retrieval unit 501, the extraction unit 502 and the compression unit 503 of the apparatus 500 for processing information based on the compression model can be referred to. Figure 2 This corresponds to step 201, step 202 and step 203 in the embodiment.

[0086] In some optional implementations of this embodiment, the extraction unit 502 is further configured to: input the sentences and the information to be processed in the document collection into a pre-trained extraction model to obtain the relevance of each sentence and the information to be processed; and output the sentences whose relevance is greater than a predetermined threshold as a summary sentence set.

[0087] In some optional implementations of this embodiment, the compression unit 503 is further configured to: input the information to be processed and the summary sentence set into a pre-trained compression model to determine the number of compressed sentences; and filter out the summary sentences with the highest number of relevance from the summary sentence set as the target sentence set for output.

[0088] In some optional implementations of this embodiment, the apparatus further includes a generation unit 504 configured to: input the information to be processed and the target sentence set into the dialogue generation model, and output suggestion information.

[0089] Further references Figure 6 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for training a compression model. Figure 4 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0090] like Figure 6As shown, the apparatus 600 for training a compression model in this embodiment includes: an acquisition unit 601, a prediction unit 602, and an adjustment unit 603. The acquisition unit 601 is configured to acquire training data, wherein the training data includes sample information, a set of sample summary sentences, and a compression rate label; the prediction unit 602 is configured to input the sample information and the set of sample summary sentences into the compression model to obtain a predicted compression rate; and the adjustment unit 603 is configured to adjust the network parameters of the compression model based on the difference between the predicted compression rate and the compression rate label.

[0091] In this embodiment, the specific processing of the acquisition unit 601, the prediction unit 602 and the adjustment unit 603 of the apparatus 600 for training the compression model can be referred to. Figure 4 This corresponds to step 401, step 402 and step 403 in the embodiment.

[0092] In some optional implementations of this embodiment, the acquisition unit 601 is further configured to: acquire sample information, a set of sample summary sentences, and true answers to the sample information, wherein the set of sample summary sentences is obtained by extracting sentences related to the sample information from a set of documents related to the sample information; construct multiple subsets based on the set of sample summary sentences, wherein the number of sentences in the multiple subsets is different; obtain predicted answers corresponding to the multiple subsets through a dialogue generation model, and determine the number of sentences in the subsets in which the predicted answers successfully match the true answers; and determine a compression rate label based on the minimum number of sentences in the successfully matched subsets.

[0093] In some embodiments, the number of the plurality of subsets is N, where N is a natural number greater than 1; the acquisition unit 601 is further configured to: select the first n sample summary sentences from the sample summary sentence set in descending order of relevance to the sample information, and generate the nth subset, n [1,N]; set n to N and perform the following matching steps: input the information to be processed and the sample summary sentences in the nth subset into the dialogue generation model, obtain the predicted answer of the nth subset, and match the predicted answer of the nth subset with the true answer; in response to a successful match and n≠1, set n to n-1 and continue to perform the matching steps; in response to a failed match and n≠N, determine that the minimum number of sentences in the successfully matched subset is n+1.

[0094] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, and storage of user personal information involved in the technical solutions disclosed herein all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken with respect to user personal information to prevent unauthorized access to user personal information data and to safeguard the security of user personal information, network security, and national security.

[0095] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device and a readable storage medium.

[0096] An electronic device comprises: one or more processors; a storage device on which one or more computer programs are stored, and when the one or more computer programs are executed by the one or more processors, the one or more processors implement the method described in process 200 or 400.

[0097] A computer-readable medium stores a computer program thereon, wherein the computer program implements the method described in process 200 or 400 when executed by a processor.

[0098] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0099] like Figure 7 As shown, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. RAM 703 may also store various programs and data required for the operation of device 700. Computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.

[0100] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0101] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the road zone planning method. For example, in some embodiments, the road zone planning method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the road zone planning method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the road zone planning method via any other suitable means (e.g., via firmware).

[0102] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0103] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0104] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0105] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0106] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0107] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology. The server may be a server in a distributed system or a server integrated with blockchain. The server may also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0108] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0109] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for processing information based on a compression model, comprising: Retrieve a collection of documents related to the information to be processed; Extracting a set of summary sentences related to the information to be processed from the document set; The information to be processed and the summary sentence set are input into a pre-trained compression model, and a target sentence set with a compressed number of sentences is output.

2. The method according to claim 1, wherein The step of extracting a set of summary sentences related to the information to be processed from the document set includes: Inputting the sentences in the document collection and the information to be processed into a pre-trained extraction model to obtain the relevance of each sentence to the information to be processed; Sentences with a relevance greater than a predetermined threshold are output as a summary sentence set.

3. The method according to claim 1, wherein The step of inputting the information to be processed and the summary sentence set into a pre-trained compression model and outputting a target sentence set with a compressed number of sentences includes: Inputting the information to be processed and the summary sentence set into a pre-trained compression model to determine the number of compressed sentences; The summary sentences of the number of sentences with the highest relevance are screened out from the summary sentence set and output as the target sentence set.

4. The method according to any one of claims 1 to 3, wherein: The method further comprises: The information to be processed and the target sentence set are input into a dialogue generation model, and suggestion information is output.

5. A method for training a compression model, comprising: Acquiring training data, wherein the training data includes sample information, a sample summary sentence set, and a compression rate label; Inputting the sample information and the sample summary sentence set into a compression model to obtain a predicted compression rate; Adjusting network parameters of the compression model according to a difference between the predicted compression rate and the compression rate label.

6. The method according to claim 5, wherein: The obtaining of training data includes: Obtaining sample information, a set of sample summary sentences, and true answers to the sample information, wherein the set of sample summary sentences is obtained by extracting sentences related to the sample information from a set of documents related to the sample information; constructing a plurality of subsets based on the sample summary sentence set, wherein the number of sentences in the plurality of subsets is different; Obtaining predicted answers corresponding to a plurality of subsets through a dialogue generation model, and determining the number of sentences in the subsets where the predicted answers successfully match the true answers; The compression ratio label is determined based on the minimum number of statements in the subset that matches successfully.

7. The method according to claim 6, wherein: The number of the plurality of subsets is N, where N is a natural number greater than 1; The constructing a plurality of subsets according to the sample summary sentence set includes: Select the first n sample summary sentences from the sample summary sentence set in descending order of relevance to the sample information to generate the nth subset, n [1,N]; and Obtaining predicted answers corresponding to a plurality of subsets through the dialogue generation model, and determining the number of sentences in the subsets in which the predicted answers successfully match the true answers, includes: Set n to N and perform the following matching step: input the information to be processed and the sample summary sentences in the nth subset into the dialogue generation model, obtain the predicted answer of the nth subset, and match the predicted answer of the nth subset with the true answer; in response to a successful match and n≠1, set n to n-1 and continue performing the matching step; In response to a matching failure and n≠N, it is determined that the minimum number of statements in the subset that matched successfully is n+1.

8. A device for processing information based on a compression model, comprising: a retrieval unit configured to retrieve a document collection related to the information to be processed; an extraction unit configured to extract a set of summary sentences related to the information to be processed from the document set; The compression unit is configured to input the information to be processed and the summary sentence set into a pre-trained compression model, and output a target sentence set with the number of sentences compressed.

9. A device for training a compression model, comprising: an acquisition unit configured to acquire training data, wherein the training data includes sample information, a sample summary sentence set, and a compression rate label; a prediction unit configured to input the sample information and the sample summary sentence set into a compression model to obtain a predicted compression rate; An adjustment unit is configured to adjust network parameters of the compression model according to a difference between the predicted compression rate and the compression rate label.

10. An electronic device comprising: one or more processors; a storage device having one or more computer programs stored thereon, When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1 to 7.

11. A computer-readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

12. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images