Question and answer processing method based on retrieval enhancement and terminal equipment

Abstract

The invention provides a question and answer processing method based on retrieval enhancement and terminal equipment, and relates to the field of natural language processing, the terminal equipment responds to question information input by a user, obtains long context information for the question information, at least partially compresses the long context information, obtains target context information, and sends the target context information to the terminal equipment; and inputting the question information and the target context information into a generation model to enable the generation model to generate reply information matched with the question information based on the target context information, and outputting the reply information.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a question-answering processing method and terminal device based on retrieval enhancement. Background Art

[0002] With the rapid development and popularization of artificial intelligence technology, question-answering systems based on artificial intelligence technology have been widely used in many fields such as online education, intelligent customer service, e-commerce, medical health, legal consulting, natural language processing research, and smart homes, providing people with convenient answers to questions and suggestions.

[0003] In the actual application of question-answering systems, information fragments related to user questions are usually obtained and combined with user questions to input into the question-answering generation model for analysis to generate answer information that meets user needs. However, these information fragments have less content, which easily leads to incomplete context semantics, seriously affecting the accuracy and comprehensiveness of the answer information and reducing the user experience. Summary of the invention

[0004] In view of the above problems, this application provides the following solutions:

[0005] The first aspect of the present application provides a question-answering processing method based on retrieval enhancement, the method comprising:

[0006] In response to question information input by a user, obtaining long context information for the question information;

[0007] Compressing at least part of the long context information to obtain target context information;

[0008] Inputting the question information and the target context information into a generation model so that the generation model generates answer information matching the question information based on the target context information;

[0009] The reply information is output.

[0010] Optionally, the method further includes: obtaining short context information for the question information;

[0011] The obtaining of long context information for the question information includes:

[0012] The long context information for the question information is obtained from the original document mapped with the short context information.

[0013] Optionally, compressing at least part of the long context information to obtain target context information includes:

[0014] Compressing at least part of the long context information to obtain compressed context information;

[0015] Concatenating the compressed context information with the short context information in the corresponding original document to obtain target context information;

[0016] or,

[0017] Concatenating the long context information with the short context information in the corresponding original document to obtain concatenated long context information;

[0018] The concatenated long context information is at least partially compressed to obtain target context information.

[0019] Optionally, at least partially compressing the long context information or the concatenated long context information includes:

[0020] Performing keyword recognition on the long context information or the concatenated long context information to obtain keywords and information to be compressed that does not contain keywords;

[0021] Input the information to be compressed into the compression module and the adaptation module for compression alignment processing to obtain compressed information; the target context information includes the short context information, the keyword and the compressed information, or the target context information includes the keyword and the compressed information;

[0022] The compressed information is semantically aligned with the generated model in the latent space.

[0023] Optionally, performing keyword recognition on the long context information or the concatenated long context information to obtain keywords and information to be compressed that does not contain keywords includes at least one of the following:

[0024] Inputting the long context information or the concatenated long context information into an entity recognition model for entity recognition, determining the recognized target entity as a keyword, and determining the long context information after removing the target entity as information to be compressed;

[0025] The long context information or the concatenated long context information is segmented, and based on the obtained similarity between each word to be recognized and the sentence in which the word to be recognized is located, keywords for the corresponding sentence are extracted from the each word to be recognized, and the sentence after the keyword is extracted and the sentence without the keyword extracted are determined as information to be compressed.

[0026] Optionally, at least partially compressing the long context information or the concatenated long context information includes:

[0027] Inputting the long context information or the concatenated long context information into a compression module for compression processing to obtain at least one sentence embedding vector;

[0028] The at least one sentence embedding vector is input into an adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector; the target context information includes the at least one first embedding vector and at least one second embedding vector representing the short context information, or the target context information includes the at least one first embedding vector.

[0029] Optionally, the compression module and the adaptation module are obtained by training an initial compression module and an initial adaptation module based on a pre-training model; the pre-training model is trained to obtain the generation model;

[0030] The process of training the initial compression module and the initial adaptation module based on the pre-trained model to obtain the compression module and the adaptation module includes:

[0031] Acquire multiple first sample information, and question-answering task information and task answer labels for the multiple first sample information;

[0032] Input the plurality of first sample information into the initial compression module, and process them in sequence through the initial compression module and the initial adaptation module to obtain corresponding plurality of second sample information;

[0033] Inputting the plurality of second sample information and the question-answering task information into the pre-training model, so that the pre-training model processes the plurality of second sample information according to the question-answering task information to generate a reference answer for the question-answering task information;

[0034] By reducing the loss between the reference answer and the task answer label, the parameters of the initial compression module and the initial adaptation module are adjusted so that the output of the adjusted compression module is semantically aligned with the pre-trained model in the latent space through the adjusted adaptation module.

[0035] Optionally, the adjusted adaptation module and the adjusted compression module are respectively a pending compression module and a pending adaptation module, and the process of obtaining the compression module and the adaptation module by training the initial compression module and the initial adaptation module based on the pre-trained model further includes:

[0036] Obtain a fine-tuning dataset; the fine-tuning dataset includes multiple fine-tuning sample information, fine-tuning question-answering task information, and fine-tuning task answer labels;

[0037] Based on the multiple fine-tuning sample information, fine-tuning question and answer task information, and fine-tuning task answer labels, the pending compression module, the pending adaptation module, and the pre-trained model are jointly trained to obtain the compression module, the adaptation module, and the generation model.

[0038] Optionally, the acquiring of the short context information for the question information and the acquiring of the long context information for the question information from the original document mapped by the short context information include:

[0039] Retrieving a plurality of short context information related to the question information from each document contained in the knowledge base;

[0040] Extracting a plurality of information fragments for the question information from the original documents mapped to the respective short context information;

[0041] The multiple information fragments are processed to obtain the long context information.

[0042] A second aspect of the present application provides a terminal device, the terminal device comprising: at least one communication element, at least one memory and at least one processor, wherein:

[0043] The communication element is used to receive question information input by the user;

[0044] The memory is used to store a plurality of computer program instructions;

[0045] The processor is configured to execute the plurality of computer program instructions to implement the following steps:

[0046] In response to the question information, obtaining long context information for the question information;

[0047] Compressing at least part of the long context information to obtain target context information;

[0048] Inputting the question information and the target context information into a generation model so that the generation model generates answer information matching the question information based on the target context information;

[0049] The reply information is output. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0051] Figure 1A flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 1 of the present application;

[0052] Figure 2 A flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 2 of the present application;

[0053] Figure 3 A flowchart of a method for acquiring target context information in a question-answering processing method based on retrieval enhancement proposed in Embodiment 2 of the present application;

[0054] Figure 4 A flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 3 of the present application;

[0055] Figure 5 A flowchart of a question-answering processing method based on retrieval enhancement proposed in Embodiment 4 of the present application;

[0056] Figure 6 A schematic diagram of a process of at least partially compressing long context information in a question-answering processing method based on retrieval enhancement proposed in Embodiment 4 of the present application;

[0057] Figure 7 This is a flowchart of the question-answering processing method based on retrieval enhancement proposed in Example 5 of the present application;

[0058] Figure 8 A flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 6 of the present application;

[0059] Fig. 9 This is a flowchart of the question-answering processing method based on retrieval enhancement proposed in Example 7 of the present application;

[0060] Fig.10 This is a flowchart of the question-answering processing method based on retrieval enhancement proposed in Example 8 of the present application;

[0061] Fig.11 This is a flowchart of the question-answering processing method based on retrieval enhancement proposed in Example 9 of the present application;

[0062] Fig.12 A flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 10 of the present application;

[0063] Fig.13 A schematic diagram of the model training process in the question-answering processing method based on retrieval enhancement proposed in an embodiment of the present application;

[0064] Fig.14 A schematic diagram of the structure of a question-answer processing device based on retrieval enhancement provided in an embodiment of the present application;

[0065] Fig.15 A schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0066] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation mode of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application. The embodiments of the present application are described below in conjunction with the drawings. It is known to those of ordinary skill in the art that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0067] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0068] Regarding the description content of the background technology part, due to the limitation of the input length of the question-answering generation model, only shorter information fragments in the document can be intercepted and input into the question-answering generation model, resulting in the reply information being incomplete and inaccurate. In order to improve the accuracy of the retrieved information fragments, a lot of time and effort is needed to design the retriever of the question-answering system, so that semantically complete information can be retrieved, thereby improving the accuracy of the reply. However, this will bring great system delays and greatly reduce the end-to-end user experience.

[0069] In order to solve the above problems, it is proposed to use the ability of long-context large language model (LLM) to process ultra-long texts, reduce the burden on the retriever, and effectively avoid information loss, fundamentally improving the recall rate and accuracy of the answer. However, long-context reasoning will bring additional system overhead. As the input length grows linearly, the consumption of video memory and computing power resources increases significantly. For terminal devices with limited video memory and computing power resources (such as mobile phones, laptops and desktop computers, etc.), it is often difficult to support the normal operation of such a question-answering system based on a long-context large language model.

[0070] In order to improve the above problems, the embodiment of the present application provides a question-answering processing method based on retrieval enhancement. Compared with obtaining short information fragments for question information, the present application will obtain long context information for the question information input by the user to improve the semantic integrity of the information, and before inputting the generation model, the long context information will be at least partially compressed to obtain the target context information to reduce video memory consumption, especially for local terminal devices with limited resources. Afterwards, the target context information and question information are input into the generation model to generate reply information matching the question information based on the target context information, thereby improving the accuracy and reliability of the reply information. In this way, when the video memory and computing power consumption are basically the same, it is allowed to increase the length of the context information input into the generation model to significantly improve the generation effect of the question-answering system. The question-answering processing method based on retrieval enhancement of the embodiment of the present application is described in detail below in conjunction with the accompanying drawings.

[0071] Reference Figure 1 , is a flowchart of a question-answering processing method based on retrieval enhancement proposed in the first embodiment of the present application. The method can be applied to terminal devices, such as mobile phones, tablet computers, wearable devices, vehicle-mounted devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPC), netbooks, smart transportation / medical equipment, smart home devices or robots, etc. The embodiment of the present application does not limit the product form of the terminal device. Based on this, Figure 1 As shown, the question-answering processing method based on retrieval enhancement proposed in this embodiment may include but is not limited to the following steps:

[0072] Step S11, in response to the question information input by the user, obtaining long context information for the question information;

[0073] In practical applications, after receiving the question information input by the user, the input unit of the electronic device can perform retrieval enhancement on the question information through natural language processing technology, machine learning model / deep learning model, or rule-based method, etc., to obtain long context information for the question information. The long context information can refer to considering a wider range of information when processing information, such as at least one complete document, chapter or long story, so as to capture a wider range of information and deep semantic connections through such a wider range of context information, so as to generate more accurate and complete reply information.

[0074] In an embodiment of the present application, in a possible implementation, in a question-and-answer scenario such as a document summary or a multi-paragraph question-and-answer task, the long context information may be the context information that the user intends to process, and the user may describe the source of the processed long context information and the information identifier and other contents in the question information, so as to read or retrieve the long context information that needs to be processed according to the user's intention. For example: the question information input by the user is "summarize the content related to product 1 released by A", and the long context information obtained may be the context information of one or more data types such as various texts, videos, pictures or voices related to product 1 released by A. The question information input by the user is "analyze the rise and fall of the xxx family in literary work B", and the long context information obtained may be literary work B, or the relevant content describing the xxx family in literary work B, etc.

[0075] Among them, for multi-hop reasoning problems between complex questions and answers, long context information can be obtained from multiple interactive questions and answers. It should be noted that in the case where the question information input by the user does not directly express the long context information identifier or source content, it may be necessary to query the long context information that may be used to solve the user's problem from one or more information sources such as knowledge bases, databases, Internet platforms or local storage devices based on the user's intention, so as to generate the answer information of the user's question through comprehensive analysis. For example, the question information input by the user is "Why is the climate getting worse and worse", it is necessary to analyze the factors affecting the climate and obtain long context information described from different aspects of the influencing factors.

[0076] It can be seen that the long context information obtained by this application may or may not directly contain reply information, and further processing is required to generate reply information, etc. As analyzed above, the long context information can be context information of background knowledge, historical data, research results or social analysis related to the question information, and can be determined based on the type of topic that the user wants to understand in depth and comprehensively expressed in the question information input, so as to feedback accurate and comprehensive reply information to the user through subsequent steps. This application does not restrict the content, acquisition method and information source of the long context information, which can be determined based on the actual question-answering task type.

[0077] Step S12, compressing at least part of the long context information to obtain target context information;

[0078] Step S13, inputting the question information and the target context information into the generation model, so that the generation model generates answer information matching the question information based on the target context information;

[0079] Combined with the above analysis of the technical concept of this application, if a long-context large language model is directly used as a generation model to process long-context information, the generation model deployed on the terminal device will not be able to run smoothly due to the limited video memory and computing power resources of the terminal device, affecting other applications running on the terminal device. In order to reduce the resource consumption of the terminal device, this application proposes to first partially or fully compress the acquired long context information to reduce the information length, ensure that the obtained target context information meets the input length requirements of the generation model, and then combine it with the problem information to input the generation model running on the terminal device, so as to avoid the generation model directly analyzing the long context information and consuming too much resources of the terminal device.

[0080] Moreover, since the target context information obtained by the present application has higher semantic integrity than the short context information only related to the question information, it can improve the recall and accuracy of the question answering, so that the answer information generated by the generation model based on the target context information is more accurate and reliable. The target context information obtained by the present application has a greatly reduced information length relative to the long context information, but the semantics are basically the same. In this way, when the video memory and computing power resources are consistent, the present application can significantly increase the information length of the input generation model without adding extra video memory and reasoning time pressure, and can be used to improve the performance of tasks that require long context understanding.

[0081] It should be noted that for the above-mentioned step S12, if the present application compresses all the long context information, the target context information obtained refers to the context information after the long context information is compressed. If the present application partially compresses the long context information, the target context information obtained by the present application includes this part of the compressed information, and another part of the uncompressed context information. For example, if the first part of the context information in the long context information is compressed, and the second part of the context information in the long context information (such as the key information in the long context information) is retained without being compressed, the target context information obtained by the present application includes the context information after the first part of the context information is compressed, and the second part of the context information. The present application does not limit the compression method used in step S12.

[0082] The question information and target context information input into the generation model can be processed into numerical representations, i.e., vector representations, and the vectors are sequentially spliced ​​and input into the generation model to generate reply information that matches the question information. In this implementation process, in a possible implementation, the question information can be used as the prompt information for this question-answering task to guide the generation model to analyze the target context information and generate reply information. In a possible implementation, a standardized prompt template can be pre-configured to define how to combine the input target context information and question information to form the input information of the generation model, so as to guide the generation model to analyze the target context information and generate reply information through the processing method of the target context information defined by the prompt words in the prompt template. This application does not elaborate on the implementation process of the generation model using the prompt word function to generate reply information, and the implementation method of step S13 includes but is not limited to the method proposed in this embodiment.

[0083] The generative model in the embodiment of the present application can be a generative model (Generative Models) based on artificial intelligence AI, such as a pre-trained large language model (LLM) or a large multimodal model (LM). It should be understood that the model involved in the present application can be a general large model, or it can be an expert large model obtained after fine-tuning based on demand to achieve answering questions in specific fields or tasks, and the embodiment of the present application does not limit this.

[0084] Step S14, outputting the reply information.

[0085] In practical applications, for the reply information generated by the generation model according to the method described above, the reply information can be output by at least one method such as voice, display based on a display screen, or network-based transmission to an external device. Among them, when the data format of the reply information output by the generation model does not match the preset output method, the reply information can be processed into a data format that matches the output method before being output. For example, the reply information output by the generation model is in text format. If the user chooses to output it in voice, the reply information in text format can be converted into voice reply information and then sent to the speaker of the terminal device for output. This application does not limit the output method of the reply information, which can be determined according to the circumstances.

[0086] To summarize, in an embodiment of the present application, after a user inputs question information into a terminal device, the question-and-answer system of the terminal device responds to the question information. In order to provide the user with accurate reply information, long context information for the question information is obtained. Compared with directly inputting the long context information and the question information into a generation model, the present application proposes to first at least partially compress the long context information to obtain target context information with a greatly reduced length, and then input the target context information and the question information into the generation model, so as to greatly reduce the consumption of video memory and computing power resources in the process of generating reply information based on the target context information, thereby improving the generation effect of the terminal-side question-and-answer system and ensuring the accuracy and reliability of the reply information.

[0087] Reference Figure 2 , is a flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 2 of this application. In order to further improve the accuracy and reliability of the answer of the question-answering system, this embodiment of the application proposes to obtain long context information and short context information for the question information input by the user to provide a more accurate and comprehensive answer. Based on this, Figure 2 As shown, the question-answering processing method based on retrieval enhancement proposed in this embodiment may include but is not limited to the following steps:

[0088] Step S21, in response to the question information input by the user, obtaining short context information for the question information;

[0089] In an embodiment of the present application, short context information may be key information for generating reply information, which may include at least one information fragment related to the question information input by the user (which is usually one or several sentences of very short length), and may query short context information with a high relevance to the question information from documents provided by at least one information source, such as identifying entities or topics related to the user's intention and the question information, and retrieving relevant short context information based on this, etc. The present application does not impose any restrictions on the method for obtaining short context information.

[0090] In one possible implementation, when the question-answering system of a terminal device adopts a RAG (Retrieval-Augmented Generation) architecture, short context information related to the question information can be retrieved from one or more information sources of a knowledge base, a database, or an Internet platform through a retriever, and can be used as an enhanced prompt, that is, a part of the prompt word engineering, integrated into a predefined prompt template to form an input of a generation model for information enhancement in combination with other information, but is not limited to this prompt word engineering implementation method.

[0091] In another possible implementation, the terminal device may also use a search engine configured by itself to obtain short context information related to the question information by using a suitable information matching algorithm / information retrieval model (such as a vector space model, a probability model, or information retrieval based on a language model) / evaluation retrieval system (such as based on retrieval effect evaluation, measuring the extent to which the documents retrieved by the system meet the user's query intent, or through relevance feedback and query expansion methods, improving the retrieval effect through user feedback, etc.). This application does not limit the implementation method of the search engine obtaining short context information.

[0092] Step S22, obtaining long context information for the question information from the original document mapped with the short context information;

[0093] Since the content of short context information is limited, it is easy to lose information and have incomplete semantics, resulting in the generation model being based only on short context information, and the generated response information being inaccurate and insufficiently comprehensive, and unable to meet the user's query intent. Considering that short context information is usually the most relevant information fragment in the original document to the question information, and not all information related to the question information, in order to obtain global information related to the question information, this application proposes mapping the obtained short context information back to the original document (original long text), and obtaining long context information for the question information from the original document, so that the long context information can reliably enhance the accuracy and comprehensiveness of the response information to the question information.

[0094] In some embodiments, the acquired long context information may be other context information in the original document except the short context information; or it may be other associated context information (which has a lower relevance to the question information) contained in the original document except the short context information (which has a higher relevance to the question information); since the semantics of consecutive paragraphs of an original document are often related, the acquired long context information may also be context information composed of multiple information fragments that are adjacent to the short context information above and below the position of the original document, etc. The present application does not limit the content of the long context information and the method of acquiring it.

[0095] Step S23, compressing at least part of the long context information to obtain compressed context information;

[0096] As analyzed above, due to the large length of the long context information, if the generation model of the terminal device directly analyzes it, it will generate a large system overhead, which is very stressful for the terminal device with limited resources. In this regard, the embodiment of the present application proposes to at least partially compress the long context information before inputting the generation model to reduce the length of the long context information and obtain the compressed long context information. The implementation process can refer to the description of the corresponding part of the above embodiment, such as the compression process described in the corresponding part of step S12, which will not be repeated in this embodiment. It should be noted that the content of the long context information obtained in this embodiment can be the same as or different from the content of the long context information obtained in step S11, but can be at least partially compressed using the same compression method.

[0097] It should be understood that if the long context information is partially compressed in step S23, that is, a part of the context information in the long context information (which can be referred to as information to be compressed for convenience of description) is compressed, and the other part of the context information is not compressed, then after the partial compression of the long context information, the information to be compressed in the long context information is compressed into compressed information with a smaller length, and the other part of the context information in the long context information that does not need to be compressed is retained, so that the compressed long context information obtained in step S23 includes the compressed information and the other part of the context information that is not compressed. The compressed information and the other part of the context information can be spliced ​​as needed, and the spliced ​​context information is used as the compressed context information, but it is not limited to splicing these two parts of information here.

[0098] Step S24, concatenating the compressed context information with the short context information in the corresponding original document to obtain target context information;

[0099] Following the above analysis, the compressed context information comes from the long context information for the question information initially obtained, and it is spliced ​​with the short context information in the same original document to obtain the target context information that is semantically complete and meets the input length requirement of the generation model. In the splicing process, for the short context information and the compressed context information belonging to the same original document, the corresponding context information can be spliced ​​in the order of their positions in the original document, so that the semantics of the target context information is complete and smooth, but it is not limited to this splicing implementation method.

[0100] Step S25, inputting the question information and the target context information into the generation model, so that the generation model generates answer information matching the question information based on the target context information;

[0101] Step S26, output the reply information.

[0102] In summary, in the embodiments of the present application, Figure 3 In the process diagram shown in the figure, when the user inputs question information into the terminal device to request the terminal device's question-answering system to generate the answer information expected or needed by the user, the terminal device performs intent recognition on the question information and obtains short context information related to the user's intent. However, since the scope of this context is small, it may only involve a single sentence or several adjacent sentences, and cannot capture long-distance dependencies (such as the relationship between different paragraphs in an article, or the connection between different plots in a long story, etc.). When facing question-answering tasks that require global understanding (which require the understanding and integration of a large amount of information), such as document summarization, multi-paragraph question-answering tasks, and multi-hop reasoning problems between complex questions and answers, it is also necessary to obtain long context information for the question information.

[0103] In a method for obtaining long context information proposed in an embodiment of the present application, as Figure 3 As shown, the long context information expected to be obtained is semantically consistent with the short context information that has been obtained, so as to supplement the lost information and semantics of the short context information, and can be used to generate correct and complete answer information for the question information. Therefore, this embodiment proposes to obtain the corresponding long context information from the original document from which the short context information comes, and can even locate the position of the original document from the short context information, read the long context information at that position in the original document, and combine it with the short context information, thereby ensuring the semantic integrity of the context information.

[0104] Afterwards, in order to reduce the resource consumption of long context information analysis, this embodiment will at least partially compress the long context information to reduce the dimensionality of the long context information, and concatenate the compressed context information with the short context information from the same original document. The obtained low-dimensional (i.e., short information length) target context information and question information are input into the generation model, so that the generation model can generate accurate and informative response information based on the semantically complete and accurate target context information. Compared with directly concatenating the short context information and the long context information or directly inputting the original document into the generation model, the resource consumption of the generation model reasoning process is greatly reduced, the calculation time is shortened, and it is ensured that complex question-answering tasks can be reliably implemented in a local environment with limited resources.

[0105] In some other embodiments proposed in the present application, according to the above process of using short context information and long context information for question information to generate accurate and comprehensive reply information, in order to be applicable to local environments with limited resources, in the process of executing the method proposed in the present application by the terminal device, in addition to the implementation method described in the second embodiment, refer to Figure 4The flowchart of the retrieval-enhanced question-answering processing method proposed in the third embodiment of the present application is shown. The present application can also splice the acquired long context information with the short context information in the corresponding original document to obtain the spliced ​​long context information, and then at least partially compress the spliced ​​long context information to obtain the target context information, such as context information that meets the input length requirement of the generation model (i.e., the vector length / dimension under the vector representation).

[0106] In a possible implementation, if the long context information and short context information from the same original document are concatenated to form the original document, the short context information is key information related to the question information, and it is believed that the content of the original document where the short context information is located can be used to generate more semantically complete and accurate response information, the obtained short context information can be mapped back to the original document, and the context information in the entire original document can be directly used as the new long context information, which is equivalent to Figure 4 The concatenated long context information shown is used to at least partially compress the new long context information formed by the original document to obtain the target context information. The implementation process is not described in detail in this embodiment.

[0107] It should be noted that when the short context information is key information, it is necessary to retain the short context information and input it into the generation model. Figure 4 In the processing flow shown, during the process of at least partially compressing the concatenated long context information, at least the short context information may be kept as a part of the context information and not compressed, so that the target context information includes the complete short context information.

[0108] In addition, for the long context information described in each of the above embodiments, before at least partially compressing it, the long context information can be preprocessed to delete invalid characters (such as punctuation marks, interjections, etc.) in the long context information, as well as information that is irrelevant to the reply information for generating the question information (which can be called interference information), and then perform the above-mentioned at least partial compression processing. The present application does not limit the method for identifying the interference information. Of course, the present application can also directly obtain context information that may be associated with the reply information for generating the question information during the process of obtaining the long context information, so as to avoid the resource consumption caused by obtaining the interference information and the impact on the accuracy of the generated reply information. The implementation method is not limited by the present application.

[0109] In a possible implementation, the method for obtaining short context information described in the above embodiments can be implemented under the RAG architecture, and each document in the knowledge base can be divided into small chunks (chunks, such as a sentence or several sentences containing a certain number of words, etc., so that each chunk contains certain information), and these chunks can be encoded into vectors and stored in a vector database to facilitate subsequent screening of required context information.

[0110] It can be seen that in this case, in the retrieval stage of short context information, search engines or knowledge graphs and other technologies can be used, using information matching algorithms such as semantic similarity calculation or appropriate retrieval models, to query the first m chunks most relevant to the question information input by the user (which can be processed into question vectors and then analyzed for correlation with each chunk in the vector library) from the vector database, if the correlation is greater than a threshold (whose value can be determined according to the RAG retrieval configuration requirements) or the correlation is sorted from high to low, the top m chunks are sorted, so that the retrieved m chunks are used as short context information for the question information. The short context information obtained in this way as key information for answering questions can be directly written into the prompt template of the generation model to be combined with other information to form the input information of the input generation model, but it is not limited to the implementation method of obtaining input information based on the prompt template.

[0111] Based on the above analysis, the short context information is mapped to the original document of the vector database, that is, the information source of the short context information, and other chunks except the short context information are obtained from it as the long context information for the question information. Figure 3 or Figure 4 The processing method shown obtains the target context information for input into the generation model, such as several token (a discrete text unit, which serves as the basic unit for the generation model to generate reply information, which can be a word, number or other language element, etc.) vectors that meet the input length requirements of the generation model. The implementation process of this embodiment will not be described in detail here.

[0112] It should be noted that the retrieval-enhanced question-answering processing method proposed in the present application is applicable to but not limited to the RAG architecture. In a question-answering system that implements specific fields or knowledge-intensive tasks, the retrieval-enhanced question-answering processing method proposed in the present application can be executed under the RAG architecture to ensure the completeness of the semantic information of the input information of the input generation model, improve the accuracy and reliability of the reply information, and avoid the input information being too long, increasing the consumption of resources such as memory and computing generated by the generation model reasoning process, bringing huge pressure to terminal devices with limited resources, and affecting the performance of the terminal-side question-answering system.

[0113] Reference Figure 5, is a flow chart of a question-answering processing method based on retrieval enhancement proposed in the fourth embodiment of the present application. This embodiment can be used in the above embodiments. Figure 3 The long context information in Figure 4 An optional refinement of at least partially compressing the concatenated long context information is described as follows. Figure 3 The at least partial compression process of the long context shown in FIG. Figure 5 As shown, the process may include but is not limited to the following steps:

[0114] Step S51, in response to the question information input by the user, obtaining short context information for the question information;

[0115] Step S52, obtaining long context information for the question information from the original document mapped with the short context information;

[0116] Regarding the implementation process of step S51 and step S52, reference may be made to the description of the corresponding parts of the above embodiment, which will not be elaborated in this embodiment.

[0117] Step S53, performing keyword recognition on the long context information to obtain keywords and information to be compressed that does not contain keywords;

[0118] In order to reduce the resource consumption of the generation model reasoning process, relieve the pressure on terminal devices with limited resources, and improve the generation effect of the end-side question answering system, when obtaining the long context information for the question information (which can be the segmented information such as Figure 6 As shown in the multiple chunks, each chunk here can be a sentence of a specified length or can be individual information fragments, but the number of chunks is relatively large), they will be at least partially compressed before being input into the generation model. During the compression process, in order to avoid the loss of key information (which is usually semantically rich vocabulary) in generating the reply information due to compression, the embodiment of the present application proposes not to compress the key information in the long context information, but only to compress the non-key information therein, so as to retain the rich semantic information in the long context information.

[0119] Based on this, this application will use a keyword recognition method to extract keywords from long context information and information to be compressed that does not contain keywords. Since the length of the information to be compressed is still very large, it is necessary to compress the information to be compressed to reduce the length of the information while ensuring its semantics as much as possible. The implementation process can adopt but is not limited to the compression processing method described in the following steps. It should be understood that if Figure 3 The compressed context information shown includes the keywords obtained in step S53 and the information to be compressed.

[0120] In a possible implementation, if the present application is applicable to a RAG system, after its retriever retrieves multiple short context information related to the question information from each document contained in the knowledge base, the enhancement module can extract multiple information fragments for the question information from the original documents mapped by each short context information, and then the multiple information fragments can be processed to obtain long context information, such as long context information composed of multiple sentences of a specified length. In this case, keyword recognition is performed on the long context information, and each sentence contained therein can be directly segmented without sentence segmentation to identify the keywords contained therein and the information to be compressed that does not contain keywords.

[0121] Step S54, inputting the information to be compressed into the compression module and the adaptation module for compression alignment processing to obtain compressed information; the compressed information is semantically aligned with the generated model in the latent space;

[0122] Step S55, concatenating the short context information, the keyword and the compressed information to obtain the target context information;

[0123] Reference Figure 6 As shown in the flow chart, the information to be compressed extracted from the long context information of the question information can be input into the compression module for compression processing, and the information output by the compression module can be input into the adaptation model for alignment processing, so that the compressed information output by the adaptation module is semantically aligned with the generation model in the latent space, that is, the generation model can accurately understand the semantic information of the long context information. This application does not elaborate on how to use the compression module and the adaptation module to treat the compression alignment processing method of the compressed information. It can refer to but is not limited to the description of the corresponding part of the following embodiment. According to the adjustment of the network structure of the corresponding module, the compression alignment processing process of step S54 can be adaptively adjusted. The embodiments of this application are not described in detail one by one.

[0124] The short context information for the question information obtained above is usually the key information of the reply information. It and the keywords extracted from the long context information in this embodiment do not need to be compressed, and can be directly converted into a vector representation that meets the input requirements of the generation model. Of course, in the process of obtaining the short context information and keywords, they have been processed into corresponding vectors, which can be directly spliced ​​with the compressed information to obtain the target context information.

[0125] In a possible implementation, since the short context information may be mapped back to multiple original documents, during the splicing process described above, the information fragments (such as Figure 6The m chunks shown or their processed vectors), keywords (or their processed vectors) and compressed information are spliced, and the spliced ​​information is taken as a whole, and then spliced ​​with the spliced ​​information corresponding to other original documents to obtain the target context information. Optionally, according to actual processing needs, in the splicing process, it is also possible to supplement which information belongs to the same original document, so that in the subsequent generation model to analyze the target context information, the supplementary description can be referred to to enhance the semantic relevance of the context information belonging to the same original document, so as to improve the accuracy of the reply information. It should be noted that the implementation method of step S55 includes but is not limited to the implementation methods listed above, which can be flexibly adjusted according to actual conditions.

[0126] Step S56, inputting the question information and the target context information into the generation model, so that the generation model generates answer information matching the question information based on the target context information;

[0127] Step S57, output the reply information.

[0128] It can be seen from this that Figure 6 As shown in the flowchart, for the long context information obtained from the original document mapped back from the short context information, before compression processing, the keywords in the long context information can be extracted by keyword recognition, and the information to be compressed that does not contain keywords in the long context information is compressed and aligned using pre-trained compression modules and adaptation modules, thereby reducing the content of the information to be compressed while ensuring that its semantic information can be understood by the generation model, thereby ensuring that the response information generated by the generation model is accurate and complete.

[0129] Afterwards, if Figure 6 As shown, the uncompressed short context information and the extracted keywords, as well as the compressed information obtained by the compression alignment process, can be spliced ​​according to the corresponding semantic order to obtain the target context information that meets the input requirements of the generation model, and then the target context information and the question information input by the user are input into the generation model. Compared with directly splicing the short context information and the long context information and then inputting them into the generation model in combination with the question information, the resource consumption of the generation model in the process of reasoning and generating reply information is greatly reduced, the reasoning time is reduced, and because some interference information is eliminated during the compression alignment process, the generation model can understand the semantic information of the information to be compressed more quickly and accurately, and improve the contextual understanding ability of the global information of the question information, thereby improving the accuracy and reliability of the generated reply information.

[0130] In some embodiments, Figure 4In the target context information acquisition method shown in FIG. 1 , if the short context information and the long context information for the question information are directly spliced ​​together, and the spliced ​​context information is obtained, then at least part of the context information is compressed. Figure 7 The flowchart of the question-answering processing method based on retrieval enhancement proposed in the fifth embodiment of the present application is shown. The present application can perform keyword recognition on the spliced ​​long context information, obtain the keywords in the spliced ​​long context information and the information to be compressed that does not contain the keywords, input the information to be compressed into the compression module and the adaptation module for compression alignment processing, and after obtaining the compressed information that is semantically aligned with the generation model in the latent space, the compressed information is spliced ​​with the keywords in the spliced ​​long context information to obtain the target context information.

[0131] In the case where the short context information is context information with a high degree of relevance to the question information, the keywords identified from the spliced ​​long context information may include words of the short context information or keywords that can represent the semantic information of the short context information, and may also include keywords contained in the long context information before splicing, such as entities such as names of people, places, and organization names, or other pre-configured keywords, so that Figure 7 The keywords in the methods shown are Figure 6 There are differences in the keyword content in the methods shown. Usually, for the same question information, as shown in the above analysis, Figure 6 The key words in the method shown are Figure 7 A portion of the keywords identified by the method shown. The present application does not limit the content of the keywords identified from the spliced ​​long context information, which can be determined according to the circumstances.

[0132] In one possible implementation, for Figure 7 In the process of obtaining the spliced ​​context information in the above, short context information related to the question information can also be queried from the original document in the process of obtaining the short context information. The long context information including the queried short context information and its upper and lower adjacent positions can be directly obtained from the original document, and the spliced ​​long context information is obtained by sequentially splicing according to the position of each context information in the original document, and then the long context information is obtained according to the above. Figure 7 The method shown compresses at least part of it to obtain target context information, and the implementation process is not described in detail in this embodiment.

[0133] In some other embodiments, the long context information obtained in the first embodiment (such as a method of obtaining the spliced ​​long context information described in the previous paragraph, but not limited to this method of obtaining) can also be referred to as follows: Figure 7The at least partial compression method shown in the figure performs at least partial compression on the long context information to obtain the target context information, that is, performs keyword recognition on the long context information obtained in step S11 to obtain the keywords and the information to be compressed that does not contain the keywords, and then inputs the information to be compressed into the compression module and the adaptation module for compression alignment processing to obtain compressed information, and splices the compressed information with the keyword information to obtain the target context information. The implementation process can refer to the description of the corresponding part of the context embodiment, and this embodiment will not be described in detail here.

[0134] It should be understood that in the different methods for acquiring target context information as described in the above embodiments, the information content of the keywords, information to be compressed and compressed information involved in each acquisition method may be different, and there may be some identical information, which can be determined based on the actual processing situation. The corresponding information should be understood in combination with the entire concept of the corresponding target context information acquisition method, but it is not limited to the above example content.

[0135] In the various methods for implementing at least partial compression of long context information described in the various embodiments above, Figure 4 The refinement implementation process of at least partially compressing the concatenated long context information shown in FIG. 1 is described as an example. Figure 8 The flowchart of a question-answering processing method based on retrieval enhancement proposed in Example 6 of the present application is shown as follows. Figure 8 As shown, the method may include but is not limited to the following steps:

[0136] Step S81, in response to the question information input by the user, obtaining short context information for the question information;

[0137] Step S82, obtaining long context information for the question information from the original document mapped with the short context information;

[0138] Step S83, concatenating the long context information with the short context information in the corresponding original document to obtain concatenated long context information;

[0139] For the description of the corresponding parts of step S81 to step S83, reference may be made to the description of the corresponding parts of the above embodiment, which will not be elaborated in this embodiment.

[0140] Step S84, inputting the concatenated long context information into a compression module for compression processing to obtain at least one sentence embedding vector;

[0141] Step S85, inputting the at least one sentence embedding vector into an adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector;

[0142] In an embodiment of the present application, the spliced ​​long context information can be compressed to obtain the target context information. In this implementation process, a pre-trained compression module can be used to reduce the dimension of the spliced ​​long context information, such as reducing the dimension of a high-dimensional vector into a low-dimensional vector, and retaining the semantic information of the high-dimensional vector as much as possible to meet the input vector length requirements of the generation model, and enable the generation model to accurately understand the semantic information in the spliced ​​long context information, so as to generate the reply information according to the method described below.

[0143] In one possible implementation, the concatenated long context information can be split into multiple sentences, and these multiple sentences are input into the compression model. The compression model reduces the dimension of the high-dimensional information of each sentence into a low-dimensional sentence embedding vector (i.e., latent space vector) based on the semantic information represented by these multiple sentences, and then inputs the pre-trained adaptation model to process the input sentence embedding vectors to align with the latent space semantics of the generation model, ensuring that the generation model can accurately understand the first embedding vector output by the adaptation module, thereby improving the performance and accuracy of the generation model in question-answering tasks based on long text information. It should be noted that this application does not limit the network structure of the above-mentioned compression module and adaptation module to implement the corresponding functions.

[0144] It should be understood that in the method proposed in Example 1, the long context information obtained can also be compressed according to the above-mentioned compression processing method for the spliced ​​context information to achieve the compression processing of the long context information obtained in step S11. The implementation process is similar and will not be described in detail in this application.

[0145] Step S86, inputting the target context information formed by the at least one first embedding vector and the question information into the generation model, so that the generation model generates answer information matching the question information based on the target context information;

[0146] Step S87, output the reply information.

[0147] In an embodiment of the present application, the spliced ​​long context information is compressed, and the target context information obtained includes the above-mentioned at least one first embedding vector, which is spliced ​​with the question vector representing the question information and input into the generation model to generate the reply information. In this implementation process, if the generation model uses prompt engineering to generate the reply information, the first embedding vectors and question vectors obtained by the above processing can be input into the corresponding fields of the prompt template to form an enhanced input sequence, and then input into the generation model, so that the generation model generates the reply information according to the enhanced input sequence, improves the accuracy of the reply information, and reduces the hallucination problem and improves data security.

[0148] In some embodiments, in combination Figure 7In the method flow shown in FIG. 1 , after executing step S83 to obtain the spliced ​​context information, as shown in FIG. Fig. 9 The flowchart of the method proposed in Embodiment 7 of the present application shown in the figure can perform keyword recognition on the spliced ​​long context information to obtain keywords and information to be compressed that does not contain keywords, input the information to be compressed into the compression module for compression processing, obtain at least one sentence embedding vector, input the at least one sentence embedding vector into the adaptation module for semantic alignment with the latent space of the generation model, and obtain at least one first embedding vector. In this processing process, the keywords identified from the spliced ​​long context information can be processed to obtain the corresponding first embedding vector representing the keywords.

[0149] Afterwards, the first embedding vectors obtained by the above processing can be spliced ​​according to the position order of the represented keywords and the information to be compressed in the spliced ​​long context information, and the obtained first embedding vector sequence is determined as the target context information. Then the target context information and the question information are input into the generation model to generate the reply information. Optionally, the first embedding vector sequence can also be spliced ​​with the question vector representing the question information, and after obtaining the input sequence, the input sequence is input into the generation model to generate the reply information. The implementation process is not described in detail in this application.

[0150] In some further embodiments, for Figure 3 and Figure 6 The long context information for the question information obtained in the method shown can be at least partially compressed according to the method described in the sixth embodiment above to obtain the corresponding target context information. Fig.10 The flowchart of the method proposed in Example 8 of the present application is shown. After obtaining the long context information for the question information, the long context information is input into the compression model for compression processing to obtain at least one sentence embedding vector, and then the at least one sentence embedding vector is input into the adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector.

[0151] In the above compression process, the short context information for the question information can be processed to obtain at least one second embedding vector representing the short context information. After that, the obtained first embedding vector and the second embedding vector can be spliced ​​to obtain the target context information, and then input into the generation model in combination with the question information to generate the reply information. Optionally, the first embedding vector, the second embedding vector and the question vector representing the question information can also be spliced ​​and input into the generation model to generate the reply information, etc. The implementation process can refer to the description of the corresponding part of the above embodiment, and this embodiment will not be described in detail here.

[0152] In a possible implementation, for the long context information obtained for the question information, in order to retain more semantic information therein and avoid compressing the key information therein and affecting the global semantic integrity, the present application proposes to partially compress the long context information. For this purpose, the present application can first perform keyword recognition on the long context information, obtain the keywords and the information to be compressed that does not contain the keywords, and then refer to Fig.11 The flowchart of the method proposed in the ninth embodiment of the present application is shown, in which the information to be compressed is input into the compression module for compression processing to obtain at least one sentence embedding vector representing the compressed information, and then the at least one sentence embedding vector is input into the adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector representing the compressed information, which meets the input vector requirements of the generation model and ensures that the generation model can accurately understand the semantic information of the compressed information.

[0153] In the above partial compression process, the keywords identified from the long context information can be processed into the first embedding vector representing the corresponding keywords, such as Fig.11 , the first embedding vector for guaranteeing keywords, at least one first embedding vector representing compressed information, and at least one second embedding vector representing short context information can be spliced ​​to obtain target context information, and then input into the generation model in combination with the question information. Optionally, the present application can also splice each obtained embedding vector with the question vector representing the question information and input into the generation model to generate the reply information. The present application does not elaborate on the generation process of the reply information.

[0154] It can be seen that according to the compression method described above, global compression of the acquired long context information will lead to a decrease in the ability to retain details of the compressed information, which is likely to cause incomplete semantic information input to the generation model, thereby reducing the accuracy and reliability of the reply information. In order to improve this problem, the embodiment of the present application proposes a processing method for locally compressing the acquired long context information, and does not compress the keywords with rich semantic information in the long context information, and selects to compress the information to be compressed that does not contain keywords, to ensure that the context information after compression is basically consistent with the semantic information of the long context information before compression, and combined with the short context information, the global context understanding ability is improved, and the foundation is laid for the subsequent generation model reasoning to generate accurate reply information, so that the present application is better used to improve the performance of tasks that require long context information understanding, and will not increase the pressure of video memory and reasoning time, and can improve the generation effect of the end-side question and answer system.

[0155] In some embodiments, the present application does not limit the implementation method of keyword recognition for the acquired long context information or the above-mentioned concatenated long context information proposed in the above embodiments. In one possible implementation, the present application can be implemented using an entity recognition model, that is, the above-mentioned acquired long context information or the concatenated long context information is input into the entity recognition model for entity recognition, the identified target entity is determined as a keyword, and the long context information after the target entity is removed is determined as information to be compressed.

[0156] Among them, the entity recognition model can be a named entity recognition (NER) model, which can identify entities with specific meanings or strong referentiality in the acquired long context information or the above-mentioned spliced ​​long context information, such as names of people, places, names of organizations, dates and times, proper nouns, and designated entities such as product names, models or prices determined according to business needs, and use them as keywords in the corresponding long context information. It can be seen that according to changes in business needs or question-and-answer scenarios, matching target entities belonging to keywords can be pre-configured. In this way, after performing entity recognition on the acquired long context information or the above-mentioned spliced ​​long context information, and identifying each entity in the corresponding long context information, the keywords belonging to the pre-configured target entities are determined, and the context information other than the keywords in the corresponding long context information is determined as information to be compressed.

[0157] Optionally, the above-mentioned entity recognition model can also be a pre-trained BERT (Bidirectional Encoder Representations from Transformers) model, or other models based on deep learning / machine learning to realize keyword recognition of long context information. The implementation process is not described in detail with examples one by one in this application.

[0158] In some other embodiments, the present application may also use a method based on word frequency statistics to determine the importance of each word in the long context information obtained above or the long context information after splicing, and select words with higher importance as keywords. Or a graph-based sorting algorithm is used to construct a graph through the co-occurrence relationship between words, and a sorting algorithm is used to calculate the importance score of the words, and the words with scores greater than the score threshold (such as 95 points, etc.) are determined as keywords. It is also possible to use an n-gram (language model) method to capture phrase information, calculate the frequency of the calculator or distribute indicators such as rarity in the long context information to determine keywords, or use a word embedding (such as Word2Vec and GloVe) model to calculate the distance between vectors to measure the importance of words to the long context information. In addition, the present application can also be implemented using other keyword extraction tools, etc. The present application does not limit the keyword recognition method for each long context information.

[0159] Based on this, in a possible implementation, the long context information obtained above or the long context information after splicing above can be segmented to obtain each word to be recognized and the sentence in which it is located, and the similarity between each word to be recognized and the sentence in which the word to be recognized is calculated, and the degree of semantic influence of the word to be recognized on the corresponding sentence is determined based on the similarity between each word to be recognized and the sentence in which the word to be recognized is located, and then the keywords for the corresponding sentence (that is, the words to be recognized that have a large semantic influence on the sentence) are extracted from each word to be recognized, and the sentence after the keyword is extracted and the sentence without the keyword extracted are determined as information to be compressed.

[0160] Optionally, the present application may use a word embedding model or a sentence embedding model to realize the recognition of the above-mentioned keywords. The sentence embedding model is used for illustration. After the word segmenter performs word segmentation on the acquired long context information or the sentences contained in the above-mentioned spliced ​​long context information, each sentence can be encoded to obtain the corresponding sentence vector, recorded as a_all, and each word to be recognized is encoded to obtain the corresponding word vector, recorded as a_word. After that, the distance between each word vector and the sentence vector of the sentence in which it is located is calculated, such as the cosine similarity algorithm. If the distance between a_word and a_all of the sentence in which it is located is greater than the distance threshold, it means that the corresponding word to be recognized has a large influence on the sentence, and the word to be recognized is a keyword and is not compressed. The content other than the keyword in the sentence is determined as the information to be compressed. Among them, the distance threshold can be determined based on the cosine distance of 5%-10% in the training set, and the present application does not limit its value.

[0161] It should be noted that the keyword recognition method used in the at least partial compression process of each long context information involved in each of the above embodiments includes but is not limited to the several methods listed above. According to needs, a combination of multiple methods can also be used to improve the accuracy and reliability of the keyword recognition results, thereby improving the accuracy of the reply information.

[0162] In summary, the methods described in the various embodiments will now be described below for the training process of the compression module, adaptation module and generation model in the method. In the present application, the above-mentioned compression module and adaptation module can be obtained by training the initial compression module and the initial adaptation module based on the pre-training model. The pre-training model can be trained to obtain a generation model, which can be any of the large models exemplified above, etc. The present application does not limit the type of pre-training model and its network results, which can be determined based on actual conditions. The initial adaptation model can be determined based on the type of pre-training model and its applicable task scope, etc., to improve training efficiency and module performance. It can be composed of a simple linear layer (linear layer) or a complex neural network structure, etc. The present application does not limit its network structure.

[0163] Based on this, refer to Fig.12 , is a flow chart of the question-answering processing method based on retrieval enhancement proposed in Example 10 of the present application, such as Fig.12 As shown, the implementation process of the compression module and the adaptation module obtained by training the initial compression module and the initial adaptation module based on the pre-training model may include but is not limited to the following steps:

[0164] Step S121, obtaining a plurality of first sample information, and question-answering task information and task answer labels for the plurality of first sample information;

[0165] In the embodiment of the present application, the plurality of first sample information may be long context information, and may be information fragments processed from one or more documents, such as Fig.13 The present application does not limit the source of the multiple first sample information. The question-answering task information may be the expected pre-trained model ( Fig.13 Only the LLM decoder is used as an example for explanation. The implementation process of other types of pre-trained models is similar. The processing task content of the multiple first sample information can be used as a prompt template to input the pre-trained model relative to the above question information. The task response label can be the response information that is actually generated after processing the multiple first sample information according to the question-and-answer task information. The question-and-answer task information and the task response label can be recorded in the form of a data pair, that is, the question-and-answer data in the field of natural language processing.

[0166] Step S122, inputting the plurality of first sample information into the initial compression module, and sequentially undergoing processing by the initial compression module and the initial adaptation module to obtain corresponding plurality of second sample information;

[0167] Regarding the process of the initial compression module and the initial adaptation module sequentially processing multiple first sample information, reference can be made to the description of the corresponding part of the above embodiment, such as the initial compression module compresses the input first sample information, obtains the corresponding sample sentence embedding vector, and then inputs it into the initial adaptation module to obtain the sample sentence embedding vector, and obtains multiple second sample information from the multiple sample sentence embedding vectors. Of course, the present application can also perform keyword recognition on multiple first sample information, not compress the keywords therein, and only compress the sample information to be compressed that does not contain keywords, obtain the compressed sample information according to the above method, and splice it with the keywords to obtain multiple second sample information. The implementation process can refer to the above description of the method for obtaining the target context information, which will not be described in detail in this embodiment.

[0168] Need to explain, Fig.13 In the training process shown in FIG. 1 , only the compression process of the compressed information is shown. If the present application extracts keywords from a plurality of first sample information, it does not need to go through the following steps: Fig.13 The sentence embedding (sentence embedding module) and adapter (adapter module) in the process are directly processed into token vectors. Fig.13 In this implementation, the token vector representing the keyword can be concatenated with the compressed information to obtain a plurality of second sample information, and then the subsequent operations are performed.

[0169] Step S123, inputting the plurality of second sample information and the question-answering task information into the pre-training model, so that the pre-training model processes the plurality of second sample information according to the question-answering task information, and generates a reference answer for the question-answering task information;

[0170] Referring to the implementation method of the above embodiment, the target context information and question information are input into the generation model. The multiple second sample information and question-answering task information after the above processing are input into the pre-trained model to generate a reference reply for the question-answering task information. The model input can be determined and the reference reply can be generated in combination with the prompt engineering. The implementation process is not described in detail in this embodiment.

[0171] Step S124, by reducing the loss between the reference answer and the task answer label, adjusting the parameters of the initial compression module and the initial adaptation module, so as to semantically align the output of the adjusted compression module with the pre-trained model in the latent space through the adjusted adaptation module, so as to obtain a pending compression module and a pending adaptation module;

[0172] During the training process, due to the compression and alignment processing of multiple first sample information by the initial compression module and the initial adaptation module, the semantic information of the multiple second sample information obtained may not be accurate, and the alignment effect with the semantic space of the pre-trained model may not be good, resulting in inaccurate reference answers generated by the pre-trained model, that is, the loss between the reference answer and the corresponding task answer label (which can be calculated using a suitable loss function, and the type of the loss function can be determined based on the question-answering task information, such as the document summarization task, the consistency algorithm can be used to implement the loss calculation; if it is a retelling of the input sample information, the sample answer label can be multiple first sample information, or its standard retelling information, and the reference retelling information generated by the pre-training model can be calculated for similarity with the sample answer label, and the loss of this training is determined based on the similarity between the two) is large, so it is necessary to train the initial compression module and the initial adaptation module. In this process, the pre-trained model is locked, that is, without adjusting the model parameters of the pre-trained model, the parameters of the initial compression module and the initial adaptation module are adjusted to obtain the adjusted adaptation module and the adjusted compression module for this training.

[0173] Afterwards, the training process described above is repeated. This process will process multiple first sample information based on the adjusted adaptation module and the adjusted compression module obtained in the last training. After obtaining multiple second sample information, they are input into the pre-training model in combination with the question-answering task information to calculate the loss between the reference answer obtained this time and the corresponding task answer label. If the training termination conditions are still not met, such as the loss is greater than the loss threshold, or the loss has not converged (not within a fixed range), or the number of training times has not reached the preset number, etc., the adjusted adaptation module and the adjusted compression module can continue to be trained until the training termination conditions are met. The adjusted adaptation module obtained by the last training is used as the pending compression module, and the adjusted compression module is used as the pending adaptation module, and then participates in subsequent fine-tuning training.

[0174] Optionally, according to the method described above, the training of the initial compression module and the initial adaptation module is completed to determine whether the training termination conditions are met. The adjusted adaptation module obtained by the final training can also be used as a compression module, and the adjusted compression module can be used as an adaptation module to implement the processing of context information in the method proposed in the above embodiment. The implementation process is not described in detail in this embodiment.

[0175] In some embodiments, during the training process of the above-mentioned initial compression module and the initial adaptation module, on the basis of locking the pre-trained model, in the initial training stage, the initial adaptation module can be locked first, and the initial compression module can be trained, that is, according to the training method described in the above steps, by reducing the loss between the reference response and the task response label, the parameters of the initial compression module are adjusted so that the output of the initial compression module is semantically aligned with the pre-trained model in the latent space through the adjusted adaptation module. After several training times, the adjusted adaptation module and the initial compression module are continued to be trained according to the method described in the above steps until the training termination conditions are met. The implementation process is similar and will not be described in detail in this application.

[0176] Step S125, obtaining a fine-tuning dataset; the fine-tuning dataset includes a plurality of fine-tuning sample information, fine-tuning question-answering task information, and a fine-tuning task answer label;

[0177] Step S126, based on multiple fine-tuning sample information, fine-tuning question and answer task information, and fine-tuning task response labels, jointly train the pending compression module, the pending adaptation module, and the pre-trained model to obtain a compression module, an adaptation module, and a generation model.

[0178] In some other embodiments proposed in the present application, in order to make the pre-trained model adapt to the question-answering task with long context information of the present application and further improve the model performance, after training the pending compression module and the pending adaptation module according to the method described above, a fine-tuning data set can be obtained to realize the joint training of the pending compression module, the pending adaptation module and the pre-trained model, that is, the full fine-tuning (FFT) method is adopted to adjust the parameters of the pending compression module, the pending adaptation module and the pre-trained model, so as to further reduce the loss of the pre-trained model output and make it better suitable for question-answering tasks based on long context information.

[0179] Among them, multiple fine-tuning sample information in the fine-tuning data set can come from multiple SFT data suitable for question-answering tasks in the SFT (Supervised Fine-Tuning) data set, or long text information can be obtained from other platforms such as databases, knowledge bases or the Internet, and determined as fine-tuning sample information, combined with fine-tuning question-answering task information and fine-tuning task response labels as question-answering pairs (i.e., QA question-answering data) to achieve full fine-tuning training. In this implementation process, multiple fine-tuning sample information is input into the pending compression module and the pending adaptation module for processing, and the processed information and the fine-tuning question-answering task information are input into the pre-training model. Based on the processed information, a fine-tuning reference response for the fine-tuning question-answering task information is generated, and the parameters of the pending compression module, the pending adaptation module and the pre-training model are adjusted by minimizing the loss between the fine-tuning reference response and the fine-tuning task response label. The training is repeated multiple times until the fine-tuning training termination condition is met, and the compression module, the adaptation module and the generation model are obtained. The implementation process can refer to the training process of the initial compression module and the initial adaptation module above, and this application will not be described in detail.

[0180] In some other embodiments, during the above-mentioned joint training process, the present application can also be implemented by using a parameter-efficient fine-tuning (PEFT) method, that is, only some parameters in each model are updated to reduce training time and cost. This method is suitable for situations where computing resources are limited. The present application does not describe the training implementation process in detail.

[0181] In summary, the model training process described in the above embodiments, as well as the compression module, adaptation module and generation model obtained based on the training, implement the retrieval-enhanced question-answering processing method proposed in this application. Compared with the RAG architecture that only allows the input of short context information when executing question-answering tasks, when the video memory and computing power resources of the terminal device are consistent, the retrieval-enhanced question-answering processing method proposed in this application can increase the input length of the generation model by at least 5 times, significantly improving the generation effect of the terminal-side system. And compared with implementing question-answering tasks based on the LongRAG architecture, this application compresses long context information and ensures that the compressed latent space vector is aligned with the latent space semantics of the generation model, thereby reducing the computational complexity of the generation model while ensuring that the generation model can accurately and comprehensively understand the semantics of the long context information, thereby ensuring the accuracy and reliability of the generated reply information.

[0182] Among them, the present application can partially compress long context information and not compress words with rich semantic information. If combined with the RAG architecture, it can also retain the retrieved short context information related to the question information, further improving the contextual understanding ability of global information, reducing the pressure on the end-side RAG system, improving model performance, ensuring that the generated response information is accurate and complete, and providing user experience.

[0183] The above introduces a question-and-answer processing method based on retrieval enhancement provided in an embodiment of the present application. The following will introduce a device for executing the above-mentioned question-and-answer processing method based on retrieval enhancement.

[0184] Reference Fig.14 , is a schematic diagram of a structure of a question-answer processing device based on retrieval enhancement provided in an embodiment of the present application. Fig.14 As shown, the question-answer processing device based on retrieval enhancement may include:

[0185] The long context information acquisition module 141 is used to acquire long context information for the question information in response to the question information input by the user;

[0186] A target context information obtaining module 142, configured to compress at least part of the long context information to obtain target context information;

[0187] A reply information generating module 143, configured to input the question information and the target context information into a generating model, so that the generating model generates reply information matching the question information based on the target context information;

[0188] The reply information output module 144 is used to output the reply information.

[0189] Optionally, the above-mentioned question-answer processing device based on retrieval enhancement may further include:

[0190] A short context information acquisition module, used to acquire short context information for the question information;

[0191] Based on this, the long context information acquisition module 141 may include:

[0192] The long context information acquisition unit is used to acquire the long context information for the question information from the original document mapped by the short context information.

[0193] Based on the above analysis, in a possible implementation, the short context information acquisition module and the long context information acquisition module 141 may include:

[0194] A retrieval unit, used to retrieve a plurality of short context information related to the question information from each document included in the knowledge base;

[0195] An extraction unit, configured to extract a plurality of information fragments for the question information from the original documents mapped to the respective short context information;

[0196] A processing unit is used to process the multiple information fragments to obtain the long context information.

[0197] Optionally, the long context information obtaining unit may include:

[0198] A first compression unit, configured to compress at least part of the long context information to obtain compressed context information;

[0199] A first concatenation unit is configured to concatenate the compressed context information with the short context information in the corresponding original document to obtain target context information;

[0200] Optionally, the long context information obtaining unit may include:

[0201] A second concatenation unit, configured to concatenate the long context information with the short context information in the corresponding original document to obtain concatenated long context information;

[0202] The second compression unit is used to compress at least part of the concatenated long context information to obtain target context information.

[0203] Based on the above analysis, the first compression unit (or the second compression unit) may include:

[0204] A keyword recognition unit, used to perform keyword recognition on the long context information (or the concatenated long context information) to obtain keywords and information to be compressed that does not contain keywords;

[0205] The compression alignment processing unit is used to input the information to be compressed into the compression module and the adaptation module for compression alignment processing to obtain compressed information; the target context information in the first compression unit includes the short context information, the keyword and the compressed information, and the target context information in the second compression unit includes the keyword and the compressed information. The compressed information is semantically aligned with the generation model in the latent space.

[0206] Optionally, the keyword recognition unit of the first compression unit (or the second compression unit) may include any of the following recognition subunits:

[0207] A first keyword recognition subunit is used to input the long context information (here, the concatenated long context information in the second compression unit) into an entity recognition model for entity recognition, determine the recognized target entity as a keyword, and determine the long context information after removing the target entity as information to be compressed;

[0208] The second keyword recognition subunit is used to perform word segmentation processing on the long context information (here, the concatenated long context information in the second compression unit), extract keywords for the corresponding sentences from the words to be recognized based on the similarity between each word to be recognized and the sentence where the word to be recognized is located, and determine the sentence after the keyword is extracted and the sentence without the keyword extracted as the information to be compressed.

[0209] In some embodiments, the first compression unit (or the second compression unit) may include:

[0210] A third compression unit is used to input the long context information (here, the concatenated long context information in the second compression unit) into a compression module for compression processing to obtain at least one sentence embedding vector;

[0211] The alignment unit is used to input the at least one sentence embedding vector into the adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector; the target context information includes the at least one first embedding vector and at least one second embedding vector representing the short context information. The target context information in the second compression unit includes the at least one first embedding vector.

[0212] In an embodiment of the present application, the compression module and the adaptation module are obtained by training the initial compression module and the initial adaptation module based on a pre-training model; the pre-training model is trained to obtain the generation model.

[0213] Based on this, the training modules of the compression module and the adaptation module obtained by training the initial compression module and the initial adaptation module based on the pre-training model may include:

[0214] A sample acquisition unit, used to acquire a plurality of first sample information, and question-answering task information and task answer labels for the plurality of first sample information;

[0215] An information processing unit, configured to input the plurality of first sample information into the initial compression module, and sequentially process the plurality of first sample information through the initial compression module and the initial adaptation module to obtain corresponding plurality of second sample information;

[0216] A reference answer generation unit, configured to input the plurality of second sample information and the question-answering task information into the pre-trained model, so that the pre-trained model processes the plurality of second sample information according to the question-answering task information to generate a reference answer for the question-answering task information;

[0217] A parameter adjustment unit, used to adjust the parameters of the initial compression module and the initial adaptation module by reducing the loss between the reference response and the task response label, so as to semantically align the output of the adjusted compression module with the pre-trained model in the latent space through the adjusted adaptation module.

[0218] If the adjusted adaptation module and the adjusted compression module are respectively a pending compression module and a pending adaptation module, the training module may further include:

[0219] A fine-tuning data acquisition unit, used to acquire a fine-tuning data set; the fine-tuning data set includes a plurality of fine-tuning sample information and fine-tuning question-answering task information and a fine-tuning task answer label;

[0220] A fine-tuning unit is used to jointly train the pending compression module, the pending adaptation module and the pre-trained model according to the multiple fine-tuning sample information, the fine-tuning question and answer task information and the fine-tuning task answer label to obtain the compression module, the adaptation module and the generation model.

[0221] An embodiment of the present application also provides a computer program product including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements any one of the retrieval-enhanced question-and-answer processing methods provided in the embodiments of the present application.

[0222] A computer-readable storage medium is also provided in an embodiment of the present application. The storage medium carries one or more computer programs. When one or more computer programs are executed by an electronic device, the electronic device can implement any one of the retrieval-enhanced question-and-answer processing methods provided in the embodiments of the present application.

[0223] Reference Fig.15 , is a schematic diagram of the hardware structure of an electronic device proposed in an embodiment of the present application, such as Fig.15 As shown, the electronic device may include: at least one communication element 151, at least one memory 152 and at least one processor 153, wherein:

[0224] At least one communication element 151, at least one memory 152 and at least one processor 153 can communicate with each other via a bus. The bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.15 Only one bidirectional line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0225] The communication element 151 can be used to obtain question information input by the user, obtain the first sample information and its corresponding question-answering task information and task answer label during training, and fine-tune the data set, etc. It can also be used to realize data or instruction transmission between internal components of the electronic device.

[0226] In the embodiment of the present application, the communication element 151 may include a communication element corresponding to wireless communication methods such as wifi, Bluetooth, 5G / 6G and other mobile communications, so that the electronic device can realize data transmission with other devices through the communication element, and may also include one or more interfaces supporting wired communication methods, such as general-purpose input / output (GPIO) interface, USB interface, universal asynchronous receiver / transmitter (UART) interface, etc., to realize data transmission between various components inside the electronic device. The present application does not limit the composition structure of the communication element 151 to realize this function and its corresponding communication transmission mechanism, which can be determined according to the circumstances.

[0227] The memory 152 can be used to store multiple computer program instructions for implementing the retrieval-enhanced question-answering processing method proposed in the embodiments of the present application; the processor 153 can load and execute the computer program instructions stored in the memory 152 to implement the various steps of the retrieval-enhanced question-answering processing method proposed in the embodiments of the present application. The implementation process can refer to the description of the corresponding part of the method embodiment above.

[0228] In the embodiment of the present application, the memory 152 may include storage media such as a floppy disk, a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk. The processor 151 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), a digital signal processor (DSP), an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0229] It should be understood that Fig.15 The structure of the electronic device shown does not constitute a limitation on the electronic device in the embodiment of the present application. In practical applications, the electronic device may include Fig.15 More or fewer components as shown, or a combination of certain components, such as when the electronic device is a terminal device, may also include microphones, speakers, displays, various sensors, antennas, power modules, RF components, external ports and other input / output components, etc., which can be determined based on processing function requirements, and this application does not give detailed examples one by one.

[0230] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0231] Through the description of the above implementation mode, in the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, a computer, a training device or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state disk (SSD)).

[0232] In addition, the various embodiments in this specification are described in a progressive or parallel manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the devices, electronic devices, products and media disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part description.

Claims

1. A question-answering processing method based on retrieval enhancement, the method comprising: In response to question information input by a user, obtaining long context information for the question information; Compressing at least part of the long context information to obtain target context information; Inputting the question information and the target context information into a generation model so that the generation model generates answer information matching the question information based on the target context information; The reply information is output.

2. The method according to claim 1, further comprising: Obtaining short context information for the question information; The obtaining of long context information for the question information includes: The long context information for the question information is obtained from the original document mapped with the short context information.

3. The method according to claim 2, compressing at least part of the long context information to obtain target context information, comprising: Compressing at least part of the long context information to obtain compressed context information; Concatenating the compressed context information with the short context information in the corresponding original document to obtain target context information; or, Concatenating the long context information with the short context information in the corresponding original document to obtain concatenated long context information; The concatenated long context information is at least partially compressed to obtain target context information.

4. The method according to claim 3, at least partially compressing the long context information or the concatenated long context information, comprising: Performing keyword recognition on the long context information or the concatenated long context information to obtain keywords and information to be compressed that does not contain keywords; Inputting the information to be compressed into the compression module and the adaptation module for compression and alignment processing to obtain compressed information; The target context information includes the short context information, the keyword and the compressed information, or the target context information includes the keyword and the compressed information; The compressed information is semantically aligned with the generated model in the latent space.

5. The method according to claim 4, performing keyword recognition on the long context information or the concatenated long context information to obtain keywords and information to be compressed that does not contain keywords, including at least one of the following: Inputting the long context information or the concatenated long context information into an entity recognition model for entity recognition, determining the recognized target entity as a keyword, and determining the long context information after removing the target entity as information to be compressed; The long context information or the concatenated long context information is segmented, and based on the obtained similarity between each word to be recognized and the sentence in which the word to be recognized is located, keywords for the corresponding sentence are extracted from the each word to be recognized, and the sentence after the keyword is extracted and the sentence without the keyword extracted are determined as information to be compressed.

6. The method according to claim 3, wherein at least partially compressing the long context information or the concatenated long context information comprises: Inputting the long context information or the concatenated long context information into a compression module for compression processing to obtain at least one sentence embedding vector; The at least one sentence embedding vector is input into an adaptation module for semantic alignment with the latent space of the generation model to obtain at least one first embedding vector; the target context information includes the at least one first embedding vector and at least one second embedding vector representing the short context information, or the target context information includes the at least one first embedding vector.

7. According to the method according to any one of claims 4 to 6, the compression module and the adaptation module are obtained by training the initial compression module and the initial adaptation module based on a pre-training model; the pre-training model is trained to obtain the generation model; in, The process of training the initial compression module and the initial adaptation module based on the pre-trained model to obtain the compression module and the adaptation module includes: Acquire multiple first sample information, and question-answering task information and task answer labels for the multiple first sample information; Input the plurality of first sample information into the initial compression module, and process them in sequence through the initial compression module and the initial adaptation module to obtain corresponding plurality of second sample information; Inputting the plurality of second sample information and the question-answering task information into the pre-training model, so that the pre-training model processes the plurality of second sample information according to the question-answering task information to generate a reference answer for the question-answering task information; By reducing the loss between the reference answer and the task answer label, the parameters of the initial compression module and the initial adaptation module are adjusted so that the output of the adjusted compression module is semantically aligned with the pre-trained model in the latent space through the adjusted adaptation module.

8. The method according to claim 7, wherein the adjusted adaptation module and the adjusted compression module are respectively a pending compression module and a pending adaptation module, and the process of obtaining the compression module and the adaptation module by training the initial compression module and the initial adaptation module based on the pre-trained model further comprises: Get a fine-tuning dataset; The fine-tuning dataset includes a plurality of fine-tuning sample information, fine-tuning question-answering task information, and fine-tuning task answer labels; Based on the multiple fine-tuning sample information, fine-tuning question and answer task information, and fine-tuning task answer labels, the pending compression module, the pending adaptation module, and the pre-trained model are jointly trained to obtain the compression module, the adaptation module, and the generation model.

9. The method according to claim 2, wherein the step of obtaining short context information for the question information and obtaining long context information for the question information from an original document mapped with the short context information comprises: Retrieving a plurality of short context information related to the question information from each document contained in the knowledge base; Extracting a plurality of information fragments for the question information from the original documents mapped to the respective short context information; The multiple information fragments are processed to obtain the long context information.

10. A terminal device, comprising: at least one communication element, at least one memory, and at least one processor, wherein: The communication element is used to receive question information input by the user; The memory is used to store a plurality of computer program instructions; The processor is configured to execute the plurality of computer program instructions to implement the following steps: In response to the question information, obtaining long context information for the question information; Compressing at least part of the long context information to obtain target context information; Inputting the question information and the target context information into a generation model so that the generation model generates answer information matching the question information based on the target context information; The reply information is output.

Images