Retrieval enhancement generation method and system based on long-term memory, electronic equipment and storage medium

Abstract

The invention provides a retrieval enhancement generation method and system based on long-term memory, electronic equipment and a storage medium, and belongs to the field of artificial intelligence. The method comprises the following steps: performing compression and semantic coding on the content of a whole database by utilizing a large language model to form global memory; generating a preliminary answer on the basis of a query proposed by a user, and taking the preliminary answer as retrieval information to retrieve related information from a database; and generating a final answer based on the retrieved related information and the global memory, and providing the final answer to the user. Firstly, global memory is formed by using a lightweight large language model, retrieval clues are generated based on query proposed by a user, and most valuable information can be quickly and accurately positioned by adjusting the detailed degree and correlation of the retrieval clues. And a final detailed answer is generated by combining the retrieved detailed evidence and the preliminary answer, so that the method has the advantages of high efficiency, accuracy and flexibility, and can also be applied to application scenes needing to process a large amount of text data and complex queries.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a retrieval enhancement generation method, system, electronic device and storage medium based on long-term memory. Background Art

[0002] The Big Language Model is a powerful AI model that has excellent text generation and comprehension capabilities. By training on large amounts of text data, it can learn language patterns and regularities, and generate coherent and reasonable responses or texts based on input prompts or questions. Big Language Models have been widely used in areas such as machine translation, automatic summarization, dialogue systems, and text generation.

[0003] Traditional retrieval-enhanced generation systems often rely on short-term, query-based direct relevance matching. This approach is obviously insufficient when dealing with tasks that require in-depth contextual understanding and complex reasoning, and cannot effectively handle long texts and implicit information needs. At the same time, in the face of complex queries across domains or multiple steps, traditional methods also have difficulty in effective information integration and reasoning, resulting in the inability to generate accurate and comprehensive answers. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a retrieval enhancement generation method, system, electronic device and storage medium based on long-term memory, which is used to fully or at least partially solve the technical problems that the above-mentioned prior art cannot effectively process long texts and implicit information needs. At the same time, in the face of complex cross-domain or multi-step queries, it is difficult to carry out effective information integration and reasoning, resulting in the inability to generate accurate and comprehensive answers.

[0005] In a first aspect, an embodiment of the present application provides a retrieval enhancement generation method based on long-term memory, comprising: Using a large language model, the contents of the entire database are compressed and semantically encoded to form a global memory; Generate a preliminary answer based on the query raised by the user, and use the preliminary answer as retrieval information to retrieve relevant information from a database; Based on the retrieved relevant information and global memory, the final answer is generated and provided to the user.

[0006] Optionally, a large language model is used to compress and semantically encode the contents of the entire database to form a global memory, including: The data in the database is read and aggregated, and a Transformer-based large language model is used to compress long texts into smaller memory units through a multi-layer attention mechanism, while retaining key semantic information to form a global memory, wherein the database includes a relational database and a non-relational database.

[0007] Optionally, read and aggregate the data in the database, and use a Transformer-based large language model to compress long text into smaller memory units through a multi-layer attention mechanism, while retaining key semantic information to form a global memory, including: A global memory construction algorithm is introduced to construct a global memory representation containing semantic information from the original long text data: Representation steps of the original input token: The original input token is processed by the Transformer model and converted into query, key and value matrices, which are used to calculate the attention weights of the query, key and value matrices; The calculation steps of the attention mechanism are: use the softmax function to calculate the attention weight, and use the attention weight to obtain the weighted value; Initialization step of memory tokens: After each context window, a preset number of memory tokens are appended and another set of query, key, and value weight matrices are initialized for the memory tokens; Update step of the memory token: use the same attention mechanism as the original input token and the weight matrix of the memory token to calculate the updated representation of the memory token; Memory formation step: After being processed by multiple layers of Transformers, the original input token is encoded into a hidden state, and after memory formation, the key-value cache of the original token is discarded, where the hidden state includes the hidden state of the original token and the hidden state of the memory token; Semantic compression step: compress long text into global memory; The training steps of the global memory construction algorithm include pre-training and supervised fine-tuning stages. In the pre-training stage, the global memory construction algorithm uses a long context dataset to learn to form memories from the original context. In the supervised fine-tuning stage, the global memory construction algorithm uses data from preset tasks to generate task clues based on the formed memory.

[0008] Optionally, the global memory is characterized as: ; In the formula, is the i-th vector in the hidden state representation, is the attention weight corresponding to the i-th word in the attention weight matrix output by the attention mechanism.

[0009] Optionally, based on the query raised by the user, a preliminary answer is generated, and the preliminary answer is used as retrieval information to retrieve relevant information from the database, including: After receiving a query from a user, the query is parsed, and a preliminary answer draft is generated according to the global memory, and the preliminary answer draft is used as a retrieval clue to characterize the information needs behind the user's query; Use large language models and prompt engineering to evaluate the generated retrieval clues. If they do not meet the requirements, regenerate them to ensure the accuracy of the retrieval clues. The retrieval clues are used to retrieve the information fragments most relevant to the user query in the database, and the retrieved information fragments are screened and integrated to remove redundant and irrelevant content to form a collection of evidence for subsequent answer generation.

[0010] Optionally, based on the retrieved relevant information and global memory, generate a final answer and provide it to the user, including: Using a large language model, the query raised by the user, the retrieved evidence set and the global memory are integrated to generate an initial answer, and the generated initial answer is post-processed to generate a final answer.

[0011] Optionally, before generating a final answer based on the retrieved relevant information and global memory and providing it to the user, the long-term memory-based retrieval enhancement generation method further includes: The Transformer-based large language model is fine-tuned based on the collected data, so that the Transformer-based large language model can better understand the user's query intention.

[0012] In a second aspect, the embodiment of the present application further provides a retrieval enhancement generation system based on long-term memory, comprising: The global memory building module is used to compress and semantically encode the contents of the entire database using a large language model to form a global memory; A search clue generation and information retrieval module is used to generate a preliminary answer based on the query raised by the user, and use the preliminary answer as search information to retrieve relevant information from the database; The answer generation module is used to generate the final answer based on the retrieved relevant information and global memory and provide it to the user.

[0013] In a third aspect, an embodiment of the present application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned long-term memory-based retrieval enhancement generation method when executing the program.

[0014] In a fourth aspect, an embodiment of the present application further provides a storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned long-term memory-based retrieval enhancement generation method are implemented.

[0015] It can be seen from the above technical solutions that the present invention has the following advantages: In the retrieval enhancement generation method, system, electronic device and storage medium based on long-term memory provided by the present application, a lightweight large language model is first used to compress and semantically encode the contents of the entire database to form a global memory. Subsequently, based on the query raised by the user, the system generates preliminary answers, which are used as retrieval clues to retrieve relevant information from the global memory. At the same time, in combination with the retrieved information, a heavyweight large language model is used for in-depth analysis and answer generation, and its powerful expression ability is used to optimize the quality and accuracy of the answer. In addition, a set of refined clue generation and retrieval strategies are designed to ensure that the most valuable information can be quickly and accurately located by adjusting the detail and relevance of the retrieval clues. Finally, the final detailed answer is generated and provided to the user in combination with the retrieved detailed evidence and preliminary answers. It has the advantages of high efficiency, accuracy and flexibility, and provides an effective solution for application scenarios that require processing large amounts of text data and complex queries. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0017] Figure 1 A flowchart of an implementation of a retrieval enhancement generation method based on long-term memory provided by an embodiment of the present invention; Figure 2 A flowchart of retrieving relevant information provided by an embodiment of the present invention; Figure 3 A detailed implementation flow chart of a retrieval enhancement generation method based on long-term memory provided by an embodiment of the present invention; Figure 4 A schematic diagram of the structure of a retrieval enhancement generation system based on long-term memory provided by an embodiment of the present invention; Figure 5 A schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In the detailed description below, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0019] Hereinafter, the terms "include" or "may include" that may be used in various embodiments of the present disclosure indicate the presence of the disclosed functions or operations, and do not limit the addition of one or more functions or operations. In addition, as used in various embodiments of the present disclosure, the terms "include", "have" and their cognates are intended only to indicate specific features, numbers, steps, operations, or combinations of the foregoing items, and should not be understood as first excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing items or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing items.

[0020] In various embodiments of the present disclosure, the expression "or" or "at least one of A or / and B" includes any combination or all combinations of the words listed at the same time. For example, the expression "A or B" or "at least one of A or / and B" may include A, may include B, or may include both A and B.

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] See also Figure 1 FIG. 1 is a flowchart of a retrieval enhancement generation method based on long-term memory in a specific embodiment, comprising the following execution steps: Step 100: Using a large language model, the contents of the entire database are compressed and semantically encoded to form a global memory.

[0023] Specifically, a large language model is used to compress and semantically encode the contents of the entire database to form a global memory, including: reading and aggregating the data in the database, and using a Transformer-based large language model to compress long texts into smaller memory units through a multi-layer attention mechanism, while retaining key semantic information to form a global memory, wherein the database includes a relational database and a non-relational database.

[0024] It should be understood that the global memory can not only effectively store the key content of the database, but also quickly respond to various query requirements in subsequent steps.

[0025] In a specific implementation, a JDBC (Java Database Connectivity) or ORM (Object-Relational Mapping) framework (such as Hibernate, MyBatis) is used to connect to a relational database such as MySQL and PostgreSQL. Write database connection configuration information, including database URL, user name, password, etc. Write SQL query statements according to business needs to extract required data from the table. Use the query language provided by the database (such as MongoDB QueryLanguage of MongoDB) or API to extract data. For complex queries, consider using an aggregation pipeline (such as MongoDB's AggregationPipeline). Remove invalid or redundant data fields. Convert data to a unified format, such as JSON or CSV, use data cleaning tools (such as Apache Spark, Pandas, etc.) to clean and convert data, and perform aggregation operations such as grouping, summing, and averaging on data according to business needs, using SQL aggregation functions (such as SUM, AVG, GROUP BY) or aggregation functions provided by the database. Store the aggregated data in a temporary table, an in-memory database (such as Redis) or a file for subsequent processing. Use natural language processing libraries (such as NLTK, spaCy) to perform preprocessing operations such as word segmentation, stop word removal, and stemming. When segmenting text, use pretrained word embedding models (such as Word2Vec, GloVe) to improve the word segmentation effect. Select a pretrained large language model based on Transformer. Use the encoder of the Transformer model to encode the preprocessed text. The encoder converts the text into a series of feature vectors, which contain the semantic information of the text. In the Transformer model, the multi-head self-attention mechanism is one of the core components. Use the API or library provided by the model to calculate the attention weights of each position in the text. Based on the attention weights, important information in the text can be extracted and irrelevant information can be ignored. Use pooling layers (such as mean pooling, maximum pooling, etc.) to compress the output matrix of the encoder into a smaller memory unit. Store the compressed memory unit in a database (such as MySQL, MongoDB, etc.) or in memory (such as Redis). According to business needs, write query statements or API calls to retrieve relevant information from the global memory. When the data in the database changes, update the global memory in time.

[0026] Step 101: Generate a preliminary answer based on the query raised by the user, and use the preliminary answer as retrieval information to retrieve relevant information from a database.

[0027] For example, the query raised by the user is such as “Please tell me the relationship between the main characters in the Harry Potter series”.

[0028] Step 102: Generate a final answer based on the retrieved relevant information and global memory and provide it to the user.

[0029] In this embodiment, a lightweight large language model (LLM) is first used to compress and semantically encode the contents of the entire database to form a global memory. Subsequently, based on the query raised by the user, the system generates preliminary answers, which serve as retrieval clues to guide the system to retrieve relevant information from the global memory. At the same time, combined with the retrieved information, the heavyweight LLM is used for in-depth analysis and answer generation, and its powerful expression ability is used to optimize the quality and accuracy of the answers. In order to optimize the retrieval performance, a set of refined clue generation and retrieval strategies are also designed to ensure that the most valuable information can be quickly and accurately located by adjusting the detail and relevance of the retrieval clues. Finally, the final detailed answer is generated and provided to the user by combining the retrieved detailed evidence and preliminary answers. It has the advantages of high efficiency, accuracy and flexibility, and provides an effective solution for application scenarios that require processing large amounts of text data and complex queries.

[0030] In an embodiment of the present invention, based on step 100, a possible embodiment is given below to illustrate its specific implementation in a non-limiting manner.

[0031] A global memory construction algorithm is introduced to construct a global memory representation containing semantic information from the original long text data, which specifically includes the following steps: S1: Representation step of the original input token: The original input token is processed by the Transformer model and converted into query, key and value matrices, which are used to calculate the attention weights of the query, key and value matrices.

[0032] , , ; In the formula, is the embedding representation of the input token, , and They are the weight matrices of query (Query, Q), key (Key, K) and value (Value, V) respectively.

[0033] S2: Calculation steps of the attention mechanism: Use the softmax function to calculate the attention weights, and use the attention weights to obtain the weighted values.

[0034] ; In the formula, is the dimension of the key vector.

[0035] S3: Initialization step of memory tokens: After each context window, a preset number of memory tokens are appended and another set of query, key, and value weight matrices are initialized for the memory tokens. For example, another set of query, key and value weight matrices are represented as ,、 and .

[0036] S4: Update step of the memory token: Using the same attention mechanism as the original input token and using the weight matrix of the memory token, the updated representation of the memory token is calculated.

[0037] , , ; ; S5: Memory formation step: After being processed by multiple layers of Transformer, the original input token is encoded into a hidden state, and after the memory is formed, the key-value cache of the original token is discarded, wherein the hidden state includes the hidden state of the original token and the hidden state of the memory token.

[0038] It should be understood that after being processed by multiple layers of Transformer, the original input token is encoded into hidden states, which include the hidden states of the original token and the hidden states of the memory token. After the memory is formed, the model discards the key-value (KV) cache of the original token, which is similar to the forgetting process of human memory.

[0039] S6: Semantic compression step: compressing long text into global memory.

[0040] S7: The training steps of the global memory construction algorithm include pre-training and supervised fine-tuning stages, wherein, in the pre-training stage, the global memory construction algorithm uses a long context dataset to learn to form memories from the original context, and in the supervised fine-tuning stage, the global memory construction algorithm uses data from preset tasks to generate task clues based on the formed memory.

[0041] Exemplarily, the global memory is characterized as: ; In the formula, is the i-th vector in the hidden state representation, is the attention weight corresponding to the i-th word in the attention weight matrix output by the attention mechanism.

[0042] In one embodiment of the present invention, refer to Figure 2 As shown, based on step 101, a possible example will be given below to illustrate its specific implementation scheme in a non-limiting manner.

[0043] S1010: After receiving a query raised by a user, the query is parsed, and a preliminary draft answer is generated based on global memory, and the preliminary draft answer is used as a retrieval clue to characterize the information needs behind the user query.

[0044] Exemplarily, query parsing includes preprocessing the query input by the user, including operations such as removing special symbols, reference resolution, and normalization of time and place.

[0045] S1011: Use a large language model and prompt engineering to evaluate the generated retrieval clues. If they do not meet the requirements, regenerate them to ensure the accuracy of the retrieval clues.

[0046] S1012: Using the retrieval clues, retrieve the information fragments most relevant to the user query in the database, and screen and integrate the retrieved information fragments to remove redundant and irrelevant content to form an evidence set for subsequent answer generation.

[0047] Exemplarily, this step utilizes a variety of retrieval methods, including sparse retrieval, dense retrieval, and re-ranking, and selects a suitable retrieval method according to the complexity of the task and the characteristics of the database. For example, when a large amount of unstructured text needs to be processed, dense retrieval can be selected.

[0048] Exemplarily, the generated answers are post-processed, such as removing redundant information, adjusting word order, and correcting grammatical errors, to improve the accuracy and fluency of the answers.

[0049] In some implementations, when executing step 102, the steps may be performed: using a large language model to fuse the query raised by the user, the retrieved evidence set, and the global memory to generate an initial answer, and post-processing the generated initial answer to generate a final answer.

[0050] In some embodiments, before generating a final answer based on the retrieved relevant information and global memory and providing it to the user, the long-term memory-based retrieval enhancement generation method process further executes the step of: fine-tuning the Transformer-based large language model based on the collected data so that the Transformer-based large language model can better understand the user's query intent.

[0051] In one embodiment, Figure 3 This is a detailed flow chart of a retrieval enhancement generation method based on long-term memory provided according to an embodiment of the present invention. This embodiment is further optimized and expanded on the basis of the above embodiments.

[0052] S300: Using a large language model, the contents of the entire database are compressed and semantically encoded to form a global memory.

[0053] S301: After receiving a query raised by a user, the query is parsed, and a preliminary draft answer is generated according to global memory, and the preliminary draft answer is used as a retrieval clue to characterize the information needs behind the user query.

[0054] S302: Use the large language model and prompt engineering to evaluate the generated search clues. If they do not meet the requirements, regenerate them to ensure the accuracy of the search clues.

[0055] S303: Using the retrieval clues, retrieve the information fragments most relevant to the user query in the database, and screen and integrate the retrieved information fragments to remove redundant and irrelevant content to form an evidence set for subsequent answer generation.

[0056] S304: Using a large language model, the query raised by the user, the retrieved evidence set and the global memory are integrated to generate an initial answer, and the generated initial answer is post-processed to generate a final answer.

[0057] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0058] like Figure 4 As shown, the following is an embodiment of the long-term memory-based retrieval enhancement generation system provided by the embodiment of the present disclosure, which belongs to the same inventive concept as the long-term memory-based retrieval enhancement generation method of the above-mentioned embodiments. For details not described in detail in the embodiment of the long-term memory-based retrieval enhancement generation system, please refer to the above-mentioned embodiment of the long-term memory-based retrieval enhancement generation method.

[0059] The global memory building module is used to compress and semantically encode the contents of the entire database using a large language model to form a global memory; A search clue generation and information retrieval module is used to generate a preliminary answer based on the query raised by the user, and use the preliminary answer as search information to retrieve relevant information from the database; The answer generation module is used to generate the final answer based on the retrieved relevant information and global memory and provide it to the user.

[0060] This application is based on the retrieval enhancement generation system of long-term memory. Figure 4 As shown, it includes multiple core modules: global memory construction module, retrieval clue generation module, information retrieval module, answer generation module, and model fine-tuning module. Among them, the global memory construction module is responsible for processing and encoding the content of the entire database, providing a comprehensive knowledge base for the system. The retrieval clue generation module uses global memory to extract key information from user queries to ensure that the generated clues can accurately guide the subsequent retrieval process. The information retrieval module retrieves the most relevant information fragments from the database through an efficient search algorithm based on these generated clues. The answer generation module builds accurate and comprehensive answers based on the retrieved information and the original query through natural language processing technology. The model fine-tuning module improves the performance and adaptability of the system for specific application scenarios and user needs. These modules work together to form the long-term memory-based retrieval enhancement generation system of this application, which significantly improves the processing capabilities of complex queries and long text data.

[0061] To facilitate understanding of the present application, an implementation example is given below, taking the user inputting "Please tell me the relationship between the main characters in the Harry Potter series." as an example: 1) Global memory building module: The system first extracts all the text content of the Harry Potter series from the database, including books, character descriptions, and plot summaries. Then, these texts are processed and encoded using the Transformer-based LLM to generate a set of memory tags.

[0062] 2) Retrieval clue generation module: When the user enters a query, the system uses global memory to extract key information from it. The system analyzes the user's query, identifies "main characters" and "relationships" as key elements, and generates relevant retrieval clues, such as "Harry and Hermione's friendship", "Ron and Harry's relationship", etc.

[0063] 3) Information Retrieval Module: The information retrieval module receives the generated retrieval clues and searches the database for information fragments related to these clues through efficient search algorithms (such as vector-based similarity search). The content retrieved by the system includes interactions between characters, dialogues, and plot descriptions.

[0064] 4) Answer Generation Module: The answer generation module uses natural language processing technology to build an accurate and comprehensive answer based on the retrieved information fragments and the user's original query. The system integrates the retrieved information and generates the final response.

[0065] 5) Model fine-tuning module: The model fine-tuning module fine-tunes the large model based on the collected data in order to continuously optimize the system's performance for specific application scenarios and user needs. Through fine-tuning, the system can better understand the user's query intent and improve the quality of response to complex queries.

[0066] Figure 5 It is a schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0067] The retrieval enhancement generation method based on long-term memory provided in the embodiment of the present application can be applied to electronic devices. It will be appreciated by those skilled in the art that the electronic device structure involved in the embodiment of the present invention does not constitute a limitation on the electronic device, and the electronic device may include more or less components than shown, or combine certain components, or different component arrangements. In an embodiment of the present invention, the electronic device includes but is not limited to a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the embodiments of the present application described herein and / or required.

[0068] The electronic device may include a processor, an external memory interface, an internal memory, a universal serial bus (USB) interface, a charging management module, a power management module, a battery, a wireless communication module, an audio module, a speaker, a microphone, a sensor module, buttons, a camera, a display, and a SIM card interface, etc.

[0069] It is to be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0070] The processor may include one or more processing units, for example, the processor may include a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated into one or more processors.

[0071] The processor can be the nerve center and command center of the electronic device. The controller can generate an operation control signal according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0072] A memory may also be provided in the processor for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. The memory may store instructions or data that the processor has just used or is cyclically used. If the processor needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor, and thus improves system efficiency.

[0073] The external memory interface can be used to connect an external memory card, such as a MicroSD card, to expand the storage capacity of the electronic device. The external memory card communicates with the processor through the external memory interface to implement data storage functions. For example, files such as music and videos can be saved in the external memory card.

[0074] The internal memory can be used to store computer executable program codes, which include instructions. The processor executes various functional applications and data processing of the electronic device by running the instructions stored in the internal memory. The internal memory may include a program storage area and a data storage area. The internal memory may include a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0075] The wireless communication function of an electronic device can be realized through an antenna, a wireless communication module, a modem processor, and a baseband processor.

[0076] The wireless communication module can provide wireless communication solutions for electronic devices, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared technology (IR), etc.

[0077] Electronic devices can implement audio functions, etc. through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0078] Electronic devices can achieve shooting functions through ISP, camera, video codec, GPU, display and application processor.

[0079] Electronic devices can achieve display functions through GPU, display screen and application processor.

[0080] The GPU is a microprocessor for image processing that connects the display screen and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor may include one or more GPUs that execute program instructions to generate or change display information.

[0081] The display screen is used to display images, videos, etc. The display screen includes a display panel.

[0082] The above-mentioned electronic device realizes the retrieval enhancement generation method based on long-term memory of the present application, which utilizes a large language model to compress and semantically encode the contents of the entire database to form a global memory; generates a preliminary answer based on the query raised by the user, and uses the preliminary answer as retrieval information to retrieve relevant information from the database; generates a final answer based on the retrieved relevant information and global memory, and provides the user with a technical solution, which achieves the beneficial effect of improving the quality and accuracy of responses to complex queries, and significantly improving the ability to handle long texts and implicit information needs.

[0083] The storage medium provided in the present application stores a program product capable of implementing a retrieval enhancement generation method based on long-term memory.

[0084] The retrieval enhancement generation method based on long-term memory includes: using a large language model to compress and semantically encode the contents of the entire database to form a global memory; generating a preliminary answer based on the query raised by the user, and using the preliminary answer as retrieval information to retrieve relevant information from the database; generating a final answer based on the retrieved relevant information and global memory, and providing it to the user.

[0085] In some possible embodiments, the subject matter of the present disclosure, namely, a method and system for retrieval enhancement generation based on long-term memory, can be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps described in the above “Exemplary Method” section of this specification according to various exemplary embodiments of the present disclosure.

[0086] The storage medium of the present disclosure can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0087] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A retrieval enhancement generation method based on long-term memory, characterized in that: include: Using a large language model, the contents of the entire database are compressed and semantically encoded to form a global memory; Generate a preliminary answer based on the query raised by the user, and use the preliminary answer as retrieval information to retrieve relevant information from a database; Based on the retrieved relevant information and global memory, the final answer is generated and provided to the user.

2. The retrieval enhancement generation method based on long-term memory according to claim 1, characterized in that: Using a large language model, the entire database content is compressed and semantically encoded to form a global memory, including: The data in the database is read and aggregated, and a Transformer-based large language model is used to compress long texts into memory units through a multi-layer attention mechanism, while retaining key semantic information to form a global memory, wherein the database includes a relational database and a non-relational database.

3. The retrieval enhancement generation method based on long-term memory according to claim 2, characterized in that: Read and aggregate data from the database, and use a Transformer-based large language model to compress long text into memory units through a multi-layer attention mechanism, while retaining key semantic information to form a global memory, including: A global memory construction algorithm is introduced to construct a global memory representation containing semantic information from the original long text data: Representation steps of the original input token: The original input token is processed by the Transformer model and converted into query, key and value matrices, which are used to calculate the attention weights of the query, key and value matrices; The calculation steps of the attention mechanism are: use the softmax function to calculate the attention weight, and use the attention weight to obtain the weighted value; Initialization step of memory tokens: After each context window, a preset number of memory tokens are appended and another set of query, key, and value weight matrices are initialized for the memory tokens; Update step of the memory token: use the same attention mechanism as the original input token and the weight matrix of the memory token to calculate the updated representation of the memory token; Memory formation step: After being processed by multiple layers of Transformers, the original input token is encoded into a hidden state, and after memory formation, the key-value cache of the original token is discarded, where the hidden state includes the hidden state of the original token and the hidden state of the memory token; Semantic compression step: compress long text into global memory; The training steps of the global memory construction algorithm include pre-training and supervised fine-tuning stages. In the pre-training stage, the global memory construction algorithm uses a long context dataset to learn to form memories from the original context. In the supervised fine-tuning stage, the global memory construction algorithm uses data from preset tasks to generate task clues based on the formed memory.

4. The retrieval enhancement generation method based on long-term memory according to claim 3, characterized in that: The global memory is characterized as: ； In the formula, is the i-th vector in the hidden state representation, is the attention weight corresponding to the i-th word in the attention weight matrix output by the attention mechanism.

5. The retrieval enhancement generation method based on long-term memory according to claim 1, characterized in that: Based on the query raised by the user, a preliminary answer is generated, and the preliminary answer is used as the search information to retrieve relevant information from the database, including: After receiving a query from a user, the query is parsed, and a preliminary answer draft is generated according to the global memory, and the preliminary answer draft is used as a retrieval clue to characterize the information needs behind the user's query; Use large language models and prompt engineering to evaluate the generated retrieval clues. If they do not meet the requirements, regenerate them to ensure the accuracy of the retrieval clues. The retrieval clues are used to retrieve the information fragments most relevant to the user query in the database, and the retrieved information fragments are screened and integrated to remove redundant and irrelevant content to form a collection of evidence for subsequent answer generation.

6. The retrieval enhancement generation method based on long-term memory according to claim 5, characterized in that: Based on the retrieved relevant information and global memory, the final answer is generated and provided to the user, including: Using a large language model, the query raised by the user, the retrieved evidence set and the global memory are integrated to generate an initial answer, and the generated initial answer is post-processed to generate a final answer.

7. The retrieval enhancement generation method based on long-term memory according to claim 1, characterized in that: Before generating a final answer based on the retrieved relevant information and global memory and providing it to the user, the long-term memory-based retrieval enhancement generation method further includes: The Transformer-based large language model is fine-tuned based on the collected data, so that the Transformer-based large language model can better understand the user's query intention.

8. A retrieval enhancement generation system based on long-term memory, characterized in that: include: The global memory building module is used to compress and semantically encode the contents of the entire database using a large language model to form a global memory; A search clue generation and information retrieval module is used to generate a preliminary answer based on the query raised by the user, and use the preliminary answer as search information to retrieve relevant information from the database; The answer generation module is used to generate the final answer based on the retrieved relevant information and global memory and provide it to the user.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the retrieval enhancement generation method based on long-term memory as described in any one of claims 1 to 7 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the retrieval enhancement generation method based on long-term memory as claimed in any one of claims 1 to 7 are implemented.

Images