A question answering method based on memory retrieval

Abstract

This application relates to the field of artificial intelligence technology, and more particularly to a question-answering method based on memory retrieval. It involves acquiring a target image containing historical interaction information that is semantically related to the question to be answered. Since the target image records an analysis log of the agent generating historical responses based on the historical questions to be answered, and this log includes at least one of the following: the question to be answered, historical responses, the agent's thought process, tool usage information, and core information from both the historical questions and responses, the question to be answered and the target image are input into the agent's target model. This allows the target model to generate the target response information for the question to be answered based on the information in the target image. Because the token length of a single image is determined by the image resolution, even if the target image contains a lot of information, it will not exceed the context limit of the target model, ensuring that the agent can effectively reuse historical interaction experience.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a question-answering method based on memory retrieval. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agents, with their powerful language understanding and generation capabilities, are widely used in many fields. Memory management, as one of the core technologies of intelligent agents, directly determines whether an intelligent agent can effectively reuse historical interaction experience to provide accurate services.

[0003] In related technologies, to enable agents to effectively reuse historical interaction experience, the entire historical conversation text is typically segmented into blocks, and then these blocks are converted into vectors and stored in a vector library. When the memory needs to be retrieved, relevant text blocks are recalled through similarity matching. While this method can preserve the integrity of historical conversation information to some extent, when the number of recalled text blocks is large, it is very easy to exceed the context limit of the large model invoked by the agent, thus causing the agent to be unable to effectively process the recalled information.

[0004] Therefore, how to ensure that intelligent agents can effectively reuse historical interaction experience has become an urgent problem to be solved. Summary of the Invention

[0005] This application provides a memory-based question-answering method to address the problem in the prior art where the agent cannot effectively process the recalled information when the number of recalled text blocks is too large.

[0006] In a first aspect, embodiments of this application provide a question-answering method based on memory retrieval, the method comprising: Search for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information; Obtain a target image saved for the target's historical interaction information. The target's historical interaction information includes at least one historical unanswered question and one historical answer. The target image records an analysis log of the agent generating the historical answer based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer, the agent's thought process, tool call information, and core information of the historical unanswered question and the historical answer. The question to be answered and the target image are input into the target big model of the agent, so that the target big model generates target answer information for the question to be answered based on the information in the target image.

[0007] Furthermore, the step of searching for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information includes: In the constructed memory network, target nodes that are semantically related to the question to be answered are searched. The memory network includes multiple nodes, and each node is used to represent the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line. Determine the target historical interaction information to which the content represented by the target node belongs.

[0008] Furthermore, the memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword. The keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated in any question and answer in the conversation unit. The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: Extract the target keywords from the questions to be answered; The semantic similarity between the target keyword and the keyword represented by each node in the keyword tag network is determined, and the node corresponding to the keyword whose semantic similarity meets the threshold requirement is determined as the target node.

[0009] Furthermore, the memory network is a message association network, where any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the message association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0010] Further, the step of determining the semantic similarity between the question to be answered and the content represented by each node in the message association network, and determining the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node, includes: Determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered, and determine the node corresponding to the content with the first semantic similarity greater than the first threshold as the first node; For each first node, determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network, and determine the node corresponding to the feature vector with the second semantic similarity greater than the second threshold as the second node; The first node and the second node are identified as the target nodes.

[0011] Furthermore, the memory network is a session association network, where any node in the session association network is used to represent the content communicated by any session unit; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the session association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0012] Furthermore, after finding the target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the method further includes: If there are multiple target historical interaction information, for each target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time; based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, determine the target ranking score corresponding to the target historical interaction information. Based on the target ranking score corresponding to each target's historical interaction information, a preset number of target historical interaction information are selected from the current multiple target historical interaction information.

[0013] Furthermore, the process of determining the target image corresponding to the target historical interaction information includes: If the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by that question-and-answer pair in the information tree; generate the target image corresponding to the target historical interaction information according to the analysis log and the preset image generation rules; If the target historical interaction information is the content communicated by the conversation unit, the information tree of the conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by each question and answer pair in the information tree is searched. Based on each analysis log found and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

[0014] Furthermore, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; After searching the analysis log of each question-answer pair included in the target historical interaction information in the saved session information tree, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, the method further includes: For each data tag in each core piece of information, determine the semantic vector corresponding to that data tag; The semantic vector is reduced in dimensionality based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. Each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space. Based on the three-dimensional intermediate vectors corresponding to all data labels of this core information, determine the maximum and minimum intermediate vectors; Determine the first difference between the three-dimensional intermediate vector corresponding to the data label and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; Determine the quotient of the first difference and the second difference; The product of the quotient and the preset value is rounded down to obtain the RGB color value corresponding to the data label; Based on the analysis logs and preset image generation rules, a target image corresponding to the target historical interaction information is generated, including: If the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0015] Furthermore, the method also includes: Retrieve the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; If a memory update instruction is received, extract the summary to be updated based on the session to be updated carried in the instruction; Calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; The target modification strategy is determined based on the pre-saved correspondence between different threshold ranges and modification strategies; Based on the target modification strategy, the analysis log of the session to be updated is displayed in the information tree of the session unit, and based on the target modification strategy, at least one node is displayed in the memory network to represent the semantics of the content communicated by the session to be updated.

[0016] Secondly, embodiments of this application provide a question-answering device based on memory retrieval, the device comprising: The retrieval module is used to search for target historical interaction information that has a semantic relationship with the question to be answered in the stored historical interaction information; The acquisition module is used to acquire a target image saved for the target's historical interaction information. The target's historical interaction information includes at least one historical unanswered question and one historical answer. The target image records an analysis log of the agent generating the historical answer based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer, the agent's thinking content, tool call information, and core information of the historical unanswered question and the historical answer. The response module is used to input the question to be answered and the target image into the target big model of the agent, so that the target big model can generate target response information for the question to be answered based on the information in the target image.

[0017] Furthermore, the retrieval module is specifically used to search for target nodes in the constructed memory network that have a semantic association with the question to be answered. The memory network includes multiple nodes, each node representing the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets a preset requirement, the two nodes are connected by a connecting line. The target historical interaction information to which the content represented by the target node belongs is determined.

[0018] Furthermore, the memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword. The keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated in any question and answer in the conversation unit. The retrieval module is specifically used to extract target keywords from the question to be answered; determine the semantic similarity between the target keywords and the keywords represented by each node in the keyword tag network; and determine the nodes corresponding to the keywords whose semantic similarity meets the threshold requirement as target nodes.

[0019] Furthermore, the memory network is a message association network, where any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The retrieval module is specifically used to determine the semantic similarity between the question to be answered and the content represented by each node in the message association network, and to determine the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node.

[0020] Further, the retrieval module is specifically used to determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered; to determine the node corresponding to the content with a first semantic similarity greater than a first threshold as a first node; for each first node, to determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network; to determine the node corresponding to the feature vector with a second semantic similarity greater than a second threshold as a second node; and to determine the first node and the second node as the target node.

[0021] Furthermore, the memory network is a session association network, where any node in the session association network is used to represent the content communicated by any session unit; The retrieval module is specifically used to determine the semantic similarity between the question to be answered and the content represented by each node in the session association network, and to determine the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node.

[0022] Furthermore, the device also includes: The filtering module is used to, if there are multiple target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, and the time difference between the creation time of the target historical interaction information and the current time for each target historical interaction information; determine the target ranking score corresponding to the target historical interaction information based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics; and select a preset number of target historical interaction information from the current multiple target historical interaction information based on the target ranking score corresponding to each target historical interaction information.

[0023] Furthermore, the device also includes: The generation module is configured to: if the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by the question-and-answer pair in that information tree; generate a target image corresponding to the target historical interaction information based on the analysis log and a preset image generation rule; if the target historical interaction information is the content communicated by a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by each question-and-answer pair in that information tree; generate a target image corresponding to the target historical interaction information based on each analysis log found and a preset image generation rule.

[0024] Furthermore, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; The generation module is further configured to: determine the semantic vector corresponding to each data tag in each core information item; perform dimensionality reduction processing on the semantic vector based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector, wherein each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space; determine the maximum intermediate vector and the minimum intermediate vector based on the three-dimensional intermediate vectors corresponding to all data tags of the core information item; determine the first difference between the three-dimensional intermediate vector corresponding to the data tag and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; determine the quotient of the first difference and the second difference; and round down the product of the quotient and a preset value to obtain the RGB color value corresponding to the data tag; if the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0025] Furthermore, the acquisition module is also used to acquire the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; The update module is configured to, upon receiving a memory update instruction, extract a summary to be updated based on the session to be updated carried in the instruction; calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; determine a target modification strategy based on the pre-saved correspondence between different threshold ranges and modification strategies; display the analysis log of the session to be updated in the information tree of the session unit based on the target modification strategy, and display at least one node in the memory network based on the target modification strategy to represent the semantics of the content communicated by the session to be updated.

[0026] Thirdly, embodiments of this application also provide an electronic device, the electronic device including a processor, the processor being configured to execute a computer program stored in a memory to implement the steps of the memory-based question-answering method as described above.

[0027] Fourthly, embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the memory-based question-answering method as described above.

[0028] Fifthly, embodiments of this application also provide a computer program product, the computer program product comprising: computer program code, which, when run on a computer, causes the computer to perform the steps of the memory-based question-answering method described above.

[0029] In this embodiment, target historical interaction information with semantically related to the question to be answered is searched from the stored historical interaction information. Then, a target image is obtained for this target historical interaction information. This target historical interaction information includes at least one historical question to be answered and one historical answer. Since the target image records the agent's analysis log when generating historical answer information based on the historical question to be answered, this analysis log includes at least one of the following: the question to be answered, the historical answer information, the agent's thought process, tool call information, and core information of the historical question to be answered and the historical answer information. Therefore, after obtaining the target image, the question to be answered and the target image are input into the agent's target model, so that the target model generates the target answer information for the question to be answered based on the information in the target image. Because the token length of a single image is determined by the image resolution, even if the target image records a lot of information, it will not exceed the context limit of the target model, ensuring that the agent can effectively reuse historical interaction experience. Attached Figure Description

[0030] To more clearly illustrate the technical solutions of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 A schematic diagram of a memory-based retrieval question-and-answer process provided for an embodiment of this application; Figure 2 This application provides a schematic diagram of the structure of a keyword tagging network. Figure 3This is a schematic diagram of the structure of a message association network provided in an embodiment of this application; Figure 4 This application provides a schematic diagram of the structure of a session association network. Figure 5 A schematic diagram of a memory retrieval process provided in an embodiment of this application; Figure 6 A schematic diagram of an information tree for a session unit provided in an embodiment of this application; Figure 7 A schematic diagram illustrating a label semantic-color space mapping process provided in an embodiment of this application; Figure 8 An example diagram of the layout of a generated image provided in this application embodiment; Figure 9 A schematic diagram illustrating a memory compression construction process provided in an embodiment of this application; Figure 10 A schematic diagram of a three-level tree-like storage structure provided in an embodiment of this application; Figure 11 A schematic diagram illustrating a memory update process provided in an embodiment of this application; Figure 12 A schematic diagram of a memory-based question-answering device provided in an embodiment of this application; Figure 13 This is a schematic diagram of an electronic device structure provided in an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art are within the scope of protection of this application.

[0033] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0034] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0035] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0036] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0037] The acquisition, transmission, storage, and use of data in this application all comply with the requirements of relevant national laws and regulations.

[0038] Current mainstream solutions for memory management in large-scale intelligent agents fall into three categories, all of which suffer from unresolved underlying flaws: Option 1: Text-based summarization memory: Representative solutions include the LangChain memory module and OpenAI's native memory function. This approach uses a large model to extract summaries and entities from conversations, converting them into structured text for storage. Drawbacks: Summarization extraction is limited by model capabilities, inevitably losing conversation context details and inference information related to tool calls. Furthermore, long conversation summaries accumulate errors over time. Option 2: Text Chunk Vectorization Memory: A representative solution is the Retrieval-Augmented Generation (RAG) scheme. The entire conversation text is chunked, vectorized, and stored in a vector library. During retrieval, relevant text is recalled through similarity matching. Drawbacks: Storing the entire text as a token is extremely costly; long conversation retrievals easily exceed the model's contextual limits; and it cannot uniformly handle multimodal memory such as images and audio. Option 3: Multimodal Memory Storage: Existing solutions process multimodal information such as text, images, and audio using separate text embedding, image embedding, and audio embedding models, and then store and retrieve them in separate databases. Drawbacks: Cross-modal retrieval suffers from high semantic alignment difficulty, low retrieval accuracy, and inability to achieve unified and interconnected storage of information across the entire conversation.

[0039] The underlying contradiction of the above solutions is that the higher the integrity of the memory, the higher the token cost and the lower the retrieval efficiency; the higher the compression degree, the more severe the information loss and the less the agent can reuse historical experience. Based on these problems, this application provides a memory-based question-answering method that solves this contradiction from the ground up through image compression and multimodal unified embedding. The following will provide a detailed description with reference to various embodiments.

[0040] Figure 1 A schematic diagram of a memory-based question-answering process provided in this application embodiment, the process including: S101: In the stored historical interaction information, find the target historical interaction information that has a semantic relationship with the question to be answered.

[0041] The memory-based question-answering method provided in this application is applied to electronic devices, such as servers, PCs, and mobile terminals.

[0042] In this embodiment, pending questions can be received in real time. These pending questions can be input by the user, sent by other electronic devices, or retrieved from a preset storage location, such as a database or cloud storage. The format of the pending questions can be any format, such as text, audio, image, or document. It should be noted that the pending questions are not limited to questions; any information sent by the user during interaction with the intelligent agent can be identified as pending questions.

[0043] To improve the accuracy of question answering, after receiving a question to be answered, the system accurately recalls the memory content related to that question and provides the agent with complete historical context information while controlling token consumption.

[0044] In this embodiment, historical interaction information can be stored in advance. This historical interaction information can be all the information generated when any user interacts with the intelligent agent; it can also be all the information generated when the user who sent the question to be answered interacts with the intelligent agent within a set time period in the past; or it can be a large amount of interaction information collected in advance and entered by relevant technical personnel.

[0045] After receiving a question to be answered, the system searches for target historical interaction information that is semantically related to the question. For example, the semantic similarity between the semantics of the question to be answered and the semantics of each piece of historical interaction information can be determined, and historical interaction information with a semantic similarity greater than a set threshold is identified as target historical interaction information with a related relationship.

[0046] In this embodiment, when determining whether a semantic relationship exists, a first semantic feature vector for the question to be answered and a second semantic feature vector for the communication content corresponding to each historical interaction can be determined. Then, the cosine similarity between the first semantic feature vector and each second semantic feature vector is determined. Historical interaction information corresponding to second semantic feature vectors with a cosine similarity greater than a preset threshold is identified as target historical interaction information. When determining whether a semantic relationship exists, a trained model can also be used for identification. This trained model can be a large model or other models. Of course, those skilled in the art can also use other semantic analysis methods to determine target historical interaction information that has a semantic relationship with the question to be answered.

[0047] S102: Obtain the target image saved for the target historical interaction information, wherein the target historical interaction information includes at least one historical unanswered question and one historical answer information, and the target image records the analysis log of the agent when generating the historical answer information based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer information, the agent's thinking content, tool call information, and core information of the historical unanswered question and the historical answer information.

[0048] In this embodiment, any historical interaction information may include only one question-and-answer pair, i.e., one historical question to be answered and one historical answer, or it may include multiple question-and-answer pairs. A corresponding image is saved for each historical interaction information, and this image records the analysis log of the agent generating historical answer information based on the historical question to be answered. This analysis log includes at least one of the following: the question to be answered, the historical answer information, the agent's thought process, tool call information, and core information of the historical question to be answered and the historical answer information. The core information may include the generation time of the historical question to be answered and the historical answer information, the interaction time, and a content summary, etc.

[0049] In other words, the information generated during any historical interaction is stored as an image. In this embodiment, all information can be recorded in the same image or in different images.

[0050] After determining the target's historical interaction information, the target image saved based on that historical interaction information can be obtained.

[0051] S103: Input the question to be answered and the target image into the target big model of the agent, so that the target big model generates target answer information for the question to be answered based on the information in the target image.

[0052] Because the target image contains analysis logs of the agent generating historical responses based on past unanswered questions, the target model invoked by the agent can improve the accuracy of question-and-answer generation by referring to these logs. Therefore, after obtaining the target image, the unanswered question and the target image can be input into the agent's target model, enabling the model to generate the target response based on the information in the target image.

[0053] Since the token length of a single image is determined by the image resolution, even if the target image contains a lot of information, it will not exceed the context limit of the target large model, ensuring that the agent can effectively reuse historical interaction experience.

[0054] In one possible implementation, if the target large model invoked by the agent is a multimodal large model, the acquired target image can be directly input into the target large model.

[0055] In one possible implementation, if the target large model invoked by the agent is a plain text large model, then the target image can be converted into structured text using a Vision-Language Model (VLM), and the core content of the structured text can be extracted. The total token consumption of the extracted core content is limited to a set value, such as within 3000. Then, the target's historical interaction information and the core content extracted from the target image are input into the target large model.

[0056] In one possible implementation, if multiple target images are obtained, then N images can be selected from these multiple target images, and these N target images, along with the question to be answered, can be input into the target large model. For example, N can be any value such as 2, 3, or 4. Those skilled in the art can configure this according to the context limit of the target large model; for example, if the total token consumption is configured to be controlled within 2000, then N can be set to 3.

[0057] In this embodiment, target historical interaction information with semantically related to the question to be answered is searched from the stored historical interaction information. Then, a target image is obtained for this target historical interaction information. This target historical interaction information includes at least one historical question to be answered and one historical answer. Since the target image records the agent's analysis log when generating historical answer information based on the historical question to be answered, this analysis log includes at least one of the following: the question to be answered, the historical answer information, the agent's thought process, tool call information, and core information of the historical question to be answered and the historical answer information. Therefore, after obtaining the target image, the question to be answered and the target image are input into the agent's target model, so that the target model generates the target answer information for the question to be answered based on the information in the target image. Because the token length of a single image is determined by the image resolution, even if the target image records a lot of information, it will not exceed the context limit of the target model, ensuring that the agent can effectively reuse historical interaction experience.

[0058] To improve the efficiency of memory retrieval, based on the above embodiments, in this embodiment, the step of searching for target historical interaction information that has a semantic relationship with the question to be answered in the stored historical interaction information includes: In the constructed memory network, target nodes that are semantically related to the question to be answered are searched. The memory network includes multiple nodes, and each node is used to represent the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line. Determine the target historical interaction information to which the content represented by the target node belongs.

[0059] In this embodiment, a memory network is pre-constructed, comprising multiple nodes. Each node represents the content communicated by any session unit or any question-and-answer pair within a session unit. A session unit can be understood as a set of multiple question-and-answer pairs; that is, a session unit includes multiple question-and-answer pairs, and the question-and-answer pairs within the same session unit belong to the same dialogue context. Since the content communicated by any session unit or any question-and-answer pair within a session unit may include a large amount of information, to save storage space, in this embodiment, the content represented by any node can be a summary of the corresponding communicated content. Of course, the content represented by any node can also be the original text of the corresponding communicated content.

[0060] When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line.

[0061] For example, when the semantic similarity between the content represented by node A and the content represented by node B is greater than a preset threshold, node A and node B can be connected using a connecting line. This preset threshold can be configured as needed by those skilled in the art; for example, the preset threshold can be any value such as 0.8, 0.9, or 0.98.

[0062] In this embodiment of the application, when searching for target historical interaction information that is semantically related to the question to be answered, the semantic similarity between the semantics of the question to be answered and the semantics of the content represented by each node can be determined in the constructed memory network, and the node corresponding to the content with a semantic similarity greater than a threshold is determined as the target node.

[0063] After identifying the target node, the content represented by the target node is obtained, and the target historical interaction information to which this content belongs is determined. In other words, it is determined from which target historical interaction information the content represented by the target node originates.

[0064] To further improve the efficiency of memory retrieval, based on the above embodiments, in this embodiment, the memory network is a keyword tagging network, any node in the keyword tagging network is used to represent a keyword, and the keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated based on any question and answer in the conversation unit; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: Extract the target keywords from the questions to be answered; The semantic similarity between the target keyword and the keyword represented by each node in the keyword tag network is determined, and the node corresponding to the keyword whose semantic similarity meets the threshold requirement is determined as the target node.

[0065] In this embodiment, the constructed memory network can be a keyword labeling network, where any node represents a keyword. The keyword is extracted based on the content communicated in any conversation unit, or it can be extracted based on the content communicated in any question-and-answer pair within the conversation unit. That is, the keyword can be extracted from a single round of dialogue messages (i.e., one question-and-answer pair), or it can be extracted from all dialogue messages in a conversation unit (i.e., multiple question-and-answer pairs).

[0066] In one possible implementation, a pre-trained model can be used to perform semantic recognition on the content communicated by question-and-answer pairs or by conversational units. This recognition identifies information such as the topic, time frame, and retrieval requirements within the content communicated by the question-and-answer pairs or conversational units, and the identified information is then designated as keywords. When constructing the keyword tagging network, any node can be used to represent the keywords of the content communicated by any question-and-answer pair or any conversational unit.

[0067] In a keyword tagging network, edges (connectors) represent the semantic similarity between keywords. Specifically, each edge can be assigned a weight value, which represents the cosine similarity of the vectors of the keywords represented by the corresponding nodes. This keyword tagging network can be used for fast keyword-level retrieval and related recommendations.

[0068] For example, when determining any node and other nodes connected to it, the semantic similarity between the keyword represented by the node and the keyword represented by each other node can be determined. Other nodes with a semantic similarity greater than a similarity threshold are connected to the node. The similarity threshold can be 0.75, or of course, other values. Those skilled in the art can configure it as needed.

[0069] Figure 2 This is a schematic diagram of the structure of a keyword tagging network provided in an embodiment of this application, such as... Figure 2 As shown, this keyword tagging network includes multiple nodes. The "tag" identified in any node can be understood as the keyword tag represented by that node. Any node in this keyword tagging network can be connected to one or more other nodes using connecting lines. Each connecting line is assigned a semantic similarity weight, which is used to represent the cosine similarity between the content represented by the corresponding nodes.

[0070] In this embodiment of the application, when searching for target nodes that are semantically related to the question to be answered in the constructed memory network, target keywords in the question to be answered can be extracted. For example, a pre-trained model can be used to perform semantic recognition on the question to be answered, identifying information such as the topic, time range, and search requirements included in the question to be answered, and the identified information is determined as target keywords.

[0071] In one possible implementation, if the question to be answered is plain text data, a pre-trained model can be used to extract the target keywords of the question to be answered.

[0072] In one possible implementation, if the question to be answered is multimodal data—that is, if the question includes multimodal files, such as images or documents—then a Vision-Language Model (VLM) can be used to extract the target keywords for the question.

[0073] After determining the target keywords, the semantic similarity between the target keywords and the keywords represented by each node in the keyword tag network is determined. The nodes corresponding to the keywords whose semantic similarity meets the threshold requirement are determined as the target nodes.

[0074] In this embodiment, a coarse search is performed using a keyword tag network, and relevant tags are quickly matched using the keyword tag network to recall conversation units and / or question-answer pairs related to the question to be answered, achieving millisecond-level response, which can be used for rapid preview and filtering.

[0075] To further improve the efficiency of memory retrieval, based on the above embodiments, in this embodiment of the application, the memory network is a message association network, and any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the message association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0076] In this embodiment, the constructed memory network can be a message association network, where any node in the message association network is used to represent the content contained in the analysis log of any question-answer pair. That is, any node in the message association network is used to represent the content included in the image corresponding to any question-answer pair.

[0077] In this embodiment, analysis logs corresponding to question-answer pairs can be obtained, and features can be extracted from these logs to obtain embedding vectors. Each node in the message association network is used to represent the extracted embedding vector. Edges (connectors) in the message association network represent the semantic similarity between question-answer pairs. Specifically, each edge can be assigned a weight value, which represents the cosine similarity between the content represented by the corresponding nodes. This message association network can be used for precise message-level retrieval, i.e., precise retrieval at the question-answer pair level.

[0078] For example, when determining any node and its connected nodes, the semantic similarity between the content represented by that node and the content represented by each other node can be determined. Other nodes with semantic similarity greater than a similarity threshold are then connected to that node. This similarity threshold can be 0.8, or other values. Alternatively, when determining any node and its connected nodes, the semantic similarity between that node and each other node can be determined. The semantic similarities are then sorted in descending order, and the top N nodes with the highest semantic similarity are selected and connected to that node. Here, N can be any integer such as 3, 5, or 6, and can be configured as needed by those skilled in the art.

[0079] Figure 3 This is a schematic diagram of the structure of a message association network provided in an embodiment of this application, such as... Figure 3As shown, this message association network includes multiple nodes. The "single-round message image embedding vector" identified in any node can be understood as the embedding vector of the analysis log of the corresponding question-answer pair represented by that node. Any node in this message association network can be connected to one or more other nodes using connecting lines. Each connecting line is assigned a semantic similarity weight to represent the cosine similarity between the content represented by the corresponding nodes.

[0080] In this embodiment of the application, when searching for a target node in the constructed memory network that is semantically related to the question to be answered, the semantic vector of the question to be answered can be extracted.

[0081] In one possible implementation, if the question to be answered is plain text data, a text embedding model can be used to generate a semantic vector for the question. For example, a 768-dimensional semantic vector can be generated. Of course, the semantic vector can also have other dimensions, which can be configured as needed by those skilled in the art.

[0082] In one possible implementation, if the question to be answered is multimodal data—that is, if the question includes multimodal files, such as images or documents—a multimodal embedding model can be used to generate embedding vectors for the files. For example, a 768-dimensional embedding vector can be generated.

[0083] After determining the semantic vector in the question to be answered, the semantic similarity between this semantic vector and the vector representing the content of each node in the message association network is determined. Nodes corresponding to content with semantic similarity meeting a threshold requirement are identified as target nodes. For example, nodes corresponding to content with semantic similarity ≥ 0.6 are identified as target nodes.

[0084] In this embodiment of the application, a fine retrieval is performed through a message association network. The semantic vector of the question to be answered is compared with the vector of the content represented by each node in the message association network by cosine similarity calculation, thereby realizing fine retrieval at the question-answer pair level.

[0085] To improve the accuracy of memory retrieval, based on the above embodiments, in this embodiment, the step of determining the semantic similarity between the question to be answered and the content represented by each node in the message association network, and determining the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node, includes: Determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered, and determine the node corresponding to the content with the first semantic similarity greater than the first threshold as the first node; For each first node, determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network, and determine the node corresponding to the feature vector with the second semantic similarity greater than the second threshold as the second node; The first node and the second node are identified as the target nodes.

[0086] In order to improve the accuracy of question answering by filtering out multiple content nodes that are semantically related to the question to be answered, in this embodiment of the application, the first semantic similarity between the content represented by each node in the message association network and the question to be answered can be determined, and the node corresponding to the content whose first semantic similarity is greater than a first threshold is determined as the first node.

[0087] Since the semantics of the content represented by other nodes associated with each first node may also be related to the semantics of the question to be answered, after determining each first node, for each first node, a second semantic similarity is determined between the content represented by that first node and the content represented by other nodes in the message association network. Nodes corresponding to feature vectors with a second semantic similarity greater than a second threshold are determined as second nodes. These second nodes can be nodes in the message association network connected to the first node by a connecting line, or nodes not connected to the first node by a connecting line.

[0088] The first and second nodes are identified as target nodes. That is, the first node that is highly semantically related to the question to be answered is first identified, and then the scope is expanded based on the first node to find other second nodes that are semantically similar to the first node, which improves the accuracy of target node recall.

[0089] To further improve the efficiency of memory retrieval, based on the above embodiments, in this embodiment, the memory network is a session association network, and any node in the session association network is used to represent the content communicated by any session unit; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the session association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0090] In this embodiment, the constructed memory network can be a session association network, where any node in the session association network is used to represent the content communicated by any session unit. That is, the session association network is constructed in advance based on existing session units.

[0091] In one possible implementation, when constructing the session association network, a summary of the content communicated by the session units can be extracted, with each node representing the extracted summary. Edges (i.e., connections) in the session association network are used to represent the semantic similarity between session units. Specifically, each edge can be assigned a weight value, which represents the cosine similarity between the content represented by the corresponding nodes. This session association network can be used for global retrieval and context reconstruction at the session level, i.e., precise retrieval at the session unit level.

[0092] For example, when determining any node and other nodes connected to it, the semantic similarity between the content represented by the node and the content represented by each other node can be determined, and other nodes with semantic similarity greater than a similarity threshold can be connected to the node. The similarity threshold can be 0.85, or of course, other values. Those skilled in the art can configure it as needed.

[0093] Figure 4 This is a schematic diagram of the structure of a session association network provided in an embodiment of this application, such as... Figure 4 As shown, this session association network includes multiple nodes. The "session summary embedding vector" identified in any node can be understood as the embedding vector corresponding to the summary of the content communicated by the session unit represented by that node. Any node in this session association network can be connected to one or more other nodes using connecting lines. Each connecting line is assigned a relevance weight to represent the cosine similarity between the content represented by the corresponding nodes.

[0094] In this embodiment of the application, when searching for target nodes that are semantically related to the question to be answered in the constructed memory network, the semantic similarity between the question to be answered and the content represented by each node in the session association network can be determined, and the nodes corresponding to the content whose semantic similarity meets the threshold requirement can be determined as target nodes.

[0095] In one possible implementation, a summary of the question to be answered can be extracted. For example, a text embedding model can be used to extract the summary of the question to be answered, and a text vector corresponding to the summary can be generated.

[0096] After determining the text vector in the question to be answered, the semantic similarity between this text vector and the vector representing the content of each node in the session association network is determined. Nodes corresponding to content with semantic similarity meeting a threshold requirement are identified as target nodes. For example, nodes corresponding to content with semantic similarity ≥ 0.6 are identified as target nodes.

[0097] In this embodiment of the application, when extracting vectors, a multimodal embedding model with image-text alignment can be used. The similarity between text vectors and image vectors can be calculated directly without converting the text to an image, thus ensuring the accuracy of cross-modal retrieval.

[0098] In one possible implementation, when a user needs the complete session context, the images corresponding to the full analysis log of the entire session are retrieved through the session association network, assembled in chronological order, and returned.

[0099] In the embodiments of this application, the constructed keyword tag network, message association network and session association network share the same semantic vector space. During retrieval, hierarchical penetration retrieval of "keyword → message → session" can be achieved, as well as precise filtering retrieval at a single level.

[0100] To further improve the accuracy of question answering, based on the above embodiments, in this embodiment, after finding the target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the method further includes: If there are multiple target historical interaction information, for each target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time; based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, determine the target ranking score corresponding to the target historical interaction information. Based on the target ranking score corresponding to each target's historical interaction information, a preset number of target historical interaction information are selected from the current multiple target historical interaction information.

[0101] Since there may be multiple target historical interaction information retrieved through semantic similarity matching, inputting images corresponding to too many target historical interaction information into the target large model may still lead to exceeding the context limit. In this embodiment, a hybrid sorting rule of "semantic similarity × time decay coefficient" can be used to filter the obtained target historical interaction information. For example, semantic similarity accounts for 70% of the weight, and time accounts for 30%, ensuring that the most relevant and recent memories are ranked first.

[0102] Therefore, in this embodiment of the application, if multiple target historical interaction information are obtained, the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time, can be determined for each target historical interaction information. Based on this time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, the target ranking score corresponding to the target historical interaction information is determined.

[0103] Specifically, the target ranking score corresponding to any target's historical interaction information can be determined based on the following formula:

[0104] in, This represents the target ranking score of the historical interaction information of the m-th target; the higher the score, the higher the ranking. This represents the semantic similarity weights for semantic preservation, such as... ; This indicates the time decay weight for saving time differences, such as, =0.3; This represents the semantic similarity between the historical interaction information of the m-th target and the question to be answered, with a value range of [0, 1]. This represents the time difference between the creation time of the m-th target's historical interaction information and the current time, in days. Represents the time decay coefficient, such as, ,Will Setting it to 0.05 ensures a gradual decay of memory weight within 30 days, while memory weight decreases rapidly after 90 days. This represents the time decay factor, which ranges from (0, 1). The newer the memory, the closer the time decay factor is to 1.

[0105] It should be noted that the examples given above for semantic similarity weight, time decay weight, and time decay coefficient are not limited to these, and those skilled in the art can configure them as needed.

[0106] After determining the target ranking score corresponding to each target historical interaction information, a preset number of target historical interaction information can be selected from the current multiple target historical interaction information based on the target ranking score corresponding to each target historical interaction information.

[0107] Specifically, the historical interaction information of each target is arranged in descending order of target ranking score, and the top N historical interaction information of the targets are selected. For example, N = 10, 5, or any other integer.

[0108] In one possible implementation, to improve the efficiency of memory retrieval, data filtering requirements can be pre-configured in this embodiment. After searching for target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the obtained target historical interaction information can be filtered based on the data filtering requirements to remove results that do not meet the requirements. For example, precise filtering can be performed on information such as user input, agent output, tool calls, time range, tags, and session IDs.

[0109] In one possible implementation, the acquired target historical interaction information may contain multiple target historical interaction information messages that all originate from the same session. Therefore, in this embodiment, the continuous dialogue information of the same session unit can be merged to avoid redundancy in the results.

[0110] The following is combined Figure 5 The memory-based question-answering process provided in this embodiment will be described. Figure 5 This is a schematic diagram of a memory retrieval process provided in an embodiment of this application.

[0111] In this embodiment, the core objective of memory retrieval is to accurately recall relevant memory content based on user input, providing the agent with complete historical context while controlling token consumption. The retrieval process consists of five standardized steps and supports multi-dimensional precise filtering retrieval.

[0112] Step 1: Query Preprocessing. The user-input query is parsed and preprocessed. Information such as the summary, keywords, and semantic vectors of the query are extracted.

[0113] Step Two: Multi-path Parallel Retrieval. Based on the preprocessed query vector and keywords, a three-layer network is used for parallel retrieval, supporting both coarse and fine retrieval modes simultaneously. Coarse search: Through a keyword tag network, it quickly matches relevant tags, retrieves the corresponding conversation and message list, and achieves millisecond-level response for rapid preview and filtering; Detailed retrieval: By using message association networks and session association networks, the query vector is compared with the image embedding vector and summary vector in the vector library using cosine similarity calculation, and the top 20 relevant memories with similarity ≥ 0.6 are retrieved.

[0114] Step 3: Structure merging and filtering.

[0115] Filtering rules: Supports precise filtering by message category (user input / agent output / tool ​​call), time range, tag, and session ID to filter out results that do not meet the requirements; at the same time, it merges consecutive messages belonging to the same session to avoid result redundancy.

[0116] Step 4: Mixed reordering.

[0117] A hybrid ranking rule of "similarity × time decay coefficient" is used to determine the target ranking score, and based on the target ranking score, a set number of target historical interaction information are selected, such as selecting the top 10 target historical interaction information.

[0118] Step 5: Assemble the memorized content.

[0119] The top 10 target historical interaction messages after reordering are assembled into content, with two modes available for selection as needed: Image mode: Directly select images of the corresponding target historical interaction information, up to 3 images, with total token consumption controlled within 2000, and directly provide them to multimodal large models; Text mode: Converts images into structured text using VLM, extracts core content, and keeps total token consumption under 3000, providing it for use by large pure text models.

[0120] Finally, the assembled memory content is returned to the target large model of the agent. Simultaneously, the keywords, recall results, and user and agent feedback are recorded for subsequent optimization of search thresholds and ranking rules.

[0121] To reduce storage costs, based on the above embodiments, the process of determining the target image corresponding to the target historical interaction information in this application embodiment includes: If the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by that question-and-answer pair in the information tree; generate the target image corresponding to the target historical interaction information according to the analysis log and the preset image generation rules; If the target historical interaction information is the content communicated by the conversation unit, the information tree of the conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by each question and answer pair in the information tree is searched. Based on each analysis log found and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

[0122] To facilitate the storage of analysis logs related to each historical interaction, an information tree is pre-constructed for each session unit in this embodiment. The root node of this information tree represents the session-level basic information of the corresponding session unit, including the session identifier, session start time, and total number of session rounds. The first-level child nodes of this information tree represent the single-round message-level information of each question-and-answer pair within the session unit, including the question-and-answer pair identifier, the session unit identifier, and a timestamp. The second-level child nodes of this information tree represent the analysis logs of the question-and-answer pairs of the corresponding parent child nodes.

[0123] In other words, in this embodiment, historical interaction information is divided into a three-level structure: "session level - single-round message level - message category level," completely preserving all interaction information without any loss. Specifically: Basic session information: includes a unique session ID, session start time, session end time, and total number of session rounds. The session start / end time is automatically generated based on the time of the user's first and last interaction. The unique session ID is used to build the root node of the tree-like memory structure. Single-round message level information: A single-round message is a closed loop of "one user input + one complete response from the agent". Each round message contains a unique round ID, the session ID to which it belongs, and the round timestamp, which serve as a first-level child node in a tree-like memory structure. Message category-level information: Each round of messages is broken down into analysis logs based on content type, specifically into 6 fixed categories, which serve as second-level child nodes in a tree-like memory structure, including: ① User input content: It is divided into two parts: one is the text information entered by the user, and the other is the multimodal files such as images, audio, documents, code, and tables uploaded by the user. The storage path and preview thumbnail of the original file are retained for the multimodal files. ② The agent's thinking content: The agent's understanding of the user's task, task breakdown, execution plan, and reasoning chain constitute the agent's complete thinking process. This process has a "planning-execution" sequence with the subsequent tool calls, and there is no overlap in content. ③ Tool call information: It includes four parts: tool name, tool input parameters, tool execution output results, and agent's observation and judgment of the results. It fully retains the entire link information of the tool call, including complete logs of tool execution failures and retries; ④ Agent's conclusion output: The final answer the agent provides to the user's question based on the task execution situation; ⑤ Session metadata: includes the timestamp of this round of messages, the duration of the interaction, and user feedback (if any). The user feedback can be understood as the user's evaluation of the response information. ⑥ Conversation summary and tags, which are the core information of the interaction.

[0124] Figure 6 This is a schematic diagram of an information tree for a session unit provided in an embodiment of this application, such as... Figure 6 As shown, this information tree consists of three levels. The first level is the root node, representing basic session-level information, including a unique session ID, session start time, session end time, and total number of sessions. The second level consists of first-level child nodes, representing single-round message-level information, including a unique round ID, the session ID it belongs to, and a round timestamp. The third level consists of second-level child nodes, representing message category-level information, i.e., analysis logs. Specifically, this includes information such as user input, agent thought processes, tool call information, agent conclusions, session metadata (session summary and tags), as shown in the curly braces below any second-level child node in the figure.

[0125] In this embodiment of the application, an information tree is used to record relevant information for each session, ensuring a clear hierarchy of session information and enabling accurate tracing from session to single-round message.

[0126] When determining the target image corresponding to the target historical interaction information, if the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, the information tree of that conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by that question-and-answer pair in that information tree is retrieved. Based on the obtained analysis log and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

[0127] If the target historical interaction information is the content communicated by the conversation unit, search for the information tree of the conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by each question and answer pair in the information tree. Based on each analysis log found and the preset image generation rules, generate the target image corresponding to the target historical interaction information.

[0128] In this embodiment of the application, image layout rules can be specified in the preset image generation rules, such as pre-defining the arrangement order of each piece of information.

[0129] To improve the clarity of image content representation, based on the above embodiments, in this application embodiment, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; After searching the analysis log of each question-answer pair included in the target historical interaction information in the saved session information tree, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, the method further includes: For each data tag in each core piece of information, determine the semantic vector corresponding to that data tag; The semantic vector is reduced in dimensionality based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. Each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space. Based on the three-dimensional intermediate vectors corresponding to all data labels of this core information, determine the maximum and minimum intermediate vectors; Determine the first difference between the three-dimensional intermediate vector corresponding to the data label and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; Determine the quotient of the first difference and the second difference; The product of the quotient and the preset value is rounded down to obtain the RGB color value corresponding to the data label; Based on the analysis logs and preset image generation rules, a target image corresponding to the target historical interaction information is generated, including: If the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0130] In this embodiment, the core information in the analysis log may include at least one of the following: summary, topic tags, entity tags, and user preference tags. To ensure the accuracy of the information description, any one of the core information items may include multiple data tags. For example, for the topic tag core information, this core information may include multiple data items used to describe the topic, and these data items used to describe the topic are the data tags. For example, the data tags included in the topic tag core information may be "Python code development," "program development," etc.; the data tags included in the entity tag core information may be "MySQL database," "database type," "database," etc.; and the data tags included in the user preference tag core information may be "prefers concise code examples."

[0131] In this embodiment, after finding the analysis log for each question-answer pair, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, a semantic vector corresponding to each data tag in each core piece of information can be determined. The dimension of this semantic vector can be configured as needed by those skilled in the art. Subsequently, the background color corresponding to the data tag can be determined based on this semantic vector to improve the readability of the image.

[0132] In this embodiment, an order-preserving mapping from "semantic vector space to RGB color space" is established for the extracted data tags to ensure the consistency between semantic relevance and color visual relevance, providing a foundation for subsequent image generation.

[0133] In this embodiment, the semantic vector is dimensionality-reduced based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. To enable the use of different colors to represent different data labels and to map semantics to label colors, in this embodiment, each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space.

[0134] Specifically, the semantic vector corresponding to any data label can be determined based on the following formula:

[0135] in, This represents the semantic vector corresponding to the i-th data label, such as a 768-dimensional high-dimensional semantic vector. This represents the i-th data label; This represents a pre-trained text embedding model used to map labeled text into semantic vectors of fixed dimensions.

[0136] After generating semantic vectors based on the above formulas, the semantic space to color space can be mapped using the following forward order-preserving dimensionality reduction mapping formula:

[0137] in, This represents the three-dimensional intermediate vector after dimensionality reduction of the i-th data label; This represents a pre-trained, fixed-parameter uniform manifold approximation and projection (U-Map) dimensionality reduction model. This model can fix the random seed and the number of iterations to ensure that the semantic relative correlation between labels is preserved during the dimensionality reduction process.

[0138] After determining the three-dimensional intermediate vectors of all data labels corresponding to the core information, the maximum and minimum intermediate vectors can be determined based on the three-dimensional intermediate vectors of all data labels corresponding to the core information.

[0139] In this embodiment, for each channel in the RGB color space, the color value corresponding to that channel can be obtained from all three-dimensional intermediate vectors. The maximum value among all color values ​​is determined as the maximum value of that channel, and the minimum value among all color values ​​is determined as the minimum value of that channel. Then, based on the maximum value of each channel, a maximum intermediate vector is constructed. Based on the minimum value of each channel, a minimum intermediate vector is constructed.

[0140] For example, any three-dimensional intermediate vector is [A, B, C], where the first component value A corresponds to channel R in the RGB color space, the second component value B corresponds to channel G in the RGB color space, and the third component value C corresponds to channel B in the RGB color space. By statistically analyzing all the three-dimensional intermediate vectors of this core information, the maximum value for channel R is RA, the minimum value for channel R is RB, the maximum value for channel G is GA, the minimum value for channel G is GB, the maximum value for channel B is BA, and the minimum value for channel B is BB. Based on the maximum value of each channel, the maximum intermediate vector can be constructed as [RA, GA, BA]. Based on the minimum value of each channel, the minimum intermediate vector can be constructed as [RB, GB, BB].

[0141] To achieve the mapping from semantic space to color space, in this embodiment, a first difference between the three-dimensional intermediate vector corresponding to the data tag and the minimum intermediate vector, and a second difference between the maximum intermediate vector and the minimum intermediate vector can be determined. The quotient of the first and second differences is then determined, and the product of this quotient and a preset value is rounded down to obtain the RGB color value corresponding to the data tag. The preset value can be 255.

[0142] Specifically, the mapping from semantic space to color space can be achieved based on the following formula:

[0143] in, Represents the RGB color value corresponding to the i-th data label, which is a 3-dimensional integer vector with a value range of [0, 255]. Represents the minimum intermediate vector; Represents the maximum intermediate vector; This represents the rounding function, matching the integer value requirements of the RGB color space; 255 in the above formula is the preset value. , Used for normalization.

[0144] When generating a target image corresponding to the target historical interaction information based on one or more analysis logs and preset image generation rules, it can be determined whether the analysis logs contain core information. If it is determined that any analysis log contains core information, then when displaying each data tag of the core information in the image, the RGB color value corresponding to the data tag is used as the background color of the corresponding data tag.

[0145] The following is combined Figure 7 The mapping process is explained. Figure 7This is a schematic diagram illustrating a label semantic-color space mapping process provided in an embodiment of this application.

[0146] First, obtain the tag set, which includes data tags. Then, generate a semantic vector corresponding to each data tag.

[0147] After obtaining each semantic vector, an order-preserving dimensionality reduction process is performed on each semantic vector. Specifically, the semantic vectors are dimensionality reduced based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector.

[0148] After order-preserving dimensionality reduction, 3D vector normalization is performed. Specifically, the first difference between the 3D intermediate vector and the minimum intermediate vector corresponding to the data label, and the second difference between the maximum and minimum intermediate vectors are determined; the quotient of the first difference and the second difference is determined; the product of the quotient and a preset value is rounded to obtain the RGB color value corresponding to the data label. The obtained RGB color value is the label-color mapping result.

[0149] In this embodiment of the application, when generating the target image, the full-link structured information of the session can be combined with the color mapping of the tags to generate a standardized image that fully preserves all information.

[0150] For example, the pre-configured image generation rules are as follows: Granularity division: Supports two image granularities, which can be selected as needed: one is to generate an image for a single round of messages (i.e., a question-and-answer pair), and the other is to generate a long image for the entire conversation unit; Standardized format: The image resolution is fixed at 1080px in width, the height adapts to the content, the background is pure white, the font is a monospaced boldface, the font size is fixed at 16px, and the line spacing is fixed at 20px. Color mapping rule: Each piece of core information corresponds to one or more core data tags, and the background color of that data tag is the color corresponding to the RGB color value mapped to that data tag. That is, the core information in the image is highlighted using the color of the corresponding data tag.

[0151] Content layout: The content is arranged in a hierarchical manner according to "basic conversation information → single-round messages → analysis logs". Each message category is separated by a dividing line, and all structured information is fully preserved without any omissions.

[0152] It should be noted that those skilled in the art can also configure other preset image generation rules as needed.

[0153] In one possible implementation, when the received question to be answered is a multimodal file such as an image, document, or table, a preview thumbnail and file path label can be generated within the corresponding image. If the question to be answered includes an audio file, a waveform diagram of the audio file and its transcribed text can be generated within the image, thus ensuring that the multimodal information is fully integrated into the image.

[0154] Figure 8 An example layout diagram of a generated image provided in this application embodiment, such as... Figure 8 As shown, the image includes a user input area, an agent thought process area, a tool call information area, an agent conclusion output area, and a summary and tag area. The user input area records the user's question (i.e., the question to be answered). Since the question includes both text and images, a preview thumbnail is displayed in the user input area. The agent thought process area records the agent's thought process based on the user's data. The tool call information area records information about all tools used by the agent to generate the answer to the user's question. The agent conclusion output area records all information displayed to the user by the agent in response to the user's question. The summary and tag area uses different background colors to display the tags and / or summary related to the user's question.

[0155] In one possible implementation, when mapping the semantic space to the color space, it is necessary to ensure that the mapping satisfies the requirements of consistency and reversibility. Consistency is also known as order preservation.

[0156] Consistency (order preservation): This ensures that the semantic similarity of data labels is positively correlated with their visual similarity in the color space; that is, the more semantically similar the data labels, the closer their mapped colors will be. Specifically, consistency can be verified based on the following formula.

[0157] First, the semantic similarity between data tags is calculated based on the following semantic similarity calculation formula:

[0158] in, The semantic similarity between the i-th data tag and the j-th data tag of any core information is represented, with a value range of [0, 1]. The closer the value is to 1, the more similar the semantics are. This represents the dot product of the semantic vectors of the i-th data label and the semantic vector of the j-th data label; This represents the L2 norm of the corresponding semantic vector.

[0159] Then, calculate the color similarity between data labels based on the following color similarity calculation formula:

[0160] in, The color similarity between the i-th data tag and the j-th data tag of any core information is represented, with a value range of [0,1]. The closer the value is to 1, the more similar the visual colors are. This represents the RGB color value corresponding to the i-th data label; This represents the RGB color value corresponding to the j-th data label.

[0161] Finally, based on the following core formula for consistency verification, we verify whether the mapping process of each data tag in the corresponding core information meets the consistency requirements:

[0162] in, The Pearson correlation coefficient, representing semantic similarity and color similarity, ranges from [-1, 1]. This represents the function for calculating the Pearson correlation coefficient. , This represents the set of semantic similarity and the set of color similarity for all data tags in the corresponding core information.

[0163] In this embodiment, after determining the RGB color value corresponding to each data tag of any core information, the Pearson correlation coefficient corresponding to that core information can be determined based on the above formula. And determine the Pearson correlation coefficient. If the preset threshold requirement is met, then the consistency requirement can be determined, and the subsequent steps of generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules can continue.

[0164] Specifically, consistency verification may include the following steps: Step 1: Use a text embedding model (such as BERT, text-embedding-ada-002) to generate a 768-dimensional semantic vector for all data labels of any core information item; Step 2: Use the U-Map dimensionality reduction algorithm to reduce the high-dimensional semantic vector to three dimensions. The dimensionality reduction process uses a fixed random seed and has ≥200 iterations to ensure that the dimensionality reduction result preserves the local and global semantic relevance. Step 3: Normalize the 3D reduced vector to the 0-255 range and directly map it to the R, G, and B channels of the RGB color space to obtain the unique RGB color value corresponding to each label; Step 4: Verify consistency using the Pearson correlation coefficient. The correlation coefficient between semantic similarity and color similarity must be ≥0.95 to ensure that the mapping is ordered. If this requirement is met, continue with the subsequent steps of generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules.

[0165] Reversibility: Ensuring bidirectional reconstruction between data tag semantics and color; that is, color can be obtained through semantic mapping of data tags, and the semantic relevance of data tags can be reversed through color. Specifically, reversibility can be verified based on the following reverse reconstruction formula (the core formula for reversibility):

[0166] in, This represents a high-dimensional semantic vector that is inversely derived from the RGB color vector; This represents the inverse transformation function of the fixed U-Map model, enabling the reverse reconstruction from a 3D color vector to a high-dimensional semantic vector; This represents the RGB color value corresponding to the i-th data label; Represents the maximum intermediate vector; This represents the smallest intermediate vector.

[0167] In this embodiment of the application, after determining the RGB color value corresponding to any data tag, the high-dimensional semantic vector corresponding to the data tag can be determined based on the above formula. And determine the high-dimensional semantic vector. The semantic similarity between the semantic vector and the semantic vector corresponding to the original extracted data label is used to determine whether the mapping process meets the reversibility requirement. If the semantic similarity is greater than a preset threshold, the mapping process can be determined to meet the reversibility requirement.

[0168] Specifically, a fixed dimensionality reduction model pre-trained with U-Map is used to achieve a forward mapping from "semantic vector to 3D color vector". At the same time, the inverse transformation of U-Map is used to achieve a reverse restoration from "3D color vector to high-dimensional semantic vector". If the cosine similarity between the reverse restored semantic vector and the original semantic vector is ≥0.9, then the reversibility requirement can be met, and the subsequent steps of generating the target image corresponding to the target historical interaction information based on the analysis log and the preset image generation rules can continue.

[0169] In one possible implementation, the background color of the newly added data tags needs to meet the requirement of dynamism. Dynamism can be understood as: after adding a tag, the color mapping can be dynamically adjusted while ensuring the semantic-color correspondence of the original tags remains stable, preventing a global color reset. Specifically, when any new data tag is added, the RGB color value corresponding to the new data tag can be determined based on the following dynamic incremental mapping formula (the core formula for dynamism):

[0170] in, This represents the RGB color value corresponding to the j-th newly added data label; This represents the high-dimensional semantic vector of the j-th newly added data label; , This represents the maximum and minimum intermediate vectors corresponding to the core information to which the newly added data label belongs. When adding a new label, the maximum and minimum intermediate vectors remain fixed to ensure that the color mapping of the original label remains completely unchanged.

[0171] Specifically, the dynamic nature of the RGB color values ​​of data tags can be ensured based on the following steps: Step 1: Pre-train a fixed U-Map dimensionality reduction model using a general semantic dataset. The mapping rules do not change with the addition of new labels. Step 2: When adding a new tag, only perform "semantic vector generation → dimensionality reduction → color mapping" on the new tag, and the colors of the original tags remain completely unchanged; Step 3: When the number of newly added tags reaches 20, perform a global mapping optimization to adjust the color distribution of all tags.

[0172] Figure 9 This is a schematic diagram of a memory compression construction process provided in an embodiment of this application, such as... Figure 9 As shown, memory compression mainly consists of four core steps, which are executed sequentially: Step 1: Tag semantics - color space mapping.

[0173] The core objective of this step is to extract data tags from the input structured conversation information, establish an order-preserving mapping between the semantic vector space and the RGB color space for the extracted data tags, ensure the consistency between semantic relevance and color visual relevance, and provide a foundation for subsequent image generation, memory association, and dynamic updates.

[0174] Step 2: Standardized layout and image generation of conversation information.

[0175] The core objective of this step is to generate standardized images by combining the structured information from the entire conversation with color mapping of tags, thus fully preserving all information.

[0176] Step 3: Multimodal vector embedding.

[0177] The core objective of this step is to convert the generated images and conversation summaries into high-dimensional vectors in a unified semantic space, providing a foundation for subsequent memory retrieval and association network construction.

[0178] Step 4: Construction of tree structure and multi-layered network.

[0179] The core objective of this step is to construct a structured "basic tree structure" and a "three-layer association network" to achieve association retrieval at different granularities and fully restore the context and association relationships of the session.

[0180] This basic tree structure can adopt a three-level tree storage structure, such as... Figure 10 As shown, Figure 10 This diagram illustrates a three-level tree-structured storage system provided in this application. The root node of this three-level tree-structured storage system is the "session ID," the first-level child nodes are the "single-turn message ID," and the second-level child nodes are "message category, message image, summary, tag, and embedding vector." This fully corresponds to the three-level structure for memory retrieval, ensuring a clear hierarchy of session information and enabling precise tracing from a session to a single-turn message. In other words, this three-level tree-structured storage system allows for quick and accurate retrieval of relevant information for any session.

[0181] Standardized compressed memory can be obtained through the processing of the above four core steps.

[0182] Based on the solutions provided in the above embodiments, the core contradiction between memory accuracy and token cost is resolved, the entire session information is compressed into standardized images, and a searchable, associative, and dynamically updatable memory network is constructed.

[0183] The core contradiction of existing technologies is that full-text memory has high accuracy, but token costs are extremely high and it is easy to exceed the context limit; summary compression has low token costs, but core information is lost, making it impossible to balance the two. The memory-based question-answering method provided in the embodiments of this application generates single / multiple fixed-format message images by standardizing the structured information of the entire conversation through standardized layout and tag semantic-color mapping, achieving three core values: ① Lossless compression: The image completely preserves the structured information, format, and color mapping features of the entire conversation, without information loss caused by summary extraction; ② Fixed token cost: The token consumption of a single image is fixed at 200-800 tokens (determined by the image resolution), reducing it by 60%-90% compared to full-text memory, completely avoiding the context limit problem; ③ Multimodal unified embedding: Through a multimodal embedding model, images can be converted into high-dimensional vectors in a unified semantic space, enabling cross-modal unified retrieval of text, images, and conversation information, while supporting multi-granularity retrieval (keyword level, message level, conversation level).

[0184] To further improve the accuracy of question-and-answer sessions, based on the above embodiments, the method in this application embodiment further includes: Retrieve the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; If a memory update instruction is received, extract the summary to be updated based on the session to be updated carried in the instruction; Calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; The target modification strategy is determined based on the pre-saved correspondence between different threshold ranges and modification strategies; Based on the target modification strategy, the analysis log of the session to be updated is displayed in the information tree of the session unit, and based on the target modification strategy, at least one node is displayed in the memory network to represent the semantics of the content communicated by the session to be updated.

[0185] In this embodiment, to achieve incremental dynamic updates of memories, ensure the consistency of semantic relevance after adding new memories, and simultaneously achieve full lifecycle management of memories to avoid redundant expansion of the memory bank, a modification strategy is pre-configured in this embodiment. This modification strategy may include replacement, merging, addition, and deletion.

[0186] In order to reasonably update the stored memory information, in this embodiment of the application, the historical communication summary of the content communicated by any question-and-answer pair in each conversation unit, as well as the historical communication summary of the content communicated by each conversation unit, can be obtained. The historical communication summary of any content can be extracted from the stored corresponding content, or it can be extracted from the obtained communication content when a memory update is required.

[0187] In this embodiment of the application, a memory update instruction can be received in real time. This instruction can be sent by other electronic devices, sent by the user, or triggered when the set update conditions are met.

[0188] For example, the triggering condition for the update instruction can be either actively extracted or passively extracted.

[0189] The trigger for proactive extraction can be understood as online real-time updates. The triggering conditions can be: ① The agent recognizes that the user explicitly issues commands such as "remember," "save," or "record"; ② The agent recognizes that the user input contains information that needs to be remembered long-term, such as user preferences, core needs, key entities, and important rules; ③ After a single-turn session completes the tool call and outputs the final conclusion.

[0190] The triggering of passive extraction can be understood as an offline asynchronous update. Its triggering conditions can be at least one of the following: ① No user interaction for more than 30 minutes in a single session; ② No user interaction for more than 2 hours in the entire session; ③ Batch processing of sessions that have not completed the summary extraction for the day during off-peak hours (such as 2 am).

[0191] If a memory update instruction is received, obtain the session to be updated carried in the instruction, and extract the update summary based on the session to be updated.

[0192] In order to determine whether the content communicated between the currently received session to be updated and any question and answer in the previously stored session units is related to the content communicated with the previously stored session units, in this embodiment of the application, the semantic similarity between the summary to be updated and each historical communication summary can be calculated, and the highest semantic similarity value can be determined.

[0193] In order to determine the update strategy when updating based on the session to be updated, in this embodiment of the application, the target modification strategy can be determined according to the correspondence between different pre-saved threshold ranges and modification strategies.

[0194] Once the target modification strategy is determined, the analysis log of the session to be updated can be displayed in the information tree of the session unit based on the target modification strategy, and at least one node can be displayed in the memory network based on the target modification strategy to represent the semantics of the content communicated by the session to be updated.

[0195] For example, a four-level update strategy is set based on the highest cosine similarity between the new memory and the existing memory. The threshold is verified through extensive experiments to ensure the accuracy and consistency of the memory. In this embodiment, the target modification strategy can be determined based on the following formula:

[0196] in, This represents the highest semantic similarity between the new memory (i.e., the summary to be updated) and all memories in the existing memory bank (i.e., historical communication summaries); An embedding vector representing the summary of the session to be updated; This represents the embedding vector of the historical communication of the k-th memory in the existing memory bank, where M is the total number of existing memories. The embedding vector represents the historical communication summary of the content communicated by any question-and-answer pair in each session unit, and the k-th historical communication summary in the historical communication summary of the content communicated by each session unit, where M is the number of stored historical communication summaries; This represents the function for calculating cosine similarity.

[0197] For ease of description, in the embodiments of this application, [0.97, ∞) is referred to as the first threshold interval, [0.85, 0.97) as the second threshold interval, [0.6, 0.85) as the third threshold interval, and [0, 0.6) as the fourth threshold interval.

[0198] If the highest semantic similarity value falls within the first threshold range, then the session information tree to be updated and the memory network to be updated corresponding to the session to be updated are determined. Using the session information tree to be updated, the information tree of the historical interaction dialogue corresponding to the highest semantic similarity value is replaced, and using the memory network to be updated, the nodes corresponding to the historical interaction dialogues with the highest semantic similarity value in the constructed memory network are replaced.

[0199] Specifically, when the highest semantic similarity value falls within the first threshold range, it can be determined that the content communicated in the session to be updated is almost identical to the content communicated corresponding to the highest semantic similarity value, with only updates in time and details (such as the user updating parameters for the same requirement). Therefore, the target modification strategy at this time can be determined as a replacement strategy. The execution rules of the replacement strategy are as follows: according to the session creation time, the corresponding old memory is replaced with the new memory, the old memory is archived and stored, and the corresponding information tree nodes, memory network nodes, stored vector indexes, and summary information are updated simultaneously.

[0200] If the highest semantic similarity value falls within the second threshold range, then the information tree for the session to be updated and the historical interaction dialogue corresponding to the highest similarity value are determined based on the session to be updated and the historical interaction dialogue corresponding to the highest similarity value. The information tree for the communication content corresponding to the highest semantic similarity value is then updated using the information tree for the session to be updated, and the nodes for the communication content corresponding to the highest semantic similarity value in the constructed memory network are replaced using the memory network to be updated.

[0201] Specifically, when the highest semantic similarity value falls within the second threshold range, it can be determined that the content communicated in the session to be updated is highly related to the topic and core content of the communication content corresponding to the highest semantic similarity value, belonging to supplementary content of the same topic (such as multiple rounds of supplementary questions from users regarding the same need). Therefore, the target modification strategy at this time can be determined as a merging strategy. The execution rules of the merging strategy are as follows: using the VLM model to understand the images corresponding to the content communicated between the old and new memories, extracting the core content, merging to generate a new analysis log and summary, while retaining the original images of the old and new memories as attachments; after merging, the original redundant memories are deleted, and the corresponding information tree nodes, memory network nodes, stored vector indexes, and summary information are updated.

[0202] If the highest semantic similarity value falls within the third threshold range, then the information tree and memory network corresponding to the session to be updated are determined. The session to be updated is added to the information tree corresponding to the highest semantic similarity value; the memory network to be updated is added to the constructed memory network, and nodes corresponding to the content with the highest semantic similarity value are connected by connecting lines.

[0203] Specifically, when the highest semantic similarity value falls within the third threshold range, it can be determined that the content communicated in the session to be updated is somewhat related to the content communicated corresponding to the highest semantic similarity value, but they belong to different independent events / requirements. Therefore, the target modification strategy at this time can be determined as the addition strategy. The execution rules of the addition strategy are as follows: directly add the memory in the memory bank, establish the corresponding tree structure node, establish an association edge between it and related memories, incrementally update the network and vector index, and keep the original memory completely unchanged.

[0204] If the highest semantic similarity value falls within the fourth threshold range, then the session information tree and memory network to be updated corresponding to the session to be updated are determined. The session information tree and memory network to be updated are then saved.

[0205] Specifically, when the highest semantic similarity value falls within this fourth threshold range, it can be determined that the content communicated in the session to be updated is unrelated to the content corresponding to the highest semantic similarity value, and belongs to a completely new topic / requirement. Therefore, the target modification strategy at this time can be determined as the archiving strategy. The execution rules of the archiving strategy are: directly add new memories, establish an independent tree structure root node, add corresponding network nodes and vector indices, and do not establish associated edges with existing memories.

[0206] The incremental update method provided in this application embodiment only calculates the newly added / modified nodes, while the association relationship of the original nodes remains unchanged, improving the update efficiency by more than 80%.

[0207] In one possible implementation, lifecycle management is performed on stored historical interaction information. In this embodiment, complete rules for memory cleanup, archiving, and deletion are configured to avoid redundant expansion of the memory bank. Automatic archiving: Memories that have not been retrieved for more than 1 year are automatically archived to cold storage, retaining only the summary and data tag vector, thus freeing up hot storage resources; Automatic cleanup: Memory that has not been accessed for more than 2 years and has no user-marked retention will be automatically deleted, and the corresponding vector index and network node will be cleaned up at the same time; Manual management: Supports users to manually mark memories as to keep / delete / pin them. Manually deleted memories will have all related data deleted simultaneously and cannot be recovered.

[0208] In one possible implementation, the extraction of abstracts and tags can be performed according to the following extraction criteria: Summary: Single-round message-level summaries are limited to 50 characters, and session-level summaries are limited to 200 characters, fully preserving core events, key requirements, and execution results without redundant information; Tags are divided into three categories: topic tags (such as "Python code development"), entity tags (such as "MySQL database"), and user preference tags (such as "likes concise code examples"). There are no more than 5 tags per round of messages and no more than 10 tags per session.

[0209] The following is combined Figure 11 Explain the process of memory updating. Figure 11 This is a schematic diagram of a memory update process provided in an embodiment of this application.

[0210] The core objective of memory updating is to achieve incremental dynamic updates of memories, ensure the consistency of semantic relevance after adding new memories, and simultaneously achieve full lifecycle management of memories to avoid redundant expansion of the memory bank. The update process consists of four core stages, along with complete replacement, merging, addition, and deletion strategies.

[0211] Step 1: Preprocessing of new memories.

[0212] Upon completion of a new session, a memory update instruction can be triggered, and the new memory (i.e., the session to be updated) carried in the instruction can be retrieved. Following the aforementioned steps of "memory retrieval → memory compression and construction," information extraction, label color mapping, message image generation, vector embedding, and tree structure construction of the new session are completed to obtain standardized new memory data.

[0213] Step 2: Full Similarity Matching. The new session's summary embedding vector and message image embedding vector are compared with the vectors in the existing memory for full similarity calculation. Based on the highest similarity result, the corresponding update strategy is triggered.

[0214] When the similarity is ≥0.97, "Strategy 1: Replacement Strategy" is triggered; When 0.85 ≤ similarity < 0.97, "Strategy 2: Merge Strategy" is triggered; When the similarity is between 0.6 and 0.85, trigger "Strategy 3: Add a new strategy"; When the similarity is less than 0.6, "Strategy 4: Archiving Strategy" is triggered.

[0215] Step 3: Incrementally update the memory network and vector index. Based on the triggered update strategy, incrementally update the tree structure and three-layer association network of the memory, without needing to recalculate the entire network. Simultaneously, for newly added / modified vectors, incrementally update the vector database index, without requiring a full reconstruction.

[0216] Step 4: Memory Lifecycle Management. Manage stored memories according to configured memory cleanup, archiving, and deletion rules to prevent redundant expansion of the memory bank.

[0217] In one possible implementation, in order to improve the efficiency of memory retrieval, based on the above embodiments, the generated images and conversation summaries can be converted into high-dimensional vectors of a unified semantic space, providing a foundation for subsequent memory retrieval and association network construction.

[0218] In the embodiments of this application, the same embedding model can be used for conversion. For example, a multimodal embedding model with image-text alignment (such as CLIP, Qwen-VL-Embedding) can be used to ensure that the vectors of text and images are in the same semantic space, and cross-modal similarity can be directly calculated.

[0219] Specifically, a 768-dimensional image embedding vector can be generated for a single image, corresponding to message-level memory retrieval; a 768-dimensional text embedding vector can be generated for a conversation-level summary, corresponding to conversation-level memory retrieval; and a corresponding semantic vector can be generated for any data tag, corresponding to keyword-level memory retrieval.

[0220] After obtaining the relevant vectors, all vectors can be stored in a vector database (such as Milvus or pgvector) and an index can be created for subsequent fast retrieval.

[0221] In one possible implementation, after obtaining the target answer information for the question to be answered, the detailed process of this answer can be recorded, such as recording the search keywords, recall results, and user and agent feedback, so as to optimize the search threshold and ranking rules based on the recorded information.

[0222] In one possible implementation, the retrieval is performed based on the methods described in the above embodiments. If the similarity of all retrieved memories is less than a set threshold (e.g., all similarities are less than 0.6), an empty memory is returned. That is, if no matching target historical interaction information is found, the corresponding target image cannot be obtained. Therefore, the question to be answered can be directly input into the agent's target large model. To facilitate subsequent recording and updates, relevant information from this response process can be recorded.

[0223] The retrieval-based question-answering method provided in this application has the following main improvements compared to related technologies: This solves the problem of information loss in textual memory: related technical solutions preserve memory through summary extraction and structured storage, but lose conversation context details, full-link information of tool calls, and original multimodal semantics, resulting in the agent being unable to fully reuse historical experience; It solves the contradiction between memory accuracy and token cost: storing the entire session text will lead to the expiration of the context token after retrieval, and storing the summary will lose core details. Related technical solutions cannot balance the two. It solves the problem of unified storage and retrieval of multimodal memory: related technical solutions either store and retrieve multimodal memory such as text, images, audio, and documents separately or convert them into text without semantics, and cannot achieve unified and accurate retrieval across modalities; The problem of semantic consistency in dynamic memory updates has been solved: when adding new memories, the relevant technical solutions either require full recalculation, which leads to low efficiency, or incremental storage, which leads to semantic space drift. The relevance between new and old memories cannot be guaranteed, and the retrieval accuracy continues to decline.

[0224] The retrieval-based question-answering method provided in this application achieves the following technical effects through six core steps: multimodal information extraction, tag semantic-color space mapping, image-based compression of conversation information, tree-structured + association network memory construction, unified multimodal retrieval, and incremental dynamic updating: In terms of memory accuracy: image compression completely preserves the entire session information (including user input, agent reasoning, and tool invocation), without the information loss caused by summary extraction; Regarding token cost: single-round / single-session memory is compressed into a single image, and the token consumption is fixed at 200-800 tokens, which is 60%-90% lower than full text memory, and completely avoids the context over-limit problem; In terms of retrieval capabilities: it has achieved multi-granularity cross-modal retrieval at the keyword level, message level, and session level, improving the retrieval recall rate by more than 30%, while also supporting precise filtering retrieval by message category, time range, and tags; Regarding dynamic updates: By using a dynamic mapping of the semantic-color space of the tags to maintain order, the consistency of semantic relevance after adding new memories is ensured, eliminating the need for a full recalculation and improving update efficiency by more than 80%.

[0225] Based on the same inventive concept, embodiments of this application provide a data manipulation device. Figure 12 A schematic diagram of a memory-based question-answering device provided in this application embodiment is shown below. Figure 12 The device includes: The retrieval module 1201 is used to search for target historical interaction information that has a semantic relationship with the question to be answered in the stored historical interaction information; The acquisition module 1202 is used to acquire a target image saved for the target historical interaction information. The target historical interaction information includes at least one historical unanswered question and one historical answer information. The target image records an analysis log of the agent when generating the historical answer information based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer information, the agent's thinking content, tool call information, and core information of the historical unanswered question and the historical answer information. The response module 1203 is used to input the question to be answered and the target image into the target big model of the intelligent agent, so that the target big model generates target response information for the question to be answered based on the information in the target image.

[0226] In one possible implementation, the retrieval module 1201 is specifically used to search for target nodes in the constructed memory network that have a semantic association with the question to be answered. The memory network includes multiple nodes, each node representing the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets a preset requirement, the two nodes are connected by a connecting line. The target historical interaction information to which the content represented by the target node belongs is determined.

[0227] In one possible implementation, the memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword, and the keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated based on any question and answer in the conversation unit. The retrieval module 1201 is specifically used to extract target keywords from the question to be answered; determine the semantic similarity between the target keywords and the keywords represented by each node in the keyword tag network; and determine the nodes corresponding to the keywords whose semantic similarity meets the threshold requirement as target nodes.

[0228] In one possible implementation, the memory network is a message association network, where any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The retrieval module 1201 is specifically used to determine the semantic similarity between the question to be answered and the content represented by each node in the message association network, and to determine the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node.

[0229] In one possible implementation, the retrieval module 1201 is specifically used to determine a first semantic similarity between the content represented by each node in the message association network and the question to be answered; to determine the node corresponding to the content with a first semantic similarity greater than a first threshold as a first node; for each first node, to determine a second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network; to determine the node corresponding to the feature vector with a second semantic similarity greater than a second threshold as a second node; and to determine the first node and the second node as the target node.

[0230] In one possible implementation, the memory network is a session association network, where any node in the session association network is used to characterize the content communicated by any session unit; The retrieval module 1201 is specifically used to determine the semantic similarity between the question to be answered and the content represented by each node in the session association network, and to determine the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node.

[0231] In one possible implementation, the device further includes: The filtering module 1204 is used to, if there are multiple target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, and the time difference between the creation time of the target historical interaction information and the current time for each target historical interaction information; determine the target ranking score corresponding to the target historical interaction information based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics; and select a preset number of target historical interaction information from the current multiple target historical interaction information based on the target ranking score corresponding to each target historical interaction information.

[0232] In one possible implementation, the device further includes: The generation module 1205 is configured to: if the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by the question-and-answer pair in that information tree; generate a target image corresponding to the target historical interaction information according to the analysis log and a preset image generation rule; if the target historical interaction information is the content communicated by a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by each question-and-answer pair in that information tree, and generate a target image corresponding to the target historical interaction information according to each analysis log found and a preset image generation rule.

[0233] In one possible implementation, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; The generation module 1205 is further configured to: determine the semantic vector corresponding to each data tag in each core information item; perform dimensionality reduction processing on the semantic vector based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector, wherein each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space; determine the maximum intermediate vector and the minimum intermediate vector based on the three-dimensional intermediate vectors corresponding to all data tags of the core information item; determine the first difference between the three-dimensional intermediate vector corresponding to the data tag and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; determine the quotient of the first difference and the second difference; and round down the product of the quotient and a preset value to obtain the RGB color value corresponding to the data tag; if the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0234] In one possible implementation, the acquisition module 1202 is further configured to acquire the historical communication summary of the content communicated by any question-and-answer pair in each session unit, and the historical communication summary of the content communicated by each session unit. The update module 1206 is configured to, upon receiving a memory update instruction, extract a summary to be updated based on the session to be updated carried in the instruction; calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold interval to which the highest semantic similarity value belongs; determine a target modification strategy according to the pre-saved correspondence between different threshold intervals and modification strategies; display the analysis log of the session to be updated in the information tree of the session unit based on the target modification strategy, and display at least one node in the memory network based on the target modification strategy to characterize the semantics of the content communicated by the session to be updated.

[0235] Based on the same inventive concept, embodiments of this application provide an electronic device that can implement the steps of the memory-retrieval-based question-answering method described above. Figure 13 This application provides a schematic diagram of an electronic device structure, such as... Figure 13 As shown, it includes: processor 1301, communication interface 1302, memory 1303 and communication bus 1304, wherein processor 1301, communication interface 1302 and memory 1303 communicate with each other through communication bus 1304; The memory 1303 stores a computer program. When the program is executed by the processor 1301, the processor 1301 performs the following steps: Search for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information; Obtain a target image saved for the target's historical interaction information. The target's historical interaction information includes at least one historical unanswered question and one historical answer. The target image records an analysis log of the agent generating the historical answer based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer, the agent's thought process, tool call information, and core information of the historical unanswered question and the historical answer. The question to be answered and the target image are input into the target big model of the agent, so that the target big model generates target answer information for the question to be answered based on the information in the target image.

[0236] In one possible implementation, searching for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information includes: In the constructed memory network, target nodes that are semantically related to the question to be answered are searched. The memory network includes multiple nodes, and each node is used to represent the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line. Determine the target historical interaction information to which the content represented by the target node belongs.

[0237] In one possible implementation, the memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword, and the keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated based on any question and answer in the conversation unit. The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: Extract the target keywords from the questions to be answered; The semantic similarity between the target keyword and the keyword represented by each node in the keyword tag network is determined, and the node corresponding to the keyword whose semantic similarity meets the threshold requirement is determined as the target node.

[0238] In one possible implementation, the memory network is a message association network, where any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the message association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0239] In one possible implementation, determining the semantic similarity between the question to be answered and the content represented by each node in the message association network, and identifying the node corresponding to the content whose semantic similarity meets a threshold requirement as the target node, includes: Determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered, and determine the node corresponding to the content with the first semantic similarity greater than the first threshold as the first node; For each first node, determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network, and determine the node corresponding to the feature vector with the second semantic similarity greater than the second threshold as the second node; The first node and the second node are identified as the target nodes.

[0240] In one possible implementation, the memory network is a session association network, where any node in the session association network is used to characterize the content communicated by any session unit; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the session association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0241] In one possible implementation, after finding target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the method further includes: If there are multiple target historical interaction information, for each target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time; based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, determine the target ranking score corresponding to the target historical interaction information. Based on the target ranking score corresponding to each target's historical interaction information, a preset number of target historical interaction information are selected from the current multiple target historical interaction information.

[0242] In one possible implementation, the process of determining the target image corresponding to the target historical interaction information includes: If the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by that question-and-answer pair in the information tree; generate the target image corresponding to the target historical interaction information according to the analysis log and the preset image generation rules; If the target historical interaction information is the content communicated by the conversation unit, the information tree of the conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by each question and answer pair in the information tree is searched. Based on each analysis log found and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

[0243] In one possible implementation, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; After searching the analysis log of each question-answer pair included in the target historical interaction information in the saved session information tree, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, the method further includes: For each data tag in each core piece of information, determine the semantic vector corresponding to that data tag; The semantic vector is reduced in dimensionality based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. Each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space. Based on the three-dimensional intermediate vectors corresponding to all data labels of this core information, determine the maximum and minimum intermediate vectors; Determine the first difference between the three-dimensional intermediate vector corresponding to the data label and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; Determine the quotient of the first difference and the second difference; The product of the quotient and the preset value is rounded down to obtain the RGB color value corresponding to the data label; Based on the analysis logs and preset image generation rules, a target image corresponding to the target historical interaction information is generated, including: If the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0244] In one possible implementation, the method further includes: Retrieve the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; If a memory update instruction is received, extract the summary to be updated based on the session to be updated carried in the instruction; Calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; The target modification strategy is determined based on the pre-saved correspondence between different threshold ranges and modification strategies; Based on the target modification strategy, the analysis log of the session to be updated is displayed in the information tree of the session unit, and based on the target modification strategy, at least one node is displayed in the memory network to represent the semantics of the content communicated by the session to be updated.

[0245] Since the principle of the above-mentioned electronic device in solving the problem is similar to that of the question-answering method based on memory retrieval, the implementation of the above-mentioned electronic device can be found in the embodiments of the method, and the repeated parts will not be described again.

[0246] The communication bus mentioned in the aforementioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 1302 is used for communication between the aforementioned electronic device and other devices. The memory can include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0247] The processors mentioned above can be general-purpose processors, including central processing units, network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits, field-programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0248] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a processor. When the program runs on the processor, it causes the processor to perform the following steps: Search for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information; Obtain a target image saved for the target's historical interaction information. The target's historical interaction information includes at least one historical unanswered question and one historical answer. The target image records an analysis log of the agent generating the historical answer based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer, the agent's thought process, tool call information, and core information of the historical unanswered question and the historical answer. The question to be answered and the target image are input into the target big model of the agent, so that the target big model generates target answer information for the question to be answered based on the information in the target image.

[0249] In one possible implementation, searching for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information includes: In the constructed memory network, target nodes that are semantically related to the question to be answered are searched. The memory network includes multiple nodes, and each node is used to represent the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line. Determine the target historical interaction information to which the content represented by the target node belongs.

[0250] In one possible implementation, the memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword, and the keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated based on any question and answer in the conversation unit. The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: Extract the target keywords from the questions to be answered; The semantic similarity between the target keyword and the keyword represented by each node in the keyword tag network is determined, and the node corresponding to the keyword whose semantic similarity meets the threshold requirement is determined as the target node.

[0251] In one possible implementation, the memory network is a message association network, where any node in the message association network is used to characterize the content contained in the analysis log of any question-answer pair; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the message association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0252] In one possible implementation, determining the semantic similarity between the question to be answered and the content represented by each node in the message association network, and identifying the node corresponding to the content whose semantic similarity meets a threshold requirement as the target node, includes: Determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered, and determine the node corresponding to the content with the first semantic similarity greater than the first threshold as the first node; For each first node, determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network, and determine the node corresponding to the feature vector with the second semantic similarity greater than the second threshold as the second node; The first node and the second node are identified as the target nodes.

[0253] In one possible implementation, the memory network is a session association network, where any node in the session association network is used to characterize the content communicated by any session unit; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the session association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

[0254] In one possible implementation, after finding target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the method further includes: If there are multiple target historical interaction information, for each target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time; based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, determine the target ranking score corresponding to the target historical interaction information. Based on the target ranking score corresponding to each target's historical interaction information, a preset number of target historical interaction information are selected from the current multiple target historical interaction information.

[0255] In one possible implementation, the process of determining the target image corresponding to the target historical interaction information includes: If the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by that question-and-answer pair in the information tree; generate the target image corresponding to the target historical interaction information according to the analysis log and the preset image generation rules; If the target historical interaction information is the content communicated by the conversation unit, the information tree of the conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by each question and answer pair in the information tree is searched. Based on each analysis log found and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

[0256] In one possible implementation, the core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; After searching the analysis log of each question-answer pair included in the target historical interaction information in the saved session information tree, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, the method further includes: For each data tag in each core piece of information, determine the semantic vector corresponding to that data tag; The semantic vector is reduced in dimensionality based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. Each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space. Based on the three-dimensional intermediate vectors corresponding to all data labels of this core information, determine the maximum and minimum intermediate vectors; Determine the first difference between the three-dimensional intermediate vector corresponding to the data label and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; Determine the quotient of the first difference and the second difference; The product of the quotient and the preset value is rounded down to obtain the RGB color value corresponding to the data label; Based on the analysis logs and preset image generation rules, a target image corresponding to the target historical interaction information is generated, including: If the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

[0257] In one possible implementation, the method further includes: Retrieve the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; If a memory update instruction is received, extract the summary to be updated based on the session to be updated carried in the instruction; Calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; The target modification strategy is determined based on the pre-saved correspondence between different threshold ranges and modification strategies; Based on the target modification strategy, the analysis log of the session to be updated is displayed in the information tree of the session unit, and based on the target modification strategy, at least one node is displayed in the memory network to represent the semantics of the content communicated by the session to be updated.

[0258] Since the principle of the computer-readable storage medium in solving the problem is similar to that of the memory-based question-answering method, the implementation of the computer-readable storage medium can be found in the implementation of the method, and the repetitions will not be repeated.

[0259] Based on the same inventive concept, this application also provides a computer program product, which includes computer program code. When the computer program code is run on a computer, it causes the computer to execute any of the memory-retrieval-based question-answering methods discussed above. Since the principle of the above-described computer program product in solving the problem is similar to that of the memory-retrieval-based question-answering method, the implementation of the above-described computer program product can refer to the implementation of the method, and repeated details will not be described again.

[0260] In this embodiment, target historical interaction information with semantically related to the question to be answered is searched from the stored historical interaction information. Then, a target image saved for this target historical interaction information is obtained. This target historical interaction information includes at least one historical question to be answered and one historical answer. Since the target image records the agent's analysis log when generating historical answer information based on the historical question to be answered, and this analysis log includes at least one of the following: the question to be answered, the historical answer information, the agent's thought process, tool call information, and core information of the historical question to be answered and the historical answer information, after obtaining the target image, the question to be answered and the target image are input into the agent's target model, so that the target model generates the target answer information for the question to be answered based on the information in the target image. Because the token length of a single image is determined by the image resolution, even if the target image records a lot of information, it will not exceed the context limit of the target model, ensuring that the agent can effectively reuse historical interaction experience.

[0261] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0262] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0263] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0264] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of user-operated steps to be executed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0265] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A question-answering method based on memory retrieval, characterized in that, The method includes: Search for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information; Obtain a target image saved for the target's historical interaction information. The target's historical interaction information includes at least one historical unanswered question and one historical answer. The target image records an analysis log of the agent generating the historical answer based on the historical unanswered question. The analysis log includes at least one of the following: the unanswered question, the historical answer, the agent's thought process, tool call information, and core information of the historical unanswered question and the historical answer. The question to be answered and the target image are input into the target big model of the agent, so that the target big model generates target answer information for the question to be answered based on the information in the target image.

2. The method according to claim 1, characterized in that, The step of searching for target historical interaction information that is semantically related to the question to be answered from the stored historical interaction information includes: In the constructed memory network, target nodes that are semantically related to the question to be answered are searched. The memory network includes multiple nodes, and each node is used to represent the content communicated by any conversation unit or the content communicated by any question-answer pair in the conversation unit. The conversation unit includes multiple question-answer pairs, and the question-answer pairs included in the same conversation unit belong to the same dialogue context. When the semantic similarity between the content represented by two nodes meets the preset requirements, the two nodes are connected by a connecting line. Determine the target historical interaction information to which the content represented by the target node belongs.

3. The method according to claim 2, characterized in that, The memory network is a keyword tagging network, where any node in the keyword tagging network is used to represent a keyword. The keyword is extracted based on the content communicated in any conversation unit, or based on the content communicated in any question and answer in the conversation unit. The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: Extract the target keywords from the questions to be answered; The semantic similarity between the target keyword and the keyword represented by each node in the keyword tag network is determined, and the node corresponding to the keyword whose semantic similarity meets the threshold requirement is determined as the target node.

4. The method according to claim 2 or 3, characterized in that, The memory network is a message association network, and any node in the message association network is used to represent the content contained in the analysis log of any question-answer pair; The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the message association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

5. The method according to claim 4, characterized in that, The step of determining the semantic similarity between the question to be answered and the content represented by each node in the message association network, and determining the node corresponding to the content whose semantic similarity meets the threshold requirement as the target node, includes: Determine the first semantic similarity between the content represented by each node in the message association network and the question to be answered, and determine the node corresponding to the content with the first semantic similarity greater than the first threshold as the first node; For each first node, determine the second semantic similarity between the content represented by the first node and the content represented by other nodes in the message association network, and determine the node corresponding to the feature vector with the second semantic similarity greater than the second threshold as the second node; The first node and the second node are identified as the target nodes.

6. The method according to claim 2 or 3, characterized in that, The memory network is a session association network, where any node in the session association network is used to represent the content communicated by any session unit. The step of searching for target nodes in the constructed memory network that are semantically related to the question to be answered includes: The semantic similarity between the question to be answered and the content represented by each node in the session association network is determined, and the node corresponding to the content whose semantic similarity meets the threshold requirement is determined as the target node.

7. The method according to claim 1, characterized in that, After finding the target historical interaction information that is semantically related to the question to be answered, and before obtaining the target image saved for the target historical interaction information, the method further includes: If there are multiple target historical interaction information, for each target historical interaction information, determine the semantic similarity between the target historical interaction information and the question to be answered, as well as the time difference between the creation time of the target historical interaction information and the current time; based on the time difference, the time decay weight saved for the time difference, the semantic similarity, and the semantic similarity weight saved for the semantics, determine the target ranking score corresponding to the target historical interaction information. Based on the target ranking score corresponding to each target's historical interaction information, a preset number of target historical interaction information are selected from the current multiple target historical interaction information.

8. The method according to claim 2, characterized in that, The process of determining the target image corresponding to the target historical interaction information includes: If the target historical interaction information is the content communicated by any question-and-answer pair in a conversation unit, search for the information tree of that conversation unit in the saved conversation information tree, and search for the analysis log of the content communicated by that question-and-answer pair in the information tree; generate the target image corresponding to the target historical interaction information according to the analysis log and the preset image generation rules; If the target historical interaction information is the content communicated by the conversation unit, the information tree of the conversation unit is searched in the saved conversation information tree, and the analysis log of the content communicated by each question and answer pair in the information tree is searched. Based on each analysis log found and the preset image generation rules, the target image corresponding to the target historical interaction information is generated.

9. The method according to claim 8, characterized in that, The core information includes at least one of the following: summary, topic tags, entity tags, and user preference tags; After searching the analysis log of each question-answer pair included in the target historical interaction information in the saved session information tree, and before generating the target image corresponding to the target historical interaction information based on the analysis log and preset image generation rules, the method further includes: For each data tag in each core piece of information, determine the semantic vector corresponding to that data tag; The semantic vector is reduced in dimensionality based on a pre-trained dimensionality reduction model to obtain a three-dimensional intermediate vector. Each component value in the three-dimensional intermediate vector corresponds one-to-one with the color value of each channel in the RGB color space. Based on the three-dimensional intermediate vectors corresponding to all data labels of this core information, determine the maximum and minimum intermediate vectors; Determine the first difference between the three-dimensional intermediate vector corresponding to the data label and the minimum intermediate vector, and the second difference between the maximum intermediate vector and the minimum intermediate vector; Determine the quotient of the first difference and the second difference; The product of the quotient and the preset value is rounded down to obtain the RGB color value corresponding to the data label; Based on the analysis logs and preset image generation rules, a target image corresponding to the target historical interaction information is generated, including: If the analysis log includes core information, then when displaying each data tag of the core information in the image, the color of the RGB color value corresponding to the data tag is used as the background color of the data tag.

10. The method according to claim 8, characterized in that, The method further includes: Retrieve the historical communication summary of the content communicated by any question-and-answer pair in each session unit, as well as the historical communication summary of the content communicated by each session unit; If a memory update instruction is received, extract the summary to be updated based on the session to be updated carried in the instruction; Calculate the semantic similarity between the summary to be updated and each historical communication summary, and determine the target threshold range to which the highest semantic similarity value belongs; The target modification strategy is determined based on the pre-saved correspondence between different threshold ranges and modification strategies; Based on the target modification strategy, the analysis log of the session to be updated is displayed in the information tree of the session unit, and based on the target modification strategy, at least one node is displayed in the memory network to represent the semantics of the content communicated by the session to be updated.

Images