Memory reading method and reading system of accompanying robot

Abstract

The invention discloses a memory reading method and reading system for an accompanying robot, and the method comprises the steps: S1, obtaining user input and corresponding environment perception, carrying out the analysis of the user input and the corresponding environment perception, and extracting key information: entities, relationships, attributes, emotion types, and environment information; the extracted key information is input to the memory center; s2, the memory center carries out long-term memory bank retrieval according to the key information; s3, converting the key information and the retrieval result of the long-term memory library into vectors, calculating semantic similarity among the vectors, obtaining 10 vectors with the highest semantic similarity, and inputting the 10 results to a reasoning and decision-making layer; and S4, the reasoning and decision-making layer performs language generation according to the 10 vectors with the highest semantic similarity and the key information vector. Through the above design, the method can achieve the coherent tracking of user information, guarantees the conversation coherence, accumulates the user preferences, and enables the accompanying robot to have the long-term memory and personalized interaction capabilities.

Description

Technical Field

[0001] This invention belongs to the field of voice artificial intelligence design, specifically relating to a memory retrieval method and system for a companion robot. Background Technology

[0002] The core value of companion robots lies in their profound "understanding" and "empathy," enabling them to establish coherent and personalized interactive relationships with users, understand their emotional fluctuations, and provide unique and continuously evolving companionship. Achieving this goal relies on the effective memorization and efficient retrieval of users' historical behavior, dialogue content, and preference information. Currently, existing companion robots generally lack systematic and intelligent memory and memory retrieval mechanisms, falling into the predicament of "seven-second goldfish memory": each interaction is isolated and fragmented, unable to understand the context of the dialogue. For example, if a user first says, "I have an important speech tomorrow, I'm very nervous," and then later says, "My heart is beating so fast," an ideal companion should be able to connect the two sentences and respond, "Don't worry, you're well-prepared for today's speech. Take a deep breath, you'll succeed." However, existing robots are likely to only offer an isolated, generic response to phrases like "Your heart is beating so fast," such as "I suggest you take a break," completely missing the crucial context of a "speech." This results in mechanical, superficial interactions between humans and companion robots, failing to build any deep connections. They also struggle to learn and adapt to personalized user preferences over the long term. For example, a user might enjoy listening to Chopin on weekends but spend weekday evenings discussing elementary school textbooks with grandchildren; a robot lacking a memory system cannot understand this habit and might simply recommend music every time. Furthermore, they cannot accumulate user-related knowledge to solve complex interaction problems. For instance, robots cannot access and cross-analyze stored historical information such as consumption categories, spending frequency, exercise levels, and sleep quality. Ultimately, existing companion robots can only achieve simple "question-and-answer" interactions or act as "command executors," failing to meet users' needs for intelligent companionship that involves continuous growth and deep adaptation. They cannot co-create a continuous, memorable "relationship history" with users, thus failing to satisfy the deep-seated human need for long-term, stable, and reliable companionship. Therefore, there is an urgent need for a memory system that can cover the entire process of "memory encoding - multi-mode storage - intelligent retrieval - feedback update", thereby enabling companion robots to move towards true "intelligence" and "humanization". Summary of the Invention

[0003] To address the shortcomings of existing companion robots, such as the lack of effective memory and memory retrieval mechanisms, which leads to isolated interactions, inability to understand context, difficulty in adapting to user preferences, and inability to accumulate user knowledge, this application designs a memory retrieval method and interaction system for companion robots. This enables companion robots to accurately memorize, efficiently retrieve, and coherently apply user information, thereby enhancing the personalization and intelligence of the interaction.

[0004] A method for reading the memories of a companion robot includes the following steps: Step S1: Obtain user input and corresponding environmental awareness, analyze them, and extract key information: entities, relationships, attributes, sentiment categories, and environmental information; and input the extracted key information into the memory center. Step S2: The memory center retrieves information from the long-term memory bank based on key information; Step S3: Convert the key information and retrieval results from the long-term memory into vectors, calculate the semantic similarity between the vectors, obtain the 10 vectors with the highest semantic similarity, and input these 10 results into the reasoning and decision-making layer; Step S4: The reasoning and decision-making layer generates language based on the 10 vectors with the highest semantic similarity and key information vectors.

[0005] Preferably, in step S1, the user input includes voice; the environmental perception includes a photo of the user's surroundings.

[0006] Preferably, the method for determining whether a long-term memory retrieval is required is as follows: According to the formula:

[0007] in, The input information can be entities, relationships, attributes, sentiment categories, or environmental information; each different type of input is a... ; This represents different categories of embedding models, where k refers to the corresponding analogy; Represents normal distribution sampling with the same dimensions as the embedding model; The vector of potential semantic information for each piece of information, indicating whether it needs to be read from memory, was calculated. It is a small neural network that determines the probability of whether or not to access memory based on the given information. Sum all probabilities; if If the value exceeds the set threshold, the memory needs to be read; if If the value is less than the set threshold, there is no need to read the memory.

[0008] Preferably, the memory center includes a short-term memory bank and a long-term memory bank; The short-term memory includes a key-value pair database; the short-term memory is used to store each dialogue sequence, and all dialogue records of that sequence are read from the short-term memory each time. The long-term memory includes: a vector database, a graph database, and a relational database.

[0009] Preferably, in step S3, the method for calculating the semantic similarity between vectors is as follows: Step S31: Using an embedding model, convert all the search results from the graph database and the relational database into vectors. Adding the vector database search results, there will be a total of k+m+n vectors. Each vector is labeled... Where k is the vector of search results from graph databases, m is the vector of search results from relational databases, and n is the vector of search results from vector databases. Step S32: Vectorize the content of the key information by embedding the model to obtain a vector p; Step S33: Calculate the semantic similarity between vectors according to the following formula: ; For the calculated k+m+n Sort the results, obtain the 10 results with the highest sort values, and input their corresponding vectors into the reasoning and decision-making layers.

[0010] Preferably, in step S4, the information generated in each step, including language, user input, and user feedback, needs to be processed as follows: Step S41: Preprocessing, summarizing and standardizing, then appending metadata and processing according to different categories such as vectors, knowledge graphs and structured information; Step S42: Then, store the processed information in a vector database, a graph database, and a relational database; Step S43: Store the original information with attached metadata directly into the key-value pair database of the short-term memory.

[0011] A memory retrieval system for a companion robot includes an input layer, a reasoning and decision-making layer, an output layer, a feedback layer, and a memory center; The input layer is used to acquire the user's voice input and photos of the user's environment, and extract key information; The memory center determines whether a long-term database search is needed based on key information. If so, the search results are converted into vectors, and the key information is also converted into vectors. The semantic similarity between the vectors is calculated, and the 10 vectors with the highest semantic similarity are obtained. These 10 results are then input into the reasoning and decision-making layer. The reasoning and decision-making layers generate language based on the 10 vectors with the highest semantic similarity and key information vectors; if long-term memory retrieval is not required, language generation is only required based on key information vectors. The output layer is used to display the language generated by the reasoning and decision-making layers, and then transmit the display results to the memory center. The feedback layer is used for user interaction and feedback, and transmits the feedback results to the memory center.

[0012] Preferably, the memory center further includes a metadata attachment module, which is used to pass the content transmitted from the output layer and the feedback layer to the memory center to the short-term memory; and to preprocess, summarize and standardize the above content, and to pass the acquired structured data, vector data and knowledge graph data to the long-term memory.

[0013] The advantages and effects of this application are as follows: This application designs a memory retrieval system for a companion robot, comprising an input layer, a memory center, a reasoning and execution layer, an output layer, and a feedback layer. Each layer is linked sequentially through a data link, forming a closed-loop system of "input-memory-reasoning-output-feedback-memory update." Through an architecture of "input parsing-multi-database storage-intelligent retrieval," analogous to the human brain, it achieves coherent tracking of the user's historical information. Furthermore, it ensures the continuity of dialogue through a short-term memory bank, accumulates user preferences through a long-term memory bank, and enables dynamic memory updates through the feedback layer. This allows the companion robot to transcend simple question-and-answer functionality and become a truly intelligent partner with long-term memory, personalized interaction capabilities, and continuous growth.

[0014] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings.

[0015] The above and other objects, advantages and features of this application will become more apparent to those skilled in the art from the following detailed description of specific embodiments in conjunction with the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In all drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0017] Figure 1 A framework diagram of the memory retrieval system for the companion robot designed for this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. In the following description, specific details such as specific configurations and components are provided merely to help fully understand the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. In addition, for clarity and brevity, descriptions of known functions and structures are omitted in the embodiments.

[0019] It should be understood that the phrase "an embodiment" or "this embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "an embodiment" or "this embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0020] Furthermore, reference numerals and / or letters may be repeated in different examples within this application. Such repetition is for the purpose of simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or settings discussed.

[0021] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, B exists alone, and A and B exist simultaneously. The term " / and" in this article describes another type of relationship between related objects, indicating that two relationships can exist. For example, A / and B can mean: A exists alone, and A and B exist alone. In addition, the character " / " in this article generally indicates that the related objects before and after it are in an "or" relationship.

[0022] In this article, the term "at least one" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, "at least one of A and B" can mean: A exists alone, A and B exist simultaneously, or B exists alone.

[0023] It should also be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion.

[0024] Example 1: Please refer to Figure 1 This embodiment mainly introduces a memory retrieval system for a companion robot, which includes an input layer, a reasoning and decision-making layer, an output layer, a feedback layer, and a memory center; The input layer is used to acquire the user's voice input and photos of the user's environment, and extract key information; The memory center determines whether a long-term database search is needed based on key information. If so, the search results are converted into vectors, and the key information is also converted into vectors. The semantic similarity between the vectors is calculated, and the 10 vectors with the highest semantic similarity are obtained. These 10 results are then input into the reasoning and decision-making layer. The reasoning and decision-making layers generate language based on the 10 vectors with the highest semantic similarity and key information vectors; if long-term memory retrieval is not required, language generation is only required based on key information vectors. The output layer is used to display the language generated by the reasoning and decision-making layers and transmit the display results to the memory center. The output layer selects the appropriate display method based on whether it is text, JSON, or speech. For example, text is displayed on the screen, speech is played through a speaker, and JSON is processed by selecting appropriate sensors to perform actions, such as shaking the head, nodding, walking, picking up objects, etc. These final outputs are then used as new memory inputs and processed by the memory center.

[0025] The feedback layer is used for user interaction and feedback, and transmits the feedback results to the memory center.

[0026] Furthermore, the memory center also includes a metadata attachment module, which is used to pass the content transmitted from the output layer and the feedback layer to the memory center to the short-term memory; and to preprocess, summarize and standardize the above content, and to pass the acquired structured data, vector data and knowledge graph data to the long-term memory.

[0027] Example 2: Based on Example 1, this example mainly introduces a method for reading the memory of a companion robot, including the following steps: Step S1: Obtain user input and corresponding environmental awareness, analyze them, and extract key information: entities, relationships, attributes, sentiment categories, and environmental information; and input the extracted key information into the memory center. Furthermore, user input will be parsed to extract text (the content of the speech), emotions (such as excitement, quietness, anger, etc.), and environmental information (such as living room, dining room, kitchen, bedroom, whether someone is alone or in a group, whether there are children, elderly people, etc.). This embodiment does not limit the method of parsing user input; various mature methods can be used. The parsed content will extract entities, relationships, and key information. This embodiment does not limit the method used; various existing methods can be used.

[0028] Step S2: The memory center retrieves information from the long-term memory bank based on key information; Furthermore, the retrieval process requires generating vector database retrieval statements, graph database retrieval statements, and SQL retrieval statements based on vector databases, graph databases, and relational databases, respectively. Simultaneously, these databases are queried in parallel. Each type of retrieval may yield multiple results. For example, k vector results, m graph retrieval results, and n relational database rows.

[0029] Step S3: Convert the key information and retrieval results from the long-term memory into vectors, calculate the semantic similarity between the vectors, obtain the 10 vectors with the highest semantic similarity, and input these 10 results into the reasoning and decision-making layer; Step S4: The reasoning and decision-making layer generates language based on the 10 vectors with the highest semantic similarity and key information vectors.

[0030] Furthermore, in step S1, the user input includes voice; the environmental perception includes a photo of the user's surroundings.

[0031] Furthermore, in step S2, the method for determining whether a long-term memory retrieval is required is as follows: According to the formula:

[0032] In the formula above, The input information can be entities, relationships, attributes, sentiment categories, environmental information, etc. Each different type of input is a... ,for example: Three entities: radish, pork ribs, and ginger; An emotion category: Excitement Two sets of environmental information: the kitchen, and the scene where the cook is being explained to the best friend. So, there are a total of 6 x's, namely...

[0033] Representing different categories of embedding models, k refers to the corresponding analogy. In the example above, entity, sentiment category, and environmental information are three different embedding models, respectively. , ,

[0034] This represents a normal distribution sampling with the same dimensions as the embedding model. The vector of potential semantic information for each piece of information, indicating whether it needs to be read from memory, was calculated.

[0035] It is a small neural network that determines whether to read the memory based on the probability value of the given information. For the neural network itself, CNN, RNN and Transformer can be selected according to different needs and data conditions.

[0036] Sum all probabilities together.

[0037] if If the value exceeds the set threshold (which can be set based on actual test data), then it is necessary to read the memory; if... If the value is less than the set threshold, there is no need to read the memory.

[0038] Furthermore, the memory center includes a short-term memory bank and a long-term memory bank; The short-term memory includes a key-value pair database; the short-term memory is used to store each dialogue sequence, and all dialogue records of that sequence are read from the short-term memory each time. The long-term memory includes: a vector database, a graph database, and a relational database.

[0039] Furthermore, in step S3, the method for calculating the semantic similarity between vectors is as follows: Step S31: Using an embedding model, convert all the search results from the graph database and the relational database into vectors. Adding the vector database search results, there will be a total of k+m+n vectors. Each vector is labeled... ; Where k is the vector of search results from graph databases, m is the vector of search results from relational databases, and n is the vector of search results from vector databases; Step S32: Vectorize the content of the key information by embedding the model to obtain a vector p; Step S33: Calculate the semantic similarity between vectors according to the following formula:

[0040] For the calculated k+m+n Sort the results, obtain the 10 results with the highest sort values, and input their corresponding vectors into the reasoning and decision-making layers.

[0041] Furthermore, the reasoning and decision-making layers include: an assembly context module, which combines user input with long-term and short-term memory to generate complete prompts according to a template; a large model generation module, which inputs the complete prompts into the large model and obtains the output from the large model; and a response generation module, which handles various formatting and parses the output from the large model to generate text, JSON, or speech structures.

[0042] Furthermore, in step S4, the information generated in each step, including language, user input, and user feedback, needs to be processed as follows: Step S41: Preprocessing, summarizing and standardizing, then appending metadata and processing according to different categories such as vectors, knowledge graphs and structured information; Step S42: Then, store the processed information in a vector database, a graph database, and a relational database; Step S43: Store the original information with attached metadata directly into the key-value pair database of the short-term memory.

[0043] This application presents a memory retrieval method for a companion robot. Through an architecture of "input parsing - multi-database storage - intelligent retrieval," analogous to the human brain, it enables the coherent tracking of the user's historical information. Furthermore, it ensures the continuity of dialogue through a short-term memory bank, accumulates user preferences through a long-term memory bank, and achieves dynamic memory updates through a feedback layer. This allows the companion robot to transcend simple question-and-answer functions and become a truly intelligent partner with long-term memory, personalized interaction capabilities, and continuous growth.

[0044] The above description is merely a preferred embodiment of the present invention and does not limit the scope of protection of the present invention. For those skilled in the art, the present invention can have various modifications and variations. Any changes, modifications, substitutions, integrations, and parameter alterations to these embodiments within the spirit and principles of the present invention, achieved through conventional substitutions or by achieving the same function without departing from the principles and spirit of the present invention, fall within the scope of protection of the present invention.

Claims

1. A method for retrieving the memory of a companion robot, characterized in that, Includes the following steps: Step S1: Obtain user input and corresponding environmental awareness, analyze them, and extract key information: entities, relationships, attributes, sentiment categories, and environmental information; and input the extracted key information into the memory center. Step S2: The memory center retrieves information from the long-term memory bank based on key information; Step S3: Convert the key information and retrieval results from the long-term memory into vectors, calculate the semantic similarity between the vectors, obtain the 10 vectors with the highest semantic similarity, and input these 10 results into the reasoning and decision-making layer; Step S4: The reasoning and decision-making layer generates language based on the 10 vectors with the highest semantic similarity and key information vectors.

2. The memory retrieval method for a companion robot according to claim 1, characterized in that, In step S1, the user input includes voice; the environmental perception includes photos of the user's surroundings.

3. The memory retrieval method for a companion robot according to claim 1, characterized in that, In step S2, the method for determining whether a long-term memory retrieval is required is as follows: According to the formula: ； in, The input information can be entities, relationships, attributes, sentiment categories, or environmental information; each different type of input is a... ; Represents different categories of embedding models, where k refers to the corresponding analogy; Represents normal distribution sampling with the same dimensions as the embedding model; The vector of potential semantic information for each piece of information, indicating whether it needs to be read from memory, was calculated. It is a small neural network that determines the probability of whether or not to access memory based on the given information. Sum all probabilities; if If the value exceeds the set threshold, the memory needs to be read; if If the value is less than the set threshold, there is no need to read the memory.

4. The memory retrieval method for a companion robot according to claim 1, characterized in that, The memory center includes a short-term memory bank and a long-term memory bank; The short-term memory includes a key-value pair database; the short-term memory is used to store each dialogue sequence, and all dialogue records of that sequence are read from the short-term memory each time. The long-term memory includes: a vector database, a graph database, and a relational database.

5. The memory retrieval method for a companion robot according to claim 4, characterized in that, In step S3, the method for calculating the semantic similarity between vectors is as follows: Step S31: Using an embedding model, convert all the search results from the graph database and the relational database into vectors. Adding the vector database search results, there will be a total of k+m+n vectors. Each vector is labeled... Where k is the vector of search results from graph databases, m is the vector of search results from relational databases, and n is the vector of search results from vector databases. Step S32: Vectorize the content of the key information by embedding the model to obtain a vector p; Step S33: Calculate the semantic similarity between vectors according to the following formula: ; For the calculated k+m+n Sort the results, obtain the 10 results with the highest sort values, and input their corresponding vectors into the reasoning and decision-making layers.

6. The memory retrieval method for a companion robot according to claim 5, characterized in that, In step S4, the information generated in each step, including user input and user feedback, needs to be processed as follows: Step S41: Preprocessing, summarizing, and standardizing, then appending metadata and processing according to different categories such as vectors, knowledge graphs, and structured information; Step S42: Then storing the processed information in vector databases, graph databases, and relational databases; Step S43: Store the original information with attached metadata directly into the key-value pair database of the short-term memory.

7. A memory retrieval system for a companion robot, characterized in that, It includes an input layer, a reasoning and decision-making layer, an output layer, a feedback layer, and a memory center; The input layer is used to acquire the user's voice input and photos of the user's environment, and to extract key information; The memory center determines whether a long-term database search is needed based on key information. If so, the search results are converted into vectors, and the key information is also converted into vectors. The semantic similarity between the vectors is calculated, and the 10 vectors with the highest semantic similarity are obtained. These 10 results are then input into the reasoning and decision-making layer. The reasoning and decision-making layers generate language based on the 10 vectors with the highest semantic similarity and key information vectors; if long-term memory retrieval is not required, language generation is only required based on key information vectors. The output layer is used to display the language generated by the reasoning and decision-making layers, and then transmit the display results to the memory center. The feedback layer is used for user interaction and feedback, and transmits the feedback results to the memory center.

8. The memory retrieval system for a companion robot according to claim 7, characterized in that, The memory center also includes a metadata attachment module, which is used to pass the content from the output layer and the feedback layer to the memory center to the short-term memory; and to preprocess, summarize and standardize the above content, and to pass the acquired structured data, vector data and knowledge graph data to the long-term memory.

Images