Multi-modal and hierarchical memory based agent pet interaction method and related device

Abstract

The application relates to the field of artificial intelligence and discloses an intelligent agent pet interaction method based on multi-modal and layered memory and related equipment, which comprises the following steps: collecting multi-source interaction data related to an intelligent agent pet and performing analysis and processing to generate a unified event object; performing key event determination on the unified event object; according to the key event determination result, writing the multi-source interaction data and the unified event object into a short-term context cache layer, a long-term semantic index layer and / or a key event atlas layer in a layered memory structure according to a preset layered memory strategy; when a new interaction request is received, jointly retrieving target memory content from the layered memory structure and generating a context input. Through unified eventization of multi-source data and layered memory storage, the application improves the memory organization efficiency in long-term interaction and reduces invalid writing, and still maintains the personality consistency and interaction continuity of the intelligent agent pet under the condition of limited context, so that the efficiency and reliability of the intelligent agent pet interaction response are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and related equipment for intelligent agent pet interaction based on multimodal and hierarchical memory. Background Technology

[0002] With the development of generative artificial intelligence technology, users can engage in long-term dialogue and interaction with intelligent agents and build digital characters with continuously growing attributes in virtual space. For example, intelligent pets are virtual companion characters that integrate artificial intelligence and virtual-real interaction. They can interact with users through hardware carriers as an interaction medium to achieve long-term companionship and growth.

[0003] However, in existing technologies, interactive systems for intelligent agents often use memory methods that summarize at regular intervals or write all data. As the amount of interactive information from different sources and long-term dialogue data continues to grow, not only does it lead to a lot of redundant information and difficulty in efficiently utilizing and organizing interactive data, but it also makes it difficult to maintain the consistency of the role's personality and the continuity of growth under the condition of limited context window. This affects the efficiency and reliability of interactive response and makes it impossible to provide users with efficient and consistent long-term companionship response. Summary of the Invention

[0004] The main objective of this invention is to provide a method and related equipment for intelligent agent pet interaction based on multimodal and hierarchical memory, aiming to solve the technical problems in the prior art where long-term interaction data is difficult to organize efficiently and personality consistency decreases under contextual constraints, thereby affecting the efficiency and reliability of intelligent agent pet interaction response.

[0005] The technical solution of the present invention is as follows: The first aspect of this invention provides an intelligent agent pet interaction method based on multimodal and hierarchical memory, comprising: Continuously collect multi-source interaction data related to the intelligent pet agent; The multi-source interactive data is subjected to semantic parsing, temporal alignment, and device status parsing to generate a unified event object; The unified event object is subjected to critical event determination to confirm whether the candidate event is a critical event. Based on the key event determination results, the corresponding multi-source interaction data and unified event objects are written into the short-term context cache layer, long-term semantic index layer and / or key event graph layer in the hierarchical memory structure according to the preset hierarchical memory strategy. Upon receiving a new interaction request, the target memory content is jointly retrieved from the short-term context cache layer, the long-term semantic index layer, and the key event graph layer, and a context input is generated to drive the intelligent agent pet to respond to the interaction request through the context input.

[0006] A second aspect of the present invention provides an intelligent agent pet interaction device based on multimodal and hierarchical memory, comprising: The data acquisition module is used to continuously collect multi-source interaction data related to the intelligent pet agent; The parsing and processing module is used to perform semantic parsing, temporal alignment, and device status parsing on the multi-source interactive data to generate a unified event object. The critical event determination module is used to determine whether the unified event object is a critical event and to confirm whether the candidate event is a critical event. The hierarchical memory writing module is used to write the corresponding multi-source interaction data and unified event objects into the short-term context cache layer, long-term semantic index layer and / or key event graph layer in the hierarchical memory structure according to the key event judgment result and a preset hierarchical memory strategy. The joint retrieval and generation module is used to jointly retrieve target memory content from the short-term context cache layer, long-term semantic index layer and key event graph layer and generate context input when a new interaction request is received, so as to drive the intelligent agent pet to respond to the interaction request through the context input.

[0007] A third aspect of the present invention provides a computer device including at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to perform the above-described intelligent agent pet interaction method based on multimodal and hierarchical memory.

[0008] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the above-described intelligent agent pet interaction method based on multimodal and hierarchical memory.

[0009] Beneficial Effects: This invention discloses an intelligent agent pet interaction method and related devices based on multimodal and hierarchical memory. Compared with existing technologies, the embodiments of this invention continuously collect multi-source interaction data related to the intelligent agent pet; perform semantic parsing, temporal alignment, and device status parsing processing on the multi-source interaction data to generate a unified event object; determine key events for the unified event object to confirm whether candidate events are key events; based on the key event determination results, write the corresponding multi-source interaction data and unified event object into the short-term context cache layer, long-term semantic index layer, and / or key event graph layer in the hierarchical memory structure according to a preset hierarchical memory strategy; when a new interaction request is received, target memory content is jointly retrieved from the short-term context cache layer, long-term semantic index layer, and key event graph layer to generate context input, so as to drive the intelligent agent pet to respond to the interaction request through the context input. This invention improves the memory organization efficiency and reduces invalid writes in long-term interactions by unifying multi-source data into events and using hierarchical memory storage. It can maintain the personality consistency and interaction continuity of the intelligent agent pet even under context-constrained conditions, thereby improving the efficiency and reliability of the intelligent agent pet's interaction response. Attached Figure Description

[0010] To more clearly illustrate the solutions in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0011] Figure 1 A schematic diagram of an application environment for the intelligent agent pet interaction method based on multimodal and hierarchical memory provided in an embodiment of the present invention; Figure 2 A flowchart of an intelligent agent pet interaction method based on multimodal and hierarchical memory provided in an embodiment of the present invention; Figure 3 A schematic diagram of the functional modules of the intelligent pet interaction device based on multimodal and hierarchical memory provided in an embodiment of the present invention; Figure 4 A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0012] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention is further described in detail below. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. The embodiments of the invention are described below with reference to the accompanying drawings.

[0013] The intelligent agent pet interaction method based on multimodal and hierarchical memory provided in this invention can be applied to, for example... Figure 1 The interactive scenario of the virtual-real interactive intelligent pet shown includes a terminal device 101, a hardware carrier 102 bound to the intelligent pet, a network 103, and a server 104. The network 103 serves as a medium to provide a communication link between the terminal device 101, the hardware carrier 102, and the server 104. The network 103 can include various connection types, such as wired and / or wireless communication links (e.g., Bluetooth, Wi-Fi, NFC, etc.).

[0014] Users can interact with server 104 via network 103 using terminal device 101 and hardware carrier 102 to receive or send messages, etc. Terminal device 101 may have a client installed that supports virtual scenarios. For example, when the virtual scenario is pet companionship, the client can be a virtual pet application, allowing users to log in to view, edit, or control the interactions of their bound intelligent pet in the virtual scenario (this is just an example). Terminal device 101 can be various electronic devices with a display screen and web browsing support, including but not limited to smartphones, tablets, and desktop computers.

[0015] The hardware carrier 102, which is bound to the intelligent pet, can serve as a real-world interaction medium between the user and the intelligent pet. It can sense the user's close-range touch, shaking, movement, and other real-world activities in real time and associate them with corresponding interactive events. The hardware carrier 102 can be a plush toy, smart wearable accessory, desktop smart ornament, smart speaker, etc., that integrates motion sensing and near-field communication recognition functions.

[0016] Server 104 can be a server that provides various services, such as a backend server that supports the content browsed by the user using terminal device 101 and hardware carrier 102 (this is just an example). The backend server can analyze and process data such as received user requests, and feed the processing results back to the user through terminal device 101 and hardware carrier 102. Server 104 can be a cloud server, a server of a distributed system, or a server combined with blockchain.

[0017] It should be understood that the number of terminal devices 101, hardware carriers 102, networks 103, and servers 104 mentioned above is merely illustrative. Depending on the implementation needs, there can be any number of terminal devices 101, hardware carriers 102, networks 103, and servers 104. For example, a single user can correspond to one terminal device 101 and one hardware carrier 102, and multiple users can achieve interactive linkage between intelligent pets through the network.

[0018] like Figure 2As shown, the intelligent agent pet interaction method based on multimodal and hierarchical memory provided in this embodiment of the invention specifically includes the following steps: S201. Continuously collect multi-source interaction data related to the intelligent pet.

[0019] In this embodiment, after activating the intelligent pet, multi-source interaction data related to it is continuously collected. This multi-source interaction data refers to all types of data related to the intelligent pet that reflect the user's interaction intent, the intelligent pet's state, and virtual-real interaction behavior. Specifically, it includes dialogue data between the user and the intelligent pet, virtual world exploration commands initiated by the user and exploration feedback data (including task completion status, acquired items, triggered events, and exploration status information, etc.), interaction data between other users and the intelligent pet, environment configuration data generated by the user creating / editing / decorating the virtual residence, and at least one of the following: motion sensing data and device interaction data generated by the hardware carrier bound to the intelligent pet. This achieves comprehensive capture of the user's interaction intent, the intelligent pet's state, and virtual-real interaction behavior, ensuring that no key interaction information is missed, and providing a complete and reliable basic data source for subsequent event processing and memory construction.

[0020] Specifically, a combination of real-time and timed data collection can be used to ensure that key interactive behaviors are not missed while reducing power consumption. For example, user and dialogue data can be collected in real time, hardware status data can be collected every 100ms, and virtual exploration data can be collected when a behavior occurs. The specific collection method and collection frequency can also be flexibly adjusted and set according to actual needs. This embodiment does not limit this.

[0021] S202. Perform semantic parsing, timing alignment, and device status parsing on the multi-source interactive data to generate a unified event object.

[0022] In this embodiment, since the multi-source interactive data is scattered and has different formats, it is difficult to achieve unified management and event recognition. Therefore, multi-dimensional parsing and processing is used to standardize the multi-source heterogeneous interactive data into a unified event object, so that different types of interactive behaviors can be represented with a consistent structure, providing a unified basis for subsequent key event judgment and hierarchical memory writing.

[0023] Semantic parsing primarily targets interactive data such as text and voice, extracting core information from the interaction, such as the interaction subject, interaction object, and context summary. Time sequence alignment mainly targets various behavior and operation logs, using timestamps as a benchmark to arrange interactive behaviors occurring at different times in chronological order, thereby eliminating time discrepancies and forming a continuous interaction sequence. Device status parsing mainly targets data generated by the hardware carrier bound to the intelligent pet, identifying the hardware's action type, working status, and interaction intent, converting raw sensor signals into corresponding event information.

[0024] Based on the results of multi-dimensional parsing, multi-source interactive data is standardized and encoded into a unified event object. This unified event object is a standardized data structure containing core interactive information, including at least fields such as event type, timestamp, participant identifier, event intensity, context summary, spatial attributes, and confidence level.

[0025] This embodiment solves the problem of scattered sources and heterogeneous formats of multi-source interactive data by standardizing various types of data into a unified event object. This enables different types of interactive behaviors to be consistently represented, which not only improves the standardization and efficiency of interactive data processing, but also helps to improve the continuity and interpretability of the growth trajectory of the intelligent agent pet.

[0026] S203. Perform critical event determination on the unified event object to confirm whether the candidate event is a critical event.

[0027] In this embodiment, after obtaining a unified event object, key events are determined. Interaction events with the intelligent pet can be categorized into key events and non-key events. Key events are those that significantly impact the intelligent pet's growth, personality development, relationship network construction, or virtual world state; these events need to be permanently stored in memory. Non-key events are mundane, everyday interactions with no long-term impact (such as simple greetings), requiring only short-term caching. Using key event determination to guide subsequent hierarchical memory writing effectively reduces invalid writes and duplicate summaries, lowering computational and storage resource consumption.

[0028] Specifically, key events can be identified and determined by combining model recognition with rule constraints. Key events include events such as the creation of new friends; the creation of new residences or significant changes in residences; the completion of adventures; real-world companionship / play events; and events whose impact on character relationships, preferences, ability growth, or world state exceeds an impact threshold, etc. This embodiment accurately filters out key events that have a long-term impact on the growth, personality development, and relationship network construction of the intelligent agent pet, distinguishes between key and non-key interactions, avoids redundant information occupying storage and computing resources, and provides accurate judgment criteria for hierarchical memory storage.

[0029] S204. Based on the key event determination results, write the corresponding multi-source interaction data and unified event objects into the short-term context cache layer, long-term semantic index layer and / or key event graph layer in the hierarchical memory structure according to the preset hierarchical memory strategy.

[0030] In this embodiment, the hierarchical structure adopts a three-layer architecture of short-term caching, long-term semantic indexing, and key event graph association. Different layers undertake different memory functions. The preset hierarchical memory strategy clarifies the memory writing rules corresponding to different judgment results (key / non-key events), ensuring the efficiency and rationality of memory organization.

[0031] Specifically, the three-layer architecture of the hierarchical memory structure has the following functions: The short-term context cache layer is used to store a preset number of recent interaction records, behavior logs, and device action sequences (such as storing the most recent N rounds of dialogue, the most recent M hardware actions, the most recent K world change operations, etc.) to support real-time response. Specifically, a first-in-first-out mechanism can be adopted to ensure the efficiency of real-time response. The long-term semantic index layer is used to store vectorized historical interaction fragments, summary fragments, and event description texts to support fast retrieval based on semantic similarity; The key event graph layer is used to store entity nodes (users, intelligent pets, friends, residences, etc.), event nodes, and their relationship edges (including relationships such as ownership, companionship, acquaintance, residence, exploration, and commemoration), enabling the association management and traceability of key events. For example, a new friend relationship established by a "tap" can be represented as: a new "acquaintance" relationship edge is added between "Hardware A" corresponding to "Intelligent Pet A" and "Hardware B" corresponding to "Intelligent Pet B", with timestamps, location tags, and interaction confidence.

[0032] Based on the functions of different layers in the hierarchical memory structure, after determining the criticality of an event, the corresponding multi-source interaction data and unified event objects are written into the short-term context cache layer, long-term semantic index layer, and / or key event graph layer of the hierarchical memory structure according to a preset hierarchical memory strategy. For example, if the event is determined to be non-critical, the multi-source interaction data and unified event objects are only temporarily stored in the short-term context cache layer and not written into long-term memory to avoid redundancy; if the event is determined to be critical, it needs to be written into both the long-term semantic index layer and the key event graph layer to achieve long-term solidification and association management of critical events. Of course, the specific hierarchical memory strategy can be flexibly set according to actual writing needs, and this embodiment does not limit it. By establishing a hierarchical memory structure of short-term context cache layer, long-term semantic index layer, and key event graph layer, and in conjunction with differentiated hierarchical memory storage strategies, the memory organization efficiency in long-term interaction scenarios can be improved under context-constrained conditions, laying the foundation for rapid retrieval and optimizing the utilization of storage and computing resources.

[0033] S205. Upon receiving a new interaction request, the target memory content is jointly retrieved from the short-term context cache layer, the long-term semantic index layer, and the key event graph layer, and a context input is generated to drive the intelligent agent pet to respond to the interaction request through the context input.

[0034] In this embodiment, if a new interaction request is received, such as various operation or dialogue requests related to the intelligent agent pet, the target memory content is jointly retrieved from the short-term context cache layer, the long-term semantic index layer, and the key event graph layer based on the hierarchical memory structure that has been continuously collected and identified and written differentially before, and context input is generated. That is, the content related to the current interaction request in the three layers of memory is integrated to generate a comprehensive and coherent context input. The intelligent agent is driven to respond to the current interaction request through this context input, ensuring that the response of the intelligent agent pet can conform to the historical interaction logic and personality settings, and reflect the continuity of growth and memory correlation.

[0035] This embodiment integrates three types of memory content: short-term, long-term, and related, to ensure that the intelligent pet's response is consistent with the latest interaction, related to historical memory, and adapted to the current scenario, avoiding problems such as interruption of growth or inconsistency in personality, and improving the consistency, accuracy, and user experience of the interaction.

[0036] In the above embodiments, this invention discloses an intelligent agent pet interaction method based on multimodal and hierarchical memory. The method involves continuously collecting multi-source interaction data related to the intelligent agent pet; performing semantic parsing, temporal alignment, and device status parsing on the multi-source interaction data to generate a unified event object; determining key events for the unified event object to confirm whether candidate events are key events; based on the key event determination results, writing the corresponding multi-source interaction data and the unified event object into the short-term context cache layer, long-term semantic index layer, and / or key event graph layer of the hierarchical memory structure according to a preset hierarchical memory strategy; upon receiving a new interaction request, jointly retrieving target memory content from the short-term context cache layer, long-term semantic index layer, and key event graph layer to generate context input, thereby driving the intelligent agent pet to respond to the interaction request through the context input. This invention, through unified eventification of multi-source data and hierarchical memory storage, can improve memory organization efficiency and reduce invalid writes in long-term interactions, maintaining the personality consistency and interaction continuity of the intelligent agent pet even under context-constrained conditions, thereby improving the efficiency and reliability of the intelligent agent pet's interaction response.

[0037] In one embodiment, step S202 includes: The multi-source interactive data is divided according to data type to obtain text interaction data, behavior and operation log data, and hardware status data; Semantic parsing is performed on the text interaction data to extract slot information related to the interaction; The behavior and operation log data are time-series aligned according to timestamps to generate a continuous sequence of events; The hardware status data is parsed to extract the device status and the original sensing signals are encoded into corresponding event structures. The slot information, sequence events, and event structure are standardized and uniformly encapsulated to obtain the unified event object.

[0038] In this embodiment, the types of multi-source interactive data vary considerably, and different types of data require different parsing methods. Therefore, the data is first classified according to data type to obtain text interaction data, behavior and operation log data, and hardware status data, ensuring the relevance and accuracy of the parsing. Text interaction data can include text dialogue data between the user and the intelligent pet, voice-to-text data, etc.; behavior and operation log data can include user-initiated virtual adventures, friend interactions, residence editing, and other operation records to reflect the user's actions in the virtual world; hardware status data includes motion sensing data generated by the hardware carrier bound to the intelligent pet, such as acceleration, angular velocity, and displacement, as well as device interaction data, such as NFC near-field contact and device identity information, to reflect real-world interaction behavior.

[0039] For the different types of data obtained from the segmentation, semantic parsing is performed on the text interaction data to extract slot information related to the interaction. Slot information is structured data that represents the core content of the interaction, including interaction subject slots, interaction object slots, and interaction time slots, etc., providing core information for the subsequent encapsulation of unified event objects. Specifically, the text interaction data can be processed through a pre-trained language model. This model is pre-trained on the labeled interaction text dataset and can automatically perform semantic understanding and extract various types of slot information. For example, semantic parsing of the text "Adventure with me" extracts the following slot information: interaction subject = user A, agent A; interaction intent = adventure; interaction object = adventure scene; interaction time = current time, etc.

[0040] The behavior and operation log data are time-series aligned according to timestamps. Since there may be time deviations in the behavior and operation log data, such as different timestamp precisions for different operations or disordered operation order records, time-series alignment is used to eliminate time deviations. The discrete operation logs are arranged in chronological order to form a continuous sequence of events, thereby clearly reflecting the sequential logic of user operations and facilitating subsequent event identification and memory association.

[0041] For hardware status data, since it is mostly raw sensor signals, such as acceleration values ​​and displacement coordinates, it cannot be directly used for event recognition and memory writing. Therefore, device status parsing is required to encode the raw signals into understandable event structures. For example, after preprocessing the raw sensor signals to eliminate noise interference, the preprocessed signals are identified according to preset hardware action recognition rules to determine the type of hardware action, such as shaking, touching, or moving. Finally, the action type, action parameters (including shaking amplitude, peak acceleration, moving distance, etc.), action duration, device identifier, and other information are encoded into event structures.

[0042] After performing targeted parsing and processing on different types of interactive data, slot information, sequence events, and event structures are standardized to convert the three types of information into a consistent format, eliminating data heterogeneity. Specific standardization processes can include field unification, numerical standardization, text normalization, etc. Finally, they are uniformly encapsulated into a unified event object to ensure that the fields are consistent and the information is complete.

[0043] This embodiment improves the accuracy and efficiency of data parsing by classifying data by type and parsing it in a targeted manner. Furthermore, it standardizes and encapsulates the parsing results to generate a unified event object, which solves the problem of heterogeneous multi-source interactive data and provides a unified foundation for subsequent key event judgment and memory writing, ensuring that different types of interactive behaviors can be effectively identified and managed.

[0044] In one embodiment, step S203 includes: The unified event object is input into a preset event classification model for event identification to obtain the event type probability of the corresponding candidate event; By combining the weight information of different event types and the event impact threshold in the preset rule engine, the event type probability of the candidate events is weighted and scored to obtain the corresponding key score; When the critical score is greater than the preset critical threshold, the corresponding candidate event is confirmed as a critical event.

[0045] In this embodiment, when determining key events, key event identification is performed based on an event classification model and a preset rule engine. The preset event classification model is a deep learning-based multi-classification model. Its input is the feature vector of a unified event object, and its output is the probability that the event belongs to various preset event types. For example, the model can adopt a Transformer architecture and be pre-trained on a dataset containing a large amount of labeled interaction data. The training dataset covers various event types such as dialogue, exploration, friend interaction, and hardware interaction. Of course, the event classification model can also be updated through incremental learning. For example, when a new event type appears, such as the addition of "residence decoration event", the model can be fine-tuned by supplementing labeled data to ensure the model's recognition accuracy and generalization ability.

[0046] Specifically, the various fields of a unified event object are first converted into feature vectors. These feature vectors are then input into a pre-defined event classification model. The model extracts the core features of the event through an attention mechanism and outputs the probability that a candidate event belongs to any particular event type. For example: When two hardware devices make near-field contact within a preset time window and the device identity verification is successful, it is determined as a candidate event for establishing a friend relationship. When the shaking amplitude, duration, and displacement pattern of a single hardware carrier meet the preset thresholds, the probability of the output event type is: 0.85 for real-world companionship candidate event, 0.10 for dialogue interaction event, and 0.05 for adventure event, and it is determined to be a real-world companionship candidate event. When a user adds a room to their residence, changes the furniture layout to a preset ratio, or updates the decoration theme, the probability of the event type is output as follows: significant residence change candidate event 0.85, dialogue interaction event 0.10, exploration event 0.05, which is judged as a significant residence change candidate event. When the intelligent pet completes map node exploration, task triggering, or item acquisition in the virtual world, the probability of the output event type is as follows: exploration completion candidate event 0.85, dialogue interaction event 0.10, companionship event 0.05, which is judged as exploration completion candidate event.

[0047] Since the probability output of an event classification model alone cannot accurately determine critical events (because some event types have little impact on agent growth, such as ordinary greetings), even if the probability is high, they should not be classified as critical events. Therefore, a pre-configured rule engine is used to accurately filter critical events through weight allocation and influence threshold settings. The rule engine is a set of rules pre-configured based on business needs and can be dynamically adjusted according to the actual application scenario.

[0048] Specifically, the rule engine predefines weight information for various event types. The weight value is positively correlated with the degree of influence of the event on the growth of the agent and the relationship network. For example, the weight of the hardware companionship event is 0.8, the weight of the friend creation event is 0.9, the weight of the ordinary dialogue event is 0.2, the weight of the residence editing event is 0.7, and so on. At the same time, the event impact thresholds for different event types are also defined. The impact threshold is positively correlated with the importance of the event type. For example, the impact threshold for key event types (friend creation, hardware companionship, adventure completion events) is 0.8, and the impact threshold for non-key event types (ordinary dialogue) is 0.2, and so on.

[0049] The probability of candidate events is weighted and scored using weight information and an event impact threshold to obtain a corresponding criticality score. The specific calculation method is: Criticality Score = Event Type Probability × Corresponding Event Type Weight + Event Impact Threshold. The calculated criticality score is compared with a preset criticality threshold. If the criticality score is greater than the preset criticality threshold, the candidate event is confirmed as a critical event and requires long-term memory fixation; if the criticality score is less than or equal to the preset criticality threshold, the candidate event is confirmed as a non-critical event and only requires short-term caching. This preset criticality threshold can be set based on a large amount of experimental data or dynamically adjusted according to actual application scenarios (such as storage resources and computing power), for example, using 0.7, 0.8, etc. This embodiment does not limit this.

[0050] This embodiment identifies key events through a dual judgment mechanism of model recognition and rule constraints, avoiding the bias of single model recognition, accurately screening out key events that have a long-term impact on the intelligent agent pet, providing an accurate basis for subsequent hierarchical memory writing, and reducing the storage and computing power consumption of non-critical events.

[0051] In one embodiment, step S204 specifically includes: If the event is determined to be non-critical, the multi-source interaction data and the unified event object are temporarily stored in the short-term context cache layer of the hierarchical memory structure. If it is determined to be a critical event, then vectorization processing and graph information extraction are performed based on the multi-source interaction data and the unified event object to obtain the vector representation and graph information of the critical event. The key event graph information is written into the key event graph layer of the hierarchical memory structure, and the vector representation of the key events is written into the long-term semantic index layer of the hierarchical memory structure.

[0052] In this embodiment, if a candidate event is determined to be a non-critical event, the multi-source interaction data and the unified event object are temporarily stored in the short-term context cache layer and not written to the long-term memory and critical event graph. Non-critical events refer to daily trivial interaction behaviors that have no long-term impact on the growth of the intelligent agent pet, such as simple greetings like "Hello" or "Are you there?". These events are only needed for recent interaction responses and do not require long-term storage. Therefore, they are temporarily stored in the short-term context cache layer to reduce storage and computing power consumption.

[0053] Specifically, the short-term context cache layer can adopt a first-in-first-out storage mechanism with a preset cache capacity (such as the most recent N rounds of dialogue, the most recent M hardware actions, the most recent K world change operations, etc.). When a new non-critical event is written, if the cache capacity is full, the earliest written non-critical event is deleted to ensure that the cache layer only retains the latest short-term interaction content.

[0054] If a candidate event is determined to be a key event, it needs to be permanently stored in memory and support efficient retrieval and related queries. Therefore, the first step is to perform vectorization processing and graph information extraction based on multi-source interaction data and a unified event object. Vectorization processing is used to convert key events into vector representations that can be used for semantic retrieval. Specifically, the text content of multi-source interaction data and the core fields of the unified event object are converted into fixed-dimensional vector representations. These vector representations can characterize the semantic features of key events, facilitating semantic similarity retrieval in the long-term semantic index layer. Graph information extraction is used to construct the association between key events and entities, facilitating subsequent graph retrieval and relationship tracing. Specifically, entity nodes (including users, intelligent pets, friends, hardware, residences, exploration areas, real-world hardware devices, etc.), event nodes, and relationship edges between entities and between entities and events (including ownership, companionship, acquaintance, residence, exploration, commemoration, etc.) are extracted from multi-source interaction data and the unified event object to form key event graph information.

[0055] Subsequently, based on the relevant information of key events, incremental updates to the long-term semantic index layer and the key event graph layer are triggered. When writing key event graph information into the key event graph layer, the extracted entity nodes, event nodes, and relationship edges are added to the existing graph structure. If the entity node already exists (e.g., user A is already in the graph), only new relationship edges and event nodes are added to avoid duplicate storage. When writing the vector representation of key events into the long-term semantic index layer, metadata such as timestamps, event types, and entity identifiers are added to the vectors, and a semantic index is built to support fast retrieval based on semantic similarity. For example, when a user queries "which friends I have added", the relevant friend creation events can be quickly found through semantic vector retrieval.

[0056] This embodiment employs a differentiated memory writing strategy to cache non-critical events for a short period and solidify critical events for a long period. This ensures both the efficiency of real-time interaction and the long-term retention of critical events. Furthermore, through vectorization and graph processing, it improves the retrieval efficiency and traceability of critical events, providing reliable support for subsequent joint retrieval and response generation.

[0057] In one embodiment, step S204 further includes: When the historical interaction fragments in the short-term context cache layer exceed the cache time window, the historical interaction fragments that exceed the cache time window are vectorized and written into the long-term semantic index layer. When the context length, call resource consumption, and / or retrieval load are detected to reach the corresponding preset thresholds, the historical interaction fragments in the short-term context cache layer are subjected to summary compression processing to generate the corresponding structured summary. The structured summary is compared with the historical summaries stored in the long-term semantic index layer. When there is an incremental difference that exceeds the similarity threshold, the newly generated structured summary is vectorized and written into the long-term semantic index layer. When a change in the relationship network or a change in the virtual world state is detected, an incremental update to the long-term semantic index layer and the key event graph layer is triggered.

[0058] In this embodiment, based on differentiated writing according to whether key events are detected, the preset hierarchical memory strategy also provides processing methods for factors such as cache time windows, summary compression, and changes in relationship networks or virtual world states.

[0059] The short-term context cache layer has a cache time window that is a preset time threshold, such as 1 hour or other time. Although historical interaction fragments (non-critical events) that exceed this window do not need to be used as the context of short-term interactions, they may still contain certain semantic information, which can help with the semantic retrieval of subsequent long-term interactions. Therefore, they can be vectorized and written into the long-term semantic index layer. For example, when the storage time of a certain historical interaction fragment exceeds the cache time window, the fragment is vectorized, the semantic vector is extracted, and the vector representation of the historical interaction fragment is written into the long-term semantic index layer and labeled with the "non-critical event semantic vector". This makes it easier to distinguish between the semantic information of critical and non-critical events during subsequent retrieval, thus achieving long-term retention of semantic information and avoiding the original data from occupying too much storage resources.

[0060] Furthermore, prolonged interactions and the continuous accumulation of interaction fragments may lead to excessively long context lengths, high resource consumption, or excessive retrieval loads, thus affecting interaction response efficiency. Therefore, pre-set thresholds are established for context length, resource consumption, and retrieval load. For example, the context length threshold could be the upper limit T1 for the current session token count; the resource consumption threshold could be the upper limit T2 for computing power utilization, or the upper limit T3 for the proportion of non-critical content, etc.; and the retrieval load threshold could be the upper limit T4 for the number of candidate memory fragments returned by retrieval, etc. When any threshold is reached, digest compression is triggered. Specifically, digest compression is initiated when the system detects that any of the following conditions are met: The current number of session tokens exceeds the threshold T1; The current device's computing power utilization rate is greater than the threshold T2; The proportion of non-critical content in the current session is greater than the threshold T3; The number of candidate memory fragments returned by the retrieval is greater than the threshold T4.

[0061] After the summary compression process is initiated, the specific summary compression can extract high-weight slot information (such as personality trait slots, preference slots, relationship change slots, and world state slots) related to the character's personality, relationship status, and world state from the historical interaction fragments to be compressed, and generate a structured summary. This structured summary adopts a fixed format and includes key information such as personality tag update fields, preference tag update fields, relationship network change fields, adventure experience summary fields, residence change summary fields, and real-life companionship record fields, but the length is greatly reduced, effectively reducing redundancy.

[0062] Based on summary compression, updates to the long-term semantic index layer must meet the summary drift condition. That is, in order to avoid repeatedly writing the same or similar summary information and save storage resources, it is necessary to compare the similarity between the newly generated structured summary and the historical summary. Long-term writing is only performed when there is a significant incremental difference, so as to ensure that the information in the long-term semantic index layer has incremental value.

[0063] Specifically, a cosine similarity algorithm can be used to calculate the similarity between the vector representation of the newly generated structured summary and the historical summary vectors already stored in the long-term semantic index layer. If the similarity is less than a preset similarity threshold, it indicates that there is an incremental difference exceeding the threshold, and the summary contains new semantic information. Therefore, it is vectorized and written into the long-term semantic index layer. If the similarity is greater than or equal to the preset similarity threshold, it indicates that there is no significant increment in the summary information, and it is not permanently written into the long-term semantic index layer. This avoids repeatedly writing similar summary information into the long-term semantic index layer, further optimizing the utilization of storage resources, ensuring that the long-term semantic index layer only retains information with incremental value, and improving the effectiveness of long-term memory.

[0064] In addition to updating the long-term semantic index layer and the key event graph layer upon detecting critical events and summary drift, incremental updates to the long-term semantic index layer and the key event graph layer are also triggered upon detecting changes in the relationship network or the virtual world state. Specifically, changes in the relationship network, such as adding or removing friends, and changes in the virtual world state, such as completing home renovations, ending exploration missions, or increasing agent growth values, all affect the associations and semantic information in long-term memory. Therefore, incremental updates are necessary to ensure that long-term memory remains consistent with the current state and to avoid deviations in response.

[0065] Specifically, when changes in relationships, such as adding or removing friends, are detected, the long-term semantic index layer updates the semantic vectors related to the changes, such as the semantic vector corresponding to a newly added friend. For the key event graph layer, entity nodes are added and their relationship edges are updated, such as adding an "acquaintance" relationship edge, ensuring the graph structure remains consistent with the current relationship network. When the virtual world state changes, the attribute data of entity nodes in the key event graph layer is updated accordingly, and the state change content is vectorized and incrementally added to the long-term semantic index layer, ensuring the virtual world state can be accurately retrieved and traced back.

[0066] This embodiment preserves valuable semantic information and reduces redundant storage through subsequent processing of short-term cache fragments. Furthermore, by using digest compression and triggered incremental updates, it optimizes the utilization of storage resources while ensuring the timeliness and consistency of long-term memory, thus guaranteeing the accuracy of interactive responses.

[0067] In one embodiment, step S205 includes: Upon receiving a new interaction request, read the most recent N interaction records from the short-term context cache layer; The interaction request is semantically vectorized, and the long-term semantic index layer is used to retrieve semantically relevant events based on the request semantic vector. Retrieve and extract target graph paths related to the current scene state from the key event graph layer; The most recent N interaction records, semantically related events, and target graph paths are integrated to generate corresponding context inputs, which are used to drive the intelligent pet to respond to the interaction requests.

[0068] In this embodiment, during retrieval and response generation, upon receiving a new interaction request, the most recent N interaction records are read from the short-term context cache layer in reverse chronological order. This content reflects the user's latest interaction intent and scenario. Reading these records ensures that the intelligent pet's response aligns with recent interactions, avoiding "amnesia." Preferably, records related to the current interaction request from the most recent N interaction records can be further filtered out, while irrelevant records (such as irrelevant greetings) are removed to ensure that the read records are targeted.

[0069] Furthermore, new interaction requests are semantically vectorized to generate request semantic vectors. The top K semantic vectors with the highest similarity to the request semantic vectors are retrieved in the long-term semantic index layer (K ​​can be 3-5, etc.). Based on the retrieved semantic vectors, the corresponding historical events are extracted, and the semantically relevant events most related to the current request are selected. Through semantic vectorization and vector recall, historical events semantically related to the current interaction request can be quickly extracted, ensuring that the response can be associated with long-term memory and reflecting the growth continuity of the intelligent pet.

[0070] Simultaneously, target graph paths related to the current scene state are retrieved and extracted from the key event graph layer. The current scene state includes character relationships (such as the relationship between the user and friends, the relationship between the user and the agent), living environment (such as the decoration status of the residence, furniture layout, etc.), and exploration progress (such as the completion status of exploration tasks, explored scenes, etc.). This information is stored in the key event graph layer. By retrieving graph paths, the associations and events related to the current scene can be extracted, ensuring that the response fits the current scene and reflects the relevance of memory. Specifically, the core entities of the current scene state can be identified first, such as user A, agent A, user A's residence, and the current exploration scene. Then, in the key event graph layer, starting from the core entities, relationship edges and event nodes related to the core entities are retrieved to construct graph paths and filter out target graph paths related to the current interaction request. By accurately extracting the associations and event information of the current scene, the context driving the agent pet to respond is ensured to fit the current scene, thereby improving the relevance and rationality of the response.

[0071] Finally, when generating context input, the most recent N interaction records, semantically related events, and target graph paths obtained from short-term, long-term, and graph relationships are integrated in logical order to generate standardized context input. This integrates short-term, long-term, and relational memory content into the context input, solving the limitations of single memory retrieval. This drives the intelligent pet to respond to the context input, ensuring that the intelligent pet's response is related to historical interactions and fits the current scenario, thus improving the coherence of the interaction and personality consistency.

[0072] In one embodiment, the method further includes: When the multi-source interactive data is motion sensing data and device interaction data generated by a pre-bound hardware carrier, feature extraction is performed on the motion sensing data and device interaction data to obtain corresponding motion interaction features. Confirm whether the motion interaction features meet the preset companion mode. If they do, map the companion activities of the hardware carrier to state changes in the virtual world based on the motion interaction features, and update the virtual world state accordingly.

[0073] In this embodiment, the hardware carrier serves as the medium for the interaction between the intelligent pet and the real world. The motion sensing data and device interaction data generated by the carrier can reflect the user's real-world behavior (such as companionship and social interaction). By extracting features from these data, the user's real-world interaction intentions can be accurately identified, providing a foundation for virtual-real interaction.

[0074] Specifically, the hardware carrier incorporates inertial sensors and near-field recognition modules, generating motion sensing data including acceleration, angular velocity, displacement trajectory, shaking amplitude, and duration. Device interaction data includes near-field contact information and device identity verification information. After preprocessing this data using signal processing and feature extraction algorithms, motion interaction features are extracted, including action type features such as shaking, touching, and moving; action parameter features such as peak shaking amplitude, duration, and acceleration sequence; and interaction object features such as the contacted hardware identifier. This provides accurate feature data for determining virtual-real interaction, ensuring its accuracy.

[0075] Based on the extracted motion interaction features, the system determines real-world companionship activities. The preset companionship mode is a predefined hardware action mode that can be identified as a user's companionship with an intelligent pet. It can include various companionship scenarios, such as rocking companionship and carrying a mobile adventure. Each mode corresponds to a specific motion interaction feature threshold. The extracted motion interaction features are matched with the feature threshold of the preset companionship mode. If the threshold of a certain companionship mode is met, the hardware activity is confirmed as a companionship activity. This maps the companionship activities of real-world hardware to changes in the state of the virtual world, realizing the linkage between real-world behavior and virtual growth.

[0076] When an activity is identified as a companion activity, it is mapped to a state change in the virtual world based on the parameters of the motion interaction characteristics. Different companion activities correspond to different state changes. For example, shaking companionship can be mapped to an increase in the mood and growth value of the intelligent pet in the virtual world; moving companionship can be mapped to an update of the intelligent pet's adventure footprints and an accumulation of companionship time in the virtual world; hardware touch companionship can be mapped to the establishment of friendship relationships and an increase in social value in the virtual world, and so on.

[0077] Once the mapping is complete, the virtual world state is updated accordingly, including the intelligent pet's emotional state, growth attributes, companionship duration, virtual world friend relationships, adventure progress, and residence status, etc. This status update information is synchronized to the user's terminal and hardware, allowing the user to see the impact of companionship behavior on the virtual intelligent pet in real time. Through feature extraction from hardware data and companionship pattern recognition, the system accurately captures the user's real-world companionship behavior. Furthermore, through virtual-real mapping and status updates, it achieves a linkage between real-world behavior and the growth of the virtual intelligent pet, enhancing the fun and immersion of the interaction and further enriching the interactive scenarios.

[0078] It should be noted that there is no necessary order between the above steps. Those skilled in the art will understand from the description of the embodiments of the present invention that the above steps may have different execution orders in different embodiments, that is, they may be executed in parallel or in turn, etc.

[0079] Further reference Figure 3 As a response to the above Figure 2 The present invention provides an embodiment of an intelligent agent pet interaction device based on multimodal and hierarchical memory, which implements the method shown. Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0080] like Figure 3 As shown, the intelligent pet interaction device 30 based on multimodal and hierarchical memory described in this embodiment includes: Data acquisition module 301 is used to continuously collect multi-source interaction data related to the intelligent pet agent; The parsing and processing module 302 is used to perform semantic parsing, temporal alignment and device status parsing processing on the multi-source interactive data to generate a unified event object; The critical event determination module 303 is used to determine the critical event of the unified event object and confirm whether the candidate event is a critical event. The hierarchical memory writing module 304 is used to write the corresponding multi-source interaction data and unified event objects into the short-term context cache layer, long-term semantic index layer and / or key event graph layer in the hierarchical memory structure according to the key event judgment result and a preset hierarchical memory strategy. The joint retrieval and generation module 305 is used to jointly retrieve target memory content from the short-term context cache layer, long-term semantic index layer and key event graph layer and generate context input when a new interaction request is received, so as to drive the intelligent agent pet to respond to the interaction request through the context input.

[0081] The module referred to in this invention is a series of computer program instruction segments that can perform specific functions. It is more suitable than a program for describing the interactive execution process of an intelligent pet. For specific implementation methods of each module, please refer to the corresponding method embodiments above, which will not be repeated here.

[0082] In one embodiment, the parsing processing module 302 includes: The data partitioning unit is used to partition the multi-source interactive data according to the data type to obtain text interaction data, behavior and operation log data, and hardware status data. The semantic parsing unit is used to perform semantic parsing on the text interaction data and extract slot information related to the interaction. The timing alignment unit is used to align the behavior and operation log data according to timestamps to generate a continuous sequence of events; The status encoding unit is used to parse the hardware status data and encode the original sensing signal into a corresponding event structure. The standardized encapsulation unit is used to standardize the slot information, sequence events, and event structure, and encapsulate them into the unified event object.

[0083] In one embodiment, the key event determination module 303 includes: An event recognition unit is used to input the unified event object into a preset event classification model for event recognition, and obtain the event type probability of the corresponding candidate event; The key weighted scoring unit is used to combine the weight information of different event types in the preset rule engine with the event impact threshold to weight and score the event type probability of the candidate event to obtain the corresponding key score. The determination unit is used to confirm the corresponding candidate event as a critical event when the criticality score is greater than a preset criticality threshold.

[0084] In one embodiment, the hierarchical memory writing module 304 is specifically used for: If the event is determined to be non-critical, the multi-source interaction data and the unified event object are temporarily stored in the short-term context cache layer of the hierarchical memory structure. If it is determined to be a critical event, then vectorization processing and graph information extraction are performed based on the multi-source interaction data and the unified event object to obtain the vector representation and graph information of the critical event. The key event graph information is written into the key event graph layer of the hierarchical memory structure, and the vector representation of the key events is written into the long-term semantic index layer of the hierarchical memory structure.

[0085] In one embodiment, the hierarchical memory writing module 304 is further configured to: When the historical interaction fragments in the short-term context cache layer exceed the cache time window, the historical interaction fragments that exceed the cache time window are vectorized and written into the long-term semantic index layer. When the context length, call resource consumption, and / or retrieval load are detected to reach the corresponding preset thresholds, the historical interaction fragments in the short-term context cache layer are subjected to summary compression processing to generate the corresponding structured summary. The structured summary is compared with the historical summaries stored in the long-term semantic index layer. When there is an incremental difference that exceeds the similarity threshold, the newly generated structured summary is vectorized and written into the long-term semantic index layer. When a change in the relationship network or a change in the virtual world state is detected, an incremental update to the long-term semantic index layer and the key event graph layer is triggered.

[0086] In one embodiment, the joint retrieval generation module 305 includes: The short-term read unit is used to read the most recent N interaction records from the short-term context cache layer when a new interaction request is received; The vector recall unit is used to semantically vectorize the interaction request, perform vector recall in the long-term semantic index layer based on the request semantic vector, and extract semantically related events. The path retrieval unit is used to retrieve and extract target graph paths related to the current scene state from the key event graph layer; The context generation unit is used to integrate the most recent N interaction records, semantically related events, and target graph paths to generate corresponding context inputs. The context inputs are used to drive the intelligent pet to respond to the interaction request.

[0087] In one embodiment, the device 30 further includes: The hardware motion feature extraction module is used to extract features from the motion sensing data and device interaction data generated by the pre-bound hardware carrier when the multi-source interaction data is motion sensing data and device interaction data generated by the pre-bound hardware carrier, so as to obtain the corresponding motion interaction features. The virtual-real linkage mapping module is used to confirm whether the motion interaction features meet the preset companion mode. If they do, the companion activities of the hardware carrier are mapped to state changes in the virtual world according to the motion interaction features, and the virtual world state is updated accordingly.

[0088] In the above embodiments, this invention discloses an intelligent agent pet interaction device based on multimodal and hierarchical memory. It continuously collects multi-source interaction data related to the intelligent agent pet; performs semantic parsing, temporal alignment, and device status parsing on the multi-source interaction data to generate a unified event object; performs key event determination on the unified event object to confirm whether candidate events are key events; based on the key event determination result, writes the corresponding multi-source interaction data and the unified event object into the short-term context cache layer, long-term semantic index layer, and / or key event graph layer in the hierarchical memory structure according to a preset hierarchical memory strategy; when a new interaction request is received, it jointly retrieves target memory content from the short-term context cache layer, long-term semantic index layer, and key event graph layer and generates context input, so as to drive the intelligent agent pet to respond to the interaction request through the context input. This invention, through unified eventification of multi-source data and hierarchical memory storage, can improve the efficiency of memory organization in long-term interactions and reduce invalid writes. Even under context-constrained conditions, it can maintain the personality consistency and interaction continuity of the intelligent agent pet, thereby improving the efficiency and reliability of the intelligent agent pet's interaction response.

[0089] Specific limitations regarding the intelligent agent pet interaction device based on multimodal and hierarchical memory can be found in the limitations of the intelligent agent pet interaction method based on multimodal and hierarchical memory mentioned above, and will not be repeated here. Each module in the aforementioned intelligent agent pet interaction device based on multimodal and hierarchical memory can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0090] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0091] Another embodiment of the present invention provides a computer device, such as... Figure 4 As shown, the computer device 40 includes: One or more processors 401 and memory 402, Figure 4 The following section uses a processor 401 as an example. The processor 401 and the memory 402 can be connected via a bus or other means. Figure 4 Taking the example of a connection between China and Israel via a bus.

[0092] The processor 401 is used to perform various control logics of the computer device 40. It can be any conventional processor, microprocessor, state machine, general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), microcontroller, ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components.

[0093] The memory 402, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions corresponding to the intelligent agent pet interaction method based on multimodal and hierarchical memory in the embodiments of the present invention. The processor 401 executes various functional applications and data processing of the computer device 40 by running the non-volatile software programs, instructions, and units stored in the memory 402, thereby realizing the intelligent agent pet interaction method based on multimodal and hierarchical memory in the above method embodiments.

[0094] Another embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by one or more processors, perform the steps of the intelligent agent pet interaction method based on multimodal and hierarchical memory in any of the above method embodiments.

[0095] It should be noted that the functions or steps that can be implemented by the computer-readable storage medium or computer device described above can be referred to the relevant descriptions in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0096] Based on the above description of the embodiments, those skilled in the art will understand that the methods described in the embodiments can be implemented using software plus necessary general-purpose hardware platforms. Of course, they can also be implemented using hardware, but in many cases, the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0097] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). The computer program can be stored in a non-volatile, computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The storage medium can be a memory, magnetic disk, floppy disk, flash memory, optical storage, etc.

[0098] It should be noted that any AI models, software tools, or components not belonging to this company appearing in the embodiments of this application are merely illustrative examples and do not represent actual use. All user personal information involved in the embodiments of this application has been authorized (with the knowledge and consent) by the relevant parties or has been fully authorized by all parties, and the executing entity may obtain it through various legal and compliant means. The collection, storage, use, processing, transmission, provision, and disclosure of the information, data, and signals involved all comply with relevant laws and regulations and do not violate public order and good morals.

[0099] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. A method for intelligent agent pet interaction based on multimodal and hierarchical memory, characterized in that, include: Continuously collect multi-source interaction data related to the intelligent pet agent; The multi-source interactive data is subjected to semantic parsing, temporal alignment, and device status parsing to generate a unified event object; The unified event object is subjected to critical event determination to confirm whether the candidate event is a critical event. Based on the key event determination results, the corresponding multi-source interaction data and unified event objects are written into the short-term context cache layer, long-term semantic index layer and key event graph layer in the hierarchical memory structure according to the preset hierarchical memory strategy. Upon receiving a new interaction request, the target memory content is jointly retrieved from the short-term context cache layer, the long-term semantic index layer, and the key event graph layer, and a context input is generated to drive the intelligent pet to respond to the interaction request through the context input. The process of performing semantic parsing, temporal alignment, and device status parsing on the multi-source interactive data to generate a unified event object includes: The multi-source interactive data is divided according to data type to obtain text interaction data, behavior and operation log data, and hardware status data; Semantic parsing is performed on the text interaction data to extract slot information related to the interaction; The behavior and operation log data are time-series aligned according to timestamps to generate a continuous sequence of events; The hardware status data is parsed to extract the device status and the original sensing signals are encoded into corresponding event structures. The slot information, sequence events, and event structure are standardized and uniformly encapsulated to obtain the unified event object. The step of writing the corresponding multi-source interaction data and unified event objects into the short-term context cache layer, long-term semantic index layer, and key event graph layer in the hierarchical memory structure according to the key event determination results and a preset hierarchical memory strategy specifically includes: If the event is determined to be non-critical, the multi-source interaction data and the unified event object are temporarily stored in the short-term context cache layer of the hierarchical memory structure. If it is determined to be a critical event, then vectorization processing and graph information extraction are performed based on the multi-source interaction data and the unified event object to obtain the vector representation and graph information of the critical event. The key event graph information is written into the key event graph layer of the hierarchical memory structure, and the vector representation of the key events is written into the long-term semantic index layer of the hierarchical memory structure. Upon receiving a new interaction request, the process of jointly retrieving target memory content from the short-term context cache layer, the long-term semantic index layer, and the key event graph layer and generating context input to drive the intelligent agent pet to respond to the interaction request via the context input includes: Upon receiving a new interaction request, read the most recent N interaction records from the short-term context cache layer; The interaction request is semantically vectorized, and the long-term semantic index layer is used to retrieve semantically relevant events based on the request semantic vector. Retrieve and extract target graph paths related to the current scene state from the key event graph layer; The most recent N interaction records, semantically related events, and target graph paths are integrated to generate corresponding context inputs, which are used to drive the intelligent pet to respond to the interaction requests.

2. The intelligent agent pet interaction method based on multimodal and hierarchical memory according to claim 1, characterized in that, The critical event determination of the unified event object, and the confirmation of whether the candidate event is a critical event, includes: The unified event object is input into a preset event classification model for event identification to obtain the event type probability of the corresponding candidate event; By combining the weight information of different event types and the event impact threshold in the preset rule engine, the event type probability of the candidate events is weighted and scored to obtain the corresponding key score; When the critical score is greater than the preset critical threshold, the corresponding candidate event is confirmed as a critical event.

3. The intelligent agent pet interaction method based on multimodal and hierarchical memory according to claim 1, characterized in that, After temporarily storing the multi-source interaction data and the unified event object in the short-term context cache layer of the hierarchical memory structure, the method includes: When the historical interaction fragments in the short-term context cache layer exceed the cache time window, the historical interaction fragments that exceed the cache time window are vectorized and written into the long-term semantic index layer. When the context length, call resource consumption, and / or retrieval load are detected to reach the corresponding preset thresholds, the historical interaction fragments in the short-term context cache layer are subjected to summary compression processing to generate the corresponding structured summary. The structured summary is compared with the historical summaries stored in the long-term semantic index layer. When there is an incremental difference that exceeds the similarity threshold, the newly generated structured summary is vectorized and written into the long-term semantic index layer. When a change in the relationship network or a change in the virtual world state is detected, an incremental update to the long-term semantic index layer and the key event graph layer is triggered.

4. The intelligent agent pet interaction method based on multimodal and hierarchical memory according to claim 1, characterized in that, The method further includes: When the multi-source interactive data is motion sensing data and device interaction data generated by a pre-bound hardware carrier, feature extraction is performed on the motion sensing data and device interaction data to obtain corresponding motion interaction features. Confirm whether the motion interaction features meet the preset companion mode. If they do, map the companion activities of the hardware carrier to state changes in the virtual world based on the motion interaction features, and update the virtual world state accordingly.

5. An apparatus employing the intelligent agent pet interaction method based on multimodal and hierarchical memory as described in claim 1, characterized in that, include: The data acquisition module is used to continuously collect multi-source interaction data related to the intelligent pet agent; The parsing and processing module is used to perform semantic parsing, temporal alignment, and device status parsing on the multi-source interactive data to generate a unified event object. The critical event determination module is used to determine whether the unified event object is a critical event and to confirm whether the candidate event is a critical event. The hierarchical memory writing module is used to write the corresponding multi-source interaction data and unified event objects into the short-term context cache layer, long-term semantic index layer and key event graph layer in the hierarchical memory structure according to the key event judgment result and a preset hierarchical memory strategy. The joint retrieval and generation module is used to jointly retrieve target memory content from the short-term context cache layer, long-term semantic index layer and key event graph layer and generate context input when a new interaction request is received, so as to drive the intelligent agent pet to respond to the interaction request through the context input.

6. A computer device, characterized in that, Includes at least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the intelligent agent pet interaction method based on multimodal and hierarchical memory as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform the intelligent agent pet interaction method based on multimodal and hierarchical memory as described in any one of claims 1-4.

Images