Memory graph display method and electronic device

Abstract

The present application relates to the technical field of terminals. Provided are a memory graph display method and an electronic device. The method is applied to the electronic device, and the method comprises: in response to a first operation, displaying a first interface, wherein the first interface is used for displaying a memory graph of a user, the memory graph is used for indicating memory information of the user, the memory information is represented by means of a plurality of node identifiers and relationships between the plurality of node identifiers, and the node identifiers are used for indicating entities or events. When a memory graph is applied to an LLM, it facilitates the effective associating of the LLM with other node identifiers and relationships corresponding to the other nodes by means of the plurality of node identifiers in the memory graph and the relationships between the node identifiers, such that richer context information is acquired, thereby deepening the understanding of a user by the LLM. In this way, in actual applications, the LLM can use the acquired context information to provide personalized services to the user, so as to meet personalized requirements of the user, thereby improving the user experience.

Description

Memory graph display method and electronic device

[0001] The present application claims priority to the Chinese patent application No. 202411271845.4, filed on September 10, 2024, and entitled "Memory graph display method and electronic device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the technical field of terminals, and in particular to a memory graph display method and an electronic device. BACKGROUND

[0003] With the rapid development of artificial intelligence technology, large language models (LLMs) have become a new hotspot of research and application. LLMs, with their large parameter scale and strong representation ability, have made remarkable achievements in various fields such as natural language processing, computer vision, and speech recognition. However, despite the impressive capabilities of LLMs in handling complex tasks, in practical applications, especially in providing personalized services for users, existing large model technology still faces many challenges. SUMMARY

[0004] The present application provides a memory graph display method and an electronic device, which solves the problem that LLMs cannot provide personalized services for users in the prior art.

[0005] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a memory graph display method, applied to an electronic device, the method comprising:

[0007] In response to a first operation, a first interface is displayed, the first interface being used to display a memory graph of a user, the memory graph being used to indicate memory information of the user, the memory information being represented by a plurality of node identifiers and relationships between the plurality of node identifiers, the node identifiers being used to indicate entities or events.

[0008] Based on the memory graph display method provided in the present application, the association relationship between entities and / or events in the user's memory information can be intuitively displayed through multiple node identifiers and the relationship between the multiple node identifiers. This display method is closer to the storage and use mode of human memory information, is easy for users to understand and accept, and can improve the user experience of using memory information. When the memory graph is applied to the LLM, the LLM can effectively associate other node identifiers and the relationships corresponding to the other node identifiers through the multiple node identifiers and the relationship between the multiple node identifiers in the memory graph, thereby obtaining richer context information and deepening the understanding of the user by the LLM. In actual application, the LLM can provide personalized services for the user by using the richer context information obtained, meet the personalized needs of the user, and improve the user experience.

[0009] In addition, compared with storing memory information in the form of a tensor or in the form of text, the above-mentioned manner of directly displaying a memory graph on a first interface facilitates the user to quickly find memory information related to an entity or an event through a node identifier according to actual needs, effectively avoids the cumbersome steps of finding whether the memory information is related to a certain entity or event in the memory information (or memory pool list) of the user in the form of a tensor or in the form of text, simplifies the steps of finding a certain entity or event, and facilitates the user to quickly view the memory information of the user.

[0010] Optionally, the electronic device can display the first interface in response to a first operation on a first application, where the first application can be any APP installed in the electronic device.

[0011] Optionally, the electronic device can also display the first interface in response to a first operation on a first service, where the first service can be any system service provided by the operating system.

[0012] In some embodiments, the method further includes: in response to a triggering operation on a first node identifier in the memory graph, displaying memory information corresponding to a first event in chronological order, the first event being an event indicated by the first node identifier.

[0013] It should be understood that the above-mentioned display of memory information corresponding to the first event in chronological order is to string the memory information corresponding to the first event through a timeline in chronological order. This facilitates the user to comprehensively and clearly view the memory information corresponding to the first event, and also facilitates the user to trace back and find all or part of the memory information related to the first event.

[0014] In some embodiments, the method further includes:

[0015] In response to a triggering operation on a second node identifier in the memory graph, display first graph content in the memory graph that is associated with a first entity, the first entity being indicated by the second node identifier.

[0016] In some embodiments, the first interface further includes a search control, and the method further includes:

[0017] In response to a triggering operation on the search control, display a second interface for receiving a third node identifier input by a user;

[0018] In response to a second operation detected on the second interface, display second graph content in the memory graph that corresponds to the third node identifier.

[0019] Based on the above possible implementation, by inputting a third node identifier in the search control, relevant memory information of an entity or event corresponding to the third node identifier can be quickly obtained.

[0020] In some embodiments, the method further includes:

[0021] In response to a third operation on a fourth node identifier in the memory graph, remove the fourth node identifier and a relationship corresponding to the fourth node identifier from the memory graph.

[0022] Based on the above possible implementation, the memory information of the user is represented in a visualized manner, facilitating direct control and management (such as filtering, editing, or deleting, etc.) of the memory information by the user. The visualized manner can also effectively improve the operation efficiency of the memory information by the user.

[0023] In some embodiments, the method further includes:

[0024] Display a fifth node identifier in the memory graph, the fifth node identifier being obtained according to memory information of a login user of the electronic device;

[0025] In response to a fourth operation on the fifth node identifier, add the fifth node identifier and a relationship between the fifth node identifier in the memory graph.

[0026] In some embodiments, the method further includes:

[0027] In response to a fifth operation on the fifth node identifier, modify a node name of the fifth node identifier in the memory graph.

[0028] In some embodiments, the method further includes:

[0029] In response to a sixth operation on a first relationship in the relationship, update a name of the first relationship.

[0030] It should be understood that in actual applications, the relationship between events or entities is not constant, and users can actively manage the relationship between entities or events based on changes in the relationship between entities or events, and improve the operability of events or entities.

[0031] In some embodiments, the first interface includes a display switching control, and the method further includes: in response to a triggering operation on the display switching control, displaying the memory information in a list display mode.

[0032] Based on the above possible implementation manners, the display mode of the memory information can be enriched to further meet the operation needs of the memory information under different actual application requirements. For example, the memory information closest to or farthest from the current event can be viewed in a list manner; for another example, the association between an entity (and / or an event) and other entities (and / or other events) can be more intuitively viewed in a graphing manner.

[0033] Optionally, the memory information can be displayed in a list display mode or a graphing display mode.

[0034] In some embodiments, the user logged in on the electronic device is a first user, and the method further includes:

[0035] displaying a synchronization device setting interface, the synchronization device setting interface indicating that the login user of a second device is also the first user;

[0036] In response to a seventh operation detected on the synchronization device setting interface, synchronizing the memory information between the electronic device and the second device.

[0037] Based on the above possible implementation manners, the memory information of multiple devices of the same user can be fused (or synchronized) based on the operation of the user opening at least one second device, and the sharing of personal memory data between multiple devices is realized. Even if the same application is used on different devices, consistent use experience can be maintained.

[0038] In some embodiments, the method further includes: displaying an application setting interface, the application setting interface displaying a first application;

[0039] In response to an eighth operation detected on the application setting interface, setting the source of the memory information to include the first application.

[0040] Based on the above possible implementation manner, the personal memory data fusion between multiple applications installed on the electronic device can be realized, the data on different applications can be integrated through the memory graph, and after the integrated data is applied to the LLM, the LLM can know the user better, so as to provide personalized services for the user more accurately and efficiently.

[0041] Optionally, the memory information and / or the memory graph of the user can be stored in the electronic device of the user, so that the risk of memory information leakage caused by network transmission can be reduced, and the privacy information (for example, interests, habits, etc.) of the user can be effectively protected. In this way, the user can quickly access the personal memory data without waiting for the network response or connecting to the server in the case of no network or unstable network. In addition, based on the memory information stored in the electronic device of the user, the application or the LLM can provide more personalized services for the user, such as recommending delicious food according to the personal eating habits of the user.

[0042] Optionally, the memory information and / or the memory graph of the user can also be stored in the cloud server. Storing the memory information of the user in the cloud server can effectively prevent the risk of data loss caused by hardware failure or software error of the electronic device, and the backup and recovery strategy of the cloud server is more perfect than that of the electronic device of the user, so that the data can be quickly recovered in the case of data loss or damage. In addition, storing the memory information of the user in the cloud server can also effectively reduce the load and storage pressure of the electronic device.

[0043] Optionally, the storage locations of the memory information and the memory graph of the user can be the same or different.

[0044] In a second aspect, an embodiment of the present application provides a memory graph display device, which comprises:

[0045] A first display module is configured to display a first interface in response to a first operation, the first interface being configured to display a memory graph of a user, the memory graph being configured to indicate memory information of the user, the memory information being represented by a plurality of node identifiers and relationships between the plurality of node identifiers, and the node identifiers being configured to indicate entities or events.

[0046] In some embodiments, the memory graph display device further comprises:

[0047] A second display module is configured to display memory information corresponding to a first event in a time sequence in response to a triggering operation on a first node identifier in the memory graph, the first event being an event indicated by the first node identifier.

[0048] In some embodiments, the memory graph display device further comprises:

[0049] The third display module is configured to display, in response to a triggering operation on the second node identifier in the memory graph, first graph content associated with a first entity in the memory graph, the first entity being an entity indicated by the second node identifier.

[0050] In some embodiments, the first interface further includes a search control, and the memory graph display apparatus further includes:

[0051] The fourth display module is configured to display, in response to a triggering operation on the search control, a second interface configured to receive a third node identifier input by a user; and display, in response to a second operation detected on the second interface, second graph content corresponding to the third node identifier in the memory graph.

[0052] In some embodiments, the memory graph display apparatus further includes:

[0053] The removal module is configured to remove, in response to a third operation on a fourth node identifier in the memory graph, the fourth node identifier and a relationship corresponding to the fourth node identifier from the memory graph.

[0054] In some embodiments, the memory graph display apparatus further includes:

[0055] The fifth display module is configured to display, in the memory graph, a fifth node identifier obtained according to memory information of a login user of the electronic device.

[0056] The determination module is configured to add, in response to a fourth operation on the fifth node identifier, the fifth node identifier and a relationship between the fifth node identifier in the memory graph.

[0057] In some embodiments, the memory graph display apparatus further includes:

[0058] The modification module is configured to modify, in response to a fifth operation on the fifth node identifier, a node name of the fifth node identifier in the memory graph.

[0059] In some embodiments, the memory graph display apparatus further includes:

[0060] The update module is configured to update, in response to a sixth operation on a first relationship in the relationship, a name of the first relationship.

[0061] In some embodiments, the first interface includes a display switching control, and the memory graph display apparatus further includes:

[0062] The switching module is configured to display, in response to a triggering operation on the display switching control, the memory information in a list display manner.

[0063] In some embodiments, a user logged in on the electronic device is a first user, and the memory graph display apparatus further includes:

[0064] The sixth display module is configured to display a synchronization device setting interface, the synchronization device setting interface indicating that a login user of the second device is also the first user.

[0065] The synchronization module is configured to synchronize the memory information between the electronic device and the second device in response to a seventh operation detected on the synchronization device setting interface.

[0066] In some embodiments, the memory atlas display apparatus further includes:

[0067] The seventh display module is configured to display an application setting interface, the application setting interface displaying the first application.

[0068] The setting module is configured to set the source of the memory information to include the first application in response to an eighth operation detected on the application setting interface.

[0069] In a third aspect, an embodiment of the present application provides an electronic device, including: one or more processors; one or more memories; and one or more computer programs; wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the method in any possible implementation manner of the first aspect.

[0070] In a fourth aspect, an embodiment of the present application provides a memory atlas display system, the memory atlas display system including at least one electronic device provided in the third aspect.

[0071] In a fifth aspect, a computer readable storage medium is provided, the computer readable medium storing a computer program (which can also be referred to as code or instructions), when the computer program code is run on a computer, causing the computer to perform the method in any aspect or any possible implementation manner of any aspect.

[0072] In a sixth aspect, a computer program product including instructions is provided, the computer program product including: a computer program (which can also be referred to as code or instructions), when the computer program is run on a computer, causing the computer to perform the method in any aspect or any possible implementation manner of any aspect.

[0073] It should be noted that the above computer program code can be stored in whole or in part on a first storage medium, wherein the first storage medium can be packaged together with the processor or packaged separately from the processor, and the present application does not make a specific limitation thereon.

[0074] In a seventh aspect, a chip system is provided, comprising: a processor and a memory, the memory being configured to store a computer program (also referred to as code or instructions), and the processor being configured to invoke and run the computer program stored in the memory, so that a device or apparatus installed with the chip system performs the method in any one of the aspects or any possible implementation manner of the aspects.

[0075] It can be understood that the beneficial effects of the above-mentioned second aspect to seventh aspect can be referred to the related description in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF DRAWINGS

[0076] FIG. 1 is a hardware structure schematic diagram of an electronic device provided by an embodiment of the present application.

[0077] FIG. 2 is a software structure block diagram of an electronic device provided by an embodiment of the present application.

[0078] FIG. 3 is a flow schematic diagram of a memory graph construction method provided by an embodiment of the present application.

[0079] FIG. 4 is a flowchart of constructing a tag library provided by an embodiment of the present application.

[0080] FIG. 5 is a structure schematic diagram of a constructed tag library provided by an embodiment of the present application.

[0081] FIG. 6 is a schematic diagram for defining the relationship of each node in a schema modeling manner provided by an embodiment of the present application.

[0082] FIG. 7 is a schematic diagram of using event class memory to link entity class memory provided by an embodiment of the present application.

[0083] FIG. 8 is a flow schematic diagram of a memory graph display method provided by an embodiment of the present application.

[0084] FIGS. 9 to 42 are scene schematic diagrams related to memory graph display provided by an embodiment of the present application.

[0085] FIG. 43 is a structure schematic diagram of a memory graph display device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0086] The technical solutions in the embodiments of the present application are described below in combination with the drawings in the embodiments of the present application and related embodiments. In the descriptions of the embodiments of the present application, if there is no special description and no logical conflict, the terms and / or descriptions of different embodiments are consistent and can be mutually referenced, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship. In the embodiments of the present application, “at least one” means one or more, and “multiple” means two or more. “And / or” describes the association relationship of the associated objects, which means that there can be three kinds of relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In the textual description of the embodiments of the present application, the character “ / ” generally represents that the front and rear associated objects have an “or” relationship; for example, A / B can represent A or B. In the present application, “first”, “second” and various numerical numbers are only for the convenience of distinguishing and do not limit the scope of the embodiments of the present application. For example, different messages are distinguished, rather than used to describe a specific order or sequence.

[0087] With the rapid development of artificial intelligence technology, LLM has become a new hotspot of research and application today. LLM has made remarkable achievements in many fields such as natural language processing, computer vision and speech recognition, etc. due to its huge parameter scale and strong representation ability. However, although large model technology has shown amazing ability in dealing with complex tasks, in practical applications, especially in providing personalized services for users, existing large model technology still faces many challenges.

[0088] For example, existing LLMs can only respond independently to user single-input questions, and cannot effectively associate the context information or historical information of user continuous questions, so as to predict or recommend the point of interest (POI) that the user may be interested in. This limitation makes LLM unable to meet the personalized needs of users.

[0089] Knowledge graph (Knowledge Graph) is a structured knowledge representation form, which presents entities, concepts and their relationships in a graphical way. If a corresponding knowledge graph can be constructed based on user's interests, preferences, experiences and habits, etc., then after applying the constructed knowledge graph to LLM, LLM can provide more accurate and personalized service responses and more close to user's daily planning and decision-making, etc. according to user's personalized needs.

[0090] To this end, an embodiment of the present application provides a memory graph display method, through a plurality of node identifiers and the relationships between the plurality of node identifiers, the association relationships between entities and / or events in the user's memory information can be intuitively displayed, that is, the user's memory information is indicated by using the memory graph. When the memory graph is applied to the LLM, it is helpful for the LLM to effectively associate to other node identifiers and the relationships corresponding to the other node identifiers through the plurality of node identifiers and the relationships between the plurality of node identifiers in the memory graph, to obtain richer context information, and to deepen the understanding of the user by the LLM. In this way, in actual application, the LLM can provide personalized services for the user by using the obtained context information, meet the personalized needs of the user, and improve the user experience.

[0091] It should be understood that the memory information in the embodiments of the present application is related information generated according to the historical operations of the user on the electronic device, and the memory information of different users is different. The memory information of the same user on different electronic devices can be the same or different. The memory information can indicate events, persons, objects, times, and places, etc., and can involve various aspects of personal life, work, and study, etc. In the embodiments of the present application, the memory information of the user can be, for example, "last week, I had hot pot with XXX" or "this Friday, I will go to Shanghai with my colleague Wang to sign a contract with customer B", and the like.

[0092] The memory graph, as the name implies, is a graph-based data structure, which can be used to represent and store the memory information of the user. It is a kind of relationship network formed by connecting different entities and / or events, and the relationships between the entities and / or events.

[0093] Further, the memory graph can be composed of nodes and edges.

[0094] Each node (or node identifier) can represent an entity or an event. Among them, the entity can refer to an object with at least one attribute, that is, a transaction in the real world. For example, the entity can include a person, a place name, a company, a phone, a pet, a book, a device, etc.

[0095] The event can refer to a certain matter in learning, life or work, such as marriage, travel, movie performance, business trip, meeting, overtime, shopping, sports and fitness, learning and training, etc.

[0096] Each edge can represent a certain relationship between entities and / or events. Specifically, an edge can represent the relationship between entities, between events, between an entity and an event, or between an event and an entity. For example, the relationship can include a friend relationship, a teacher-student relationship, a classmate relationship, a colleague relationship, a partner relationship, a family relationship, a marriage relationship, a lover relationship, etc.

[0097] It should be understood that the memory graph can represent the memory information of the user in a structured form, so that machines (such as computers, etc.) can more easily understand and process the information. In addition, the memory graph can be visually displayed using a graphical interface, which can enable the user to intuitively understand different entities and / or events and their relationships.

[0098] The memory graph can be applied to an artificial intelligence agent (AI Agent) or an LLM. For example, when the memory graph is applied to an AI Agent, the AI Agent can simulate the role of a personal assistant of the user, helping the user to manage the personal affairs of the user (such as scheduling, important matter reminders, weather forecast queries, ordering takeout, etc.). For example, in actual application, when the user asks the AI Agent “What do I have tomorrow?”, the AI Agent can call the memory information in the memory graph to check the calendar of the user and answer the related question.

[0099] The memory graph can also be applied to scenarios such as personal affairs management (such as information recording, task progress tracking, daily planning or decision making, etc.), intelligent search, dialogue questioning, text analysis, sentiment analysis, data management, etc., and the embodiments of the present application do not make any limitation thereto.

[0100] The memory graph display method provided by the embodiments of the present application can be applied to an electronic device. For example, the electronic device can include but is not limited to a personal computer (PC), a mobile phone, a tablet computer, a wearable device (such as a watch, a bracelet, a helmet, smart glasses, etc.), an augmented reality (AR) / virtual reality (VR) device, a mixed reality (MR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a vehicle-mounted device, a smart screen, a smart home device (such as a smart TV, a smart speaker, a smart camera, and an electronic door lock, etc.), a server, and a motion sensing game machine in a human-computer interaction scene, etc. The embodiments of the present application do not make any limitation on the specific type of the electronic device.

[0101] Referring to FIG. 1, a schematic diagram of a hardware structure of an electronic device 100 is provided. The electronic device 100 can include a processor 110, an external memory interface 120, an internal memory 131, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headset jack 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 can include a pressure sensor 180A, a gyroscope sensor 180B, a barometric sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0102] It can be understood that the structure illustrated in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 can include more or fewer components than illustrated, or combine certain components, or split certain components, or different arrangement of components. The illustrated components can be implemented in hardware, software, or a combination of software and hardware.

[0103] As an example, when the electronic device 100 is a mobile phone or a tablet computer, it can include all the components illustrated, or only include part of the components illustrated.

[0104] The processor 110 can include one or more processing units, for example: the processor 110 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0105] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0106] The processor 110 can also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can store instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to use the instructions or data again, it can be directly called from the memory. This avoids repeated access and reduces the waiting time of the processor 110, thereby improving the efficiency of the system.

[0107] In some embodiments, the processor 110 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0108] In some embodiments, the audio module 170 can also deliver audio signals to the wireless communication module 160 through the PCM interface. Both the I2S interface and the PCM interface can be used for audio communication.

[0109] In other embodiments, the power management module 141 can also be provided in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be provided in the same device.

[0110] The wireless communication function of the electronic device 100 can be realized through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor, and the baseband processor, etc.

[0111] Antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of antennas. For example, antenna 1 can be multiplexed as a diversity antenna for wireless local area networks. In some other embodiments, antennas can be used in combination with tuning switches.

[0112] Mobile communication module 150 can provide solutions for wireless communication including 2G / 3G / 4G / 5G, etc. applied on electronic device 100. Mobile communication module 150 can include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. Mobile communication module 150 can receive electromagnetic waves by antenna 1, and perform filtering, amplification, etc. on the received electromagnetic waves, and transmit the processed signals to a modem processor for demodulation. Mobile communication module 150 can also amplify signals modulated by the modem processor, and convert the signals into electromagnetic waves radiated by antenna 1.

[0113] In some embodiments, at least part of the functional modules of mobile communication module 150 can be arranged in processor 110. In some embodiments, at least part of the functional modules of mobile communication module 150 can be arranged in the same device as at least part of the modules of processor 110.

[0114] Wireless communication module 160 can provide solutions for wireless communication including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR) technology, etc. applied on electronic device 100. Wireless communication module 160 can be one or more devices integrated with at least one communication processing module. Wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and transmits the processed signals to processor 110. Wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification on the signals, and convert the signals into electromagnetic waves radiated by antenna 2.

[0115] In some embodiments, the antenna 1 and the mobile communication module 150 of the electronic device 100 are coupled, and the antenna 2 and the wireless communication module 160 are coupled, so that the electronic device 100 can communicate with a network and other devices through wireless communication technology. The wireless communication technology can include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS can include a global positioning system (GPS), a global navigation satellite system (GLONASS), a beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a satellite based augmentation systems (SBAS).

[0116] The electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, which is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 can include one or more GPUs, which execute program instructions to generate or change display information.

[0117] The display screen 194 is configured to display images, videos, and the like. For example, various display interfaces and the like in the embodiments of the present application. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flex light-emitting diode (FLED), a Miniled, a MicroLed, a Micro-oLed, a quantum dot light emitting diodes (QLED), and the like. In some embodiments, the electronic device 100 can include one or N display screens 194, where N is a positive integer greater than 1.

[0118] The electronic device 100 can implement a photographing function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, and an application processor.

[0119] The electronic device 100 can implement an audio function through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and an application processor.

[0120] The audio module 170 is configured to convert a digital audio signal into an analog audio signal for output, and to convert an analog audio input into a digital audio signal. The audio module 170 can also be configured to encode and decode audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0121] The speaker 170A, also known as a "loudspeaker", is configured to convert an audio electrical signal into an acoustic signal. The electronic device 100 can listen to music or take a hands-free call through the speaker 170A, for example, the speaker can play the comparison and analysis result provided by the embodiments of the present application.

[0122] The receiver 170B, also known as a "earpiece", is configured to convert an audio electrical signal into an acoustic signal. When the electronic device 100 is on a call or receiving a voice message, the receiver 170B can be placed close to the ear to receive the voice.

[0123] Microphone 170C, also called "microphone", "sound pickup", is used to convert sound signal into electrical signal. When making a call or sending voice message, the user can speak into the microphone 170C by putting the mouth close to the microphone 170C, and input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 can be provided with two microphones 170C, in addition to collecting sound signals, noise reduction function can also be realized. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C, in addition to collecting sound signals, noise reduction, it can also identify the source of sound, and realize the function of directional recording, etc.

[0124] The earphone interface 170D is used to connect wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0125] The key 190 includes the power-on key, the volume key, etc. The key 190 can be a mechanical key. It can also be a touch key. The electronic device 100 can receive key input, and generate key signal input related to user settings and function control of the electronic device 100.

[0126] The motor 191 can generate vibration prompt. The motor 191 can be used for incoming call vibration prompt, and also can be used for touch vibration feedback. For example, the touch operation acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The touch operation acting on different regions of the display screen 194, the motor 191 can also correspond to different vibration feedback effects. Different application scenarios (such as time reminder, receiving information, alarm, game, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

[0127] The indicator 192 can be an indicator light, which can be used to indicate the charging state, the power change, and also can be used to indicate messages, missed calls, notifications, etc.

[0128] The SIM card interface 195 is configured to connect a SIM card. The SIM card can be connected to or disconnected from the electronic device 100 by being inserted into or pulled out of the SIM card interface 195. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than one. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195. The types of the multiple cards can be the same or different. The SIM card interface 195 can be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external storage cards. The electronic device 100 interacts with a network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.

[0129] Referring to FIG. 2, a schematic diagram of a software structure of an electronic device according to an embodiment of the present application is shown. The operating system of the electronic device can be an Android system, an iOS system, a Windows system, an OS system, or the like. In this embodiment, the operating system of the electronic device is taken as the HarmonyOS system as an example for illustration. Alternatively, The operating system of the electronic device is taken as the HarmonyOS system as an example for illustration.

[0130] In some embodiments, the HarmonyOS system can be divided into four layers, including a kernel layer, a system service layer, a framework layer, and an application layer, and the layers communicate with each other through software interfaces.

[0131] As shown in FIG. 2, the kernel layer includes a kernel abstract layer (KAL) and a driver subsystem. The KAL includes multiple kernels, such as a Linux Kernel of a Linux system, a LiteOS of a lightweight Internet of Things system, and the like. The driver subsystem can include a hardware driver foundation (HDF). The hardware driver foundation can provide a unified peripheral access capability and a driving development and management framework. The kernel layer with multiple kernels can select a corresponding kernel for processing according to the requirements of the system.

[0132] The system service layer is a core capability set of the HarmonyOS system, and the system service layer provides services to the application program through the framework layer. The layer can include a system basic capability subsystem set, a basic software service subsystem set, an enhanced software service subsystem set, and a hardware service subsystem set.

[0133] The system basic capability subsystem set provides basic capabilities for the running, scheduling, migration and other operations of distributed applications on the devices of the HarmonyOS system. It can include distributed software bus, distributed data management, distributed task scheduling, Ark multi-language runtime, public basic library, multi-mode input, graphics, security, artificial intelligence (AI), user program framework, and the like. Among them, the Ark multi-language runtime provides a C or C++ or JavaScript (JS) multi-language runtime and basic system class library, and can also provide a runtime for Java programs (i.e., application programs or parts developed using Java language in the framework layer) that are statically compiled using the Ark compiler.

[0134] The basic software service subsystem set provides common and universal software services for the HarmonyOS system. It can include event notification, telephone, multimedia, design for X (DFX), MSDP&DV, and the like.

[0135] The enhanced software service subsystem set provides differentiated capability-enhanced software services for different devices for the HarmonyOS system. It can include smart screen specific services, wearable specific services, and Internet of Things (IoT) specific service subsystems.

[0136] The hardware service subsystem set provides hardware services for the HarmonyOS system. It can include location services, biometric identification, wearable specific hardware services, IoT specific hardware services, and the like.

[0137] The framework layer provides user program frameworks and capability frameworks for the HarmonyOS system application development in multiple languages such as Java, C, C++, JS, and the like, two user interface (UI) frameworks (including a Java UI framework for Java language and a JS UI framework for JS language), and multiple language framework application programming interfaces (APIs) exposed by various software and hardware services. Depending on the componentization degree of the system, the APIs supported by the HarmonyOS system device can also be different.

[0138] The application layer includes system applications and third-party applications (or referred to as extension applications). The system applications can include applications such as desktop, control bar, settings, telephone, and the like that are installed by default on the electronic device. The extension applications can be applications such as electronic device manager, device migration, notes, weather, and the like that are developed and designed by the manufacturer of the electronic device and are not necessary. The third-party non-system applications can be applications such as games, navigation, social networking, shopping, and the like that are developed by other manufacturers but can run on the HarmonyOS system.

[0139] The capability of running tasks in the background and unified data access abstraction are provided. A process area (PA) mainly provides support for a functional application (FA), such as providing computing capability as a background service or providing data access capability as a data warehouse. An application developed based on the FA or the PA can implement a specific business function, support cross-device scheduling and distribution, and provide a consistent and efficient application experience for a user.

[0140] Hardware assistance and resource sharing can be achieved among a plurality of electronic devices running a microkernel system through a distributed software bus, distributed device virtualization, distributed data management, and distributed task scheduling.

[0141] It should be noted that the software structure of the electronic device shown in FIG. 2 is only illustratively described, and does not limit the specific structure of the electronic device.

[0142] The memory map display method provided in the embodiments of the present application will be illustratively described below in combination with the electronic device shown in FIG. 1.

[0143] In the embodiments of the present application, before the user uses the electronic device to quickly open the memory map, the memory map corresponding to the user's memory information can be constructed first. As shown in FIG. 3, a flowchart of a memory map construction method 300 provided in the embodiments of the present application is shown, and the execution subject of the method 300 can be an electronic device. Referring to FIG. 3, the memory map construction method 300 can include the following steps S301-S304.

[0144] In step S301, the electronic device acquires multi-modal data of a user.

[0145] It should be understood that the multi-modal data of the user can refer to various data used for constructing the memory map, including but not limited to one or more of text data, picture data, audio data, or video data. In the embodiments of the present application, the multi-modal data can be data generated by the user of the electronic device on a social media platform, a shopping platform, an online forum, or the like.

[0146] The multi-modal data can also be diversified data including short message information, address book information, visual materials in a gallery, and health and behavior detected by a smart wearable device.

[0147] The multi-modal data can also be obtained from log information formed through human-computer interaction information between the user and the electronic device (such as a mobile phone, a computer, or the like). The log information records the user's conversation content, shopping records, ticket booking records, browsing records, search history, and the like.

[0148] The multi-modal data can also be obtained from photos, videos, texts and other content published by the user on a social media platform, and interaction information (e.g., chat records, etc.) with other users.

[0149] The multi-modal data can also be obtained from personal files and data (e.g., documents, photos, music, videos, etc.) saved in storage devices such as cloud storage, hard disks, USB flash drives, etc. The embodiments of the present application do not limit the manner of obtaining multi-modal data and the specific content obtained.

[0150] It should be noted that the multi-modal data obtained by the electronic device in step S301 can also come from other devices in addition to the electronic device.

[0151] For example, assuming that the electronic device in this embodiment is a mobile phone terminal, the user can log in to the same account information on the mobile phone terminal and other devices (e.g., tablet computers, smart watches, etc.), realizing cross-device data sharing and integration. Based on this, the mobile phone terminal can not only process the multi-modal data collected by itself, but also access and integrate the multi-modal data obtained from other devices that have logged in to the same account, thereby obtaining more rich and comprehensive data.

[0152] In step S302, the electronic device performs identification processing on the multi-modal data to obtain memory information.

[0153] It should be understood that the memory information identified from the multi-modal data can also include one or more modalities of text data, picture data, audio data, or video data, etc.

[0154] The multi-modal data obtained by the electronic device can not include memory information. In order to avoid processing irrelevant data, reduce the consumption of computing resources, and improve data processing efficiency, after the electronic device obtains the multi-modal data of the user, the multi-modal data can be identified to obtain memory information.

[0155] For example, assuming that the multi-modal data obtained by the electronic device is "The scenery here is really good", since there is no specific event, person, or other related information in "The scenery here is really good", the above information does not belong to memory information.

[0156] It should be noted that in the embodiments of the present application, the memory information can not change in terms of events, relationships or attributes, etc. in a long period of time. The memory information can have the characteristics of long-time (or permanent) storage, facilitating subsequent data utilization (e.g., analysis and decision-making, business optimization, etc.). That is, if the relevant information changes in terms of events, relationships or attributes, etc. after a preset time, it can be determined that the relevant information does not belong to memory information, wherein the electronic device can determine the preset time from the obtained multi-modal data.

[0157] By way of example and not limitation, a one-time triggered memory can not belong to the memory information. For example, "help me set an alarm for 30 minutes later", the event is invalid after 30 minutes, the recall cannot continue subsequent data utilization, and it does not have the characteristics of long-term storage, so the related information can not belong to the memory information.

[0158] Short-term persistent memory or short-term persistent memory can also not belong to the memory information. For example, "help me remember that my parking space in the mall is A507", in the foregoing information, after the preset time period, my car is no longer parked in the parking space corresponding to A507 after I leave the mall, at which time the relationship between my car and the parking space corresponding to A507 changes, the information cannot be utilized, and there is no need to store the memory for a long time, so the related information can also not belong to the memory information. The present application does not limit this.

[0159] In some embodiments, supervised fine-tuning (SFT) or prompt can be used to determine the memory information from the multi-modal data; any algorithm in long short-term memory (LSTM), elastic weight consolidation (EWC), attention mechanism, etc. can be used to determine the memory information from the multi-modal data.

[0160] In some embodiments, the memory information can be stored in the electronic device, which can reduce the risk of memory information leakage caused by network transmission and effectively protect the user's privacy information (such as interests, habits, etc.). In this way, the user can quickly access personal memory data without waiting for network response or connecting to the server in the case of no network or unstable network. In addition, based on the memory information stored on the electronic device, the application or LLM can provide more personalized services for the user, such as recommending delicious food according to the user's personal eating habits.

[0161] In some embodiments, the memory information can also be stored in the cloud server. Storing the memory information of the user in the cloud server can effectively prevent the risk of data loss caused by hardware failure or software error in the electronic device, and the backup and recovery strategy of the cloud server is more perfect than that of the user's electronic device, which can ensure quick recovery in case of data loss or damage. In addition, storing the memory information of the user in the cloud server can also effectively reduce the load and storage pressure of the electronic device.

[0162] Step S303, the electronic device classifies the memory information to obtain the category label to which the memory information belongs.

[0163] It should be understood that after the recognition processing is performed on the multi-modal data to obtain the memory information, the memory information can be classified by using a pre-configured label library (which can also be referred to as a label system or a label system), to further determine the category to which the memory information belongs, and obtain the category label to which the memory information belongs.

[0164] In the embodiments of the present application, the sources of the labels in the label library are diverse. The label library can not only include labels formed based on big data analysis and processing, but also include labels obtained according to the memory information of the logged-in user on the electronic device. That is, the labels in the label library can be further enriched and refined on the basis of the basic labels in combination with the unique memory information of each user, so as to build a more comprehensive label system. In addition, the above-mentioned manner can also effectively expand the labels in the label information library, so that the memory graph of different users can contain the node identification feedback by the user, further meeting the personalized display needs of the user for the memory graph.

[0165] In some embodiments, as shown in FIG. 4, a flowchart for constructing a label library is provided in the embodiments of the present application. Referring to FIG. 4, the process of constructing the label library can include: obtaining memory information; determining a first label according to the memory information; determining whether the first label is a label satisfying a preset condition, and if the first label is a label satisfying the preset condition, storing the first label; otherwise, if the first label is a label not satisfying the preset condition, determining whether to store the first label in response to a user operation.

[0166] It should be understood that the memory information of the user can be obtained from the multi-modal data based on the foregoing steps S301 and S302, and the specific content of the memory information can be understood by referring to the foregoing content, which will not be described here.

[0167] In some embodiments, any one of a rule matching and keyword extraction algorithm, a machine learning algorithm (such as Naive Bayes, support vector machine, deep learning, etc.), a clustering analysis algorithm, a social attribute and behavior feature inference algorithm, etc. can be used to determine the first label according to the memory information. It is not difficult to understand that in actual application, the number of first labels determined by the electronic device according to the memory information can include at least one.

[0168] In some embodiments, the preset condition can be determined according to the frequency of the first label appearing in the memory information of the user.

[0169] It should be understood that the first label satisfying the preset condition can be stored in the label library, which is used to construct the label library corresponding to the memory information of the user, so as to subsequently classify and extract information from the memory information by using the labels in the label library, and thus obtain the memory graph of the user.

[0170] For example, if the frequency of the first label in the memory information of the user is greater than a preset frequency, the first label can be directly stored. The preset frequency can be pre-configured or customized. That is, if the first label is determined to be a high-frequency label according to the memory information, the first label is directly stored.

[0171] Correspondingly, if the frequency of the first label in the memory information of the user is less than or equal to the preset frequency, it can be determined whether to store the first label in response to a user operation. In other words, if the first label is determined to be a low-frequency label according to the memory information, it can be determined whether to store the first label through a user operation in a human-computer interaction manner.

[0172] The user operation can be an operation of storing the first label. For example, the operation of storing the first label can be a confirmation operation on the first label, or a modification operation and a confirmation operation on the first label.

[0173] When the operation of storing the first label is the confirmation operation on the first label, the electronic device can directly store the first label after detecting the confirmation operation.

[0174] When the operation of storing the first label is the modification operation and the confirmation operation on the first label, the user can modify the first label (specifically, the name of the first label) after the electronic device detects the modification operation, and then the electronic device stores the modified first label after detecting the confirmation operation.

[0175] In addition, the user operation can also be an operation of not storing the first label. For example, the operation of not storing the first label can be an ignore operation or a delete operation, etc. For example, the electronic device can not store the first label after detecting the ignore operation.

[0176] As shown in FIG. 5 is a structural schematic diagram of a tag library provided by an embodiment of the present application. Referring to FIG. 5, the tag library can include at least one first-level tag such as an event tag, a relationship tag, and an attribute tag. Each first-level tag can include at least one second-level tag. For example, the event tag can include at least one second-level tag such as marriage, travel, dining, performance and movie, work, car, shopping, medical treatment, sports and fitness, and learning and training. The relationship tag can include at least one second-level tag such as friend, teacher, student, classmate, colleague, partner, family, marital relationship, legal relationship, and lover relationship. The attribute tag can include at least one second-level tag such as person and pet. Each second-level tag can further include at least one third-level tag. For example, the work tag can include at least one third-level tag such as business trip, meeting, and overtime work. The family tag can include at least one third-level tag such as father, mother, and wife. The person tag can include at least one third-level tag such as name, age, place of origin, gender, and occupation. The pet tag can include at least one third-level tag such as name, species, and age.

[0177] It can be understood that in the tag library, the tags can be constructed according to a hierarchical relationship. Each tag can include one or more large category tags, each large category tag can further include one or more small category tags, and each small category tag can further include one or more sub-tags. In this way, the electronic device or system (for example, an LLM that can call the tag library) can accurately understand and process the tags in the tag library, thereby improving the data processing efficiency.

[0178] In addition, compared with storing tags in the form of a tensor or a text list, the tag storage method shown in FIG. 5 is also beneficial to the operation (such as a screening operation) of the tags in the tag library.

[0179] Based on the method shown in FIG. 4, the tag library can be constructed. Then, the constructed tag library can be used to classify and process the memory information of the user, so as to obtain the category tag corresponding to the memory information.

[0180] It should be understood that the category tag after the classification processing can be a tag in the tag library shown in FIG. 5. As an example but not limitation, assuming that the electronic device receives a short message information "You have successfully made an appointment for the medical service of Dr. Li in Department A of XYZ Hospital on XX day of XX month of XX year at 10:00 am, and the appointment number is 123456", the short message information can be identified and classified to obtain that the short message information is a memory information, and the category tag corresponding to the memory information is "medical treatment".

[0181] In some embodiments, the SFT manner or the prompt manner can also be used to determine the category label corresponding to the memory information. In addition, the method for determining the category label corresponding to the memory information can also include, but is not limited to, a Naive Bayes Classifier, a Support Vector Machine (SVM), a Decision Tree, a Random Forest, a Neural Network, a K-means Clustering algorithm, a Hierarchical Clustering algorithm, an Association Rule Mining algorithm, and a Topic Model algorithm, etc.

[0182] In step S304, the electronic device performs information extraction on the memory information according to the category label, obtains a plurality of node identifiers and relationships between the plurality of node identifiers, and constructs a memory graph according to the plurality of node identifiers and the relationships between the plurality of node identifiers.

[0183] It should be understood that the node identifier is used to indicate an entity or an event. Correspondingly, the relationship between the plurality of node identifiers can include a relationship between entities, a relationship between an entity and an event, and a relationship between events.

[0184] In some embodiments, in the process of performing information extraction on the memory information based on the category label, the information extracted by different labels can be determined according to a pre-defined schema modeling manner corresponding to different labels. That is, different labels extract different information, and different information extracted by different labels can depict different memory information.

[0185] In the embodiments of the present application, the pre-defined schema modeling manner can follow the modeling manner of a property graph, and can specifically include five types of nodes, i.e., a concept node, an entity node, an abstract event node, a specific event node, and a memory information node.

[0186] The specific type of the concept node can be a concept type. The label corresponding to the concept node can be a label in a label library, for example, travel. The name corresponding to the concept node can be consistent with the label, for example, travel. The concept node is mainly used for classification, and can not be instantiated in the process of constructing the memory graph.

[0187] The specific type of the entity node can be an entity type; the label corresponding to the entity node can correspond to an entity label in a label library, for example, a person; the name corresponding to the entity node can be a name corresponding to the entity, for example, Zhang San; the attribute corresponding to the entity node can be an attribute possessed by the entity, for example, a person can have attributes such as name, gender, age, and the like; and the entity node can be used to instantiate a specific entity object. In the process of constructing the memory graph, the entity node is usually a type of node that needs to be instantiated.

[0188] The abstract event node corresponds to an abstract event type; the label in the abstract event node can be obtained according to an event type under an event label in a label library, for example, the label is a travel event. The name corresponding to the abstract event node can be a name of a specific event node, for example, a Thailand trip. The abstract event node can be used to represent an event. The attribute corresponding to the abstract event node can be an attribute of a specific event, for example, an event trigger word or a time. In the process of constructing the memory graph, the abstract event node is an optional node, that is, there can be an abstract event node for specific memory information, or there can be no abstract event node. The abstract event node is also usually a type of node that needs to be instantiated.

[0189] The specific event node corresponds to a specific event type; the label in the specific event node can be obtained according to the label corresponding to the abstract event node, for example, the label of the abstract event node is a business trip to Shanghai, and the label corresponding to the specific event node can be a business trip to Shanghai on XX, X, X. The name corresponding to the specific event node can be consistent with the label name of the specific event node, for example, a business trip to Shanghai on XX, X, X. The specific event node can be used to represent a specific event. The attribute corresponding to the specific event node can also be an attribute of a specific event, for example, an event trigger word or a time. In the process of constructing the memory graph, the specific event node can be obtained after the abstract event node is instantiated.

[0190] The type of the memory information node can be a memory content type node; the label of the memory information node can be a memory type obtained according to an event type label, for example, a travel memory; the memory information node can be used to indicate specific memory content (i.e., user memory information); the attribute of the memory information node can correspond to an attribute corresponding to the memory content, for example, a memory time or a place; and the memory information node can be used to represent specific event memory content. In the process of constructing the memory graph, the memory information node is also usually a type of node that needs to be instantiated.

[0191] As shown in FIG. 6 is a related schematic diagram of a pre-defined schema modeling manner provided by the embodiment of the present application, referring to FIG. 6, the concept node can be represented by using the abstract event node, the specific event node and the entity node, wherein the concept node and the abstract event node, the concept node and the specific event node, the abstract event node and the specific event node, and the concept node and the entity node can have the belonging relationship; the concept node and the concept node can have the superior-inferior relationship, the cause and effect, the correlation and the like.

[0192] The memory information node and the specific event node can have the explanation, the details and the like relationship, and the memory information node and the entity node can have the involvement relationship.

[0193] The entity node can be represented by using the label of the entity type, and the attribute possessed by the entity corresponding to the entity node can be used as the relationship between the entity nodes (corresponding to the entity relationship in FIG. 6) to describe the entity class memory. Further, in the memory graph, the entity class memory can also be linked by using the specific event node, that is, the entity node and the specific event node can have the event argument, the involvement and the like relationship, wherein the event argument can also be referred to as the event element, which is the component part of describing the specific occurrence of the event, and can usually include the time, the place, the subject, the object and the like of the event occurrence.

[0194] Based on FIG. 6, the memory information can be information extracted according to the category label to which the memory information belongs, and the multiple node identifiers in the memory information and the relationship between the multiple node identifiers are obtained. Then, the memory graph of the user can be constructed according to the multiple node identifiers and the relationship between the multiple node identifiers.

[0195] As an example but not limitation, as shown in FIG. 7 is a schematic diagram of using the event class memory to link the entity class memory provided by the embodiment of the present application, referring to FIG. 7, it is assumed that the electronic device obtains two pieces of memory information of the user, wherein one piece of memory information is “going to Shanghai for business trip with colleague Xiao Wang on Friday, and signing with customer B”, and the other piece of memory information is “having hot pot with Xiao Li last Wednesday”.

[0196] Firstly, the foregoing two pieces of memory information can be classified and processed based on the label library shown in FIG. 5 to obtain the category label to which each piece of memory information belongs. Exemplarily, referring to FIG. 7, the first-level label corresponding to the foregoing two pieces of memory information can be obtained as “event label”, further, the second-level label corresponding to “going to Shanghai for business trip with colleague Xiao Wang on Friday, and signing with customer B” can be “work”, and the second-level label corresponding to “having hot pot with Xiao Li last Wednesday” can be “catering”; still further, the third-level label corresponding to “going to Shanghai for business trip with colleague Xiao Wang on Friday, and signing with customer B” can be “business trip”.

[0197] Secondly, based on FIG. 6, information extraction is performed on each memory information according to the schema modeling manner corresponding to the predefined different category labels, to obtain a plurality of node identifiers in each memory information and the relationships between the plurality of node identifiers. Based on the foregoing example, the plurality of entities involved in the foregoing two pieces of memory information and the relationships between the plurality of entities can be extracted, to obtain a plurality of entity node identifiers and the relationships between the plurality of entity node identifiers.

[0198] In the foregoing example, it is assumed that the schema modeling manner corresponding to the predefined different category labels specifically includes that the “event label”, “work”, “dining”, and “business trip” can correspond to a concept node, the “Shanghai business trip” abstract event node can be further included under the concept node of “business trip”, and the “2024-06-14 Shanghai business trip” specific event node can be further included under the abstract event node of “Shanghai business trip”, and further, the complete memory information “Friday, go to Shanghai for business trip with colleague Xiao Wang, and sign a contract with customer B” can be further included under the specific event node of “2024-06-14 Shanghai business trip”.

[0199] Similarly, based on another piece of memory information “last Wednesday, had hot pot with Xiao Li”, the “dining” concept node can include the “hot pot” abstract event node, further, the “2024-06-05 hot pot” specific event node can be included under the abstract event node of “hot pot”, and further, the complete memory information “last Wednesday, had hot pot with Xiao Li” can be included under the specific event node of “2024-06-05 hot pot”.

[0200] According to the information extraction performed on the memory information “Friday, go to Shanghai for business trip with colleague Xiao Wang, and sign a contract with customer B” according to the foregoing predefined schema modeling manner, three entity node identifiers of “I”, “B company”, and “Xiao Wang” can be obtained, and in combination with FIG. 6, it can be obtained that the memory information node and the foregoing three entity node identifiers have a “involved” relationship, the two entity nodes of “I” and “Xiao Wang” can have a “colleague” relationship, based on the “employment” attribute value possessed by the entity node of “I”, the “employment” relationship between “I” and “A company” can be established, and further, the two entity nodes of “A company” and “B company” have a “customer” relationship; based on the “colleague” relationship between the two entity nodes of “I” and “Xiao Wang”, it can be further determined that the “employment” relationship also exists between “Xiao Wang” and “A company”.

[0201] Similarly, for another piece of memory information "ate hot pot with Xiao Li last Wednesday", information extraction can be performed to obtain two entity node identifiers "I" and "Xiao Li", and the two entity node identifiers and the corresponding memory information node have a "involved" relationship, and the two entity node identifiers "I" and "Xiao Li" can have a "classmate" relationship.

[0202] Finally, based on the obtained multiple entity node identifiers and the relationships between the multiple entity node identifiers, a corresponding memory graph can be constructed. Based on the above two pieces of memory information, the memory graph shown in the dashed box in FIG. 7 can be constructed.

[0203] It should be noted that in the process of program implementation, the extracted multiple node identifiers (or nodes) and the relationships between the multiple node identifiers can be represented by triples, where the triple can be represented as: <head, relation, tail>, where head can be an entity or an event; tail can also be an entity or an event; and relation can represent the relationship between head and tail, that is, at least one of the relationships between entities, between entities and events, or between events.

[0204] In some embodiments, the memory graph can be stored in the user's electronic device. In other embodiments, the memory graph can also be stored in a cloud server. The embodiments of the present application do not limit this.

[0205] Optionally, the storage locations of the user's memory information and the memory graph can be the same. For example, the user's memory information and the memory graph are stored in the user's electronic device. The storage locations of the user's memory information and the memory graph can also be different. For example, the user's memory information can be stored in a cloud server, and the memory graph can be stored in the user's electronic device. In actual applications, the electronic device can obtain the memory information stored in the cloud server to construct or modify the memory graph.

[0206] It is not difficult to understand that based on the above steps S301 to S304, the corresponding memory graph can be constructed based on the user's memory information. The human brain stores memory information by connecting memories through dendrites and axons of neurons. Based on the above method, the association relationship between entities and / or events in the user's memory information is intuitively displayed through the multiple node identifiers and the relationships between the multiple node identifiers, so that the constructed memory graph is closer to the human brain's way of storing memory information, is easy for users to understand and accept, and improves the aggregation efficiency of memory information.

[0207] A memory graph display method provided in an embodiment of the present application is introduced below. As shown in FIG. 8, a flowchart of the memory graph display method provided in an embodiment of the present application is shown. The memory graph display method 800 can be applied to an electronic device. As shown in FIG. 8, the memory graph display method 800 can include step S801.

[0208] In step S801, the electronic device displays a first interface in response to a first operation. The first interface is used to display a memory graph of a user. The memory graph is used to indicate memory information of the user. The memory information is represented by a plurality of node identifiers and relationships between the plurality of node identifiers. The node identifier is used to indicate an entity or an event.

[0209] It should be understood that the first operation can be a trigger operation for displaying the first interface. The trigger operation can be a click operation, a long press operation, a long press and click operation, a double-click operation, a touch operation, or a sliding operation. The trigger operation can also be a selection operation input by a physical button on the electronic device. The trigger operation can also be a gesture operation, a face recognition operation, or a voice instruction operation detected by a camera, a microphone, or other sensors of the electronic device.

[0210] In some embodiments, the electronic device can display the first interface in response to a first operation on a first application. The first application can be any application (APP) installed in the electronic device. For example, the first application can be a personal assistant APP.

[0211] In some embodiments, the electronic device can also display the first interface in response to a first operation on a first service. The first service can be any system service provided by an operating system. For example, the first service can be a memory information management service. In actual applications, the memory information management service can be started by an APP such as a smart assistant.

[0212] The memory graph display method provided in an embodiment of the present application is described below by taking a mobile phone terminal as an example of the electronic device and in combination with the two display methods of the first interface described above.

[0213] In some embodiments, referring to FIG. 9, the desktop 900 of the mobile terminal can display a plurality of pre-installed APPs, including a clock APP, a calendar APP, a gallery APP, a memo APP, a file management APP, an email APP, a music APP, a calculator APP, a video playing APP, a sports health APP, a weather APP, a browser APP, a smart life APP, a setting APP, a sound recorder APP, an application store APP, a personal assistant APP 901, a camera APP, a contact APP, a phone APP, and an information APP, etc. Of course, other APPs, such as a mobile management APP, etc., can also be pre-installed in the mobile terminal shown in FIG. 9, which is not limited in the present application.

[0214] The first kind, the mobile terminal displays the first interface in response to the first operation on the first application.

[0215] For example, referring to FIG. 9, assuming that the first application is the personal assistant APP 901, i.e., the personal assistant APP 901 is the APP corresponding to the memory map of the user, the user can touch the application icon corresponding to the personal assistant APP 901 on the desktop 900 of the mobile terminal to select and start the personal assistant APP 901. After the personal assistant APP 901 is started, the display interface 1000 shown in FIG. 10 can be directly displayed. In the display interface 1000 shown in FIG. 10, the memory map of the user can be displayed in a list display manner.

[0216] In some embodiments, after the personal assistant APP 901 is started, the memory map display interface 1100 shown in FIG. 11 can also be directly displayed. In the memory map display interface 1100 shown in FIG. 11, the memory map of the user can be displayed in a map display manner.

[0217] The second kind, the mobile terminal displays the first interface in response to the first operation on the first service.

[0218] For example, assuming that the first service is a memory information management service, and the smart assistant starting the foregoing memory information management service is a voice assistant, wherein the voice wake-up instruction of the voice assistant is Xia A, the user can send the voice wake-up instruction "Xia A" to the mobile terminal. After the mobile terminal receives the voice wake-up instruction "Xia A", the voice assistant can be started. After the voice assistant is started, the voice wake-up interface 1200 shown in FIG. 12 can be displayed. Then, the user can send the voice instruction "open memory information management" to the started voice assistant. Referring to FIG. 13, when the voice assistant recognizes the voice instruction "open memory information management", the corresponding voice recognition interface 1300 can be displayed. After the voice assistant recognizes the voice instruction "open memory information management", the display interface 1000 shown in FIG. 10 or the memory map display interface 1100 shown in FIG. 11 can be displayed.

[0219] In the embodiment of the present application, the first interface can be a memory atlas display interface 1100 as shown in FIG. 11. The display switching control can be displayed in the first interface. The mobile terminal can switch the display mode of the memory information in response to the triggering operation of the display switching control.

[0220] For example, the mobile terminal can switch the display mode of the memory information from the atlas display mode to the list display mode in response to the triggering operation of the display switching control. Alternatively, the mobile terminal can switch the display mode of the memory information from the list display mode to the atlas display mode in response to the triggering operation of the display switching control.

[0221] It should be understood that the number of display switching controls can correspond to the number of display modes. For example, the display modes can include a list display mode and an atlas display mode. Referring to FIG. 11, the number of corresponding display switching controls in the memory atlas display interface 1100 can be two, specifically, a “list display” control 1101 and an “atlas display” control 1102. The “list display” control 1101 is used to display the memory information in the list display mode, and the “atlas display” control 1102 is used to display the memory information in the atlas display mode. The mobile terminal can display the memory information in the list display mode after detecting the user operation of the “list display” control 1101. The mobile terminal can display the memory information in the atlas display mode after detecting the user operation of the “atlas display” control 1102.

[0222] Of course, the number of display switching controls can also be one. For example, the display modes of the memory information can be preconfigured in a preset order. Assuming that the preset order is list display and atlas display, one display switching control can be set in the first interface. The mobile terminal can display the memory information in the list display mode after detecting the first user operation of the display switching control. The mobile terminal can display the memory information in the atlas display mode after detecting the second user operation of the display switching control. The embodiment of the present application does not make any limitation on the specific setting form of the display switching control.

[0223] It should be noted that if the display interface 1000 as shown in FIG. 10 is displayed based on the two possible examples described above, the “list display” control 1001 and the “atlas display” control 1002 can be set in the display interface 1000. The mobile terminal can display the first interface after detecting the user operation of the “atlas display” control 1002.

[0224] In the embodiments of the present application, the display mode of the memory information includes but is not limited to a list display mode and a graph display mode. Based on the display switching control provided in the first interface, the display mode of the memory information can be enriched, and the operation of the memory information under different actual application requirements can be further met. For example, the nearest or farthest memory information from the current event can be viewed in the list mode; for another example, the association between an entity (and / or an event) and other entities (and / or other events) can be more intuitively viewed in the graph mode.

[0225] It can be understood that in the first interface, the memory information of the user can be represented by a plurality of node identifiers (or nodes) and the relationships between the plurality of node identifiers. Each node identifier can be used to indicate an entity or an event. Taking the memory graph display interface 1100 shown in FIG. 11 as an example of the first interface, the memory graph display interface 1100 shown in FIG. 11 can include six node identifiers. Specifically, five node identifiers are used to indicate corresponding entities, and one node identifier is used to indicate an event. Specifically, the five node identifiers used to indicate entities are “I”, “Zhang San”, “Li Si”, “Wang Wu” and “Zhao Liu”, and the one node identifier used to indicate an event is “out travel”.

[0226] In the memory graph display interface 1100, the memory information of “I” can be clearly understood through the corresponding relationships between the six node identifiers. Specifically, “I” can refer to the user corresponding to the mobile terminal. The relationships between the six node identifiers can include: “I” and “Zhang San” have a “boss” relationship, “I” and “Li Si” have a “girlfriend” relationship, “I” and “Wang Wu” have a “classmate” relationship, “I” and “Zhao Liu” have a “friend” relationship, “I” and “out travel” have a “traveler” relationship, “Zhang San” and “Li Si” have a “boss” relationship, “Zhang San” and “Wang Wu” have a “father” relationship, and “Li Si” and “out travel” have a “traveler” relationship.

[0227] It is worth noting that in the memory graph display interface 1100, at least one entity and / or at least one event related to “I” and the relationships between the at least one entity and / or at least one event can be displayed in the center of “I” according to a preset rule.

[0228] For example, at least one entity and / or at least one event related to “I” and the relationships between the at least one entity and / or at least one event can be displayed in the center of “I” according to the weight of the relationship between the other entities or events and “I”.

[0229] Alternatively, a preset number of other entities and / or events related to "I" and the relationships between a preset number of entities or events can be displayed in the center of "I".

[0230] Alternatively, a corresponding number of entities and / or events related to "I" and the relationships between the entities and / or events can be displayed in the center of "I" according to the screen size of the mobile terminal.

[0231] In some embodiments, the mobile terminal can display first map content associated with a first entity in the memory map in response to a triggering operation on a second node identifier in the memory map, the first entity being an entity indicated by the second node identifier, and the second node identifier being one of a plurality of node identifiers in the memory map.

[0232] It should be understood that the first map content can be a part of the map (or a local map) associated with the first entity in the memory map.

[0233] Still taking the memory map in the memory map display interface 1100 shown in FIG. 11 as an example, assuming that the second node identifier is a node identifier corresponding to "Li Si", the mobile terminal can display a display interface 1400 as shown in FIG. 14 after detecting the user operation on "Li Si". In the display interface 1400, "Li Si" can be displayed in the center, and other entities associated with "Li Si", the relationships between "Li Si" and other entities, and the relationships between other entities can be displayed.

[0234] For example, in the display interface 2000, five entities related to "Li Si" can be displayed, which can specifically include "Lao Li", "Xiao Zhang", "Xiao Yang", "Xiao Wang", and "X"; "Li Si" and "Lao Li" have a "father" relationship; "Li Si" and "Xiao Zhang" have a "father" relationship; "Li Si" and "Xiao Yang" have a "friend" relationship; "Li Si" and "Xiao Wang" have a "colleague" relationship; "Li Si" and "X" have a "company" relationship; "Lao Li" and "Xiao Zhang" have a "husband and wife" relationship; "Xiao Wang" and "X" have a "company" relationship, and the like.

[0235] In the above example, the first entity indicated by the second node identifier is Li Si; the triggering operation is the user operation on "Li Si" to display the display interface 1400. The first map content, which corresponds to the memory map displayed in the dashed line in the display interface 1400, can include "Li Si", five entities associated with "Li Si", the relationships between "Li Si" and the five entities, and the relationships between the five entities.

[0236] In addition to displaying the memory graph of the first entity corresponding to the second node identifier in the above manner, in other possible embodiments, a search control can be provided in the first interface, so that the mobile terminal displays a second interface in response to a triggering operation on the search control, the second interface being configured to receive a third node identifier input by the user. Then, the mobile terminal can display the second graph content in the memory graph corresponding to the third node identifier in response to a second operation detected on the second interface.

[0237] It should be understood that the third node identifier can be used to indicate an entity or an event. Correspondingly, if the third node identifier indicates an entity, the second graph content can be a part of the graph in the memory graph associated with the aforementioned entity; if the third node identifier indicates an event, the second graph content can be a part of the graph in the memory graph associated with the aforementioned event.

[0238] By way of example but not limitation, in the memory graph display interface 1100 shown in FIG. 11, a "search" control 1103 can also be displayed. After the mobile terminal detects a user operation on the "search" control 1103, the search interface 1500 shown in FIG. 15 can be displayed. The search interface 1500 can display a search box 1501 and a "cancel" button 1502, etc. After the mobile terminal detects a user operation of inputting corresponding search information (i.e., a third node identifier) in the search box 1501, the memory graph interface corresponding to the search information can be displayed.

[0239] Suppose the user inputs "Xiaoming" as the corresponding search information in the search box 1501. When the mobile terminal detects that the user inputs "Xiaoming" in the search box 1501, the memory graph display interface 1600 shown in FIG. 16 can be displayed. The memory graph display interface 1600 can display at least one entity and / or at least one event related to "Xiaoming" and the relationship between the at least one entity and / or at least one event, with "Xiaoming" as the center.

[0240] By way of example, four entities and one event related to "Xiaoming" can be displayed in the memory graph display interface 1600. The four entities can include "Xiaozhang", "Xiaochen", "Xiaobao", and "Xiaosun", and the one event can include "out-of-town travel". Specifically, "Xiaoming" and "Xiaozhang" can have a "husband and wife" relationship, "Xiaoming" and "Xiaochen" can have a "brother" relationship, "Xiaoming" and "Xiaobao" can have a "child" relationship, "Xiaoming" and "Xiaosun" can have a "friend" relationship, "Xiaoming" and "out-of-town travel" can have a "traveler" relationship, and "Xiaozhang" and "Xiaosun" can have a "friend" relationship, etc.

[0241] It is understandable that in the above example, the third node identifier is used to indicate the entity "Xiaoming". The triggering operation is that the user operates the "search" button 1103 to trigger the operation of the search interface 1500; the second graph content is the memory graph displayed in the dashed line in the memory graph display interface 1600 shown in FIG. 16, and can specifically include "Xiaoming", four entities and an event associated with "Xiaoming", the relationship between "Xiaoming" and the four entities and the event, and the relationship between the four entities and the event.

[0242] In the embodiments of the present application, in addition to displaying the memory graph corresponding to the first entity, the memory information corresponding to the first event can also be displayed. Further, the mobile terminal can display the memory information corresponding to the first event in chronological order in response to the triggering operation on the first node identifier in the memory graph, the first event being the event indicated by the first node identifier.

[0243] Still taking the memory graph in the memory graph display interface 1100 shown in FIG. 11 as an example, in the memory graph display interface 1100, it is assumed that the first event indicated by the first node identifier is an outing, and the mobile terminal can also detect the user operation on "outing" and display the outing display interface 1700 shown in FIG. 17, which can display "memory theme" display information 1701 and a time axis 1702, etc. The "memory theme" display information 1701 can correspond to the first event indicated by the first node identifier, for example, can also be "outing".

[0244] The time axis 1702 can be used to sequentially display the memory information corresponding to "outing" in chronological order. The "year" button, the "month" button, the specific content of the "memory information", the "record time" display information, the memory theme information, etc. can be set on the time axis 1702.

[0245] In actual applications, the user can slide up and down (or turn the page left and right) in the outing display interface 1700 to view the memory information related to the "outing" memory theme.

[0246] In some embodiments, the mobile terminal can also remove the fourth node identifier and the relationship corresponding to the fourth node identifier from the memory graph in response to a third operation on the fourth node identifier in the memory graph.

[0247] It should be understood that the third operation can refer to the operation of removing the fourth node identifier from the memory graph. The fourth node identifier can be used to indicate an entity or an event.

[0248] Still taking the memory graph in the memory graph display interface 1100 shown in FIG. 11 as an example, in the memory graph display interface 1100, assuming that the entity indicated by the fourth node identifier is Li Si, after the mobile terminal detects that the user operates "Li Si", the mobile terminal can display an operation window 1800 as shown in FIG. 18. The operation window 1800 can include a "delete" button 1801, a "modify" button 1802 and an "add" button 1803. The "delete" button 1801 can be used to delete the node identifier corresponding to "Li Si" in the memory graph and the relationship between the node identifier and other node identifiers associated with "Li Si". The "modify" button 1802 can be used to modify the node name of the node identifier corresponding to "Li Si" in the memory graph. The "add" button 1803 can be used to add other node identifiers associated with "Li Si" based on the node identifier corresponding to "Li Si" and the relationship between the other node identifiers and the node identifier corresponding to "Li Si".

[0249] Assuming that the mobile terminal detects that the user operates the "delete" button 1801, the mobile terminal can display a display interface 1900 as shown in FIG. 19. The display interface 1900 deletes the node identifier corresponding to "Li Si" and the relationship between other entities or events and "Li Si". In this example, the third operation can include the operation of the user operating "Li Si" to display the operation window 1800 and the triggering operation of the user operating the "delete" button 1801.

[0250] Of course, the third operation can also include a series of operations before displaying the memory graph in the memory graph display interface 1100 shown in FIG. 11, such as the operation of the user on the first application or the operation of the user on the first application and the operation of displaying the switch button.

[0251] It should be understood that the user operation of displaying the operation window 1800 as shown in FIG. 18 can be different from the user operation of displaying the display interface 1400 as shown in FIG. 14.

[0252] In another example, the memory graph display interface 1100 shown in FIG. 11 can also display a "modify mode" button 1104. After the mobile terminal detects that the user operates the "modify mode" button 1104, the mobile terminal can enter a modification state and display a modification interface 2000 as shown in FIG. 20. After the mobile terminal detects that the user operates "Li Si", the mobile terminal can display an operation window 2100 as shown in FIG. 21. The operation window 2100 can also include a "delete" button 2101, a "modify" button 2102 and an "add" button 2103.

[0253] The functions of the buttons in the operation window 2100 can be the same as those of the buttons in the operation window 1800 shown in FIG. 18, and thus will not be described herein. Of course, more or fewer buttons than those in the operation window 1800 shown in FIG. 18 can be displayed in the operation window 2100, and embodiments of the present application do not limit this.

[0254] For example, after detecting the user operation of the "modify" button 2102, the mobile terminal can enter a modification state, and then, in response to the user modifying "Li Si" to "Sun Qi", the mobile terminal can modify the node name corresponding to the node identifier from "Li Si" to "Sun Qi".

[0255] It can be understood that in actual applications, after modifying the node name, the mobile terminal can also update the relationship between the modified node identifier and other node identifiers based on the relationship between the modified node identifier and the entity corresponding to the other node identifiers or events.

[0256] Similarly, in some embodiments, the mobile terminal can also update the name of the first relationship in response to a sixth operation on the first relationship in the relationships between the plurality of node identifiers.

[0257] It should be understood that the first relationship can be a relationship between any entity (or event) and another entity (or event) in the memory map. The sixth operation can refer to a modification operation on any relationship in the relationships between the plurality of node identifiers.

[0258] For example, in the memory map in the memory map display interface 1100 shown in FIG. 11, after detecting the "girlfriend" relationship between "I" and "Li Si", the mobile terminal can display the operation window 2200 shown in FIG. 22. The operation window 2200 can also include a "delete" button 2201, a "modify" button 2202, and an "add" button 2203. The "delete" button 2201 can be used to delete the "girlfriend" relationship between "I" and "Li Si", that is, delete the "girlfriend" relationship between the node identifier corresponding to "I" and the node identifier corresponding to "Li Si" in the memory map. The "modify" button 2202 can be used to modify the "girlfriend" relationship between "I" and "Li Si", that is, modify the "girlfriend" relationship between the node identifier corresponding to "I" and the node identifier corresponding to "Li Si" in the memory map to another relationship. The "add" button 2203 can be used to add a relationship between "I" and "Li Si", that is, there can be two or more relationships between "I" and "Li Si", for example, the "girlfriend" relationship and the "manager" relationship.

[0259] For example, referring to FIG. 22, after detecting that the user operates the "add" button 2203, the mobile terminal enters an editing state, and in response to the operation of adding the "manager" relationship, the mobile terminal displays the "girl friend" relationship and the "manager" relationship between "I" and "Li Si".

[0260] It should be understood that in actual design, in addition to the above-mentioned operation of the user on the first relationship between the plurality of node identifiers to update the name of the first relationship, the user can also operate the connection line corresponding to the first relationship to represent the corresponding relationship between the two node identifiers corresponding to the first relationship using arrows with different directions.

[0261] For example, after detecting that the user triggers the connection line, the mobile terminal can display the operation window 2200 shown in FIG. 22, which can also include the "delete" button 2201, the "modify" button 2202, and the "add" button 2203. After detecting that the user operates the "add" button 2203, the mobile terminal enters an editing state, and in response to the operation of adding the "manager" relationship, the mobile terminal displays the display interface 2300 shown in FIG. 23, which can display the "girl friend" relationship and the "manager" relationship between "I" and "Li Si", wherein "I" is the "manager" of "Li Si", and "Li Si" is the "girl friend" of "I".

[0262] In addition, the number and position of the buttons displayed in any two or three of the operation window 1800 shown in FIG. 18, the operation window 2100 shown in FIG. 21, and the operation window 2200 shown in FIG. 22 can be the same or different. The embodiments of the present application do not limit this.

[0263] In other possible embodiments, after obtaining the memory information to be edited, the mobile terminal can also automatically perform an editing operation on the node identifier or the relationship corresponding to the node identifier in the memory graph.

[0264] It should be understood that the editing operation can include any one of the adding operation, the updating operation, the deleting operation, and the searching operation.

[0265] Specifically, after obtaining the memory information to be modified, the mobile terminal can determine a first node identifier (or node) corresponding to the memory information to be edited from the memory graph, and then determine a second node identifier to be edited and a relationship between the second node identifier based on the first node identifier and the memory information to be edited. Then, the mobile terminal generates an edited memory graph according to the second node identifier and the relationship between the second node identifier.

[0266] In some embodiments, the method of determining the first node identifier (or node) corresponding to the memory information to be edited from the memory graph can include at least one of an LLM, a retrieval augmented generation (RAG) model.

[0267] As an example but not as a limitation, as shown in FIG. 24, a schematic diagram of a memory graph of a user Joey provided by an embodiment of the present application is shown. As shown in FIG. 24, in the memory graph of Joey, there can be included a "relationship" label, a "preference" label, an "event" label, and an "attribute" label. Under the "relationship" label, there can be included a "colleague" label. Under the "event" label, there can be included an "arrangement" label. Under the "colleague" label, there can be specifically included a "boss" node identifier. Under the "arrangement" label, there can be specifically included an "Amsterdam" node identifier. The "boss" and the "Amsterdam" have a "travel destination" relationship. In addition to the "Amsterdam" node identifier, the "EK349" node identifier and the "CP" node identifier can also be related to the "Amsterdam" node identifier. The "Amsterdam" and the "EK349" have a "scheduled flight" relationship. The "Amsterdam" and the "CP" have a "scheduled hotel" relationship.

[0268] Further, the "01:30 2024-05-13" node identifier and the "EK349" node identifier can have a "departure time" relationship. In addition to the "CP" node identifier, the "voucher $20" node identifier, the "CS" node identifier, the "2024-05-18" node identifier, and the "2024-05-13" node identifier can also be related to the "CP" node identifier. The "CP" and the "voucher $20" have a "expiration time 2024-05-14" relationship. The "CP" and the "CS" have a "proximity location" relationship. The "CP" and the "2024-05-18" have a "checkout time" relationship. The "CP" and the "2024-05-13" have a "check-in time" relationship, and the like. Of course, other node identifiers and corresponding relationships can also be included in the memory graph shown in FIG. 24. For example, the "Shanghai" node identifier and the "D1234" node identifier can also be included under the "event" label. The "Shanghai" and the "D1234" can have a "scheduled train" relationship, and the like. The present application does not limit this.

[0269] In one example, assuming that the memory information to be edited received by the mobile terminal is "EK349 flight reservation seat is 32D", based on FIG. 24, after receiving the above memory information to be edited, the mobile terminal can determine the first node identifier as "EK349" based on the memory information to be edited, determine the second node identifier to be edited as "32D" based on the above "EK349 flight reservation seat is 32D", and there can be a "reservation seat" relationship between "EK349" and "32D". Based on this, the node identifier "32D" can be added in the memory map shown in FIG. 24, and the "reservation seat" relationship between "EK349" and "32D" can be added, to obtain the edited memory map as shown in FIG. 25.

[0270] In another example, assuming that the memory information to be edited received by the mobile terminal is "20 dollar hotel voucher will expire on May 14, 2024", and the system time obtained by the mobile terminal is "May 15, 2024", based on FIG. 24, after receiving the above memory information to be edited, the mobile terminal can determine the first node identifier as "Voucher $20" based on the memory information to be edited, determine the second node identifier to be edited as "Voucher $20" based on the above "20 dollar hotel voucher will expire on May 14, 2024", and there is a "expiration time 2024-05-14" relationship between "Voucher $20" and "CP". Based on this, the node identifier "Voucher $20" and the "expiration time 2024-05-14" relationship between it and "CP" can be removed in the memory map shown in FIG. 24, to obtain the edited memory map as shown in FIG. 26.

[0271] In yet another example, assuming that the memory information to be edited received by the mobile terminal is "EK349 flight has been rescheduled to Amsterdam on May 14, 2024 01:40", based on FIG. 24, after receiving the above memory information to be edited, the mobile terminal can determine the first node identifier as "EK349" based on the memory information to be edited, determine the second node identifier to be edited as "01:302024-05-13" based on the above "EK349 flight has been rescheduled to Amsterdam on May 14, 2024 01:40", and there is a "departure time" relationship between "EK349" and "01:302024-05-13". Based on this, the name corresponding to the node identifier "01:302024-05-13" can be modified to "01:402024-05-14" in the memory map shown in FIG. 24, to obtain the edited memory map as shown in FIG. 27.

[0272] In other examples, taking the voice recognition interface 1200 shown in FIG. 12 as an example, the user can input the voice dialogue instruction "What time does my boss's flight take off from Amsterdam?", and a voice dialogue interface 2800 as shown in FIG. 28 can be displayed. After the voice assistant recognizes the aforementioned voice dialogue instruction, it can be determined from the memory graph shown in FIG. 24 that the first node identifier is "Amsterdam", and according to the "predetermined flight" relationship between "Amsterdam" and "EK349" in the voice dialogue instruction, it can be further determined that the second node identifier is "EK349". In combination with the "departure time" relationship between "EK349" and "01:30 2024-05-13" in the memory graph, it can be obtained that the flight "EK349" takes off from "Amsterdam" at "01:30 2024-05-13", and based on this, a dialogue display interface 2900 as shown in FIG. 29 can be displayed. The mobile terminal can display the dialogue information "Your boss EK349 flight departure time is 2024-05-13 01:30" in the dialogue display interface 2900.

[0273] It is not difficult to understand that in the examples described in FIGS. 28 and 29, the memory graph shown in FIG. 24 can also be used as a memory library of the LLM, so that the user can initiate a dialogue question to the LLM, and the LLM can show the corresponding dialogue information to the user based on the node identifiers corresponding to "Amsterdam", "EK349" and "01:30 2024-05-13" and the relationships between the node identifiers after receiving the dialogue question.

[0274] In some embodiments, the mobile terminal can also display an application setting interface, and the first application is displayed in the application setting interface. The mobile terminal can set the source of the memory information to include the first application in response to a eighth operation detected on the application setting interface.

[0275] It should be understood that one or more first applications can be displayed in the application setting interface. The eighth operation can be used to specify one or more first applications as the source of the memory information.

[0276] In some embodiments, at least one application displayed in the application setting interface can have the same application category. For example, at least one APP whose category belongs to the utility tool is displayed in the application setting interface, specifically, such as the memo APP, the sound recorder APP, the browser APP, the smart assistant APP and the clock APP, etc.

[0277] In addition, at least one application displayed in the application setting interface can also have different application categories. For example, all APPs pre-installed in the mobile terminal are displayed in the application setting interface.

[0278] Based on the examples described in the foregoing FIG. 9 to FIG. 11, after the personal assistant APP 901 is started, an application setting interface 3000 as shown in FIG. 30 can be displayed, and at least one APP can be displayed in the application setting interface 3000. Based on a setting operation of the user on the at least one APP, the mobile terminal can obtain memory information from the APP corresponding to the setting operation.

[0279] It is assumed that referring to FIG. 30, the application setting interface 3000 can display an "all automatic authorization" option 3001, a setting option 3002 corresponding to an "information" APP, a setting option 3003 corresponding to a "phone" APP, a setting option 3004 corresponding to a "smart life" APP, a setting option 3005 corresponding to a "video playing" APP, a setting option 3006 corresponding to a "sports" APP, a setting option 3007 corresponding to a "browser" APP, a "cancel" button, and a "confirm" button, and the like.

[0280] It is not difficult to understand that each of the above setting options can be used by the user to select whether to set the data in the corresponding APP as the source of the memory information. If the mobile terminal detects that the user operates the "all automatic authorization" option 3001 and the "confirm" button, the memory information of the user can be obtained from the log data or the interaction data generated by all the APPs installed in the mobile terminal, so as to construct the memory map of the user.

[0281] In this example, after the mobile terminal obtains the memory information of the user from the log data or the interaction data generated by all the APPs, the display interface 1000 as shown in FIG. 10 or the memory map display interface 1100 as shown in FIG. 11 can be displayed.

[0282] If the mobile terminal detects that the user operates at least one APP corresponding to the setting option in the "information" APP corresponding setting option 3002, the "phone" APP corresponding setting option 3003, the "smart life" APP corresponding setting option 3004, the "video playing" APP corresponding setting option 3005, the "sports" APP corresponding setting option 3006, the "browser" APP corresponding setting option 3007, and the like, and the "confirm" button, the memory information of the user can be obtained from the log data or the interaction data generated by the at least one APP, so as to construct the corresponding memory map.

[0283] Similarly, after the mobile terminal obtains the memory information of the user from the log data or the interaction data generated by the APP corresponding to the at least one option, the display interface 1000 as shown in FIG. 10 or the memory map display interface 1100 as shown in FIG. 11 can also be displayed.

[0284] Based on the examples described in FIG. 12 and FIG. 13, after the voice assistant recognizes the voice instruction of "open memory information management", an application setting interface 3000 as shown in FIG. 30 can also be displayed. The specific content of the application setting interface 3000 in this example can be understood in the foregoing examples and will not be described here again.

[0285] Based on the above possible implementation, the fusion of memory data between multiple applications installed on the electronic device can be realized. After the data on different applications is integrated and applied to the LLM, the LLM can be more aware of the user, thereby providing more accurate and efficient personalized services for the user.

[0286] In a possible implementation, the user logged in on the mobile terminal is a first user. The mobile terminal can also display a synchronization device setting interface on the mobile terminal. The synchronization device setting interface can indicate that the login user of the second device is also the first user. The mobile terminal can synchronize the memory information between the mobile terminal and the second device in response to a seventh operation detected on the synchronization device setting interface.

[0287] It should be understood that the second device can be a device having the same login user as the mobile terminal. In actual applications, the login information of the second device can be the same as the login information on the mobile terminal, where the login information can include a mobile phone number, an email address, an identity code, etc. The seventh operation can be an operation of synchronizing the memory information between the second device and the mobile terminal.

[0288] In some embodiments, the second device can include one or more second devices, and the login user on the one or more second devices and the login user on the mobile terminal are the same user, i.e., the first user.

[0289] In one example, the mobile terminal can display a synchronization device setting interface 3100 as shown in FIG. 31. The synchronization device setting interface 3100 can include a "device 1" option 3101, a "WATCH" option 3102, a "confirm" button, and a "skip" button. The "device 1" option 3101 and the "WATCH" option 3102 can be used for the user to select whether to synchronize the memory information in the corresponding device to the mobile terminal. If the mobile terminal detects that the user operates at least one of the "device 1" option 3101 and the "WATCH" option 3102 and the "confirm" button, the memory information stored in the "device 1" and / or the "WATCH" device can be shared to the mobile terminal.

[0290] In actual design, the mobile terminal can display the synchronization device setting interface 3100 shown in FIG. 31 in response to a login operation of the user. Taking the example of the example described in FIG. 9, after the personal assistant APP 901 is started, the login interface 3200 shown in FIG. 32 can also be displayed, which can include an "account" option 3201, a "password" option 3202, a "login" button 3203, and a "registration" button 3204, etc. The user can input login information in the "account" option 3201 and the "password" option 3202, and operate the "login" button 3203. When the mobile terminal detects that the user operates the "login" button 3203, the synchronization device setting interface 3100 shown in FIG. 31 can be displayed, wherein the login information input by the user can include: the "account" option 3201: 123xxxx4567; the "password" option 3202: ABCXXXXYZ.

[0291] In some embodiments, after the mobile terminal displays the synchronization device setting interface 3100 shown in FIG. 31, when the mobile terminal detects that the user operates the "confirm" button or the "skip" button, the display interface 1000 shown in FIG. 10 or the memory atlas display interface 1100 shown in FIG. 11 can also be displayed.

[0292] It should be noted that if the mobile terminal is the first device for the user to log in the aforementioned login information to the electronic device, the synchronization device setting interface can be displayed on a third device, which is a device that is not the first device to log in the same login information to the electronic device, and the login information is also: the "account" option 3201: 123xxxx4567; the "password" option 3202: ABCXXXXYZ.

[0293] For example, assuming that the third device is a tablet computer, as shown in FIG. 33, a plurality of APPs pre-installed on the desktop 3300 of the tablet computer can be displayed, including a music APP, a weather APP, a theme APP, an application download APP, a payment APP, a contact APP, a my device APP, an album APP, a camera APP, a phonebook APP, a phone APP, a setting APP 301, and a personal assistant APP 3301, etc. Of course, other APPs such as a video player APP can also be pre-installed in the tablet computer shown in FIG. 27, which is not limited in the present application.

[0294] For example, referring to FIG. 33, the user can touch the application icon corresponding to the personal assistant APP 3301 on the desktop 3300 of the tablet computer to select and start the personal assistant APP 3301. After the personal assistant APP 3301 is started, the login interface 3400 shown in FIG. 34 can be displayed. The login interface 3400 shown in FIG. 34 can include the "account" option 3401, the "password" option 3402, the "login" button 3403, the "registration" button 3404, and the like. The user can input the login information in the "account" option 3401 and the "password" option 3402: the "account" option 3201: 123xxxx4567; the "password" option 3202: ABCXXXXYZ, and operate the "login" button 3403. After the mobile terminal detects the user operation of the "login" button 3403, the synchronization device setting interface 3500 shown in FIG. 35 can be displayed.

[0295] It is assumed that the "device 1" option 3501, the "WATCH" option 3502, the "confirm" button 3503, and the "skip" button 3504 can be displayed in the synchronization device setting interface 3500. The user can operate the "device 1" option 3501 and / or the "WATCH" option 3502 to share (or synchronize) the memory information stored in the device 1 and / or the WATCH to the tablet computer. It is not difficult to understand that in this example, the device 1 can correspond to the mobile terminal in the foregoing embodiments.

[0296] Conversely, when the tablet computer detects the user operation of the "skip" button 3504, the memory information stored in the device 1 and / or the WATCH can not be shared (or synchronized) to the tablet computer.

[0297] Of course, in the synchronization device setting interface 3500 shown in FIG. 35, when the tablet computer detects the user operation of the "confirm" button or the "skip" button, the display interface 1000 shown in FIG. 10 or the memory atlas display interface 1100 shown in FIG. 11 can also be displayed.

[0298] Based on the foregoing possible implementation, the memory information between multiple devices of the same user can be fused based on the second device selected by the user, the sharing of the memory information between multiple devices is realized, and even if the same application is used on different devices, the consistent use experience can be maintained.

[0299] In other possible implementations, the mobile terminal can also display a to-do list prompt interface. The to-do list prompt interface can display a first label. The mobile terminal can determine whether to store the first label in response to the user operation detected on the to-do list prompt interface.

[0300] It is understandable that the first label corresponds to the first label in the embodiment described in the foregoing FIG. 4.

[0301] In the embodiment of the present application, the first label is obtained according to the memory information of the logged-in user (i.e., the first user) on the mobile terminal. Since each mobile terminal usually corresponds to a different logged-in user, and the memory information (such as browsing history, preference settings, etc.) of each user is unique (i.e., the memory information of different users is different), the generated label information is also different. This differentiated processing not only enhances the pertinence and effectiveness of the service, but also ensures that each user can enjoy customized services that meet personal preferences, thereby meeting the diverse needs of individualization.

[0302] For example, assuming that the currently logged-in user on the mobile terminal is A, then a first label corresponding to A can be obtained by clustering the memory information of A on the mobile terminal based on the memory information of A on the mobile terminal, using, for example, clustering analysis algorithms. The first label corresponding to A can reflect the specific needs and interests of A.

[0303] In other possible implementations, the first label can also be obtained according to the multi-modal data of the user obtained by the mobile terminal. The embodiments of the present application do not limit this.

[0304] In the embodiments of the present application, the user operation can be a confirmation operation of the user on the first label; the user operation can also be a confirmation operation performed by the user after modifying the first label; or it can also be an ignore, delete, or other operation of the user on the first label.

[0305] In one example, the to-do list prompt interface can include a confirmation control and an ignore control. The mobile terminal can directly store the first label in response to a triggering operation on the confirmation control. The mobile terminal can also not store the first label in response to a triggering operation on the ignore control.

[0306] In some embodiments, the first label can be directly stored in the constructed label library, to be used for information extraction of the memory information based on the category label after the classification processing of the memory information and the determination of the category label corresponding to the memory information, so as to further enrich the node identifiers in the memory graph and the relationships between the node identifiers.

[0307] In other embodiments, the first label can also be directly stored in the memory graph of the user. It should be noted that if the first label is stored in the memory graph of the user, the first label can be directly displayed in the memory graph of the user.

[0308] Based on the above possible implementations, the user can also feed back the node identifiers in the label library or the memory graph, further increasing the interactivity and operability of the node identifiers for the user, and improving the user engagement.

[0309] In one example, the to-do list prompt interface can include a modification control and a confirmation control, and the corresponding user operation can include a triggering operation on the modification control and a triggering operation on the confirmation control; the mobile terminal can modify the first label in response to the triggering operation on the modification control; and then the mobile terminal can store the modified first label in response to the triggering operation on the confirmation control.

[0310] It should be understood that the modified first label can also be stored in the constructed label library, for constructing the label library corresponding to the memory information of the user, so that after determining the category label of the memory information, the memory information is information extracted according to the category label, to increase the node identifier and the relationship between the node identifiers in the memory graph. The modified first label can also be directly stored in the memory graph of the user. Similarly, if the modified first label is stored in the memory graph of the user, the modified first label can also be displayed in the memory graph of the user.

[0311] Based on the foregoing example, referring to FIG. 32, the mobile terminal can also display the interface 3600 shown in FIG. 36 after detecting the user operation of the “login” button 3203. The interface 3600 can include a “memory information display” option 3601 and a “to-do” option 3602, and the mobile terminal can display the to-do list prompt interface 3700 shown in FIG. 37 after detecting the user operation of the “to-do” option 3602. The to-do list prompt interface 3700 can display a “new label” prompt information 3701, a “confirm” button 3702, a “modify” button 3703, and a “ignore” button 3704. The label corresponding to the “new label” prompt information 3701 is the first label obtained according to the memory information of the logged-in user on the mobile terminal.

[0312] For example, assuming that the “new label” prompt information 3701 displayed in the to-do list prompt interface 3700 is “travel”, the mobile terminal can directly store “travel” after detecting the user operation of the “confirm” button 3702.

[0313] Alternatively, the mobile terminal can display the modification interface 3800 shown in FIG. 38 after detecting the user operation of the “modify” button 3703. The modification interface 3800 can include a “new label” modification information 3801, a “confirm” button 3802, and a “cancel” button. Assuming that the user modifies the “new label” modification information 3801 to “tourism”, the mobile terminal can directly store “tourism” after detecting the user operation of the “confirm” button 3802.

[0314] It should be understood that in the to-do list prompt interface 3700 shown in FIG. 37, the mobile terminal can modify the display information corresponding to the to-do option 3602 in the interface 3600 shown in FIG. 36 after detecting the user operation of the "ignore" button 3704. For example, the to-do option 3602 can be displayed with the corresponding to-do quantity reduced by one, and specifically, the display information corresponding to the to-do option 3602 can be displayed as "to-do 0".

[0315] In some embodiments, after detecting the user operation of the "memory information display" option 3601 in the interface 3600 shown in FIG. 36, the mobile terminal can display the display interface 1000 shown in FIG. 10 or the memory map display interface 1100 shown in FIG. 11.

[0316] By way of example but not limitation, it is assumed that after detecting the user operation of the "memory information display" option 3601, the mobile terminal can display the display interface 1000 shown in FIG. 10, and the display interface 1000 can display the "list display" button 1001 and the "map display" button 1002, and after detecting the user operation of the "map display" button 1002, the mobile terminal can display the memory map display interface 1100 shown in FIG. 11.

[0317] Correspondingly, the memory map display interface 1100 can also display the "list display" button 1101 and the "map display" button 1102, and after detecting the user operation of the "memory information display" option 3001, the mobile terminal can display the memory map display interface 1100 shown in FIG. 11, and then after detecting the user operation of the "list display" button 1101, the mobile terminal can display the display interface 1000 shown in FIG. 10. In this case, the "list display" button 1101 is a display switching control for switching the display mode of the memory map.

[0318] In some embodiments, a fifth node identifier can also be displayed in the memory map, and in response to a fourth operation on the fifth node identifier, the fifth node identifier and the relationship between the fifth node identifier and another node identifier are added in the memory map.

[0319] In one example, the fifth node identifier can be a first label obtained according to the memory information of a logged-in user on the mobile terminal. Based on this, the related content of the fifth node identifier can be understood with reference to the corresponding content of the aforementioned first label, which will not be described here again.

[0320] In another example, the fifth node identifier can also be a node identifier obtained by using the memory map construction method described in FIG. 3.

[0321] It should be understood that the fourth operation can be a triggering operation on the fifth node identifier in the memory graph, so that the mobile terminal can add the fifth node identifier and the relationship between the fifth node identifier in the memory graph in response to the triggering operation.

[0322] It should be noted that before the fourth operation, the display mode of the fifth node identifier in the memory graph can be different from other node identifiers, for example, the fifth node identifier can be different from other node identifiers in color, animation effect, shape, etc. After the fourth operation, the display mode of the fifth node identifier is the same as that of other node identifiers.

[0323] In this embodiment, the memory graph can be a graph content corresponding to a second subject associated with the fifth node identifier, and the second subject can include a logged-in user on the mobile terminal, such as "I" in the memory graph display interface 1100 shown in FIG. 11. In other possible embodiments, the memory graph can also be a graph content corresponding to a third subject or an event that satisfies a preset relationship with the fifth node identifier. For example, the preset relationship can include that the interaction frequency between the entity or event indicated by the fifth node identifier and the third subject or event is greater than a preset threshold, and the like.

[0324] By way of example and not limitation, assuming that the fifth node identifier is "basketball", taking the memory graph display interface 1100 shown in FIG. 11 as an example, referring to FIG. 39, the fifth node identifier corresponding to "basketball" can be displayed in a thicker color in the memory graph display interface 1100. The mobile terminal can display the display interface 3900 in response to the fourth operation (such as double-clicking the fifth node identifier corresponding to "basketball") on the fifth node identifier corresponding to "basketball".

[0325] Compared with the memory graph display interface 1100 shown in FIG. 11, the display interface 3900 shown in FIG. 39 adds the node identifier "basketball" and the "interest" relationship between "I" and "basketball".

[0326] In other embodiments, the mobile terminal can also modify the node name corresponding to the fifth node identifier in the memory graph in response to the fifth operation on the fifth node identifier.

[0327] It should be understood that the fifth operation can be a modification operation on the node name of the fifth node identifier.

[0328] Taking the display interface 3900 shown in FIG. 39 as an example, assuming that the user wants to modify the node name of the "basketball" fifth node identifier to "sports", the user can perform a modification operation (for example, long-pressing the node identifier corresponding to "basketball") on the "basketball" fifth node identifier in the display interface 3900, and then modify the node name from "basketball" to "sports". The mobile terminal can display the display interface 4000 in response to the user's long-pressing and modifying the fifth node identifier corresponding to "basketball".

[0329] It should be noted that in the example shown in FIG. 39, after modifying the node name of the fifth node identifier, the relationship between the entity (or event) indicated by the fifth node identifier and the entities (or events) indicated by other node identifiers can remain unchanged.

[0330] Compared with the example described in the foregoing FIGS. 18 to 23, the manner shown in the foregoing FIG. 40 can speed up the modification of the node name of the fifth node identifier.

[0331] In other embodiments, the mobile terminal can also delete the fifth node identifier in the memory graph in response to a deletion operation on the fifth node identifier.

[0332] For example, based on the foregoing example, referring to FIG. 41, the mobile terminal can display a display interface 4100 in response to the user's long-pressing the fifth node identifier corresponding to "basketball" and dragging the fifth node identifier corresponding to "basketball" to the right boundary of the screen display area of the mobile terminal, and the display interface 4100 deletes the fifth node identifier corresponding to "basketball".

[0333] It should be understood that in the foregoing example, the deletion operation can be the user's long-pressing the fifth node identifier corresponding to "basketball" and dragging the fifth node identifier to the right boundary of the screen display area of the mobile terminal.

[0334] In the foregoing embodiments, the fifth node identifier can be displayed in the memory graph display interface 1100 shown in FIG. 11. In other embodiments, the fifth node identifier can also be displayed in a to-do list prompt interface.

[0335] By way of example but not limitation, the mobile terminal can display a to-do list prompt interface, and the to-do list prompt interface includes the fifth node identifier. The mobile terminal can add the fifth node identifier and the relationship between the fifth node identifier in the memory graph in response to a user operation detected in the to-do list prompt interface.

[0336] It is understandable that the user operation in this embodiment can also include a confirmation operation on the fifth node identifier or a modification operation on the fifth node identifier and a confirmation operation. The user operation and the related content of the fifth node identifier can refer to the corresponding content in the embodiments described with reference to FIGS. 39 to 41, which will not be described here again.

[0337] In one possible example, the to-do list prompt interface can further include a confirmation button and a modification button.

[0338] For example, assuming that the graph content displayed in the memory graph display interface 1100 shown in FIG. 11 is a memory graph, and the content corresponding to the fifth node identifier is “basketball”, the to-do list prompt interface displayed by the mobile terminal can refer to the display interface 4200 shown in FIG. 42. The display interface 4200 can include the graph content displayed in the memory graph display interface 1100, the node name “basketball” corresponding to the fifth node identifier, a confirmation button 4201, and a modification button 4202, and the like. The confirmation button 4201 can be used to directly add the node identifier corresponding to “basketball” to the memory graph; and the modification button 4202 can be used to modify the node name corresponding to “basketball”.

[0339] That is, the mobile terminal can add the fifth node identifier corresponding to “basketball” and the relationship between the fifth node identifier and other node identifiers in the memory graph in response to the triggering operation of the user on the confirmation button 4201. The mobile terminal can also modify the node name “basketball” corresponding to the fifth node identifier in response to the triggering operation of the user on the modification button 4202 (for example, modify it to “ball game”). After the modification is completed, in response to the triggering operation of the user on the confirmation button 4201, the node name corresponding to the fifth node identifier is modified from “basketball” to “ball game”, and the fifth node identifier corresponding to “ball game” and the relationship between the fifth node identifier and other node identifiers are added to the memory graph.

[0340] Of course, the display interface 4200 can include an ignore button 4203, which can directly delete the fifth node identifier corresponding to “basketball” displayed in the display interface 4200. Correspondingly, the mobile terminal can directly delete the fifth node identifier corresponding to “basketball” in the display interface 4200 in response to the triggering operation of the user on the ignore button 4203.

[0341] In the above several possible embodiments, the mobile terminal can also automatically delete (or add) the fifth node identifier and the relationship between the fifth node identifier from the memory graph after the fifth node identifier is displayed for a preset time length.

[0342] Based on the above possible implementation manner, the node identifier and the relationship corresponding to the node identifier are added to the corresponding memory graph according to the user operation, so that the user can clearly view the changes of the memory graph, and the problem that the user needs to switch back and forth between different pages or complex operation processes to understand the related information without understanding the related information is avoided, the operation convenience and fluency are improved, and the user experience is further improved.

[0343] It should be noted that in the foregoing embodiments of the present application, the user operation, the trigger operation, the first operation, the second operation, the third operation, the fourth operation, the fifth operation, the sixth operation, the seventh operation and the eighth operation detected by the mobile phone terminal can include but are not limited to pressing, clicking, double-clicking, touching, long pressing, long pressing and clicking, etc. The present application does not make any limitation thereto.

[0344] It can be understood that in the embodiments of the present application, the association relationship between entities and / or events in the user's memory information is intuitively displayed through the plurality of node identifiers and the relationships between the plurality of node identifiers, which is closer to the storage and use mode of human memory information, easy for users to understand and accept, and can improve the user experience of using memory information. When the memory graph is applied to LLM, it is helpful for LLM to effectively associate to other node identifiers and the relationships corresponding to the other node identifiers through the plurality of node identifiers and the relationships between the plurality of node identifiers in the memory graph, to obtain richer context information and deepen the understanding of the user by LLM. In actual application, LLM can provide personalized services for the user by using the obtained context information, meet the personalized needs of the user, and improve the user experience.

[0345] In addition, compared with storing memory information in the form of a tensor or in the form of text, the above manner of directly displaying the memory graph on the first interface facilitates the user to quickly find the memory information related to the entity or event through the node identifier according to the actual needs, effectively avoids the cumbersome steps of searching whether the memory information is related to a certain entity or event in the memory information (or memory pool list) of the user stored in the form of a tensor or in the form of text, simplifies the searching steps for a certain entity or event, and facilitates the user to quickly view the memory information of the user.

[0346] It should be understood that the size of the serial number of each step in the above embodiments does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0347] FIG. 43 is a structural schematic diagram of a memory graph display apparatus according to an embodiment of the present application. The memory graph display apparatus 4300 can be deployed in the electronic device of the foregoing embodiments. As shown in FIG. 43, the memory graph display apparatus 4300 can include a first display module 4301.

[0348] The first display module 4301 is configured to display a first interface in response to a first operation, the first interface being configured to display a memory graph of a user, the memory graph being configured to indicate memory information of the user, the memory information being represented by a plurality of node identifiers and relationships between the plurality of node identifiers, the node identifiers being configured to indicate entities or events.

[0349] In some embodiments, the memory graph display apparatus 4300 further includes a second display module configured to display, in a time sequence, memory information corresponding to a first event in response to a triggering operation on a first node identifier in the memory graph, the first event being an event indicated by the first node identifier.

[0350] In some embodiments, the memory graph display apparatus 4300 further includes a third display module configured to display, in response to a triggering operation on a second node identifier in the memory graph, first graph content associated with a first entity in the memory graph, the first entity being an entity indicated by the second node identifier.

[0351] In some embodiments, the first interface further includes a search control, and the memory graph display apparatus 4300 further includes a fourth display module configured to display, in response to a triggering operation on the search control, a second interface configured to receive a third node identifier input by the user, and display, in response to a second operation detected on the second interface, second graph content corresponding to the third node identifier in the memory graph.

[0352] In some embodiments, the memory graph display apparatus 4300 further includes a removal module configured to remove, in response to a third operation on a fourth node identifier in the memory graph, the fourth node identifier and relationships corresponding to the fourth node identifier from the memory graph.

[0353] In some embodiments, the memory graph display apparatus 4300 further includes:

[0354] A fifth display module is configured to display, in the memory graph, a fifth node identifier, the fifth node identifier being obtained according to memory information of a login user of the electronic device.

[0355] A determination module is configured to add, in response to a fourth operation on the fifth node identifier, the fifth node identifier and relationships between the fifth node identifier in the memory graph.

[0356] In some embodiments, the memory graph display apparatus 4300 further includes:

[0357] The modification module is configured to modify a node name of the fifth node identified in the memory graph in response to a fifth operation on the fifth node.

[0358] In some embodiments, the memory graph display apparatus 4300 further includes:

[0359] The update module is configured to update a name of the first relationship in response to a sixth operation on the first relationship in the relationship.

[0360] In some embodiments, the first interface includes a display switching control, and the memory graph display apparatus 4300 further includes:

[0361] The switching module is configured to display the memory information in a list display mode in response to a triggering operation on the display switching control.

[0362] In some embodiments, a user logged in on the electronic device is a first user, and the memory graph display apparatus 4300 further includes:

[0363] The sixth display module is configured to display a synchronization device setting interface, and the synchronization device setting interface indicates that a login user of a second device is also the first user.

[0364] The synchronization module is configured to synchronize the memory information between the electronic device and the second device in response to a seventh operation detected on the synchronization device setting interface.

[0365] In some embodiments, the memory graph display apparatus 4300 further includes:

[0366] The seventh display module is configured to display an application setting interface, and the application setting interface displays the first application.

[0367] The setting module is configured to set a source of the memory information to include the first application in response to an eighth operation detected on the application setting interface.

[0368] It should be noted that the division of the modules in the memory graph display apparatus provided in the above embodiments is illustrative, and is merely a logical function division. In actual implementation, another division manner can be used. In addition, each function module in each embodiment of the present application can be integrated in one processor, or can be physically separated, or two or more modules can be integrated into one module. The above integrated module can be implemented in the form of hardware or in the form of a software function module.

[0369] ​The integrated module, if implemented in the form of a software function module and sold or used as an independent product, can be stored in a readable storage medium. Based on such an understanding, the technical solutions of the embodiments of the present application essentially or say the parts that make contributions to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The software product is stored in a storage medium and includes several instructions to make a device or a processor execute all or part of the steps of the methods of the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0370] In addition, the memory atlas display device and the memory atlas display method provided by the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments. For easy reading, the device embodiments will not repeat the details of the foregoing method embodiments one by one, but it should be clear that the device in the embodiments can correspondingly implement all the contents in the foregoing method embodiments.

[0371] Based on the same inventive concept, the embodiments of the present application also provide an electronic device, including one or more processors; one or more memories and one or more computer programs; wherein the one or more computer programs are stored in the one or more memories, and the one or more computer programs include instructions, when the instructions are executed by the electronic device, the electronic device executes the method shown in the above various embodiments.

[0372] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method shown in the above various embodiments.

[0373] The embodiments of the present application also provide a computer program product, which stores a computer program, and the computer program is executed by an electronic device to enable the electronic device to execute the method shown in the above various embodiments.

[0374] The embodiments of the present application also provide a chip system, which includes a processor and a memory, and the memory stores a computer program, and the computer program is executed by the processor to implement the method shown in the above various embodiments.

[0375] It should be appreciated that the processor referred to in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0376] It should also be understood that the memory referred to in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a RAM used as an external cache. By way of example but not limitation, many forms of RAM can be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM).

[0377] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0378] In the embodiments described above, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (for example: coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example: infrared, wireless, microwave, etc.) mode. The computer readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. integrated with one or more available media. The available media can be magnetic media (for example: floppy disk, hard disk, magnetic tape), optical media (for example: digital versatile disc (DVD)), or semiconductor media (for example: solid state disk (SSD)) and the like.

[0379] It can be understood that various numerical numbers involved in the embodiments of the present application are only distinguished for convenience of description, and do not limit the scope of the embodiments of the present application. The size of the serial number of each process does not mean the order of execution, and the execution order of each process should be determined by its function and inherent logic.

[0380] In the present application, the reference to "one embodiment" or "some embodiments" and the like means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0381] In the embodiments of the present application, the words "exemplary" or "for example" are used to mean "an example of" or "an example, only. Any implementation or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more suitable or advantageous than other implementations or design solutions. In fact, any implementation or design solution described as "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0382] Finally, it should be noted that the above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the technical solutions thereof. Even though the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features thereof can be replaced equivalently, and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A memory map display method characterized by comprising: The method is applied to an electronic device, and the method comprises: in response to a first operation, displaying a first interface, the first interface being used to display a memory graph of a user, the memory graph being used to indicate memory information of the user, the memory information being represented by a plurality of node identifiers and relationships between the plurality of node identifiers, the node identifiers being used to indicate entities or events.

2. The memory map display method according to claim 1, characterized by, The method further comprises: in response to a triggering operation on a first node identifier in the memory graph, displaying memory information corresponding to a first event in a time sequence, the first event being an event indicated by the first node identifier.

3. The memory map display method according to claim 1 or 2, characterized by, The method further comprises: in response to a triggering operation on a second node identifier in the memory graph, displaying first graph content associated with a first entity in the memory graph, the first entity being an entity indicated by the second node identifier.

4. The memory map display method according to any one of claims 1 to 3, characterized by, The first interface further comprises a search control, and the method further comprises: in response to a triggering operation on the search control, displaying a second interface, the second interface being used to receive a third node identifier input by the user; in response to a second operation detected on the second interface, displaying second graph content corresponding to the third node identifier in the memory graph.

5. The memory map display method according to any one of claims 1 to 4, characterized by, The method further comprises: in response to a third operation on a fourth node identifier in the memory graph, removing the fourth node identifier and relationships corresponding to the fourth node identifier from the memory graph.

6. The memory map display method according to any one of claims 1 to 5, characterized by, The method further comprises: displaying a fifth node identifier in the memory graph, the fifth node identifier being obtained according to memory information of a login user of the electronic device; in response to a fourth operation on the fifth node identifier, adding the fifth node identifier and relationships between the fifth node identifier in the memory graph.

7. The memory map display method according to claim 6, characterized by, The method further comprises: in response to a fifth operation on the fifth node identifier, modifying a node name of the fifth node identifier in the memory graph.

8. The memory map display method according to any one of claims 1 to 7, characterized by, The method further comprises: in response to a sixth operation on a first relationship in the relationships, updating a name of the first relationship.

9. The memory map display method according to any one of claims 1 to 8, characterized by, The first interface comprises a display switching control, and the method further comprises: in response to a triggering operation on the display switching control, displaying the memory information in a list display manner.

10. The memory map display method according to any one of claims 1 to 9, characterized by, A user logged in on the electronic device is a first user, and the method further comprises: displaying a synchronization device setting interface, the synchronization device setting interface indicating that a login user of a second device is also the first user; in response to a seventh operation detected on the synchronization device setting interface, synchronizing the memory information between the electronic device and the second device.

11. The memory map display method according to any one of claims 1 to 10, characterized by, The method further comprises: displaying an application setting interface, the application setting interface displaying a first application; in response to an eighth operation detected on the application setting interface, setting a source of the memory information to include the first application.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method in any one of claims 1 to 11 when executing the computer program.

13. A computer-readable storage medium, characterized in that, The computer program is stored on the computer-readable storage medium and, when executed on a computer, causes the computer to perform the method in any one of claims 1 to 11.

14. A chip system, characterized by comprises: a processor and a memory for storing a computer program, the processor being configured to invoke and run the computer program stored in the memory, so that a device or apparatus in which the chip system is installed performs the method according to any one of claims 1 to 11.

Images