Intelligent retrieval method and system based on memory database

Abstract

The invention belongs to the technical field of database information retrieval, and provides an intelligent retrieval method and system based on a memory database, and the method can extract specific data stored in different databases from obtained event data, and generates retrieval conditions by analyzing a retrieval instruction during user retrieval, so that the retrieval efficiency is improved. And searching different databases according to a sequence based on a search condition to obtain related memory data, so as to integrate information through a large language model when the memory data meets a requirement to form reply content displayed to a user. According to the scheme, data acquisition, intelligent storage, semantic understanding, cross-modal retrieval and personalized generation are connected in series, a strategy of matching feature extraction with hierarchical storage is innovatively adopted, data are deconstructed into various different forms and are stored respectively, a user is allowed to use a daily language to perform complex query in the retrieval process, and the user experience is improved. The method supports long-period and fuzzy memory retrieval, greatly optimizes the retrieval efficiency and precision, and improves the reliability and practicability of reply.

Description

Technical Field

[0001] This invention belongs to the field of database information retrieval technology, specifically relating to an intelligent retrieval method and system based on a memory database. Background Technology

[0002] With the rapid development of artificial intelligence and big data technologies, the digital memory data generated by individuals and families is growing explosively. This data usually exists in multimodal form, including video, audio, images and accompanying text information. Traditional data management systems and retrieval technologies face many bottlenecks when processing such complex and unstructured memory data.

[0003] At the data storage level, traditional databases struggle to efficiently accommodate and manage multimodal data, and cannot achieve an effective balance between storage efficiency and information integrity. Raw data such as video and audio occupy a huge amount of space, and directly storing raw streaming media is not conducive to fast retrieval. If only information such as text summaries is stored, there will be problems with incomplete information storage.

[0004] At the information retrieval level, existing retrieval systems suffer from unnatural retrieval methods, limited retrieval dimensions, weak cross-modal retrieval capabilities, and unfriendly result presentation. Users need to use specific query languages ​​or search based on precise keywords and timestamps, resulting in a high barrier to entry. Moreover, most systems only support retrieval based on text keywords or meta tags, failing to achieve multi-dimensional related searches based on visual content semantics, audio sentiment, behavioral patterns, and more. When users query information involving different modalities, there is a lack of effective cross-modal alignment and joint retrieval mechanisms, leading to fragmented and low-precision retrieval results. The final retrieval results are usually presented in the form of discrete data lists, requiring users to browse, piece together, and understand them themselves, rather than directly obtaining a coherent, complete, and easily understandable answer. Summary of the Invention

[0005] To overcome the shortcomings of the prior art, this invention proposes an intelligent retrieval method based on a memory database, the method comprising: Continuously acquire event data and extract the data to be recorded from the event data based on preset logic; Text data is extracted from the data to be recorded and stored in the first database; the data to be recorded is encoded to obtain vector data and stored in the second database; and the data to be recorded is stored in the third database. In response to a search command, the search command is parsed using a large language model, and several scalar information items are extracted from the search command to generate search conditions accordingly. Based on the search criteria, keyword matching search is performed sequentially in the first database, vector similarity search is performed in the second database, and structured query search is performed in the third database to obtain memory data related to the search criteria. When all the retrieved memory data meets the preset relevance threshold and / or completeness requirements, the large language model integrates and generates information based on each memory data to form a response and displays it to the user.

[0006] Specifically, the step of extracting the data to be recorded from the event data based on preset logic includes: Multimodal analysis is performed on the event data to identify the identities, emotions, and activities of the individuals involved. Based on the recognition results, it is determined whether the event data meets the preset conditions for recording. When the conditions are met, the event data is extracted as data to be recorded. The conditions for recording include: the event data contains a person whose identity has been marked or whose identity can be identified; the person in the event data has at least one preset emotion category; and / or, the event data includes at least one preset hot activity category.

[0007] Furthermore, the multimodal analysis of the event data to identify the identities, emotions, and activities of individuals within the event data includes: Voiceprint recognition and speech-to-text processing are performed on the audio data in the event data to extract the dialogue content into text content and determine the speaker's identity. And / or, perform scene recognition, event recognition, face recognition, human behavior tracking and key point detection on the video data in the event data to identify the identity of the person, the person's emotions and the content of the activity in the video data.

[0008] Preferably, the large language model is associated with an identity information database for storing personal identities, and the method further includes: If, during the multimodal analysis of the event data, an individual whose identity is not recorded in the identity information database is identified, the text data in the event data is extracted and the identity of the individual is inferred, and / or, a labeling request is sent to the user; If a labeling instruction is received from the user within a preset time, respond and label the person's identity according to the labeling instruction; otherwise, label the person's identity based on the reasoning result. The identity information database is updated based on the annotation results, and after the update is completed, the step of performing multimodal analysis on the event data is continued through the large language model.

[0009] Preferably, the method for identifying active content in the video data includes: The video data is input into the trained human behavior recognition model, and prompt words corresponding to preset hot activity categories are configured. The human behavior recognition model is obtained by supervised fine-tuning training through a training dataset labeled with video samples and corresponding time slice annotation information. The time slice annotation information defines the start and end timestamps and activity categories of specific hot activity segments in the video samples. The video data is analyzed using the human behavior recognition model to identify activity segments that match the description of the prompt words, and the start and end timestamps and corresponding activity category tags of each activity segment are output.

[0010] Specifically, the step of parsing the search command using a large language model, extracting several scalar information items from the search command, and generating search conditions accordingly includes: By combining the search instruction with its contextual information using the large language model, the complete semantics of the search instruction are analyzed, and at least one scalar information involved in the search instruction is determined; the scalar information includes time entities, person entities, location entities, and / or event entities; Based on the type of each scalar information and the complete semantics of the retrieval instruction, at least one of the scalar information is combined into the retrieval condition, which includes query fields and constraint values.

[0011] Furthermore, the method also includes: After displaying the response content to the user, the search instruction, each of the memory data, and the response content are recorded as a set of historical interaction data; The large language model is used to periodically analyze each set of historical interaction data to identify and summarize the user's continuously focused interest topics in the current period. The preset logic is dynamically adjusted based on each interest topic to improve the sensitivity and extraction priority of event data that match the interest topic. When the search command is parsed by a large language model, the context information of the search command is supplemented by each interest topic to improve search efficiency and accuracy.

[0012] Preferably, the step of integrating and generating information based on the memory data using the large language model to form a response and display it to the user includes: The memory data from the first database, the second database, and the third database are fused to obtain a set of associated event information. The associated event information set and the preset response style parameters are input into the large language model, so that the large language model generates the response content that conforms to the response style parameters and displays it to the user.

[0013] This invention also proposes an intelligent retrieval system based on a memory database, the system comprising: The analysis module is used to continuously acquire event data and extract data to be recorded from the event data based on preset logic; The storage module is used to extract text data from the data to be recorded and store it in a first database, encode the data to be recorded to obtain vector data and store it in a second database, and store the data to be recorded in a third database; The recognition module is used to respond to the retrieval command, parse the retrieval command through a large language model, extract several scalar information from the retrieval command, and generate retrieval conditions accordingly. The retrieval module is used to perform keyword matching retrieval in the first database, vector similarity retrieval in the second database, and structured query retrieval in the third database based on the retrieval conditions, so as to obtain memory data related to the retrieval conditions. The response module is used to integrate and generate information based on each memory data through the large language model when all the retrieved memory data meets the preset relevance threshold and / or completeness requirements, forming response content and displaying it to the user.

[0014] The present invention also proposes a computer-readable storage medium storing executable instructions that, when executed by a processor, implement the intelligent retrieval method based on a memory database as described above.

[0015] The present invention has at least the following beneficial effects: The proposed solution, through systematic architectural innovation and technological integration, achieves end-to-end optimization from multimodal data acquisition to intelligent question answering. By employing a feature extraction and hierarchical storage strategy, data is deconstructed into three forms: text, vectors, and raw data. The text database ensures rapid retrieval, the vector database stores semantic features with high information density to save space, and the raw database ensures data integrity and traceability. This achieves an optimal balance between storage cost and retrieval performance without sacrificing information value. Furthermore, the sequential collaborative retrieval strategy enables the system to simultaneously cover three retrieval modes: exact matching, semantic fuzzy search, and structured filtering, significantly improving the retrieval accuracy and recall for complex queries. The system ensures that no key memory fragments are missed; it uses a large language model to parse user commands, automatically transforming natural language questions into structured search conditions. This effectively handles fuzzy time descriptions and enables semantic association retrieval across months or even years, eliminating the burden on users to learn complex query syntax and expanding the available user base. It introduces relevance thresholds and completeness requirements as dynamic quality gating to ensure that a response is generated only when sufficiently relevant and logically complete information is retrieved. This effectively avoids the risk of the model fabricating answers based on insufficient information, significantly improving the credibility and reliability of the responses. It connects data collection, intelligent storage, semantic understanding, cross-modal retrieval, and personalized generation into an intelligent closed-loop solution. Furthermore, this solution pre-defines intelligent data extraction logic based on multimodal analysis, which ensures that the memory data stored in the database contains content with related people, emotional warmth, and event value, laying the foundation for subsequent retrieval and generation. It can also build and maintain a private social relationship graph belonging to the user. When an unknown person is identified, it can combine contextual reasoning and user annotation to learn the identity and update the identity information database. It uses a large model finely tuned with time slice data and configures prompt words to identify and segment video activities, which can accurately identify specific event activities. Building upon this foundation, this solution can also analyze complete semantics within the context, extract scalar information, and combine it into structured search conditions. It can accurately transform ambiguous user language into composite query conditions that can be applied to text, vectors, and raw databases simultaneously. By recording interaction history and analyzing user interests, the solution can proactively learn and focus on recording data that users care about, continuously optimizing the relevance and accuracy of search results. By fusing multi-source memory data and combining it with style parameters to generate the final response, information can be transformed into warm communication that meets the preferences of different scenarios and users, thereby improving user satisfaction.

[0016] Therefore, this invention proposes an intelligent retrieval method and system based on a memory database. The proposed solution links data acquisition, intelligent storage, semantic understanding, cross-modal retrieval, and personalized generation. It innovatively adopts a strategy of feature extraction combined with hierarchical storage to deconstruct data into multiple different forms and store them separately. During the retrieval process, users can use everyday language to perform complex queries. It supports long-term and fuzzy memory retrieval, significantly optimizing retrieval efficiency and accuracy, while improving the credibility and practicality of the responses. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A schematic diagram of the intelligent retrieval method based on a memory database provided in Example 1; Figure 2 This is a schematic diagram of the overall process of the intelligent retrieval method; Figure 3 A schematic diagram illustrating the method for extracting data to be recorded; Figure 4 A flowchart illustrating the process of identifying a person; Figure 5 A schematic diagram illustrating the method for generating search criteria; Figure 6 A schematic diagram illustrating the overall process in response to a user's search request; Figures 7(a) and 7(b) are example diagrams of responses to user retrieval requests; Figure 8 This is a module architecture diagram of the intelligent retrieval system based on a memory database provided in Example 2. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0020] Various embodiments of the invention will be described more fully below. The invention may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the invention to the specific embodiments disclosed herein, but rather the invention should be understood to cover all modifications, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of the invention.

[0021] In the following, the terms “comprising” or “may include” as used in various embodiments of the invention indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of the invention, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0022] In various embodiments of the invention, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0023] The expressions used in the various embodiments of the present invention (such as "first," "second," etc.) may modify various constituent elements in the various embodiments, but do not limit the corresponding constituent elements. For example, the above expressions do not limit the order and / or importance of the elements. The above expressions are only used for the purpose of distinguishing one element from other elements. For example, a first user device and a second user device refer to different user devices, although both are user devices. For example, a first element may be referred to as a second element without departing from the scope of the various embodiments of the present invention, and similarly, a second element may also be referred to as a first element.

[0024] It should be noted that, in this invention, unless otherwise explicitly specified and defined, terms such as "installation," "connection," and "fixation" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0025] In this invention, those skilled in the art should understand that the terms indicating orientation or positional relationship in the text are based on the orientation or positional relationship shown in the accompanying drawings, and are only for the purpose of facilitating the description of this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this invention.

[0026] The terminology used in the various embodiments of the invention is for the purpose of describing particular embodiments only and is not intended to limit the various embodiments of the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the various embodiments of the invention pertain. The terms (such as those defined in a generally used dictionary) are to be interpreted as having the same meaning as in the context of the relevant technical field and are not to be interpreted as having an idealized or overly formal meaning, unless clearly defined in the various embodiments of the invention.

[0027] Example 1 This embodiment proposes an intelligent retrieval method based on a memory database. This method can automatically analyze input data. Compared to traditional database query methods limited by structured query language and precise time indexing, this method supports retrieval results with monthly and yearly time units, and allows users to query memory in natural language. Retrieval results are obtained directly by analyzing data in the database. Please refer to [link to relevant documentation]. Figures 1-2 The methods specifically include: S100: Continuously acquires event data and extracts the data to be recorded from the event data based on preset logic.

[0028] In this embodiment, event data may include, but is not limited to, video data and audio data.

[0029] S200: Extract text data from the data to be recorded and store it in the first database; encode the data to be recorded to obtain vector data and store it in the second database; and store the data to be recorded in the third database.

[0030] It should be noted that text databases can mainly store summary text content. In subsequent retrieval steps, text databases are faster to retrieve data. Vector databases have the advantages of small memory footprint and large information storage capacity. Raw databases can store raw data such as audio and video data obtained in step S100.

[0031] S300: In response to a search command, it parses the search command using a large language model, extracts several scalar information from the search command, and generates search conditions accordingly.

[0032] In step S300, the large language model can parse the search command to determine the user's search intent, and then analyze the memory data that may be related to the user's query. For example, when a user asks the question "Did Zhang San eat lunch yesterday?" on April 11, the large language model can extract multiple scalar information such as "Noon on April 10", "Zhang San", and "eat" from the search command, and then generate search conditions for various databases based on these scalar information.

[0033] S400: Based on the search criteria, keyword matching search is performed in the first database, vector similarity search is performed in the second database, and structured query search is performed in the third database to obtain memory data related to the search criteria.

[0034] In this embodiment, the first database includes a text database, the second database includes a vector database, and the third database includes a raw database. The large language model is associated with the text database, the vector database, and the raw database. Based on this, the large language model can also be linked to a knowledge graph database that stores user profiles. The knowledge graph database stores data such as user relationships, recent behaviors, and interest hotspots.

[0035] Preferably, step S300 will parse the retrieval command through a large language model to determine the user's retrieval intent, and then in step 400, query the corresponding memory database, organize the memory and return it to the user. At the same time, the user's knowledge graph is constructed to update the knowledge graph database, thereby improving the accuracy and efficiency of data analysis in subsequent retrievals.

[0036] S500: When all retrieved memory data meets the preset relevance threshold and / or completeness requirements, the large language model integrates and generates information based on each memory data to form a response and displays it to the user.

[0037] For example, the relevance threshold can be jointly determined by entity matching degree, time relevance, vector similarity and knowledge graph relevance strength. Entity matching degree is determined based on whether the search results contain key people, places and event entities in the query instructions. For searches with time constraints, time relevance is determined by whether the search results fall within the effective time window or the adjacent interval. Vector similarity is determined by the cosine similarity score between the encoding of the memory data retrieved from the vector database and the query semantic vector. Knowledge graph relevance strength is determined by the path distance between the search results and the user's interest topic or query topic node in the knowledge graph. The integrity requirement can be linked to temporal continuity, causal chain, and multimodal corroboration. For a search querying activities within a period of time, temporal continuity is determined by whether the search results cover the main part or key nodes of that period. For a search querying causes, causal chain is determined by whether the search results contain potential antecedent events. Multimodal corroboration requires memory data from at least two different databases to support the same event content to enhance credibility.

[0038] Specifically, please see Figures 3-4 The step S100, which involves extracting the data to be recorded from the event data based on preset logic, includes: S110: Perform multimodal analysis on event data to identify the identities, emotions, and activities of individuals within the event data.

[0039] Specifically, step S110 involves performing multimodal analysis on the event data to identify the identities of individuals, their emotions, and the content of their activities within the event data, including: The system performs voiceprint recognition and speech-to-text processing on the audio data in the event data to extract the dialogue content into text content and determine the speaker's identity; and / or performs scene recognition, event recognition, face recognition, human behavior tracking, and key point detection on the video data in the event data to identify the person's identity, emotions, and activities in the video data.

[0040] In an optional implementation, step S110 can obtain the text content of audio data such as dialogue audio by performing voiceprint recognition, ASR translation, etc. on the audio data, and store the text content that conforms to the preset logic into the first database accordingly. Furthermore, the emotion of the face in the video data can be detected by the emotion recognition model. For example, the different emotion categories such as happy, sad, angry, and excited (happy, sad, angry, excited, etc.) can be predicted by the neural network inference in the emotion recognition model in combination with the confidence mechanism.

[0041] Preferably, the large language model is associated with an identity information database for storing the identities of individuals. If, during the execution of the multimodal analysis of the event data in step S110, an individual whose identity is not recorded in the identity information database is identified, the method proposed in this embodiment can extract the text data from the event data and infer the identity of the individual, and / or, initiate a labeling request to the user. If a user's annotation instruction is received within a preset time, the system responds and annotates the person's identity according to the annotation instruction; otherwise, it annotates the person's identity based on the inference result, updates the identity information database based on the annotation result, and continues to perform multimodal analysis of the event data through a large language model after the update is completed.

[0042] Preferably, the method for identifying active content in video data includes: The video data is input into the trained human behavior recognition model, and prompt words corresponding to preset hot activity categories are configured. The human behavior recognition model analyzes the video data to identify activity segments that match the prompt word descriptions, and outputs the start and end timestamps of each activity segment and the corresponding activity category label.

[0043] It should be noted that the human behavior recognition model can be obtained through supervised fine-tuning training using a training dataset labeled with video samples and corresponding time slice annotation information. The time slice annotation information defines the start and end timestamps and activity categories of specific hot activity segments in the video samples.

[0044] In a specific implementation scenario, the method proposed in this embodiment can continuously acquire surveillance videos in home settings as event data. During the model training phase, a large number of video samples containing common family activities such as dancing, singing, arguing, playing, and exercising can be collected. Human annotators can then accurately label specific activity segments appearing in each video with time slices. For example, a family gathering video can be labeled as "[00:00-00:30] Activity category: dancing" and "[00:30-01:15] Activity category: singing". Based on these labeled data, a selected open-source large language model (such as Qwen-VL) can be supervised and fine-tuned to understand the prompt instruction "find and segment [a certain activity] segment" and accurately output the corresponding timestamp and category. After training, a dedicated full-modal human behavior recognition large model can be obtained as the human behavior recognition model described in this method.

[0045] In the practical application stage of the human behavior recognition model, the method proposed in this embodiment inputs a surveillance video to be analyzed into the trained model, and simultaneously configures the prompt: "Please find all the dancing, singing or noisy activity segments in the video". After receiving the video and the prompt, the human behavior recognition model will perform frame-by-frame or segment-by-segment understanding of the video frames, and combine visual features with the semantics of the prompt to identify the activity segments that meet the requirements. Each output contains a precise start and end timestamp and the corresponding activity category label. For example, the model can output after analysis: "[02:15-02:45] Activity category of segment: dancing; [03:10-03:35] Activity category of segment: singing; [05:20-05:50] Activity category: noisy".

[0046] S120: Based on the recognition result, determine whether the event data meets the preset conditions for recording, and if so, extract the event data as data to be recorded.

[0047] In this embodiment, the conditions to be recorded may include, but are not limited to, the existence of individuals whose identities have been marked or are identifiable in the event data, the individuals in the event data having at least one preset emotion category, and / or, the event data including at least one preset hot topic activity category.

[0048] In an optional implementation, step S120 can use the presence of a known person in the video data, the presence of a person exhibiting a preset emotional tone, or the presence of a person engaging in enthusiastic activities as the basis for determining whether the preset recording conditions are met. For example, when a known person appears in the video data, a person exhibits a joyful, angry, sad, or happy emotional tone, or a person engages in enthusiastic activities such as singing or dancing, it can be determined that the video data meets the preset recording conditions.

[0049] Specifically, please see Figures 5-6 The step S300, which involves parsing the retrieval command using a large language model, extracting several scalar information items from the retrieval command, and generating corresponding retrieval conditions, includes: S310: By combining the retrieval instruction and its contextual information through a large language model, the complete semantics of the retrieval instruction is analyzed, and at least one scalar piece of information involved in the retrieval instruction is determined.

[0050] In this embodiment, scalar information may include, but is not limited to, time entities, person entities, location entities, and / or event entities.

[0051] S320: Based on the type of each scalar information and the complete semantics of the retrieval instruction, combine at least one scalar information into a retrieval condition that includes query fields and constraint values.

[0052] For example, when a user asks the question "What does Zhang San usually eat recently?" on April 11, the large language model can resolve "recently" into a general time range in which a retrieval operation can be performed, such as from February 1 to April 11, while identifying the core scalar information, namely Zhang San as a person and what he eats as an event. Then, in step S320, the text database can be quickly filtered and searched using keywords such as "Zhang San" and "eat" to obtain relevant text data. If the data in the text database is insufficient to provide an answer, the similarity search can be performed using vectors of "Zhang San" and "food" to obtain relevant vector data. If the data obtained at this point is still insufficient to provide an answer, a structured query can be performed in the original database in combination with time range constraints to retrieve highly relevant video or audio data. The overall retrieval process of this method does not require the user to accurately remember or input specific dates, but can automatically process fuzzy time descriptions and associate relevant events spanning several months from massive amounts of data, demonstrating powerful semantic understanding and long-term association capabilities.

[0053] Please refer to Figures 7(a)-7(b). Based on this, the large language model can also use the character relationships and event chains in the knowledge graph database for retrieval. This retrieval method can go beyond simple keyword matching and realize event-based causal analysis and attribution. By connecting discrete emotion recognition results, dialogue content, and behavior records, a "storyline" that conforms to human cognitive logic is formed, answering the "why" question and demonstrating full intelligent information integration capabilities.

[0054] For example, when a user asked the question "Why is Zhang San unhappy today?" on April 11th, the large language model can retrieve fragments in the recorded data that directly record "Zhang San is unhappy," and use the person relationships and event chains in the knowledge graph database to retrieve all interactions and events related to Zhang San in the past few days. It may also retrieve text data such as "Last night, Zhang San's child's exam results were not ideal" from the text database, image encoding of Zhang San's gloomy expression yesterday from the vector database, and audio data of an anxious conversation between Zhang San and others about family expenses today from the original database. After integrating and analyzing this information, the large language model can generate an insightful response: "Zhang San's mood today may be affected by two things: first, he learned last night that his child's exam results had declined; second, they discussed the recent increase in family expenses today. Both of these things may have brought some stress to Zhang San." Furthermore, the method proposed in this embodiment also includes: S600: After displaying the response to the user, the search command, various memory data, and the response content are recorded as a set of historical interaction data.

[0055] S700: Periodically analyzes historical interaction data from various groups using a large language model to identify and summarize the topics of interest that users continue to focus on during the current period.

[0056] It should be noted that step S700 can use a large language model for logical judgment, analyze the user's interest topics during the current period, and update the knowledge graph database accordingly. For example, the questions recently asked by the user and topics representing user relationships. The knowledge graph database can serve as an active reasoning engine in the retrieval process, which can improve the accuracy of extracting memory data from event data and the retrieval speed when searching various databases by utilizing the user's continuously followed interest topics during the current period. For example, interest topics can be injected as enhanced contextual information into the retrieval conditions of the database.

[0057] Therefore, the method proposed in this embodiment can transform discrete memory data into an associated network through a knowledge graph database, enabling the system to perform leapfrog associations and relational queries, upgrading retrieval from content-based matching to semantic and relational reasoning, thereby answering more complex and implicit user needs and achieving a qualitative leap in retrieval capabilities.

[0058] S800: Based on each interest topic, the preset logic is dynamically adjusted to improve the sensitivity and extraction priority of event data that match the interest topic. When the search command is parsed by the large language model, the context information of the search command is supplemented by each interest topic to improve the search efficiency and search accuracy.

[0059] Preferably, a large language model is used to integrate and generate information based on various memory data to form a response and display it to the user, including: S510: The memory data from the first database, the second database, and the third database are fused to obtain a set of related event information.

[0060] Memory fragments from different data sources may overlap or conflict in time. The fusion process in step S510 first establishes a unified timeline based on the timestamp information of each fragment and arranges the relevant events in chronological order. When contradictory information occurs, the big language model will make intelligent judgments and choices based on the confidence scoring mechanism or contextual relevance to ensure the coherence of the timeline logic. In addition, the same event may be recorded in the form of a summary in a text database, encoded in the form of visual features in a vector database, and have a complete video in the original database. Fusion processing can identify these duplicate records that point to the same core event, and generate a more informative and comprehensive related event information by deduplication and integration. For data with missing information, large language models can try to infer the content of the missing information based on relevant information in other databases. For example, when Zhang San's dialogue data is missing, the content of Zhang San's expression can be inferred from Zhang San's lip movements, emotional state at the time, and common expression habits in the video data, and the missing information can be supplemented based on the inference results.

[0061] S520: Input the associated event information set and the preset response style parameters into the large language model, so that the large language model generates response content that conforms to the response style parameters and displays it to the user.

[0062] For example, in step S520, based on the response style parameter, the large language model can use warmer words as prompts to generate response content with a warmer tone, thereby enhancing the affinity and emotional warmth of the interaction, ensuring the appropriateness and personalization of the response, so that the response content output by the model is not only correct information, but also high-quality communication that conforms to the context and meets the user's psychological expectations, especially suitable for elderly users and child care scenarios.

[0063] Example 2 This embodiment proposes an intelligent retrieval system based on a memory database to implement the intelligent retrieval method based on a memory database as proposed in Embodiment 1. Please refer to [link to previous document]. Figure 8 The system specifically includes: Analysis module 10 is used to continuously acquire event data and extract data to be recorded from the event data based on preset logic; Storage module 20 is used to extract text data from the data to be recorded and store it in a first database, encode the data to be recorded to obtain vector data and store it in a second database, and store the data to be recorded in a third database; The recognition module 30 is used to respond to the retrieval command, and parse the retrieval command through a large language model, extract several scalar information from the retrieval command and generate retrieval conditions accordingly; The retrieval module 40 is used to perform keyword matching retrieval in the first database, vector similarity retrieval in the second database, and structured query retrieval in the third database based on the retrieval conditions, so as to obtain memory data related to the retrieval conditions. The response module 50 is used to integrate and generate information based on each memory data through the large language model when all the retrieved memory data meet the preset relevance threshold and / or completeness requirements, form response content and display it to the user; The recording module 60 is used to record the retrieval command, various memory data and the response content as a set of historical interaction data after displaying the response content to the user; Hotspot module 70 is used to periodically analyze each group of historical interaction data through a large language model in order to identify and summarize the topics of interest that users continue to pay attention to in the current period. The optimization module 80 is used to dynamically adjust the preset logic based on each interest topic to improve the sensitivity and extraction priority of event data that match the interest topic. When parsing the search command through the large language model, it supplements the context information of the search command with each interest topic to improve search efficiency and search accuracy.

[0064] Specifically, the analysis module 10 can perform multimodal analysis on the event data to identify the identities, emotions, and activities of the people in the event data. Based on the identification results, it can determine whether the event data meets the preset conditions for recording. When the conditions for recording are met, the event data is extracted as data to be recorded.

[0065] The recognition module 30 can analyze the complete semantics of the retrieval instruction by combining the retrieval instruction and the context information of the retrieval instruction through a large language model, and determine at least one scalar information involved in the retrieval instruction. Based on the type of each scalar information and the complete semantics of the retrieval instruction, the at least one scalar information is combined into a retrieval condition that includes query fields and constraint values.

[0066] In this embodiment, the first database includes a text database, the second database includes a vector database, and the third database includes a raw database. The large language model is associated with the text database, the vector database, and the raw database. In addition, the large language model can also be associated with a knowledge graph database that stores user profiles. The knowledge graph database stores data such as user relationships, recent behaviors, and interest hotspots.

[0067] Example 3 This embodiment also proposes a computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the intelligent retrieval method based on a memory database proposed in Embodiment 1 above.

[0068] It should be noted that computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0069] In summary, this invention proposes an intelligent retrieval method and system based on a memory database. The proposed solution integrates data acquisition, intelligent storage, semantic understanding, cross-modal retrieval, and personalized generation. It innovatively adopts a strategy of feature extraction combined with hierarchical storage, deconstructing data into multiple different forms and storing them separately. During the retrieval process, users can use everyday language to perform complex queries, supporting long-term and fuzzy memory retrieval, significantly optimizing retrieval efficiency and accuracy, while improving the credibility and practicality of the responses.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An intelligent retrieval method based on a memory database, characterized in that, The method includes: Continuously acquire event data and extract the data to be recorded from the event data based on preset logic; Text data is extracted from the data to be recorded and stored in the first database; the data to be recorded is encoded to obtain vector data and stored in the second database; and the data to be recorded is stored in the third database. In response to a search command, the search command is parsed using a large language model, and several scalar information items are extracted from the search command to generate search conditions accordingly. Based on the search criteria, keyword matching search is performed sequentially in the first database, vector similarity search is performed in the second database, and structured query search is performed in the third database to obtain memory data related to the search criteria. When all the retrieved memory data meets the preset relevance threshold and / or completeness requirements, the large language model integrates and generates information based on each memory data to form a response and displays it to the user.

2. The intelligent retrieval method based on a memory database according to claim 1, characterized in that, The step of extracting the data to be recorded from the event data based on preset logic includes: Multimodal analysis is performed on the event data to identify the identities, emotions, and activities of the individuals involved. Based on the recognition results, it is determined whether the event data meets the preset conditions for recording. When the conditions are met, the event data is extracted as data to be recorded. The conditions for recording include: the event data contains a person whose identity has been marked or whose identity can be identified; the person in the event data has at least one preset emotion category; and / or, the event data includes at least one preset hot activity category.

3. The intelligent retrieval method based on a memory database according to claim 2, characterized in that, The multimodal analysis of the event data to identify the identities, emotions, and activities of individuals within the event data includes: Voiceprint recognition and speech-to-text processing are performed on the audio data in the event data to extract the dialogue content into text content and determine the speaker's identity. And / or, perform scene recognition, event recognition, face recognition, human behavior tracking and key point detection on the video data in the event data to identify the identity of the person, the person's emotions and the content of the activity in the video data.

4. The intelligent retrieval method based on a memory database according to claim 3, characterized in that, The large language model is associated with an identity information database for storing personal identities, and the method further includes: If, during the multimodal analysis of the event data, an individual whose identity is not recorded in the identity information database is identified, the text data in the event data is extracted and the identity of the individual is inferred, and / or, a labeling request is sent to the user; If a labeling instruction is received from the user within a preset time, respond and label the person's identity according to the labeling instruction; otherwise, label the person's identity based on the reasoning result. The identity information database is updated based on the annotation results, and after the update is completed, the step of performing multimodal analysis on the event data is continued through the large language model.

5. The intelligent retrieval method based on a memory database according to claim 3, characterized in that, The method for identifying active content in the video data includes: The video data is input into the trained human behavior recognition model, and prompt words corresponding to preset hot activity categories are configured. The human behavior recognition model is obtained by supervised fine-tuning training through a training dataset labeled with video samples and corresponding time slice annotation information. The time slice annotation information defines the start and end timestamps and activity categories of specific hot activity segments in the video samples. The video data is analyzed using the human behavior recognition model to identify activity segments that match the description of the prompt words, and the start and end timestamps and corresponding activity category tags of each activity segment are output.

6. The intelligent retrieval method based on a memory database according to claim 1, characterized in that, The process of parsing the search command using a large language model, extracting several scalar information items from the search command, and generating corresponding search conditions includes: By combining the search instruction with its contextual information using the large language model, the complete semantics of the search instruction are analyzed, and at least one scalar information involved in the search instruction is determined; the scalar information includes time entities, person entities, location entities, and / or event entities; Based on the type of each scalar information and the complete semantics of the retrieval instruction, at least one of the scalar information is combined into the retrieval condition, which includes query fields and constraint values.

7. The intelligent retrieval method based on a memory database according to claim 6, characterized in that, The method further includes: After displaying the response content to the user, the search instruction, each of the memory data, and the response content are recorded as a set of historical interaction data; The large language model is used to periodically analyze each set of historical interaction data to identify and summarize the user's continuously focused interest topics in the current period. The preset logic is dynamically adjusted based on each interest topic to improve the sensitivity and extraction priority of event data that match the interest topic. When the search command is parsed by a large language model, the context information of the search command is supplemented by each interest topic to improve search efficiency and accuracy.

8. The intelligent retrieval method based on a memory database according to claim 1, characterized in that, The process of integrating and generating information based on the memory data using the large language model to form a response and display it to the user includes: The memory data from the first database, the second database, and the third database are fused to obtain a set of associated event information. The associated event information set and the preset response style parameters are input into the large language model, so that the large language model generates the response content that conforms to the response style parameters and displays it to the user.

9. An intelligent retrieval system based on a memory database, characterized in that, The system includes: The analysis module is used to continuously acquire event data and extract data to be recorded from the event data based on preset logic; The storage module is used to extract text data from the data to be recorded and store it in a first database, encode the data to be recorded to obtain vector data and store it in a second database, and store the data to be recorded in a third database; The recognition module is used to respond to the retrieval command, parse the retrieval command through a large language model, extract several scalar information from the retrieval command, and generate retrieval conditions accordingly. The retrieval module is used to perform keyword matching retrieval in the first database, vector similarity retrieval in the second database, and structured query retrieval in the third database based on the retrieval conditions, so as to obtain memory data related to the retrieval conditions. The response module is used to integrate and generate information based on each memory data through the large language model when all the retrieved memory data meets the preset relevance threshold and / or completeness requirements, forming response content and displaying it to the user.

10. A computer-readable storage medium, characterized in that, It stores executable instructions for use by a processor to implement the intelligent retrieval method based on a memory database as described in any one of claims 1-8.

Images