Data retrieval method and device, equipment and medium

Abstract

The embodiment of the invention relates to a data retrieval method and device, equipment and a medium, and the method comprises the steps: obtaining a target problem, generating a retrieval cue word based on the target problem and a retrieval cue word template, inputting the retrieval cue word into a memory retrieval model, a plurality of target memory retrieval sub-tasks are generated for the target question through the memory retrieval model so as to retrieve the multiple memory databases to obtain a target memory retrieval result corresponding to the target question, and each target memory retrieval sub-task is used for retrieving one memory database. By the adoption of the technical scheme, compared with retrieval with a single retrieval method or a plurality of retrieval methods and a fixed process in the related technology, the probability of retrieving accurate data is greatly improved, information missing or wrong matching caused by limitation of the retrieval methods is avoided, and related data of problems can be retrieved more comprehensively and accurately.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and in particular, to a data retrieval method, apparatus, device, and medium. Background Art

[0002] In order to promote more efficient personalized interaction between large language models (LLMs) and users, a memory retrieval system is considered to be a key component. The memory retrieval system can help the large language model refer to previous historical interaction information when responding to user requests, thereby enhancing decision-making ability and personalized interaction ability. In related technologies, the retrieval method of the memory retrieval system is single, or a retrieval with a fixed process composed of multiple retrieval methods. This method may not be able to retrieve when the user's requests are complex and diverse, and even if retrieved, the accuracy is relatively low, which needs to be improved. Summary of the Invention

[0003] To solve the above technical problems, the present disclosure provides a data retrieval method, apparatus, device, and medium.

[0004] An embodiment of the present disclosure provides a data retrieval method, the method comprising:

[0005] Obtaining a target problem;

[0006] Generating a retrieval prompt word based on the target problem and a retrieval prompt word template;

[0007] Inputting the retrieval prompt word into a memory retrieval model, and through the memory retrieval model generating a plurality of target memory retrieval subtasks for the target problem to perform retrieval processing on multiple memory databases to obtain a target memory retrieval result corresponding to the target problem, wherein each target memory retrieval subtask is used to retrieve one memory database.

[0008] An embodiment of the present disclosure further provides a data retrieval apparatus, the apparatus comprising:

[0009] An obtaining module, configured to obtain a target problem;

[0010] A first generating module, configured to generate a retrieval prompt word based on the target problem and a retrieval prompt word template;

[0011] A first input module, configured to input the retrieval prompt word into a memory retrieval model, and through the memory retrieval model generating a plurality of target memory retrieval subtasks for the target problem to perform retrieval processing on multiple memory databases to obtain a target memory retrieval result corresponding to the target problem, wherein each target memory retrieval subtask is used to retrieve one memory database.

[0012] An embodiment of the present disclosure also provides an electronic device, which includes: a processor; a memory for storing executable instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the data retrieval method provided by the embodiment of the present disclosure.

[0013] An embodiment of the present disclosure also provides a computer-readable storage medium, which stores a computer program for executing the data retrieval method provided by the embodiment of the present disclosure.

[0014] The technical solution provided by the embodiment of the present disclosure has the following advantages compared with the prior art: The data retrieval solution provided by the embodiment of the present disclosure obtains a target problem, generates a retrieval prompt word based on the target problem and a retrieval prompt word template, inputs the retrieval prompt word into a memory retrieval model, and the memory retrieval model generates multiple target memory retrieval subtasks for the target problem to perform retrieval processing on multiple memory databases to obtain a target memory retrieval result corresponding to the target problem, where each target memory retrieval subtask is used to retrieve one memory database. By adopting the above technical solution, for the obtained target problem, the memory retrieval model can split the target problem into multiple target memory retrieval subtasks, and perform data retrieval on multiple memory databases through the multiple target memory retrieval subtasks to obtain the target memory retrieval result, realizing the adaptive dynamic arrangement according to the problem for different retrieval methods for different databases. Compared with the single retrieval method or the fixed process of multiple retrieval methods in the related art, the probability of retrieving accurate data is greatly improved, and information omission or incorrect matching caused by the limitation of the retrieval method is avoided, and the relevant data of the problem can be retrieved more comprehensively and accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In combination with the drawings and with reference to the following specific embodiments, the above and other features, advantages and aspects of the embodiments of the present disclosure will become more obvious. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original components and elements are not necessarily drawn to scale.

[0016] Figure 1 It is a flowchart of a data retrieval method provided by some embodiments of the present disclosure;

[0017] Figure 2 It is a flowchart of another data retrieval method provided by some embodiments of the present disclosure;

[0018] Figure 3 It is a flowchart of yet another data retrieval method provided by some embodiments of the present disclosure;

[0019] Figure 4Schematic flowchart of another data retrieval method provided by some embodiments of the present disclosure;

[0020] Figure 5 Schematic flowchart of yet another data retrieval method provided by some embodiments of the present disclosure;

[0021] Figure 6 Schematic structural diagram of a data retrieval device provided by some embodiments of the present disclosure;

[0022] Figure 7 Schematic structural diagram of an electronic device provided by some embodiments of the present disclosure. Detailed implementation manners

[0023] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not used to limit the protection scope of the present disclosure.

[0024] It should be understood that the various steps recited in the method embodiments of the present disclosure can be executed in a different order and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard.

[0025] As used herein, the term "including" and its variants are open-ended, that is, "including but not limited to". The term "based on" is "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0026] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependent relationships.

[0027] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive. Those skilled in the art should understand that, unless otherwise clearly specified in the context, it should be understood as "one or more".

[0028] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only for illustrative purposes and are not used to limit the scope of these messages or information.

[0029] Currently, there are endless personalized memory retrieval systems, methods, and frameworks for large language model applications.

[0030] Some embodiments of the memory retrieval system achieve efficient management and retrieval of long-term memory through an intelligent self-improving memory layer, using a hybrid database method and a custom search algorithm. In some other embodiments, a vector database is used to store memory fragments for efficient retrieval of relevant information. By decomposing the long-term memory ability into multiple independent working units, each unit responsible for a specific task, and organizing these working units into a workflow, core capabilities such as efficient information retrieval and memory consolidation or retrieval of a fixed process composed of multiple retrieval methods are achieved. In still some other embodiments, various types of memory modules are used to solve the problem of stateful long conversations according to different scenarios. In still some other embodiments, by introducing an explicit memory mechanism combined with model parameters, a sparse attention mechanism is adopted to significantly improve the inference efficiency, and an efficient storage and fast retrieval are achieved through a phased training strategy. In still some other embodiments, a global memory is created through a memory model and clues are generated, and an information retrieval and accurate answer with rich context are achieved by combining a retrieval generation model. However, the above retrieval methods are single or a fixed process of multiple retrieval methods, which may not be retrievable when the user's requests are complex and diverse, and even if retrieved, the accuracy is low and needs to be improved.

[0031] To solve the above problems, the embodiments of the present disclosure provide a data retrieval method, which will be introduced below in combination with specific embodiments.

[0032] Figure 1 The following is a schematic flowchart of a data retrieval method provided by the embodiments of the present disclosure. This method can be executed by a data retrieval device, where the device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 1 shown, the method includes:

[0033] Step 101, obtain a target problem.

[0034] The data retrieval method in the embodiments of the present disclosure is applied to a memory retrieval system. The memory retrieval system can store, process, and retrieve the historical records of the user's interaction with the model. When responding to the user's request, it can refer to the user's previous historical interaction information, thereby enhancing its decision-making ability and personalized interaction ability. The memory retrieval system is particularly crucial in fields such as companion conversation applications, adaptive learning systems, and enterprise office intelligence, and is widely used in various scenarios such as emotional companionship, learning and education, and enterprise efficiency improvement. The above model can include, for example, a large language model. The large language model is a natural language-based processing model. Through the large language model, the rules and structures of the language can be automatically learned, the meaning of the language can be understood, and coherent text with correct grammar and semantics can be generated according to the understood meaning.

[0035] In the embodiments of the present disclosure, the target question can be any question that the user needs to perform data retrieval on, such as "Please summarize the work content of my business trip to XX yesterday". Specifically, after receiving the user's request, the memory retrieval system parses the user's request to obtain the target question.

[0036] Step 102: Generate a retrieval prompt word based on the target question and the retrieval prompt word template.

[0037] Among them, the retrieval prompt word template is a pre-designed text pattern with a specific structure and format, which is used to guide and standardize the retrieval behavior during the data retrieval process to help the user obtain the required information more accurately and efficiently. Specifically, the retrieval prompt word template usually consists of a fixed part and replaceable variables or placeholders. The fixed part defines the basic logic, domain scope, task type, etc. of the retrieval; the variable part is filled according to specific retrieval requirements. The retrieval prompt word refers to the natural language text obtained by splicing the user's request into the retrieval prompt word template, which is used to guide the memory retrieval model to complete a specific retrieval task.

[0038] In the embodiments of the present disclosure, after obtaining the target question, the target question is combined with the pre-designed retrieval prompt word template. For example, by filling, the target question is filled into the corresponding position of the retrieval prompt word template to generate the retrieval prompt word.

[0039] Figure 2 It is a schematic flowchart of another data retrieval method provided by some embodiments of the present disclosure. As Figure 2 shown, the above step 102 may include step 201 and step 202, specifically including:

[0040] Step 201: Obtain the memory overview information of the memory raw data and the database configuration information corresponding to multiple memory databases.

[0041] Among them, the memory of original data refers to the historical interaction data related to the user. Specifically, the memory of original data can be the information obtained by the memory retrieval model through perceiving, learning, and understanding the user's request during the interaction between the user and the memory retrieval model. This information is stored by the memory retrieval model in a certain way so that it can be quickly retrieved and used when needed. The memory overview information refers to an abstract and general description of the memory of original data. Specifically, the memory overview information includes the description of the theme and storage method of the memory of original data. Among them, the theme is the core content or purpose of the data. The storage method refers to the organization and storage form of the data. For example, if the memory of original data is the attendance record, then the memory overview information of the memory of original data includes the theme related to attendance and the storage method in the form of a table.

[0042] In an optional implementation, obtaining the memory overview information of the memory of original data may include: using a theme extraction model to cluster and extract themes from the memory of original data, obtaining multiple themes, determining the storage method corresponding to each theme, and determining the multiple themes and the storage method corresponding to each theme as the memory overview information.

[0043] The theme extraction model refers to a model that obtains the corresponding theme by clustering and extracting themes from the memory of original data, and no limitation is imposed on the type of the theme extraction model. Clustering refers to the process of dividing the memory of original data into multiple classes composed of similar objects. Theme extraction refers to the process of identifying and extracting the core theme from the memory of original data.

[0044] In the embodiments of the present disclosure, the memory retrieval system analyzes the characteristics of the memory of original data using a theme extraction model, divides similar data into several clusters (i.e., clustering), each cluster represents a class of data with similar attributes, and then performs theme extraction separately in each cluster to obtain multiple themes. After obtaining each theme, a suitable storage method is determined according to the characteristics of the theme, and the multiple themes and their corresponding storage methods are determined as the memory overview information to clearly display the content and storage method of each theme for subsequent efficient management and use.

[0045] A memory database refers to a collection that stores memory raw data according to a certain data structure. Specifically, multiple memory databases include at least two of a text database, a graph database, a structured database, and a vector database. A text database is a database that stores and manages data in text form. A graph database refers to a database stored and managed in the form of a knowledge graph, which organizes and represents data through nodes, edges, and attributes. Among them, nodes represent data entities; edges represent the relationships between entities; attributes are used to describe the characteristics of nodes or edges. A structured database refers to a database that organizes, stores, and manages data according to a certain data structure and schema. A vector database is a database stored and managed in the form of vector data.

[0046] The database configuration information corresponding to the memory database refers to a set of parameters and settings related to the memory database, which are used to define and manage the operating environment, connection method, storage structure, and supported retrieval methods of the memory database to ensure that the memory database can operate efficiently and stably and support various retrieval requirements. Specifically, the database configuration information includes multiple retrieval methods corresponding to multiple memory databases, and one retrieval method is used to describe the retrieval method for one memory database.

[0047] Among them, the retrieval method refers to the method of searching for and obtaining information in the memory database. Specifically, the retrieval method includes the database type, database name, field structure, and retrieval function information. The database type refers to the classification of databases according to different data storage and organization methods. The database name refers to the name of the database, which is used to uniquely identify a database for access and management. The field structure refers to the detailed definition and description of each field in the database table. The retrieval function information refers to the use of specific functions to process, analyze, and extract data in the memory database during the information retrieval process to obtain information that meets the user's needs.

[0048] Specifically, multiple memory databases correspond to multiple retrieval methods. For the graph database, the adopted retrieval method is graph retrieval, which can effectively handle the relationships between nodes and edges, enabling users to quickly query relevant data based on the structural relationships. For the structured database, the adopted retrieval method is structured retrieval, which refers to using predefined tables and fields to quickly obtain the required data through precise query conditions. For the text database, the adopted retrieval method is keyword retrieval; keyword retrieval means quickly locating the data containing the keywords by matching the keywords input by the user to help users efficiently find relevant information. For the vector database, the adopted retrieval method is semantic vector retrieval, which captures the semantic information by converting the data into high-dimensional vectors. Thus, the memory retrieval system can adaptively orchestrate multiple different retrieval methods such as structured retrieval, graph retrieval, keyword retrieval, and semantic vector retrieval according to the user's request to obtain accurate memory retrieval results.

[0049] Step 202: Fill the target problem, memory overview information, and database configuration information into the retrieval prompt template to obtain the retrieval prompt.

[0050] In the embodiments of the present disclosure, after obtaining the target problem, memory overview information, and database configuration information, the memory retrieval system fills the target problem, memory overview information, and database configuration information into the corresponding positions of the retrieval prompt template respectively to obtain the retrieval prompt.

[0051] Step 103: Input the retrieval prompt into the memory retrieval model. After the memory retrieval model generates multiple target memory retrieval subtasks for the target problem to perform retrieval processing on multiple memory databases to obtain the target memory retrieval result corresponding to the target problem, where each target memory retrieval subtask is used to retrieve one memory database.

[0052] The memory retrieval model refers to a model that processes the retrieval prompt to obtain the target memory retrieval result corresponding to the target problem, so as to adaptively retrieve complex memory raw data and continuously improve the memory retrieval result. The memory retrieval model of the embodiments of the present disclosure can be trained based on the large language model in the large model. The target memory retrieval result refers to the relevant data retrieved based on the target problem during the data retrieval process. The target memory retrieval subtask refers to the task steps including the retrieval method for a certain memory database formulated by comprehensively considering the characteristics of the temporality, relevance, and content of the memory raw data for the target problem. By generating multiple target memory retrieval subtasks, the entire memory retrieval plan for the problem can be split into smaller and specific task steps to more accurately complete the data retrieval.

[0053] The target memory retrieval subtasks can be executed in parallel or serially, and special identifiers <dep_task name> can be used between the target memory retrieval subtasks to indicate the mutual dependency relationship. Specifically, the output format of the target memory retrieval subtask can be: <task number>: <task dependency><task description>.

[0054] Exemplarily, taking the target question "Please summarize the work content of my business trip in City A yesterday" as an example, the corresponding target memory retrieval subtasks are: Target memory retrieval subtask 1: Perform time and space filtering to lock the memory data in City A yesterday (20XX-XX-XX). Target memory retrieval subtask 2: <dep_target memory retrieval subtask 1> Extract the memory data related to the work content from the memory data in City A yesterday. Taking the target question "List the matters related to event a handled with A" as an example, the corresponding target memory retrieval subtasks are: Target memory retrieval subtask 1: Find the memory data related to "A". Target memory retrieval subtask 2: <dep_target memory retrieval subtask 1> Obtain the memory data related to event a from the memory data related to "A".

[0055] In the above two examples, the target memory retrieval subtasks are serial, that is, target memory retrieval subtask 2 depends on the result of target memory retrieval subtask 1. In actual situations, the retrieval subtasks can also be executed in parallel. For example, taking the target question "What content did I learn in Course B and Course C" as an example, the memory retrieval plan is: Target memory retrieval subtask 1: Retrieve the course content of Course B that has been learned. Target memory retrieval subtask 2: Retrieve the course content of Course C that has been learned.

[0056] Specifically, each target memory retrieval subtask is used to retrieve a memory database. Taking the target memory retrieval subtask "Find the memory data related to 'A'" as an example, this target memory retrieval subtask is used to retrieve the graph database.

[0057] In the embodiments of the present disclosure, after obtaining the retrieval prompt word, the retrieval prompt word is input into the memory retrieval model. After the memory retrieval model processes the target question, the target question is split into multiple target memory retrieval subtasks, and the target memory retrieval results are obtained through the multiple target memory retrieval subtasks to retrieve data from multiple memory databases.

[0058] Figure 3 It is a schematic flowchart of another data retrieval method provided by some embodiments of the present disclosure. As Figure 2 shown, the above step 103 may include step 301, step 302, step 303, and step 304, specifically including:

[0059] Step 301: Based on the memory overview information in the retrieval prompt, the memory retrieval model decomposes the target problem into multiple target memory retrieval subtasks.

[0060] In the embodiments of the present disclosure, the memory retrieval system refers to the memory overview information in the retrieval prompt through the memory retrieval model, decomposes the target problem from different data dimensions, and obtains multiple target memory retrieval subtasks. For example, one target memory retrieval subtask is generated for each data dimension, and each target memory retrieval subtask serves as the smallest retrieval task unit, that is, one target memory retrieval subtask retrieves from one data dimension.

[0061] Among them, the data dimensions include at least one of time, space, relationship, and content. Time refers to the time point or time period when the event occurs. Space refers to the geographical location where the event occurs. Relationship refers to the relationship between the event and the entity. Content refers to semantic similarity and involves the theme, nature, or type of the event.

[0062] Continuing with the example where the target problem is "List the matters related to event a handled with Xiao A", target memory retrieval subtask 1 is: Find the memory data related to "Xiao A", that is, the memory retrieval system decomposes the target problem from the time dimension by referring to the memory overview information in the retrieval prompt through the memory retrieval model. Target memory retrieval subtask 2 is: <dep_ target memory retrieval subtask 1> From the memory data related to "Xiao A", obtain the memory data related to event a, that is, the memory retrieval system decomposes the target problem from the content dimension by referring to the memory overview information in the retrieval prompt through the memory retrieval model.

[0063] Step 302: Generate multiple retrieval codes corresponding to the multiple target memory retrieval subtasks based on the database configuration information in the retrieval prompt.

[0064] The retrieval code refers to the code generated for the target memory retrieval subtask using a programming language. In order to generate accurate retrieval codes, it is necessary to configure the corresponding database configuration information according to the existing memory database types (such as graph databases, document databases, vector databases, etc.).

[0065] For ease of understanding, the following is an example of the retrieval method of the memory database included in a database configuration information, and this retrieval method is represented by a YAML (Yet Another Markup Language) file:

[0066] database_type: <memory database type, such as graph database / structured database / vector database, etc.>

[0067] database: <memory database name>

[0068] data_schema: <Field structure of the memory database>

[0069] func_name: <Retrieval function name>

[0070] description: <Description of the retrieval function, briefly describing its function>

[0071] example: <Usage example of the retrieval function>

[0072] input_query: <Query statement input to the retrieval function>

[0073] returns: <Return value of the retrieval function>

[0074] For example, for a graph database, its retrieval method is a query statement written in query language c; while for a structured database, the retrieval method is a query statement written in query language d.

[0075] In the embodiments of the present disclosure, after obtaining multiple target memory retrieval subtasks, according to the memory database type, the field structure of the memory database, the retrieval function name, etc. in the database configuration information in the retrieval prompt words, multiple target memory retrieval subtasks are generated into executable retrieval codes.

[0076] In the process of generating retrieval codes, due to the dependency relationship between target memory retrieval subtasks, therefore, the retrieval codes of the front and back dependent target memory retrieval subtasks need to be generated in sequence to keep the front and back codes coherent and consistent. In an optional implementation manner, there is a dependency relationship between multiple target memory retrieval subtasks, and multiple retrieval codes are executed in sequence according to the dependency relationship when executed.

[0077] Among them, the dependency relationship means that the execution of the retrieval code of a certain target memory retrieval subtask depends on the result of the previous target memory retrieval subtask. This dependency relationship requires that after the previous target memory retrieval subtask is successfully completed, the subsequent target memory retrieval subtask can proceed normally.

[0078] Exemplarily, continuing with the target question being the above "List the matters related to event a handled with Little A" as an example, the target memory retrieval subtask 2 corresponding to this target question depends on the target memory retrieval subtask 1. First, execute the retrieval code 1 corresponding to the target memory retrieval subtask 1, and then execute the retrieval code 2 corresponding to the target memory retrieval subtask 2 based on the retrieval result of the retrieval code 1.

[0079] In the embodiments of the present disclosure, there is a dependency relationship between multiple target memory retrieval subtasks, and the corresponding retrieval codes are executed according to the dependency relationship between multiple target memory retrieval subtasks.

[0080] In the embodiments of the present disclosure, the memory retrieval model can automatically generate and execute retrieval codes according to a retrieval plan (i.e., multiple target memory retrieval subtasks), and can configure a corresponding tool set according to the database type. The retrieval code generation process takes into account the dependency relationships of the target memory retrieval subtasks, and connects multiple databases to execute the target memory retrieval subtasks in parallel or serially, reducing manual intervention and conversion costs during the retrieval process, making the entire retrieval process more efficient and smooth, and being able to quickly respond to user requests and provide relevant memory raw data.

[0081] Step 303: Execute multiple retrieval codes to perform retrieval in the memory databases corresponding to the respective target memory retrieval subtasks according to the corresponding retrieval methods, and obtain multiple memory data segments.

[0082] Among them, a memory data segment refers to an information block corresponding to a target memory retrieval subtask extracted from the memory raw data.

[0083] In the embodiments of the present disclosure, after generating multiple retrieval codes corresponding to multiple target memory retrieval subtasks, retrieval is performed respectively in the memory databases corresponding to the respective target memory retrieval subtasks according to the corresponding retrieval methods to obtain multiple memory data segments. Since the user's requests are complex and diverse, it is usually impossible to directly hit with a single retrieval method. Therefore, it is necessary to split the target memory retrieval subtasks to achieve dynamic orchestration of the memory retrieval methods, rather than using a fixed process for retrieval, to avoid information omission or incorrect matching caused by the limitations of the retrieval method, and to be able to retrieve more comprehensively and accurately the relevant data of the problem.

[0084] Step 304: Determine the multiple memory data segments as the target memory retrieval results corresponding to the target problem.

[0085] In the embodiments of the present disclosure, after obtaining multiple memory data segments, the set composed of the multiple memory data segments is used as the target memory retrieval result corresponding to the target problem.

[0086] Taking the target question "Please summarize the work content of my business trip to City A yesterday" as an example, the user requests to summarize the work content of the business trip to City A yesterday. The key points of retrieval are time, space, and content. In this case, it is necessary to dynamically combine structured retrieval methods based on time, space, and content filtering. Specifically, the memory retrieval system can first lock the memory data records of the user in City A yesterday through the time and space ranges, and then extract the specific work content (i.e., the target memory retrieval result) by combining the semantic vector retrieval method. Taking the target question "List the matters related to Event A handled together with Little A" as an example, what the user requests is to list the matters related to a specific topic (matters related to Event A) discussed with a specific person (Little A), which involves relationships and content. Specifically, the memory retrieval system can use graph retrieval to locate the nodes related to "Little A", and then use keyword retrieval or semantic vector retrieval methods to specifically extract the matters related to Event A discussed.

[0087] In the embodiments of the present disclosure, since the memory retrieval model can comprehensively consider the spatio-temporal, relevance, and content characteristics of the memory original data, formulate a retrieval plan for the target memory retrieval subtask including multiple retrieval methods, such as reasonably applying structured retrieval, graph retrieval, keyword retrieval, and semantic vector retrieval in different case scenarios, etc., it greatly improves the probability of retrieving accurate memory original data for different user requests. For example, in the case of the target question "Please summarize the work content of my business trip to City A yesterday", the work content is accurately located and extracted through time filtering combined with semantic vector retrieval; in the case of the target question "List the matters related to Event A handled together with Little A", first use graph retrieval to locate the relevant person nodes and then combine the semantic vector retrieval method to obtain the matters related to the specific Event A. Compared with the traditional single retrieval method, it can more accurately hit the corresponding memory original data and avoid information omission or incorrect matching caused by the limitation of the retrieval method.

[0088] The data retrieval solution provided by the embodiments of the present disclosure obtains a target question, generates a retrieval prompt word based on the target question and a retrieval prompt word template, inputs the retrieval prompt word into a memory retrieval model, and the memory retrieval model generates multiple target memory retrieval subtasks for the target question to perform retrieval processing on multiple memory databases to obtain a target memory retrieval result corresponding to the target question. Each target memory retrieval subtask is used to retrieve one memory database. By adopting the above technical solution, for the obtained target question, the memory retrieval model can split the target question into multiple target memory retrieval subtasks, and perform data retrieval on multiple memory databases through the multiple target memory retrieval subtasks to obtain the target memory retrieval result, realizing the dynamic arrangement of different retrieval methods for different databases according to the problem adaptively. Compared with the single retrieval method or the fixed process of multiple retrieval methods in the related art, the probability of retrieving accurate data is greatly improved, and information omission or incorrect matching caused by the limitation of the retrieval method is avoided, and the relevant data of the question can be retrieved more comprehensively and accurately.

[0089] In order to support the retrieval of different databases, in some embodiments, the data retrieval method may further include: the memory retrieval system preprocesses the memory original data to generate multiple memory databases, where the preprocessing includes at least one of data extraction, knowledge graph construction, and vectorization processing.

[0090] Among them, preprocessing refers to the process of processing, transforming, and organizing the memory original data to generate different types of databases, so that the data can be efficiently stored and retrieved in the corresponding databases. Data extraction refers to the process of extracting relevant information from the memory original data and converting it into a format suitable for storage and analysis. This process includes text extraction and spatio-temporal data extraction to generate different types of databases, such as text databases and structured databases. Text extraction refers to the process of extracting text data from the memory original data to generate a text database. Spatio-temporal data extraction refers to the process of extracting spatio-temporal data from the memory original data to generate a structured database. Spatio-temporal data refers to data including time and space attributes. Knowledge graph construction refers to the process of extracting entities and relationships as nodes and edges for the memory original data to construct a knowledge graph. Vectorization processing refers to the process of performing vectorization processing on the text content of the memory original data. Specifically, the text content of the memory original data can be block-processed and then the block-processed data can be vectorized.

[0091] In the embodiments of the present disclosure, the memory retrieval system performs data extraction on the memory original data to generate a text database and a structured database, performs knowledge graph construction on the memory original data to generate a graph database, and performs vectorization processing on the memory original data to generate a vector database to support the retrieval of different databases.

[0092] In some embodiments, the data retrieval method may further include: inputting a plurality of memory data segments included in the target memory retrieval result into a relevance model, determining the relevance scores of each memory data segment with respect to the target question, and deleting the memory data segments with relevance scores less than the relevance threshold in the target memory retrieval result.

[0093] Among them, the relevance model refers to a model that obtains the relevance scores of each memory data segment with respect to the target question through relevance analysis of each memory data segment and the target question, and no limitation is imposed on the type of the relevance model. The relevance score is a quantitative index used to measure the degree of association between a memory data segment and the target question. The relevance threshold is a critical value preset when measuring the degree of association between a memory data segment and the target question, and is used to judge the strength of relevance or whether there is significant relevance.

[0094] In the embodiments of the present disclosure, the memory retrieval system uses the relevance model to score the combination of the target question and each retrieved memory data segment, determines the relevance scores of each memory data segment with respect to the target question, and then compares the relevance scores with the relevance threshold. If the relevance score is not less than the relevance threshold, the memory data segments with relevance scores not less than the relevance threshold are retained; if the relevance score is less than the relevance threshold, the memory data segments with relevance scores less than the relevance threshold are deleted. Thus, by setting the relevance threshold, the memory data segments that are obviously irrelevant to the target question are filtered out to further improve the accuracy of data retrieval.

[0095] To further improve the accuracy of data retrieval, in some embodiments, after the above step 103 or the step of deleting the memory data segments with relevance scores less than the relevance threshold in the target memory retrieval result, the data retrieval method of the embodiments of the present disclosure may further include: generating an evaluation prompt word based on the target memory retrieval result and the target question, inputting the evaluation prompt word into a memory result evaluation model to obtain a memory evaluation result, and in response to the memory evaluation result being passed, returning the target memory retrieval result to the client for display.

[0096] An evaluation prompt is a natural language text obtained by splicing the target memory retrieval result and the target question into an evaluation prompt template, which is used to guide the memory result evaluation model to complete a specific evaluation task. The evaluation prompt template is used to generate evaluation prompts and is a pre-designed text pattern with a specific structure and format, which is used to guide and standardize the evaluation process to ensure the accuracy of the evaluation task. The memory result evaluation model is a model that obtains the corresponding memory evaluation result by processing the evaluation prompt, and no limitation is imposed on the type of the memory result evaluation model. The memory evaluation result is a conclusion and evaluation description for judging whether the target memory retrieval result is suitable for answering the target question during the evaluation process. The evaluation description is a text that details and elaborates on the evaluation result during the evaluation process.

[0097] In an embodiment of the present disclosure, the memory retrieval system splices the target memory retrieval result and the target question into an evaluation prompt template to obtain an evaluation prompt, and inputs the evaluation prompt into the memory result evaluation model. After being processed by the memory result evaluation model, a memory evaluation result is output. If the memory evaluation result is passed, indicating that the target memory retrieval result meets the requirements of the evaluation criteria, the target memory retrieval result is returned to the client for display.

[0098] In an alternative embodiment, in response to the memory evaluation result being not passed, the evaluation description in the memory evaluation result is input into the memory retrieval model as a new target question until the memory evaluation result is passed or the number of evaluations reaches the number threshold to stop the retrieval.

[0099] Among them, the evaluation description is a text that details and elaborates on the evaluation result during the evaluation process. For example, the target question is "Please summarize the work content of my business trip in City A yesterday", and the location space information of "City A" is not pointed out in multiple target memory retrieval subtasks generated by the memory retrieval model for the target question. After being evaluated by the memory result evaluation model, an evaluation description will be output, such as "The retrieval subtask lacks the location information of City A". The number of evaluations refers to the specific number of times the memory result evaluation model performs the evaluation process of the memory retrieval result of the question, that is, the number of times the evaluation prompt is input into the memory result evaluation model. The number threshold is a preset maximum allowable number limit during the evaluation process of the evaluation prompt by the memory result evaluation model. The number threshold can be determined according to the scenario requirements. For example, it can be set to 5 or larger in the offline asynchronous scenario, and 1 in the real-time synchronous scenario.

[0100] In the embodiments of the present disclosure, if the memory evaluation result fails, it indicates that the target memory retrieval result does not meet the requirements of the evaluation criteria. Then, the evaluation description in the memory evaluation result is used as the new target question and input into the memory retrieval model again. According to the above steps 101 - 103, a new target memory retrieval result is obtained, and then the memory result evaluation model is used for evaluation to obtain the memory evaluation result. If the memory evaluation result passes, the retrieval is stopped; if the memory evaluation result still fails, the above steps are repeated; if the number of evaluations reaches the threshold and the memory evaluation result still fails, the retrieval is stopped. It can be seen that by adopting this technical solution, after obtaining the memory retrieval result, the memory retrieval result is continuously evaluated to avoid the situation of missing, irrelevant, and redundant memory retrieval results, and a feedback loop can be formed to further correct or improve the memory retrieval result. Therefore, in terms of adaptability and flexibility, the memory raw data retrieval mechanism with feedback enables the memory retrieval system to dynamically adjust the retrieval plan according to the evaluation of the retrieval result, continuously optimize the retrieval strategy, and continuously improve the retrieval effect, providing users with complete and accurate memory retrieval results.

[0101] Figure 4 It is a schematic flowchart of another data retrieval method provided by some embodiments of the present disclosure. As Figure 4 shown, the method includes:

[0102] Specifically, when the memory retrieval model receives a user request, it first analyzes the content of the user request (i.e., the target question) and formulates a memory retrieval plan (i.e., decomposes the target question into multiple target memory retrieval subtasks) to complete the decomposition of the target question. Then, for each target memory retrieval subtask, different retrieval codes are generated respectively, and the target memory retrieval subtasks are automatically executed to obtain multiple memory data segments. Finally, the memory retrieval model is responsible for evaluating whether the retrieved memory evaluation result data can answer the user's request. If not, the memory retrieval plan is continuously improved.

[0103] Step 401: Formulate a memory retrieval plan (i.e., decompose the target question into multiple target memory retrieval subtasks).

[0104] When the memory retrieval model receives the target question, it analyzes the target question to analyze which memory raw data may be involved in the target question, and then formulates a memory retrieval plan, that is, decomposes the target question into multiple target memory retrieval subtasks. The main goal of decomposing the target question is to decompose the original user request (i.e., the target question) into the smallest retrieval task units, and each target memory retrieval subtask uniquely corresponds to a retrieval method of a certain memory database, so as to generate executable retrieval codes subsequently.

[0105] Specifically, when determining which memory original data needs to be involved, it is necessary to obtain the memory overview information of the memory original data in advance. Obtaining the memory overview information of the memory original data includes: obtaining the theme of the memory original data. Specifically, the content of the existing memory original data is converted into a semantic vector, clustering is performed on the converted vector data. After clustering, for each cluster after clustering, a theme extraction model is used to extract several relevant themes from the corresponding text of the memory original data. Through the theme, the memory retrieval model can understand the overall situation of the memory original data; obtaining the storage method and organization form. Specifically, determine what storage methods the memory original data has, such as a graph database, a structured database, or a vector database, etc. The memory overview information will be filled in the retrieval prompt word template of the memory retrieval model as context, so that the memory retrieval model can know the background information and then formulate an accurate memory retrieval plan.

[0106] Step 402, generation of retrieval codes.

[0107] According to the memory retrieval plan formulated in step 401, the memory retrieval model further generates retrieval codes corresponding to each target memory retrieval subtask. The retrieval codes are written in a programming language. In order to generate accurate retrieval codes, it is necessary to configure the corresponding database configuration information according to the existing memory database type (such as a graph database, a document database, a vector database, etc.). Specifically, the database configuration information is filled in the retrieval prompt word template as context. During the process of generating retrieval codes, due to the dependency relationship between target memory retrieval subtasks, the retrieval codes of the front and back dependent target memory retrieval subtasks need to be generated in sequence to maintain the integrity and consistency of the front and back codes.

[0108] Step 403, execution of target memory retrieval subtasks.

[0109] After the retrieval codes corresponding to each target memory retrieval subtask are generated, the memory retrieval model will automatically execute each retrieval code in sequence in a sandbox environment according to the dependency order, and obtain the corresponding memory data fragments, reducing manual intervention and conversion costs during the retrieval process, making the entire retrieval process more efficient and smooth, and being able to quickly respond to user requests and provide relevant memory original data. The sandbox environment is a containerized isolation environment that connects different types of databases in the background, mainly including graph databases, document databases, vector databases, and elastic search services, etc., and can support graph retrieval, structured retrieval, semantic vector retrieval, and keyword retrieval.

[0110] To support the retrieval of different databases, the memory original data needs to be preprocessed. Specifically, the organization form of the memory original data is as Figure 5 shown. Exemplarily, Figure 5Schematic flowchart of yet another data retrieval method provided by some embodiments of the present disclosure. The memory raw data undergoes three preprocessing processes and is stored in three different forms of databases respectively.

[0111] Specifically, for the memory raw data, entities and relationships are extracted as nodes and edges to construct a knowledge graph, which is then stored in a graph database. This graph database supports "relational" retrieval of memories, such as finding memory raw data related to "Little A". Temporal and spatial metadata such as the timestamp and geographical location information of the memory raw data are stored in a structured database, supporting "temporal and spatial" retrieval of memories, such as filtering memory raw data in City A yesterday. The text content of the memory raw data should be chunked and vectorized and stored in a vector database and an elastic search service, supporting "content-based" retrieval of memories, such as finding memory raw data related to event a. None of the above three databases store the original content of the memory raw data, but store reference links. As Figure 5 shown by the dotted line, the three databases jointly point to the final memory raw data to ensure that the retrieved data is consistent.

[0112] Step 404: Evaluation of memory retrieval results.

[0113] After the data retrieval is completed, several memory data segments R 1 , R 2 ,... R n can be obtained. The evaluation is divided into the following two parts:

[0114] Step 404-1: Relevance model scoring: The memory retrieval system uses a relevance model to score the combination <Q, R i > composed of the user request (i.e., the above target question) and each retrieved memory data segment, s = reanker(<Q, R i >). Here, s represents the relevance score of the i-th memory data segment to the target question, Q represents the target question, and R i represents the i-th memory data segment. The relevance scores of each memory data segment to the target question are determined, and by setting a relevance threshold T, memory data segments that are obviously irrelevant to the user request (i.e., memory data segments corresponding to s < T) are filtered out to further improve the accuracy of data retrieval.

[0115] Step 404-2, Memory Evaluation Model Evaluation: After the above-mentioned relevance model scoring, based on the remaining memory data segments and the user request, an evaluation prompt word is generated, and the evaluation prompt word is input into the memory result evaluation model to output a memory evaluation result, such as "whether the user's request can be answered and an explanation is given". The output format is <whether the request can be answered>: <evaluation explanation>. If it is determined that the retrieved memory data segment can answer the user's question, the retrieval process is completed. If it is determined that the retrieved memory data segment cannot answer the user's question, the evaluation explanation is input back into the memory retrieval model as a new question, requiring it to modify the original retrieval plan and continue to execute the retrieval process until the memory evaluation result is passed or the number of evaluation times reaches the threshold to stop the retrieval. For example, the target question is "Please summarize the work content during my business trip to City A yesterday", and the multiple target memory retrieval subtasks generated by the memory retrieval model for the target question do not specify the location information of "City A". After being evaluated by the memory result evaluation model, an evaluation explanation such as "the retrieval subtask lacks the location information of City A" will be output, and this feedback will be filled back into the conversation history of the memory retrieval model and used as context together with the previously formulated retrieval plan to guide it to modify the corresponding target memory retrieval subtask. It can be seen that through the memory raw data retrieval mechanism with feedback, the memory retrieval system can dynamically adjust the retrieval plan according to the evaluation of the retrieval results, continuously optimize the retrieval strategy, and continuously improve the retrieval effect, providing users with complete and accurate memory retrieval results.

[0116] It can be seen that the embodiment of the present disclosure effectively solves the problem that it is difficult to accurately obtain complex memory raw data in the prior art by introducing a memory retrieval model for retrieval by a single retrieval method or a fixed process composed of multiple retrieval methods.

[0117] Figure 6 The following is a schematic structural diagram of a data retrieval device provided by some embodiments of the present disclosure. The device can be implemented by software and / or hardware and is generally integrated in an electronic device. As Figure 6 shown, it includes:

[0118] An acquisition module 601, configured to acquire a target question;

[0119] A first generation module 602, configured to generate a retrieval prompt word based on the target question and a retrieval prompt word template;

[0120] A first input module 603, configured to input the retrieval prompt word into a memory retrieval model. After the memory retrieval model generates multiple target memory retrieval subtasks for the target question to perform retrieval processing on multiple memory databases to obtain a target memory retrieval result corresponding to the target question, where each target memory retrieval subtask is used to retrieve one memory database.

[0121] In an alternative embodiment, the first generation module 602 includes:

[0122] An acquisition sub-module, configured to acquire the memory overview information of the memory original data and the database configuration information corresponding to the multiple memory databases;

[0123] A filling sub-module, configured to fill the target problem, the memory overview information, and the database configuration information according to the retrieval prompt word template to obtain a retrieval prompt word.

[0124] In an alternative embodiment, the acquisition sub-module includes:

[0125] An extraction sub-module, configured to perform clustering and theme extraction on the memory original data by using a theme extraction model to obtain multiple themes;

[0126] A first determination sub-module, configured to determine the storage method corresponding to each of the themes, and determine the multiple themes and the storage method corresponding to each of the themes as the memory overview information.

[0127] In an alternative embodiment, the database configuration information includes multiple retrieval methods corresponding to the multiple memory databases.

[0128] In an alternative embodiment, the first input module 603 includes:

[0129] A decomposition sub-module, configured to decompose the target problem into multiple target memory retrieval sub-tasks based on the memory overview information in the retrieval prompt word by using the memory retrieval model;

[0130] A generation sub-module, configured to generate multiple retrieval codes corresponding to the multiple target memory retrieval sub-tasks based on the database configuration information in the retrieval prompt word;

[0131] A retrieval sub-module, configured to execute the multiple retrieval codes to perform retrieval in the memory databases corresponding to the multiple target memory retrieval sub-tasks according to the corresponding retrieval methods to obtain multiple memory data segments;

[0132] A second determination sub-module, configured to determine the multiple memory data segments as the target memory retrieval result corresponding to the target problem.

[0133] In an alternative embodiment, there is a dependency relationship between the multiple target memory retrieval sub-tasks, and the multiple retrieval codes are executed sequentially according to the dependency relationship when executed.

[0134] In an alternative embodiment, the device further includes:

[0135] A determination module, configured to input multiple memory data segments included in the target memory retrieval result into a relevance model, and determine the relevance scores of each memory data segment and the target question;

[0136] A deletion module, configured to delete the memory data segments with relevance scores less than a relevance threshold in the target memory retrieval result.

[0137] In an optional implementation manner, the device further includes:

[0138] A second generation module, configured to generate an evaluation prompt word based on the target memory retrieval result and the target question;

[0139] A second input module, configured to input the evaluation prompt word into a memory result evaluation model to obtain a memory evaluation result;

[0140] A return module, configured to, in response to the memory evaluation result being passed, return the target memory retrieval result to the client for display.

[0141] In an optional implementation manner, the device further includes:

[0142] A third input module, configured to, in response to the memory evaluation result being not passed, input the evaluation description in the memory evaluation result as a new target question into the memory retrieval model until the memory evaluation result is passed or the number of evaluation times reaches a threshold number of times to stop the retrieval.

[0143] In an optional implementation manner, the multiple memory databases include at least two of a text database, a graph database, a structured database, and a vector database.

[0144] In an optional implementation manner, the device further includes:

[0145] A third generation module, configured to preprocess memory original data to generate the multiple memory databases, where the preprocessing includes at least one of data extraction, knowledge graph construction, and vectorization processing.

[0146] The data retrieval device provided by the embodiments of the present disclosure can execute the data retrieval method provided by any embodiment of the present disclosure, and has corresponding functional modules and beneficial effects for executing the method.

[0147] The embodiments of the present disclosure further provide a computer program product, including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the data retrieval method provided by any embodiment of the present disclosure is implemented.

[0148] Figure 7 It is a schematic structural diagram of an electronic device provided for some embodiments of the present disclosure.

[0149] Specifically, with reference to Figure 7 , which shows a schematic structural diagram of an electronic device 700 suitable for implementing the embodiments of the present disclosure. The electronic device 700 in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), in-vehicle terminals (such as in-vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is merely an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0150] As Figure 7 shown, the electronic device 700 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0151] Generally, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or wirelessly to exchange data. Although Figure 7 the electronic device 700 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be implemented or had.

[0152] Specifically, according to the embodiments of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication device 709, or installed from the storage device 708, or installed from the ROM 702. When the computer program is executed by the processing device 701, the above functions defined in the data retrieval method of the embodiments of the present disclosure are executed.

[0153] It should be noted that the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and this computer-readable signal medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0154] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0155] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist separately and not be assembled into the electronic device.

[0156] The above computer-readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: obtain a target problem; generate a retrieval prompt word based on the target problem and a retrieval prompt word template; input the retrieval prompt word into a memory retrieval model to obtain a target memory retrieval result corresponding to the target problem, wherein the memory retrieval model is used to generate a plurality of memory retrieval subtasks for a problem to retrieve a memory retrieval result from a plurality of memory databases, and each of the memory retrieval subtasks is used to retrieve from one memory database.

[0157] Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages or combinations thereof. The programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0158] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0159] The units described in the embodiments of the present disclosure may be implemented in software or in hardware. In some cases, the name of the unit does not constitute a limitation on the unit itself.

[0160] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. By way of example, and without limitation, the types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

[0161] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium can be either a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a Read-Only Memory (ROM), an Erasable Programmable Read-Only Memory (EPROM or Flash memory), an optical fiber, a portable Compact Disc Read-Only Memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] It should be understood that before using the technical solutions disclosed in the embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the information involved in this disclosure should be informed to users and user authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0163] The above description is only a preferred embodiment of this disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of disclosure involved in this disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosure concept. For example, a technical solution formed by mutually replacing the above features with technical features having similar functions (but not limited to) disclosed in this disclosure.

[0164] In addition, although the operations are depicted in a particular order, this should not be construed as requiring that the operations be performed in the particular order shown or in sequential order. In certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of separate embodiments can also be implemented combinatorially in a single embodiment. Conversely, the various features described in the context of a single embodiment can also be implemented separately or in any suitable sub-combination in multiple embodiments.

[0165] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A data retrieval method, characterized in that: include: Get the target question; Generate a search prompt word based on the target question and the search prompt word template; The retrieval prompt word is input into a memory retrieval model, and the memory retrieval model generates multiple target memory retrieval subtasks for the target question to perform retrieval processing on multiple memory databases to obtain the target memory retrieval result corresponding to the target question, wherein each of the target memory retrieval subtasks is used to search a memory database.

2. The method according to claim 1, characterized in that: Generating a search prompt word based on the target question and the search prompt word template includes: Acquire memory overview information of the memory original data and database configuration information corresponding to the multiple memory databases; The target question, the memory overview information and the database configuration information are filled in according to a search prompt word template to obtain a search prompt word.

3. The method according to claim 2, characterized in that The obtaining of memory overview information of the original memory data includes: Using a topic extraction model to cluster and extract topics from the original memory data to obtain multiple topics; Determine a storage method corresponding to each of the themes, and determine the multiple themes and the storage method corresponding to each of the themes as the memory overview information.

4. The method according to claim 2, characterized in that: The database configuration information includes multiple retrieval methods corresponding to the multiple memory databases.

5. The method according to claim 1, characterized in that The retrieval prompt word is input into the memory retrieval model, and the memory retrieval model generates a plurality of target memory retrieval subtasks for the target question to perform retrieval processing on a plurality of memory databases to obtain a target memory retrieval result corresponding to the target question, including: Decomposing the target problem into a plurality of target memory retrieval subtasks based on the memory overview information in the retrieval prompt words by the memory retrieval model; Generate multiple retrieval codes corresponding to the multiple target memory retrieval subtasks based on the database configuration information in the retrieval prompt word; Executing the plurality of retrieval codes to search the memory database corresponding to each of the target memory retrieval subtasks according to the corresponding retrieval method to obtain a plurality of memory data segments; The multiple memory data segments are determined as target memory retrieval results corresponding to the target question.

6. The method according to claim 5, characterized in that There is a dependency relationship between the multiple target memory retrieval subtasks, and the multiple retrieval codes are executed in sequence according to the dependency relationship when executed.

7. The method according to claim 1, characterized in that The method further comprises: Inputting a plurality of memory data segments included in the target memory retrieval result into a relevance model to determine a relevance score between each of the memory data segments and the target question; Delete the memory data segments whose relevance scores in the target memory retrieval results are less than the relevance threshold.

8. The method according to claim 1 or 7, characterized in that: The method further comprises: generating an evaluation prompt word based on the target memory retrieval result and the target question; Inputting the evaluation prompt words into the memory result evaluation model to obtain the memory evaluation result; In response to the memory evaluation result being passed, the target memory retrieval result is returned to the client for display.

9. The method according to claim 8, characterized in that The method further comprises: In response to the memory evaluation result being a failure, the evaluation instructions in the memory evaluation result are continuously input into the memory retrieval model as new target questions until the memory evaluation result is a pass or the number of evaluations reaches a threshold and the retrieval is stopped.

10. The method according to claim 1, characterized in that The multiple memory databases include at least two of a text database, a graph database, a structured database, and a vector database.

11. The method according to claim 10, characterized in that The method further comprises: The memory raw data is preprocessed to generate the multiple memory databases, wherein the preprocessing includes at least one of data extraction, knowledge graph construction, and vectorization processing.

12. A data retrieval device, characterized in that: include: An acquisition module is used to obtain the target problem; A first generating module, used for generating a search prompt word based on the target question and a search prompt word template; The first input module is used to input the retrieval prompt word into the memory retrieval model, and the memory retrieval model generates multiple target memory retrieval subtasks for the target problem to perform retrieval processing on multiple memory databases to obtain the target memory retrieval result corresponding to the target problem, wherein each of the target memory retrieval subtasks is used to search a memory database.

13. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the data retrieval method described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the data retrieval method described in any one of claims 1 to 11.

Images