Database construction method and device for nonlinear data based on large model

Abstract

The invention discloses a database construction method and device for nonlinear data based on a large model. The method comprises the steps that a node is selected in an initial graph structure to input request data of a user, a retrieval result of the request data is retrieved and generated through an RAG system, and reply data of the request data is generated through the large model; writing the request data, the reply data and the retrieval result into a new node, determining a nearest node associated with the new node in the existing nodes, and forming a branch path with topicality by taking the nearest node as a father node of the new node; storing the new node in a database; wherein all the branch paths form a tree-shaped structure. According to the method, each dialogue node has a traceable father-son relationship by constructing a database storage mode of a multi-branch tree structure, and a user can branch a new dialogue path from any historical node. And the user can freely select different topic paths for dialogue, so that the dialogue context content of the large model is more accurate.

Description

Technical Field

[0001] The present application relates to the field of auxiliary retrieval technology, and in particular to a method and device for constructing a database of nonlinear data based on a large model. Background Art

[0002] Currently, many AI tools rely on text prompts to communicate with users in a linear, one-way manner. If users want to make precise modifications to existing designs in the later iteration stages, they must re-describe each detail in text. However, this pure text description interaction method is not intuitive enough. In order for the large language model (LLM) to understand complex modification points, users often need to enter lengthy instruction descriptions. Otherwise, the LLM may produce results that are inconsistent with expectations, causing users to constantly repeat and tedious explanations. The communication cost between users and LLM is too high, affecting iteration efficiency and accuracy.

[0003] In addition, existing intelligent interaction tools do not support nonlinear iterative interaction with LLM. "Nonlinear iteration" means that the multi-round interaction process between users and LLM does not always proceed in a linear time sequence, but often requires "going back to the previous state" or "exploring parallel branches", that is, users may want to readjust, partially modify, or switch between multiple branch results without destroying the existing conversation context. Existing tools are mainly linear chats and cannot effectively support this multi-branch, multi-path dynamic conversation management. When nonlinear context is forcibly embedded in the linear conversation history, it is difficult for users to clearly tell LLM which previous intermediate product to use as a new starting point, and LLM is also difficult to correctly extract relevant information from the long and messy conversation history for generation. This makes it difficult for users to efficiently manage context in multi-round, multi-branch LLM interactions, and the context construction is incomplete, resulting in model understanding deviation or simply superimposing linear historical records or relying on a single RAG retrieval, resulting in redundant, lost or one-sided context information, affecting the accuracy of LLM's answers in complex scenarios. Summary of the invention

[0004] The present disclosure provides a method, device, equipment and storage medium for constructing a database of nonlinear data based on a large model, so as to at least solve the above technical request data existing in the prior art.

[0005] According to a first aspect of the present application, a method for constructing a database of nonlinear data based on a large model is provided, the method comprising:

[0006] Select a node in the initial graph structure to input the user's request data, and use the RAG system to search and generate the search results of the request data; wherein each node represents a dialogue round between the user input and the large model output;

[0007] generating reply data for the request data using a large model based on the search result and the request data;

[0008] Writing the request data, the reply data and the search result into a new node, determining the nearest node associated with the new node among the existing nodes, and taking the nearest node as the parent node of the new node to form a branch path;

[0009] The new node is stored in the database, wherein all branch paths form a tree structure.

[0010] In one possible implementation, each node further includes:

[0011] Unique identifier, reference relationship of the previous node, timestamp and business label;

[0012] The search results include document ID, summary information, source information and embedding vector ID.

[0013] In one possible implementation manner, the using the RAG system to search and generate the search result of the requested data includes:

[0014] Determine an associated parent node based on the user's request data; the associated parent node is a parent node in which the request data, the reply data, and the search result have similarities with the request data;

[0015] Calculating the similarity between the request data and the request data, reply data and search results in the parent node;

[0016] Based on the similarity, a text segment is determined as a retrieval result of the user's request data.

[0017] In one possible implementation manner, the generating the response data of the request data by using the big model includes:

[0018] Get the selected node information and parent node link from the database;

[0019] Traverse the parent node link upward from the selected node to obtain a preset number of upper-level nodes or reach a number of dialogue data;

[0020] Extracting the user's historical input data and the historical response data of the large model based on the preset number of upper-level nodes or the number of dialogue data, and generating a historical context;

[0021] Constructing a Promp text based on the user's request data, historical context, and information about the selected node;

[0022] The Promp text is input into the large model to generate reply data.

[0023] In one possible implementation, before constructing the Promp text, it also includes:

[0024] Data cleaning is performed on the user's request data, historical context, and selected node information.

[0025] In one possible implementation, after traversing the parent node link upward from the selected node, the method further includes:

[0026] When information is missing, the RAG system is called to obtain supplementary data.

[0027] In one possible implementation, the determining the nearest node associated with the new node among the existing nodes includes:

[0028] Determine whether the parent node of the new node is a node associated with the current conversation;

[0029] If the upper-level node is not an associated node of the new node, the nearest node associated with the new node among the remaining existing nodes is re-determined.

[0030] In one possible implementation, before writing a new node, the method further includes:

[0031] The user's request data and the information in the previous node are verified through the large model. If the verification result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

[0032] In one possible implementation, the database adopts a graph database or a database for storing documents.

[0033] According to a second aspect of the present application, there is provided a database construction device for nonlinear data based on a large model, comprising:

[0034] A data retrieval module is used to select a node in the initial graph structure to input the user's request data, and use the RAG system to search and generate a retrieval result of the request data; wherein each node represents a dialogue round between the user input and the large model output;

[0035] A data generation module, configured to generate reply data of the request data using a large model based on the search result and the request data;

[0036] A path generation module, used to write the request data, the reply data and the search result into a new node, determine the nearest node associated with the new node among the existing nodes, use the nearest node as the parent node of the new node, and form a branch path;

[0037] The node storage module is used to store new nodes in the database; wherein all branch paths form a tree structure.

[0038] According to a third aspect of the present application, an electronic device is provided, including:

[0039] at least one processor; and

[0040] a memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0042] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the present application.

[0043] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program or instructions, which implement the method described in the present application when executed by a processor.

[0044] Using the technical solution of the present application, users of the present application can select different topic paths from a graph data structure to obtain the selected node and its associated historical context, generate RAG search results in combination with real-time search enhancement, integrate historical information with new search content, and form an accurate and complete context input to the large model LLM, thereby solving the problems of information loss and redundancy in complex conversations, and ensuring that LLM provides high-quality and accurate answers in branching or backtracking scenarios.

[0045] It should be understood that the data described in this section are not intended to identify the key or important features of the embodiments of the present application, nor are they intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0047] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0048] Figure 1 The schematic diagram shows the implementation process of the method for constructing a database of nonlinear data based on a large model in the embodiment of the present application. Figure 1 ;

[0049] Figure 2 The schematic diagram shows the implementation process of the method for constructing a database of nonlinear data based on a large model in the embodiment of the present application. Figure 2 ;

[0050] Figure 3 A structural block diagram of a database construction device based on nonlinear data of a large model in an embodiment of the present application is shown;

[0051] Figure 4 A schematic diagram of the structure of an electronic device in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0053] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0054] A method and device for constructing a database of nonlinear data based on a large model provided by the present application will be described below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the present application provides a method for constructing a database of nonlinear data based on a large model, the method comprising:

[0056] S101, selecting a node in the initial graph structure to input the user's request data, and using the RAG system to search and generate a search result of the request data; wherein each node represents a dialogue round between the user input and the large model output;

[0057] It is understandable that the initial graph structure may be a linear structure formed by a dialogue, or the initial graph structure may be a non-linear structure that has been stored, such as a tree structure. The initial graph structure includes multiple nodes, and the user may select one of the nodes through interface operation to input the user's request data, and the request data may be, for example, a question.

[0058] Among them, each node represents a conversation turn (including user input, search results and LLM output).

[0059] When the user needs to make a new request, he can select a node by operating the interface button, indicating that a new question starts from this node, realizing nonlinear iteration. It can be understood that the node selected by the user when operating the interface button is relevant to the user's new request. For example, if the conversation content contains car-related topics or travel-related topics, the user can select the node with the related topic for the next search.

[0060] ‌RAG system (retrieval enhanced generation system) is a technology that combines a large language model (LLM) and an external knowledge source, aiming to enhance the ability of LLM to generate coherent and contextually relevant responses. The RAG system provided in this application can be a Faiss vector retrieval system. The Faiss vector retrieval system can retrieve context related to the request data in the database and obtain retrieval results.

[0061] S102, generating response data of the request data by using a large model based on the search result and the request data;

[0062] Before the big model LLM answers, the RAG system is used to retrieve the relevant context of the request data. After obtaining the relevant context and request data, the retrieved context and request data are then input into the big model to obtain the reply data.

[0063] S103, writing the request data, the reply data and the search result into a new node, determining the nearest node associated with the new node among the existing nodes, taking the nearest node as the parent node of the new node, and forming a branch path;

[0064] After the big model answer is generated, the reply data, request data and search results are written into a new node and the parent_id of this node is set as the starting node of this conversation (i.e. the upstream node selected by the user). It can be understood that the parent_id is the closest node associated with the new node, thus forming a branch path. Among them, the node association indicates that the data between the two nodes are similar.

[0065] like Figure 2 As shown, assuming that the selected node is node 2 and the new node written is node 5, at this time, if the nearest node associated with node 5 is node 2, node 5 and node 2 form a branch path. If it is found that the nearest node associated with node 5 is node 3, node 5 and node 3 form a branch path.

[0066] S104, storing the new node in the database; wherein all branch paths form a tree structure.

[0067] Finally, the new nodes are stored in the database. It is understandable that the database also stores other nodes and the relationships between the nodes. In other words, the nodes and the relationships between the nodes form a tree structure and are stored in the database. When the RAG system queries the basic information of a node from the database, it can query not only the information of the node, but also the associated RAG context information and the parent node link (tracing back to the initial dialogue node). This provides context information for the big model to request data, so that the big model can output more accurate answers.

[0068] The database provided in this application can be a graph database (such as Neo4j) or a document-oriented NoSQL database (such as MongoDB) to store dialogue nodes and their relationships. Among them, the graph database can directly store nodes and edges in a graph structure, which is convenient for branch query and hierarchical visualization.

[0069] The database construction method based on nonlinear data of large models provided by this application stores dialogue nodes and their parent-child relationships through a graph-like data structure, supports nonlinear branching, backtracking and version management, solves the limitations of linear processes, and makes users more flexible and efficient in complex iterative scenarios. By integrating historical context backtracking with RAG retrieval, historical nodes and their associated contexts are merged with new search results to provide LLM with complete and accurate context, significantly improving the coherence and accuracy of answers. When generating a new node, the external context source of the RAG retrieval is recorded and associated with the node content, so that the source of the answer can be traced, which is convenient for users to verify the accuracy and basis of the results and improve credibility.

[0070] In some embodiments, each node further comprises:

[0071] Unique identifier, reference relationship of the previous node, timestamp and business label;

[0072] The search results include document ID, summary information, source information and embedding vector ID.

[0073] It can be understood that each node in the present application also includes: node_id: unique identifier; parent_id or parents: reference to the previous level node (supporting single or multiple parent nodes); timestamp: timestamp, used for display on the timeline; user_input: user request content; model_output: model answer content; rag_contexts: RAG retrieval results used for this round of answers, where the retrieval results include document ID, summary information, embedded vector references, etc.; metadata: business tags, where business tags include topics, fields, version numbers, etc.

[0074] It should be noted that the time axis displays the time sequence of nodes vertically, such as Figure 2 As shown, branches are represented by expandable side lines or hanging nodes. When a user clicks a node, the front-end requests the back-end API to obtain the upstream and downstream information and RAG fragments of this node, and displays them in a pop-up window or sidebar. When the user confirms that he wants to ask a new question based on this node, the front-end sends the node_id and user input content to the back-end, triggering the context generation and LLM call process.

[0075] In some embodiments, the step of using the RAG system to search and generate a search result of the requested data includes:

[0076] Determine an associated parent node based on the user's request data; the associated parent node is a parent node in which the request data, the reply data, and the search result have similarities with the request data;

[0077] Calculating the similarity between the request data and the request data, reply data and search results in the parent node;

[0078] Based on the similarity, a text segment is determined as a retrieval result of the user's request data.

[0079] Specifically, in this application, after a user makes a new request, the request is first passed to the Faiss vector retrieval system together with the selected parent node context (user_input, model_output and rag_contexts references). The Faiss vector retrieval system calculates the similarity between the new request data and the information in the parent node, and selects the most relevant text fragment as the RAG context, that is, the retrieval result, by sorting the similarity.

[0080] It should be noted that the context information retrieved by the RAG system is stored in the rag_contexts field, including the recalled document ID, content, source information, and optionally stored embedding vector ID. When a new node is written to the database in this application, the RAG search results are also recorded to facilitate future tracing back to which document content the answer referenced.

[0081] When generating a new node, this application records the external knowledge fragments used each time the RAG search is called, and associates them with the historical dialogue nodes to form a traceable context reference chain. This enables the verifiability and traceability of LLM answer results, helping users check the source and logic and improving reliability.

[0082] In some embodiments, the step of generating the response data of the request data by using the large model includes:

[0083] Get the selected node information and parent node link from the database;

[0084] Traverse the parent node link upward from the selected node to obtain a preset number of upper-level nodes or reach a number of dialogue data;

[0085] Extracting the user's historical input data and the historical response data of the large model based on the preset number of upper-level nodes or the number of dialogue data, and generating a historical context;

[0086] Constructing a Promp text based on the user's request data, historical context, and information about the selected node;

[0087] The Promp text is input into the large model to generate reply data.

[0088] In some embodiments, after traversing the parent node link upward from the selected node, the method further includes:

[0089] When information is missing, the RAG system is called to obtain supplementary data.

[0090] Specifically, in this application, the basic information of the node (user_input, model_output), its associated RAG context information, i.e., retrieval information, and the parent node link (tracing back to the initial dialogue node) are queried from the database. Then, starting from the selected node, the parent node chain is traversed upward until a certain number of upstream nodes are found or the upper limit of the dialogue history limit is reached. The necessary user input and model output fragments are extracted from these nodes to form a historical context. Then, the RAG system is used for retrieval. Specifically, if the semantic span is large or information is missing during the upward traversal, RAG can be called again to obtain supplementary information.

[0091] It is understandable that in this application, when querying the basic information of the node and its associated RAG context information and parent node link from the database, it can be achieved through the index structure, specifically by setting a dialogue node table, which includes node_id, parent_id, timestamp, user_input, model_output, rag_contexts, metadata. Establish a parent_id index to quickly query child nodes and trace back the parent node chain.

[0092] This application can obtain the selected node and its associated historical context from the graph data structure, and integrate the historical information with the new search content in combination with the real-time RAG (search enhancement generation) search results to form an accurate and complete context input to the LLM. This application effectively solves the problem of information loss and redundancy in complex conversations through the context integration mechanism, ensuring that the LLM provides high-quality and accurate answers in branching or backtracking scenarios.

[0093] In some embodiments, before constructing the Promp text, the following steps are further included:

[0094] Data cleaning is performed on the user's request data, historical context, and selected node information.

[0095] It is understandable that when constructing the final Prompt for LLM, the user's current input and several refined historical conversations (important summaries of the selected node and its parent node) and the RAG search fragment of the selected node are input to LLM as prompts. Then, simple rules (such as retaining only key information, retaining only the context of the previous round and the starting point of the branch) can be used to reduce context redundancy. If there are too many branches, users can also independently check the upstream nodes they want to keep for reference on the UI to help the system select the correct context. In the end, the large model LLM can still accurately give answers according to user intentions in complex nonlinear iterations based on the given Prompt (including the selected node path context and RAG fragment) when generating new answers.

[0096] Specifically, when the relevant context is retrieved, the node node_id and the newly input request data of the user are selected, the parent node chain is traced back based on the request data, and the necessary model_output fragments, user_input fragments and rag_contexts references are collected. If it is found that the information is insufficient at this time, RAG is called to search again, and then the collected information is summarized and refined to reduce the length of the Prompt, and finally the final context Prompt is returned to the LLM, where the Prompt is in JSON or text format.

[0097] In some embodiments, determining the nearest node associated with the new node among the existing nodes includes:

[0098] Determine whether the parent node of the new node is a node associated with the current conversation;

[0099] If the upper-level node is not an associated node of the new node, the nearest node associated with the new node among the remaining existing nodes is re-determined.

[0100] It is understandable that after the large model generates the reply data, it writes the answer (together with the user request and RAG retrieval information) into a new node, and sets the parent_id of this node to the starting node of this conversation (that is, the upstream node selected by the user through the interface operation). If the upstream node selected by the user when creating a new conversation is not the most recent node (it may be an old node in an earlier conversation), then this new node will use the old node as the parent node to form a branch path.

[0101] This application stores multi-round dialogue nodes and their parent-child relationships through a graph data structure, supporting users to backtrack, branch, and create versions at any historical node, thereby achieving non-linear, multi-path dialogue management.

[0102] In some embodiments, before writing a new node, the process further includes:

[0103] The user's request data and the information in the previous node are verified through the large model. If the verification result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

[0104] In this application, before establishing a new node, LLM can be used to perform a simple check on the user's new input and the upstream node topic. If the detected context is irrelevant, the user is prompted to add more information. After the LLM call is successful, the result is stored together with the context to ensure the traceability of subsequent backtracking.

[0105] As a specific embodiment, the user selects an existing node on the interface, which is a branch, enters a new question in the existing node, uses the RAG system to search the new question and the selected node to obtain the context, and then enters the context and request data into the big model. The big model LLM generates a new answer based on the context; then the answer is added to the database in the form of a new node, and the parent_id is set to the node_id of the selected node.

[0106] Existing technologies mostly use a linear, multi-round conversation history recording method, so that users can only proceed in chronological order when making iterative modifications. When users need to form a new plan or compare multiple parallel versions starting from an early node, there is often a lack of effective means. The technical solution provided by this application stores the conversation process through a graph data structure, so that each conversation node has a traceable parent-child relationship, and supports users to branch out new conversation paths from any historical node. Users can freely choose different topic paths to conduct conversations, making the context content of large model conversations more accurate, without the need to repeatedly search in lengthy linear histories. Through the technical solution of this application, users can more intuitively compare multiple versions of results in complex creation and design scenarios, and flexibly select branch paths, thereby greatly improving user experience and work efficiency.

[0107] Secondly, the traditional solution simply superimposes historical input and output in multiple rounds of dialogue. When the scene is complex or the dialogue history is long, it is difficult for LLM to extract the correct context information from it. Although other existing solutions have intent recognition and context supplementation, they are usually single-directional, either relying only on the latest information retrieved by RAG, or only using linear historical records. Before inputting to LLM, the present application organically integrates the historical node context selected by the user (including past dialogue points and previously used RAG materials) with the new round of RAG search results. Through centralized processing, the history is provided to LLM together with the new search content. Therefore, through the technical solution of the present application, LLM can obtain complete, refined and accurate context support when answering, including historical branch information and the latest material retrieval results. Because the context is more targeted and logically related, the accuracy, coherence and professionalism of the answer can be significantly improved, reducing lengthy and repeated descriptions and reducing the user's expression costs.

[0108] Thirdly, in traditional scenarios based on text prompts, users often need to repeat a lot of existing information or rebuild the context through lengthy descriptions in order for LLM to understand complex modification requirements. However, the present application utilizes a graph data structure and RAG retrieval to automatically extract the context and information related to the node after the user selects a historical node, without the user having to re-describe the context in detail. Users only need to add new change points or questions, and the system can integrate the existing information. Therefore, the technical solution of the present application greatly reduces the user's communication burden, and there is no need to describe the requirements from scratch every time, thereby improving iteration efficiency, reducing error rates, and allowing users to achieve their desired goals faster.

[0109] In addition, this application uses a timeline and a graphical visual interface to intuitively display the dialogue branch path, allowing users to select historical nodes, view RAG context, and quickly backtrack or create branches by clicking. Importance: This mechanism greatly reduces the user's operation complexity, improves the ease of use and intuitiveness of the system, and meets the interaction requirements of actual usage scenarios.

[0110] like Figure 3 As shown, the embodiment of the present application provides a database construction device for nonlinear data based on a large model, and the device includes:

[0111] The data retrieval module 301 is used to select a node in the initial graph structure to input the user's request data, and use the RAG system to search and generate the retrieval results of the request data; wherein each node represents a dialogue round between the user input and the large model output;

[0112] A data generation module 302, configured to generate response data of the request data using a large model based on the search result and the request data;

[0113] A path generation module 303 is used to write the request data, the reply data and the search result into a new node, determine the nearest node associated with the new node among the existing nodes, and use the nearest node as the parent node of the new node to form a branch path;

[0114] The node storage module 304 is used to store new nodes in the database; wherein all branch paths form a tree structure.

[0115] The working principle of the database construction device based on nonlinear data of the big model provided in the present application is that the data retrieval module 301 selects a node in the initial graph structure to input the user's request data, and uses the RAG system to retrieve and generate the retrieval result of the request data; wherein each node represents a dialogue round between the user input and the big model output; the data generation module 302 generates the reply data of the request data based on the retrieval result and the request data using the big model; the path generation module 303 writes the request data, the reply data and the retrieval result into a new node, determines the nearest node associated with the new node among the existing nodes, and uses the nearest node as the parent node of the new node to form a branch path; the node storage module 304 stores the new node in the database; wherein all branch paths form a tree structure.

[0116] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0117] The electronic device includes at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for constructing a database of nonlinear data based on a large model described in the present application. The computer instructions are used to enable the computer to execute the method for constructing a database of nonlinear data based on a large model described in the present application.

[0118] The present application also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the database construction method of the present application based on large model nonlinear data.

[0119] Figure 4A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0120] like Figure 4 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 to a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0121] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as a database construction method for nonlinear data based on a large model. For example, in some embodiments, the database construction method for nonlinear data based on a large model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the database construction method for nonlinear data based on a large model described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured in any other appropriate manner (eg, by means of firmware) to execute the database construction method based on large model nonlinear data.

[0123] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0125] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the above data. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above data.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0127] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0128] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0129] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for constructing a database of nonlinear data based on a large model, characterized in that: The method comprises: Select a node in the initial graph structure to input the user's request data, and use the RAG system to search and generate the search results of the request data; wherein each node represents a dialogue round between the user input and the large model output; generating reply data for the request data using a large model based on the search result and the request data; Writing the request data, the reply data and the search result into a new node, determining the nearest node associated with the new node among the existing nodes, and taking the nearest node as the parent node of the new node to form a branch path; The new node is stored in the database, wherein all branch paths form a tree structure.

2. The method according to claim 1, characterized in that Each node also includes: Unique identifier, reference relationship of the previous node, timestamp and business label; The search results include document ID, summary information, source information and embedding vector ID.

3. The method according to claim 1, characterized in that The method of using the RAG system to search and generate the search result of the requested data includes: Determine an associated parent node based on the user's request data; the associated parent node is a parent node in which the request data, the reply data, and the search result have similarities with the request data; Calculating the similarity between the request data and the request data, reply data and search results in the parent node; Based on the similarity, a text segment is determined as a retrieval result of the user's request data.

4. The method according to claim 1, characterized in that: The step of using the large model to generate the response data of the request data includes: Get the selected node information and parent node link from the database; Traverse the parent node link upward from the selected node to obtain a preset number of upper-level nodes or reach a number of dialogue data; Extracting the user's historical input data and the historical response data of the large model based on the preset number of upper-level nodes or the number of dialogue data, and generating a historical context; Constructing a Promp text based on the user's request data, historical context, and information about the selected node; The Promp text is input into the large model to generate reply data.

5. The method according to claim 4, characterized in that Before constructing the Promp text, also include: Data cleaning is performed on the user's request data, historical context, and selected node information.

6. The method according to claim 4, characterized in that After traversing the parent node link upward from the selected node, it also includes When information is missing, the RAG system is called to obtain supplementary data.

7. The method according to claim 1, characterized in that The determining of the nearest node associated with the new node among the existing nodes includes: Determine whether the parent node of the new node is a node associated with the current conversation; If the upper-level node is not an associated node of the new node, the nearest node associated with the new node among the remaining existing nodes is re-determined.

8. The method according to claim 1, characterized in that: Before writing to a new node, it also includes: The user's request data and the information in the previous node are verified through the large model. If the verification result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

9. The method according to claim 1, characterized in that: The database adopts a graph database or a database for storing documents.

10. A database construction device for nonlinear data based on a large model, characterized in that: The device comprises: A data retrieval module is used to select a node in the initial graph structure to input the user's request data, and use the RAG system to search and generate a retrieval result of the request data; wherein each node represents a dialogue round between the user input and the large model output; A data generation module, configured to generate reply data of the request data using a large model based on the search result and the request data; A path generation module, used to write the request data, the reply data and the search result into a new node, determine the nearest node associated with the new node among the existing nodes, use the nearest node as the parent node of the new node, and form a branch path; The node storage module is used to store new nodes in the database; wherein all branch paths form a tree structure.

Images