Database construction method and device for nonlinear data based on large model

Abstract

The application discloses a database construction method and device based on large model nonlinear data, wherein the method comprises the following steps: selecting the request data of the node input user in the initial graph structure, searching and generating the search result of the request data by using the RAG system, and generating the reply data of the request data by using the large model; writing the request data, the reply data and the search result into a new node, determining the nearest node associated with the new node in the existing node, taking the nearest node as the parent node of the new node, and forming a branch path with a topic; storing the new node to the database; wherein all branch paths form a tree structure. The application stores the database in a tree structure with multiple branches, so that each dialogue node has a traceable parent-child relationship, and the user can branch out a new dialogue path from any historical node. The user can freely select different topic paths for dialogue, so that the large model dialogue context content is more accurate.

Description

Technical Field

[0001] This application relates to the field of assisted retrieval technology, and in particular to a database construction method and apparatus based on large-scale nonlinear data. Background Technology

[0002] Currently, many AI tools rely primarily on text prompts for linear, one-way communication with users. If users want to make precise modifications to an existing design in later iterations, they must re-describe every detail in text. This purely text-based interaction is not intuitive enough. Users often need to input lengthy instructions to help the large Language Model (LLM) understand complex changes; otherwise, the LLM may produce results that do not meet expectations. This leads to repeated, tedious explanations from the user, resulting in excessively high communication costs between the user and the LLM, impacting iteration efficiency and accuracy.

[0003] Furthermore, existing intelligent interaction tools do not support non-linear iterative interaction with LLM. "Non-linear iteration" refers to the fact that the multi-turn interaction process between the user and the LLM does not always proceed in a linear time sequence, but often requires "backtracking to previous states" or "exploring parallel branches." This means that users may want to readjust, partially modify, or switch between multiple branch results without disrupting the existing dialogue context. Existing tools are primarily linear chat tools and cannot effectively support this kind of multi-branch, multi-path dynamic dialogue management. When a non-linear context is forcibly embedded in a linear dialogue history, it is difficult for the user to clearly tell the LLM which previous intermediate product to use as a new starting point, and the LLM also struggles to correctly extract relevant information from a long and complex dialogue history for generation. This makes it difficult for users to efficiently manage the context in multi-turn, multi-branch LLM interactions. Incomplete context construction leads to model misunderstandings, or simply overlaying linear historical records or relying on a single RAG retrieval results in redundant, lost, or incomplete contextual information, affecting the accuracy of the LLM's responses in complex scenarios. Summary of the Invention

[0004] This disclosure provides a method, apparatus, device, and storage medium for constructing a database based on large-scale nonlinear data, to at least address the aforementioned technical data requests existing in the prior art.

[0005] According to a first aspect of this application, a method for constructing a database based on nonlinear data from a large model is provided, the method comprising:

[0006] In the initial graph structure, nodes are selected to input the user's request data, and the RAG system is used to retrieve and generate the retrieval results of the request data; wherein, each node represents one dialogue turn between user input and large model output;

[0007] Based on the search results and the request data, a large model is used to generate response data for the request data;

[0008] Write the request data, the response data, and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path;

[0009] The new node is stored in the database; all branch paths form a tree structure.

[0010] In one possible implementation, each node further includes:

[0011] Unique identifier, reference relationship with the parent node, timestamp, and business tag;

[0012] The search results include document ID, summary information, source information, and embedding vector ID.

[0013] In one possible implementation, generating retrieval results for the requested data using the RAG system includes:

[0014] The associated parent node is determined based on the user's request data; the associated parent node is one whose request data, response data, and search results are similar to the request data.

[0015] Calculate the similarity between the request data and the request data, response data, and search results in the parent node;

[0016] The text fragments are determined as the retrieval results for the user's requested data based on the similarity.

[0017] In one possible implementation, generating response data using a large model for the request data includes:

[0018] Retrieve information about the selected node and its parent node link from the database;

[0019] Traverse the parent node link upwards from the selected node to obtain a preset number of parent nodes or reach the required number of dialogue data.

[0020] Based on the preset number of parent nodes or the amount of dialogue data reached, extract the user's historical input data and the historical response data of the large model to generate historical context;

[0021] The Promp text is constructed based on the user's request data, historical context, and selected node information.

[0022] The Promp text is input into the large model to generate response data.

[0023] In one possible implementation, before constructing the Promp text, the following is also included:

[0024] Data cleaning is performed on the user's request data, historical context, and selected node information.

[0025] In one possible implementation, after traversing the parent node links upwards from the selected node, the process further includes...

[0026] When information is missing, the RAG system is invoked to obtain supplementary data.

[0027] In one possible implementation, determining the nearest node among existing nodes associated with the new node includes:

[0028] Determine whether the parent node of the new node is a related node in this dialogue;

[0029] If the parent node is not an associated node of the new node, then the nearest node associated with the new node among the remaining existing nodes is re-determined.

[0030] In one possible implementation, before writing a new node, the method further includes:

[0031] The large model validates the user's request data and the information in the previous node. If the validation result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

[0032] In one possible implementation, the database is a graph database or a database for storing documents.

[0033] According to a second aspect of this application, a database construction apparatus for nonlinear data based on a large model is provided, comprising:

[0034] The data retrieval module is used to select nodes in the initial graph structure to input user request data, and use the RAG system to retrieve and generate retrieval results for the request data; wherein, each node represents a dialogue turn between user input and large model output;

[0035] The data generation module is used to generate response data for the request data based on the search results and the request data using a large model;

[0036] The path generation module is used to write the request data, the response data, and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path.

[0037] The node storage module is used to store new nodes in the database; all branch paths form a tree structure.

[0038] According to a third aspect of this application, an electronic device is provided, comprising:

[0039] At least one processor; and

[0040] A memory communicatively connected to the at least one processor; wherein,

[0041] The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in this application.

[0042] According to a fourth aspect of this application, a non-transitory computer-readable storage medium is provided storing computer instructions for causing the computer to perform the methods described in this application.

[0043] According to a fifth aspect of this application, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method described in this application.

[0044] Using the technical solution of this application, users can select different topic paths from the graph data structure to obtain the selected node and its associated historical context. Combined with real-time retrieval enhancement, RAG retrieval results are generated, and historical information is integrated with new retrieval content to form an accurate and complete context input to the large model LLM. This solves the problem of information loss and redundancy in complex dialogues and ensures that LLM provides high-quality and accurate answers in branching or backtracking scenarios.

[0045] It should be understood that the data described in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0046] The above and other objects, features, and advantages of exemplary embodiments of this application will become readily apparent from the following detailed description taken in conjunction with the accompanying drawings. Several embodiments of this application are illustrated in the drawings by way of example and not limitation, in which:

[0047] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0048] Figure 1 This illustration shows the implementation flow of the database construction method for nonlinear data based on a large model in an embodiment of this application. Figure 1 ;

[0049] Figure 2 This illustration shows the implementation flow of the database construction method for nonlinear data based on a large model in an embodiment of this application. Figure 2 ;

[0050] Figure 3 This paper shows a structural block diagram of a database construction device based on large-scale nonlinear data in an embodiment of this application.

[0051] Figure 4 A schematic diagram of the composition structure of the electronic device in an embodiment of this application is shown. Detailed Implementation

[0052] To make the objectives, features, and advantages of this application more apparent and understandable, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] The following description, in conjunction with the accompanying drawings, introduces a database construction method and apparatus based on large-scale nonlinear data provided in this application.

[0055] like Figure 1 As shown, this application provides a database construction method for nonlinear data based on a large model, the method comprising:

[0056] S101, Select nodes in the initial graph structure to input user request data, and use the RAG system to retrieve and generate retrieval results for the request data; wherein, each node represents one dialogue turn between user input and large model output;

[0057] It is understandable that the initial graph structure can be a linear structure formed by a dialogue, or it can be a pre-stored non-linear structure, such as a tree structure. The initial graph structure includes multiple nodes, and the user can select one of the nodes through interface operations to input the user's request data, such as a question.

[0058] Each node represents a dialogue turn (including user input, search results, and LLM output).

[0059] When a user needs to make a new request, they can select a node through interface buttons to indicate that a new question should begin from that node, enabling non-linear iteration. It's understood that the node selected by the user through these button operations is relevant to their new request. For example, if the conversation contains topics related to cars or travel, the user can choose the node with the relevant topic for further retrieval.

[0060] A Retrieval Enhancement (RAG) system is a technique that combines a Large Language Model (LLM) with external knowledge sources to enhance the capabilities of the LLM, enabling it to generate coherent and context-relevant responses. The RAG system provided in this application can be a Faiss vector retrieval system. The Faiss vector retrieval system can retrieve context related to the requested data from a database to obtain retrieval results.

[0061] S102, Based on the search results and the request data, generate response data for the request data using a large model;

[0062] Before the large model LLM responds, the RAG system is used to retrieve relevant context for the request data. After obtaining the relevant context and request data, the retrieved context and request data are then input into the large model to obtain the response data.

[0063] S103, write the request data, the response data and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path;

[0064] After the large model generates the response, the response data, request data, and search results are written together into a new node, and the parent_id of this node is set as the starting node of this dialogue (i.e., the upstream node selected by the user). It can be understood that this parent_id is the nearest node associated with the new node, thus forming a branching path. Node association indicates the similarity of data between two nodes.

[0065] like Figure 2 As shown, assuming the selected node is node 2 and the newly written node is node 5, if the nearest associated node of node 5 is node 2, then node 5 and node 2 form a branch path. However, if the nearest associated node of node 5 is found to be node 3, then node 5 and node 3 form a branch path.

[0066] S104, store the new node in the database; where all branch paths form a tree structure.

[0067] Finally, the newly formed nodes are stored in the database. This database also stores other nodes and the relationships between them. In other words, the nodes and their relationships form a tree structure stored in the database. When the RAG system queries the database for basic information about a node, it can not only retrieve the node's information but also its associated RAG context information and the parent node link (tracing back to the initial dialogue node). This provides the large model with contextual information for requesting data, enabling it to output more accurate answers.

[0068] The database provided in this application can be a graph database (such as Neo4j) or a document-oriented NoSQL database (such as MongoDB) to store dialogue nodes and their relationships. Among them, the graph database can directly store nodes and edges in a graph structure, which facilitates branch queries and hierarchical visualization.

[0069] This application provides a database construction method for large-scale model-based nonlinear data. It stores dialogue nodes and their parent-child relationships using a graph-like data structure, supporting nonlinear branching, backtracking, and version management. This overcomes the limitations of linear processes, enabling users to operate more flexibly and efficiently in complex iterative scenarios. By integrating historical context backtracking with RAG retrieval, historical nodes and their associated contexts are merged with new search results, providing LLM with complete and accurate context, significantly improving the coherence and accuracy of responses. When generating new nodes, the external context source of the RAG retrieval is recorded and associated with the node content, making the source of the response traceable. This facilitates user verification of the accuracy and basis of the results, improving credibility.

[0070] In some embodiments, each node further includes:

[0071] Unique identifier, reference relationship with the parent node, timestamp, and business tag;

[0072] The search results include document ID, summary information, source information, and embedding vector ID.

[0073] Understandably, each node in this application also includes: node_id: a unique identifier; parent_id or parents: a reference to the parent node (supporting single or multiple parent nodes); timestamp: a timestamp used for display along a timeline; user_input: the user's request content; model_output: the model's response content; rag_contexts: the RAG search results used in this round of responses, including document ID, summary information, and embedding vector references; and metadata: business tags, including topics, domains, and version numbers.

[0074] It should be noted that the timeline displays the chronological order of the nodes vertically, such as... Figure 2 As shown, branches are represented by expandable sidelines or dangling nodes. When a user clicks a node, the frontend requests the backend API to obtain upstream and downstream information and RAG fragments for that node, displaying them as a pop-up or sidebar. When the user confirms that they want to ask a new question based on this node, the frontend sends the node_id and the user's input to the backend, triggering context generation and the LLM call process.

[0075] In some embodiments, the step of using the RAG system to retrieve the search results for the requested data includes:

[0076] The associated parent node is determined based on the user's request data; the associated parent node is one whose request data, response data, and search results are similar to the request data.

[0077] Calculate the similarity between the request data and the request data, response data, and search results in the parent node;

[0078] The text fragments are determined as the retrieval results for the user's requested data based on the similarity.

[0079] Specifically, in this application, after a user submits a new request, the request, along with the selected parent node context (references to user_input, model_output, and rag_contexts), is first sent to the Faiss vector retrieval system. The Faiss vector retrieval system calculates the similarity between the new request data and the information in the parent node, and selects the most relevant text fragments as the RAG context, which is the retrieval result, based on the similarity ranking.

[0080] It should be noted that the context information retrieved by the RAG system is stored in the `rag_contexts` field, including the recalled document ID, content, source information, and optional embedded vector ID. In this application, when a new node is written to the database, the RAG search results are also recorded, facilitating future retrospective analysis of which documents were referenced by the answer.

[0081] This application records the external knowledge fragments used each time RAG retrieval is invoked when a new node is generated, and associates them with historical dialogue nodes to form a traceable contextual reference chain. This enables the verifiability and traceability of LLM response results, helping users to check the source and logic, and improving reliability.

[0082] In some embodiments, generating response data using a large model for the request data includes:

[0083] Retrieve information about the selected node and its parent node link from the database;

[0084] Traverse the parent node link upwards from the selected node to obtain a preset number of parent nodes or reach the required number of dialogue data.

[0085] Based on the preset number of parent nodes or the amount of dialogue data reached, extract the user's historical input data and the historical response data of the large model to generate historical context;

[0086] The Promp text is constructed based on the user's request data, historical context, and selected node information.

[0087] The Promp text is input into the large model to generate response data.

[0088] In some embodiments, after traversing the parent node link upwards from the selected node, the process further includes...

[0089] When information is missing, the RAG system is invoked to obtain supplementary data.

[0090] Specifically, this application queries the database for the node's basic information (user_input, model_output), its associated RAG context information (i.e., retrieval information), and the parent node chain (tracing upwards all the way to the initial dialogue node). Then, starting from the selected node, it traverses the parent node chain upwards until a certain number of upstream nodes are found or the dialogue history limit is reached. Necessary user input and model output fragments are extracted from these nodes to form the historical context. The RAG system is then used for retrieval. Specifically, if a large semantic span or missing information is found during the upward traversal, RAG can be invoked again to obtain supplementary information.

[0091] It is understandable that in this application, querying the basic information of a node, its associated RAG context information, and its parent node chain from the database can be achieved through an index structure. Specifically, a dialogue node table is set up, which includes node_id, parent_id, timestamp, user_input, model_output, rag_contexts, and metadata. An index is created for parent_id to facilitate fast querying of child nodes and tracing back the parent node chain.

[0092] This application can extract the selected node and its associated historical context from a graph data structure, and combine it with real-time RAG (Retrieval Enhancement Generation) retrieval results to integrate historical information with new retrieval content, forming an accurate and complete context input for the LLM. This application effectively solves the problems of information loss and redundancy in complex dialogues through a context integration mechanism, ensuring that the LLM provides high-quality and accurate answers in branching or backtracking scenarios.

[0093] In some embodiments, the following steps are included before constructing the Promp text:

[0094] Data cleaning is performed on the user's request data, historical context, and selected node information.

[0095] Understandably, when constructing the final prompt for the LLM, the user's current input, several refined historical dialogues (important summaries of the selected node and its parent nodes), and the RAG retrieval fragment of the selected node are all fed to the LLM as prompt input. Then, simple rules (such as retaining only key information, only retaining the context of the previous round and the starting point of the branch) can reduce contextual redundancy. If there are too many branches, users can also manually select the upstream nodes they want to retain on the UI to help the system select the correct context. Ultimately, when generating new answers, the large-scale LLM model, based on the given prompt (containing the selected node's path context and RAG fragment), can still accurately provide answers according to the user's intent even in complex nonlinear iterations.

[0096] Specifically, when the relevant context is retrieved, the node_id and the newly entered request data of the user are selected. Based on the request data, the parent node chain is traced back and the necessary model_output fragments, user_input fragments and rag_contexts references are collected. If the information is insufficient, RAG is called to retrieve it again. Then, the collected information is summarized and refined to reduce the length of the Prompt. Finally, the final context Prompt is returned to the LLM. The Prompt is in JSON or text format.

[0097] In some embodiments, determining the nearest node among existing nodes associated with the new node includes:

[0098] Determine whether the parent node of the new node is a related node in this dialogue;

[0099] If the parent node is not an associated node of the new node, then the nearest node associated with the new node among the remaining existing nodes is re-determined.

[0100] Understandably, after the large model generates the response data, it writes the response (along with the user request and RAG retrieval information) into a new node and sets the parent_id of this node to the starting node of this conversation (i.e., the upstream node selected by the user through the interface). If the upstream node selected by the user when creating a new conversation is not the most recent node (it may be an old node in an earlier conversation), then this new node will take that old node as its parent node, forming a branch path.

[0101] This application uses a graph data structure to store multi-turn dialogue nodes and their parent-child relationships, supporting users to backtrack, branch, and create versions at any historical node, thereby achieving non-linear, multi-path dialogue management.

[0102] In some embodiments, the process of writing a new node also includes:

[0103] The large model validates the user's request data and the information in the previous node. If the validation result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

[0104] In this application, before creating a new node, LLM can be used to perform simple validation on the user's new input and the topic of the upstream node. If the context is found to be irrelevant, the user is prompted to add more information. After a successful LLM call, the result along with the context is stored to ensure the traceability of subsequent backtracking.

[0105] As a specific implementation, the user selects an existing node on the interface, which serves as a branch. The user then enters a new question into the existing node. The RAG system retrieves the context from the new question and the selected node. The context and request data are then input into the large model (LLM). The large model (LLM) generates a new answer based on the context. This answer is then added to the database as a new node, and the parent_id is set to the node_id of the selected node.

[0106] Existing technologies often employ linear, multi-turn dialogue history methods, limiting users to iterative modifications by only proceeding chronologically. When users need to develop a new solution from an earlier point or compare multiple parallel versions, effective methods are often lacking. The technical solution provided in this application, however, stores the dialogue process using a graph data structure, giving each dialogue node a traceable parent-child relationship, allowing users to branch out new dialogue paths from any historical node. Users can freely choose different topic paths for dialogue, making the context of large-scale model dialogues more accurate, eliminating the need for repeated searches through lengthy linear histories. Through this technical solution, users can more intuitively compare multiple versions of results in complex creation and design scenarios and flexibly choose branch paths, thereby significantly improving user experience and work efficiency.

[0107] Secondly, traditional solutions simply overlay historical inputs and outputs in multi-turn dialogues. When the scenario is complex or the dialogue history is lengthy, LLM struggles to extract the correct contextual information. While other existing solutions include intent recognition and contextual supplementation, they are usually one-way, relying either solely on the latest information retrieved from the RAG database or only on linear historical records. This application, however, organically integrates the user-selected historical node context (including key points from past dialogues and previously used RAG data) with the results of the new round of RAG retrieval before providing input to the LLM. Through centralized processing, the historical and new search content are provided to the LLM together. Therefore, with this technical solution, the LLM can obtain complete, concise, and accurate contextual support when answering, including historical branch information and the latest data retrieval results. Because the context is more targeted and logically connected, the accuracy, coherence, and professionalism of the answer are significantly improved, reducing lengthy and repetitive descriptions and lowering the user's expression costs.

[0108] Furthermore, in traditional text-based scenarios, users often need to repeat a large amount of existing information or reconstruct the context through lengthy descriptions to help LLM understand complex modification requirements. This application, however, utilizes a graph data structure and RAG retrieval to automatically extract the context and information related to a selected historical node, eliminating the need for users to re-describe the background in detail. Users only need to add new changes or questions, and the system can integrate existing information. Therefore, this technical solution significantly reduces the communication burden on users, eliminating the need to describe requirements from scratch each time, thereby improving iteration efficiency, reducing error rates, and enabling users to achieve their goals faster.

[0109] In addition, this application employs a timeline and graph-structured visual interface to intuitively display the dialogue branch path, allowing users to select historical nodes, view RAG context, and quickly backtrack or create branches through click operations. Importance: This mechanism significantly reduces the complexity of user operations, improves the system's usability and intuitiveness, and meets the interaction needs of real-world usage scenarios.

[0110] like Figure 3 As shown in the figure, this application provides a database construction apparatus for nonlinear data based on a large model, the apparatus comprising:

[0111] The data retrieval module 301 is used to select nodes in the initial graph structure to input user request data, and use the RAG system to retrieve and generate retrieval results for the request data; wherein, each node represents one dialogue turn between user input and large model output;

[0112] Data generation module 302 is used to generate response data for the request data based on the search results and the request data using a large model;

[0113] The path generation module 303 is used to write the request data, the response data and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path;

[0114] The node storage module 304 is used to store new nodes in the database; all branch paths form a tree structure.

[0115] The working principle of the database construction device based on large-scale model nonlinear data provided in this application is as follows: the data retrieval module 301 selects nodes in the initial graph structure to input user request data, and uses the RAG system to retrieve and generate retrieval results for the request data; wherein, each node represents a dialogue turn between user input and large-scale model output; the data generation module 302 generates response data for the request data based on the retrieval results and the request data using the large-scale model; the path generation module 303 writes the request data, the response data, and the retrieval results into a new node, determines the nearest node associated with the new node among the existing nodes, and uses the nearest node as the parent node of the new node to form a branch path; the node storage module 304 stores the new node in the database; wherein, all branch paths form a tree structure.

[0116] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0117] The electronic device includes at least one processor and a memory communicatively connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed, enable the at least one processor to perform the database construction method for large-model-based nonlinear data as described in this application. The computer instructions are used to cause the computer to execute the database construction method for large-model-based nonlinear data as described in this application.

[0118] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the database construction method for nonlinear data based on a large model of this application.

[0119] Figure 4A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of this application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0120] like Figure 4 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0121] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0122] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as a database construction method for large-model-based nonlinear data. For example, in some embodiments, the database construction method for large-model-based nonlinear data can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the database construction method for large-model-based nonlinear data described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to perform a database construction method based on nonlinear data of a large model.

[0123] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0124] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0125] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the data described above. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the data described above.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0127] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0128] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A database construction method for nonlinear data based on a large model, characterized in that, The method includes: In the initial graph structure, nodes are selected to input the user's request data, and the RAG system is used to retrieve and generate the retrieval results of the request data; wherein, each node represents one dialogue turn between user input and large model output; Based on the search results and the request data, a large model is used to generate response data for the request data; Write the request data, the response data, and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path; Determining the nearest node among existing nodes that is associated with the new node includes: Determine whether the parent node of the new node is a related node in this dialogue; If the parent node is not an associated node of the new node, then the nearest node associated with the new node among the remaining existing nodes is re-determined. The new node is stored in the database; all branch paths form a tree structure. Before writing the new node, the following is also included: The large model validates the user's request data and the information in the previous node. If the validation result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

2. The method according to claim 1, characterized in that, Each node also includes: Unique identifier, reference relationship with the parent node, timestamp, and business tag; The search results include document ID, summary information, source information, and embedding vector ID.

3. The method according to claim 1, characterized in that, The retrieval results generated using the RAG system to retrieve the requested data include: The associated parent node is determined based on the user's request data; the associated parent node is one whose request data, response data, and search results are similar to the request data. Calculate the similarity between the request data and the request data, response data, and search results in the parent node; The text fragments are determined as the retrieval results for the user's requested data based on the similarity.

4. The method according to claim 1, characterized in that, The process of generating response data using a large model includes: Retrieve information about the selected node and its parent node link from the database; Traverse the parent node link upwards from the selected node to obtain a preset number of parent nodes or reach the required number of dialogue data. Based on the preset number of parent nodes or the amount of dialogue data reached, extract the user's historical input data and the historical response data of the large model to generate historical context; The Promp text is constructed based on the user's request data, historical context, and selected node information. The Promp text is input into the large model to generate response data.

5. The method according to claim 4, characterized in that, Before building the Promp text, the following is also included: Data cleaning is performed on the user's request data, historical context, and selected node information.

6. The method according to claim 4, characterized in that, After traversing the parent node links upwards from the selected node, the process also includes... When information is missing, the RAG system is invoked to obtain supplementary data.

7. The method according to claim 1, characterized in that, The database is a graph database or a database used to store documents.

8. A database construction device for nonlinear data based on a large model, characterized in that, The device includes: The data retrieval module is used to select nodes in the initial graph structure to input user request data, and use the RAG system to retrieve and generate retrieval results for the request data; wherein, each node represents a dialogue turn between user input and large model output; The data generation module is used to generate response data for the request data based on the search results and the request data using a large model; The path generation module is used to write the request data, the response data, and the search results into a new node, determine the nearest node among the existing nodes that is associated with the new node, and use the nearest node as the parent node of the new node to form a branch path. Determining the nearest node among existing nodes that is associated with the new node includes: Determine whether the parent node of the new node is a related node in this dialogue; If the parent node is not an associated node of the new node, then the nearest node associated with the new node among the remaining existing nodes is re-determined. The node storage module is used to store new nodes in the database; all branch paths form a tree structure. Before writing the new node, the following is also included: The large model validates the user's request data and the information in the previous node. If the validation result shows that the context is irrelevant, a prompt message is sent to the user to add more information.

Images