Data Retrieval Enhancement Generation Method, Apparatus and Device for Data Space

Abstract

The present application provides a data retrieval enhanced generation method, apparatus and device for a data space, which relates to the technical field of data generation and aims to improve the accuracy of retrieval tasks in the data space of the data networking. This method is applied to a data retrieval enhanced generation system for the data space of the data networking. The system includes a proxy node and multiple nodes. The method includes: receiving a query request in the form of natural language through the proxy node and determining the query intention according to the query request; matching the query intention with the query capabilities of each node, and determining multiple candidate nodes according to the matching degree. The node represents a data source in the data space of the data networking, and the query capability represents the query content and supported fields provided by the node; sending the query request to the multiple candidate nodes for retrieval generation to obtain query results of the multiple candidate nodes; and the proxy node generating a final query result according to the query results of the multiple candidate nodes.

Description

Technical Field

[0001] This application relates to the technical field of data generation, and particularly to a data retrieval enhanced generation method, device, and equipment for a data space. Background Art

[0002] In the data space constructed based on the data Internet of Things, data has multi-source heterogeneity, its storage is scattered and cannot be processed by centralized aggregation; and there is a weak coupling relationship between data and applications, and users cannot predict the specific data content in the data space before retrieval. Currently, the query of the data space mainly relies on keyword retrieval, and this method has significant semantic limitations: First, keywords are difficult to accurately express the user's intention, and the system needs to convert them into potential structured queries, but the conversion process has uncertainty; second, the query results usually contain a large amount of irrelevant data, and the system needs to evaluate the results to screen out effective information. In addition, due to the lack of a strict data schema in the data space, the relationships between data need to be dynamically established, resulting in the query results may only be sub-optimal or approximate solutions.

[0003] The metadata stored in each node in the data space can be publicly accessed, while the entity data needs to be authorized to obtain and is prohibited from freely flowing across nodes. The related technology is to perform retrieval enhanced generation for a single node of the data Internet of Things, that is, the user submits a metadata query request to a single node, and the node returns the metadata results it stores; if a global query is required, it depends on the top-level node (the top-level node aggregates all the metadata). However, the information contained in the metadata is limited and it is difficult to comprehensively reflect the content of the data entity. Especially when it comes to multiple data from multiple sources, it will result in a low query accuracy rate. Therefore, how to improve the accuracy of the retrieval task in the data space of the data Internet of Things is a technical problem to be solved urgently. Summary of the Invention

[0004] In view of the above problems, the embodiments of this application provide a data retrieval enhanced generation method, device, and equipment for a data space, so as to overcome the above problems or at least partially solve the above problems.

[0005] In the first aspect of the embodiments of this application, a data retrieval enhanced generation method for a data space is disclosed, which is applied to a data retrieval enhanced generation system of the data Internet of Things. The system includes a proxy node and multiple nodes. The method includes:

[0006] Receiving a query request in the form of natural language through the proxy node, and determining the query intention according to the query request;

[0007] Matching the query intention with the query capabilities of each node, and determining multiple candidate nodes according to the matching degree. The node represents a data source in the data space of the data Internet of Things, and the query capability represents the query content and support field provided by the node;

[0008] Send the query request to the multiple candidate nodes for retrieval generation to obtain the query results of the multiple candidate nodes;

[0009] The proxy node generates a final query result according to the query results of the multiple candidate nodes.

[0010] Optionally, the method further includes:

[0011] The node generates a capability summary of the node according to the retrieval enhancement scheme deployed by itself, and the capability summary describes the query content and supported fields provided by the node in the form of natural language;

[0012] Register the capability summary and the basic node information to the proxy node;

[0013] Match the query intent with the query capabilities of each node, including:

[0014] The proxy node determines the query capabilities of each node according to the registered capability summary and the basic node information, and matches the query intent with the query capabilities of each node.

[0015] Optionally, the method further includes:

[0016] The node constructs a vector library according to the retrieval enhancement scheme deployed by itself, and the vector library represents the data space of the node;

[0017] Sending the query request to the multiple candidate nodes for retrieval generation to obtain the query results of the multiple candidate nodes includes:

[0018] The proxy node processes the query request into a target query request and sends it to the multiple candidate nodes, and the target query request includes a query description and keywords in the form of natural language;

[0019] Each candidate node performs retrieval generation based on the retrieval enhancement scheme deployed by itself, the target query request, and the vector library to obtain the query results of the multiple candidate nodes.

[0020] Optionally, each candidate node performs retrieval generation based on the retrieval enhancement scheme deployed by itself, the target query request, and the vector library to obtain the query results of the multiple candidate nodes, including:

[0021] When the retrieval enhancement scheme deployed by the candidate node itself has the ability of retrieval enhancement generation, retrieve in the vector library according to the query description of the target query request, and generate the query result of the candidate node according to the large model;

[0022] When the retrieval enhancement solution deployed on the candidate node itself does not have the ability of retrieval enhanced generation, perform metadata retrieval in the vector library according to the keywords in the target query request to obtain the query result of the candidate node.

[0023] Perform retrieval in the vector library according to the query description of the target query request, and generate the query result of the candidate node according to the large model, including:

[0024] Perform retrieval in the vector library according to the query description of the target query request to obtain the first target digital object, and generate the query result of the candidate node through the large model according to the first target digital object and the identifier of the first target digital object;

[0025] Perform metadata retrieval in the vector library according to the keywords in the target query request to obtain the query result of the candidate node, including:

[0026] Perform metadata retrieval in the vector library according to the keywords in the target query request to obtain the second target digital object, and obtain the query result of the candidate node according to the metadata of the second target digital object and the identifier of the second target digital object.

[0027] Optionally, match the query intent with the query capabilities of each node, and determine multiple candidate nodes according to the matching degree, including:

[0028] Match the query intent with the query capabilities of each node, and sort the nodes in descending order according to the matching degree to obtain the first sorting result;

[0029] Take the first N nodes in the first sorting result as candidate nodes, where N is a positive integer greater than 0.

[0030] Optionally, the proxy node generates the final query result according to the query results of the multiple candidate nodes, including:

[0031] Match the query result of each candidate node with the query request, and sort the query results of the candidate nodes in descending order according to the matching degree to obtain the second sorting result;

[0032] Take the query results of the first M candidate nodes in the second sorting result as the target query result, where M is a positive integer greater than 0;

[0033] Generate the final query result according to the target query result.

[0034] Optionally, generate the final query result according to the target query result, including:

[0035] Generate a final query result based on the target query result and the candidate nodes corresponding to the target query result.

[0036] In a second aspect of the embodiments of the present application, a data retrieval enhancement generation device for a data space is disclosed. The device includes:

[0037] An intent determination module, configured to receive a query request in natural language form and determine a query intent according to the query request;

[0038] A node matching module, configured to match the query intent with the query capabilities of each node, and determine a plurality of candidate nodes according to the matching degree. The node represents a data source in the data network data space, and the query capability represents the query content and supported fields provided by the node;

[0039] A retrieval generation module, configured to send the query request to the plurality of candidate nodes for retrieval generation to obtain query results of the plurality of candidate nodes;

[0040] A query generation module, configured to generate a final query result according to the query results of the plurality of candidate nodes.

[0041] In a third aspect of the embodiments of the present application, an electronic device is disclosed, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the steps of the data retrieval enhancement generation method for a data space described in the first aspect of the embodiments of the present application are implemented.

[0042] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is disclosed, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data retrieval enhancement generation method for a data space described in the first aspect of the embodiments of the present application are implemented.

[0043] In a fifth aspect of the embodiments of the present application, a computer program product is disclosed, including a computer program. When the computer program is executed by a processor, the steps of the data retrieval enhancement generation method for a data space described in the first aspect of the embodiments of the present application are implemented.

[0044] The embodiments of the present application have the following advantages:

[0045] In the embodiments of the present application, by receiving a query request in the form of natural language through a proxy node and parsing the query intention, the semantic limitations of keyword retrieval are overcome, the user needs can be captured more accurately, and the query deviation caused by vague keyword expression can be reduced; the query intention is matched with the query capabilities of each node, and multiple candidate nodes are determined according to the matching degree, avoiding blindly querying all nodes, reducing the computational overhead, and improving the retrieval efficiency; the query request is sent to multiple candidate nodes for retrieval generation, the query results of multiple candidate nodes are obtained, and the query results of multiple candidate nodes are integrated through the proxy node to generate the final query result, which can cover a wider data source, reduce the query omission caused by the data limitation of a single node, and improve the comprehensiveness and accuracy of the results.

[0046] In this way, by setting up a proxy node and nodes in the data space of the data network to achieve two-level content generation, since the final query result is generated by the proxy node based on the query results of multiple candidate nodes, the transmission of raw data between nodes is avoided, ensuring the data security of each node in the data space of the data network, and at the same time ensuring the accuracy of the final query result. Therefore, a cross-node retrieval enhancement generation in the data space of the data network is realized, and the accuracy of the retrieval task in the data space of the data network is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0048] Figure 1 is a schematic diagram of an existing retrieval enhancement system for the data space of the data network;

[0049] Figure 2 is a schematic diagram of a retrieval enhancement system for the data space of the data network provided by the embodiments of the present application;

[0050] Figure 3 is a flowchart of the steps of a data retrieval enhancement generation method for the data space provided by the embodiments of the present application;

[0051] Figure 4 is a flowchart of the steps of another data retrieval enhancement generation method for the data space provided by the embodiments of the present application;

[0052] Figure 5 is a schematic diagram of the structure of a data retrieval enhancement generation device for the data space provided by the embodiments of the present application;

[0053] Figure 6It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0054] To make the above objects, features, and advantages of the present application more obvious and understandable, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts belong to the scope of protection of the present application.

[0055] To better understand the technical solutions of the present application, the technical concepts involved in the present application will be described.

[0056] Data Internet of Things: Based on software-defined, through an open software architecture centered on data and a standardized interoperability protocol, various heterogeneous data platforms and systems are connected to form a "virtual / data" network on top of the "physical / machine" Internet. Under the architecture of the Data Internet of Things, a data space of "data interconnection, on-demand scheduling, intra-domain autonomy, and inter-domain collaboration" can be formed.

[0057] Data space: A data space is a collection of data related to a subject and its relationships. The data in the data space comes from multiple different data sources, and the data formats are diverse. At the same time, the relationships between the data are complex, dynamic, and evolving. Through the Data Internet of Things technology, a data space can be constructed, in which the data has a unified data model and supports a unified interoperability protocol.

[0058] Retrieval-Augmented Generation (RAG): It refers to the technology of referring to private or authoritative knowledge bases outside the training data to assist the output of large models. Large Language Models (LLMs) are trained with massive amounts of data and use billions of parameters to generate raw outputs for tasks such as answering questions, translating languages, and completing sentences. Based on the already powerful general task execution capabilities of LLMs, RAG extends them to be able to access the internal knowledge bases of specific domains or organizations, all without retraining the model. Therefore, Retrieval-Augmented Generation is a cost-effective method to improve the output of LLMs, enabling them to maintain relevance, accuracy, and practicality in various scenarios.

[0059] Digital Object Architecture: The digital object architecture is a typical technical route of the Data Internet of Things. The digital object architecture unifies and standardizes the data resources of the Internet through digital objects, uses the digital object interface protocol and the digital object identifier resolution protocol to standardize data interaction behaviors, and forms an open software architecture based on the three core systems of digital objects, realizing the interconnection and interoperability of heterogeneous, geographically dispersed, and owner-different data, and forming a data space with the characteristics of "data interconnection, on-demand scheduling, intra-domain autonomy, and inter-domain collaboration". The three elements of digital objects each undertake part of the tasks: the identifier, as the only globally unique identifier of the digital object, plays a positioning role; the data entity plays the role of digitalizing and persisting the original information of the data resources, and is the resource content actually represented by the digital object; the metadata is a collection of descriptive information about data items, and the metadata is used in the actual retrieval process to retrieve digital objects under specific description conditions.

[0060] The digital object architecture is a brand-new software architecture system centered on data. For the access to data objects, the currently more applicable solution is to use metadata as the search domain set, and through the input of the actual query words and short sentences by the user, perform some natural language matching with the metadata name, description, etc. This matching method segments the input with the partial field control model of the metadata, and uses the search middleware to perform text-level matching to match the predefined industry, business, and other metadata content.

[0061] As Figure 1 shown, Figure 1 is a schematic diagram of an existing enhanced system for retrieving the data space of the Data Internet of Things. The metadata stored in each node in the data space can be publicly accessed, while the entity data requires authorization to access, and the entity data cannot flow freely between nodes. The existing retrieval process: The user queries the metadata from a certain node in the Data Internet of Things, and the node returns the query result according to the metadata stored therein; if global data needs to be queried, it is necessary to query through the top-level node (the top-level node aggregates all the metadata). However, due to the limited information contained in the metadata, it is difficult to comprehensively reflect the content of the data entity, especially when it comes to multiple data from multiple sources, which will result in a low accuracy rate of the query results.

[0062] The related technology performs retrieval enhancement generation for individual nodes in the data web, without considering the data query scenarios in the data space formed by multiple nodes. Therefore, this single-node retrieval enhancement generation scheme based on text-based metadata matching, although it can complete the task of retrieving digital objects using metadata, still has some problems that can be improved: (1) Limited metadata information restricts the retrieval effect: Currently, users can add industry metadata, business metadata, etc. to assist actual users in retrieval. This method has drawbacks: a. There may be a cognitive gap between the users who define the metadata and the users who use the data retrieval function, and the users cannot use the predefined metadata for matching retrieval well; b. Other methods for generating additional metadata can be supplemented, but this method can only be input manually, lacking an efficient metadata generation method that can be retrieved; c. Combining with the current development trend of natural language processing, there are optimization points for introducing large models to improve the overall performance in the existing methods. (2) Unable to handle cross-node retrieval enhancement problems: The current digital object retrieval enhancement generation methods only deal with the data in a single node. When multiple nodes' data are needed, since data entities usually cannot be aggregated to a single node for processing, the existing digital object retrieval enhancement generation methods cannot function.

[0063] Since the information carried by metadata is relatively limited and metadata has strong domain characteristics, in order to further improve the accuracy of queries, it is necessary to consider using entity data. With the emergence of large model technology, the RAG technology has been proposed to improve the presentation form of retrieval. However, there are certain problems in directly applying the RAG technology to the data web scenario. When multiple data sources are involved, this method needs to aggregate multiple matching data to the large model, and then the large model generates content based on the matching data. Since entity data usually cannot be directly aggregated, this brings difficulties to the application of the RAG technology.

[0064] To overcome the limitations of the related technology, the embodiments of this application provide a data retrieval enhancement generation method for the data space, which can solve the following problems: (1) Solve the cross-node retrieval enhancement generation problem in the data space. Since the data is scattered in different entities, and each data entity has its own control requirements for data usage, when performing retrieval enhancement generation, the relevant raw data retrieved from the data controlled by each entity cannot be directly aggregated to a certain entity and then enhanced generated. The embodiments of this application perform retrieval enhancement generation on the data in the data web space without aggregating the raw data, which can achieve efficient retrieval of the data in the data space while protecting the users' effective control over their own data.

[0065] (2)Solve the problem of multi-level data traceability during the retrieval process. Since the data in the data space comes from multiple data sources and has multiple different types, the association between data is weak, which causes difficulties in using the data. After introducing the retrieval enhancement generation function, due to the hallucination problem of the large model, the content fabricated out of thin air by the large model may exist in the returned results. Therefore, in order to improve the credibility of the generated results, it is necessary to trace the source of the returned results. Since there is a multi-level data generation process during the generation process, the embodiments of the present application implement a multi-level data traceability method to ensure that the generated content is well-founded.

[0066] The following will describe the data retrieval enhancement generation method for the data space provided by the embodiments of the present application with reference to the accompanying drawings.

[0067] The data retrieval enhancement generation method for the data space provided by the embodiments of the present application is applied to the data retrieval enhancement generation system of the data networking data space, as Figure 2 shown. Figure 2 FIG. is a schematic diagram of a data networking data space retrieval enhancement system provided by the embodiments of the present application. The system includes a proxy node and multiple nodes. Among them, large models are deployed on both the proxy node and each node. The node represents the data source in the data networking data space. Each node can generate a query result corresponding to the query request based on its own data. The proxy node can generate a final query result according to the query results from the nodes. This method does not require aggregating entity data and performs two-level content generation based on the proxy node and the nodes in the data space.

[0068] Referring to Figure 3 shown. Figure 3 FIG. is a flowchart of the steps of a data retrieval enhancement generation method for the data space provided by the embodiments of the present application. As Figure 3 shown, the data retrieval enhancement generation method for the data space may include steps S310 to S340:

[0069] Step S310: Receive a query request in the form of natural language through the proxy node and determine the query intent according to the query request.

[0070] In the embodiments of the present application, a user sends a query request in the form of natural language to an agent node. The agent node receives the query request in the form of natural language, and parses the query request through intent recognition to determine the query intent corresponding to the query request. In some embodiments, the agent node may use an intent recognition model to parse the query request. For example, the agent node may parse the query request based on a text classifier of BERT, where BERT refers to Bidirectional Encoder Representations from Transformers, a pre-trained natural language processing model based on the Transformer architecture.

[0071] Since the query request is provided in the form of natural language, it overcomes the semantic limitations of traditional keyword retrieval, can capture user needs more accurately, and reduce query deviation caused by ambiguous keyword expressions.

[0072] Step S320: Match the query intent with the query capabilities of each node, and determine multiple candidate nodes according to the matching degree. The node represents a data source in the digital networking data space, and the query capability represents the query content and supported fields provided by the node.

[0073] In the embodiments of the present application, considering that different data sources have different requirements for data preparation (segmentation), embedding models, and re-ranking algorithms, different nodes adopt their own independent data segmentation, embedding models, and re-ranking algorithms. That is, each node deploys its own independent retrieval enhancement generation scheme according to its own needs. Therefore, the query capabilities of each node may be different. Among them, the supported field of the node can be understood as the field or industry served by the node.

[0074] The agent node matches the query intent with the query capabilities (query content, supported fields) of each node, which can be achieved by calculating the semantic similarity between the query intent and the query capabilities of the node (for example, using cosine similarity to match keyword vectors). The higher the matching degree, the more the query capabilities of the node match the query intent. Multiple candidate nodes are determined according to the matching degree between the query intent and the query capabilities of each node. Among them, the candidate node represents a node whose query capabilities match the query intent.

[0075] In some embodiments, matching the query intent with the query capabilities of each node and determining multiple candidate nodes according to the matching degree specifically includes: matching the query intent with the query capabilities of each node, and sorting the nodes in descending order according to the matching degree to obtain a first sorting result; taking the first N nodes in the first sorting result as candidate nodes, where N is a positive integer greater than 0.

[0076] Among them, the first N nodes in the first sorting result can be understood as N nodes whose matching degree between the query intention and the query capabilities of the nodes is greater than or equal to the target matching degree threshold. For example, if there are 6 nodes, and the matching degrees between the query intention and the 6 nodes (node 1, node 2, node 3, node 4, node 5, node 6) are calculated as: 0.8, 0.3, 0.5, 0.9, 0.95, 0.7, then the nodes are sorted in descending order of the matching degree, that is, 0.95 (node 5), 0.9 (node 4), 0.8 (node 1), 0.7 (node 6), 0.5 (node 3), 0.3 (node 2). If the target matching degree threshold is 0.8, then the three nodes, namely node 1, node 4, and node 5, can be used as candidate nodes.

[0077] In this way, by matching the query intention with the query capabilities of each node and determining multiple candidate nodes according to the matching degree, blind querying of all nodes is avoided, the computational overhead is reduced, and the retrieval efficiency is improved.

[0078] Step S330: Send the query request to the multiple candidate nodes for retrieval generation to obtain the query results of the multiple candidate nodes.

[0079] In the embodiment of the present application, after the proxy node determines multiple candidate nodes, it sends the query request to the multiple candidate nodes. After each candidate node receives the query request, it performs retrieval generation based on the retrieval enhancement scheme deployed by itself to obtain the query results of the candidate nodes. Specifically, the candidate node performs local data retrieval according to the query request and generates a query result according to the retrieval result.

[0080] Step S340: The proxy node generates a final query result according to the query results of the multiple candidate nodes.

[0081] In the embodiment of the present application, after each candidate node generates a query result, it sends the query result to the proxy node. The proxy node generates a final query result based on the retrieval enhancement scheme deployed by itself according to the query results of the multiple candidate nodes.

[0082] Specifically, the proxy node generates a final query result according to the query results of the multiple candidate nodes, which may be to generate a final query result by combining indicators such as data relevance and authority, effectively filtering out noise data, providing a high-quality answer that better meets the user's needs, and avoiding the problem of returning sub-optimal or approximate solutions in traditional methods. In this way, by integrating the query results of multiple candidate nodes through the proxy node to generate a final query result, a wider range of data sources can be covered, the query omission caused by the data limitation of a single node can be reduced, and the comprehensiveness and accuracy of the results can be improved.

[0083] Adopting the technical solution of the embodiment of the present application, two-level content generation is achieved by setting up proxy nodes and nodes in the data space of the data networking. Since the final query result is generated by the proxy node based on the query results of multiple candidate nodes, the transmission of raw data between nodes is avoided, ensuring the data security of each node in the data space of the data networking, and at the same time ensuring the accuracy of the final query result. Therefore, a cross-node retrieval enhancement generation in the data space of the data networking is realized, improving the accuracy of the retrieval task in the data space of the data networking.

[0084] Combined with the above embodiments, in one implementation, the embodiment of the present application also provides a data retrieval enhancement generation method for the data space. In this method, in addition to the above steps, the following steps are further included:

[0085] Step A1: The node generates an ability summary of the node according to the retrieval enhancement scheme deployed by itself. The ability summary describes the query content and the supported fields provided by the node in the form of natural language; the ability summary and the basic information of the node are registered to the proxy node.

[0086] Step A2: The node constructs a vector library according to the retrieval enhancement scheme deployed by itself. The vector library represents the data space of the node.

[0087] In the embodiment of the present application, before retrieval, the proxy node and each node are initialized to complete data preparation and related registration work. Each node generates an ability summary of the node according to the retrieval enhancement scheme deployed by itself, and registers the ability summary and the basic information of the node to the proxy node. Thus, the proxy node can determine the query capabilities of each node and the basic information of each node according to the registered information. Among them, the basic information of the node can be the node address information, etc. At the same time, each node can also complete data accuracy and vector library construction according to the retrieval enhancement scheme deployed by itself. When subsequent nodes perform retrieval generation, they can perform vector retrieval based on the constructed vector library and generate query results according to the generation of the large model.

[0088] Further, the "matching the query intention with the query capabilities of each node" in the above step S320 specifically includes: the proxy node determines the query capabilities of each node according to the registered ability summary and the basic information of the node, and matches the query intention with the query capabilities of each node.

[0089] In the embodiment of the present application, when performing retrieval, after receiving a query request and determining the query intention, the proxy node can determine the query capabilities (query content, supported fields) of each node according to the registered ability summary and the basic information of the node, so as to determine multiple candidate nodes according to the matching degree between the query intention and the query capabilities of each node, avoiding blindly querying all nodes, reducing the calculation overhead, and improving the retrieval efficiency.

[0090] Further, the step of "sending the query request to the multiple candidate nodes for retrieval generation to obtain the query results of the multiple candidate nodes" in step S330 may specifically include sub-steps S330-1 to S330-2:

[0091] Step S330-1: The proxy node processes the query request into a target query request and sends it to the multiple candidate nodes. The target query request includes a query description in natural language form and keywords.

[0092] Step S330-2: Each candidate node performs retrieval generation based on its own deployed retrieval enhancement scheme, the target query request, and the vector library to obtain the query results of the multiple candidate nodes.

[0093] In the embodiment of the present application, the proxy node uses keyword technology to extract keywords from the query request to obtain keywords, and takes the keywords and the corresponding query description together as the target query request and sends it to the candidate nodes. After receiving the target query request, the candidate nodes perform retrieval generation according to their own deployed retrieval enhancement scheme and vector library to obtain the corresponding query results. Among them, the retrieval enhancement scheme deployed by the node itself may or may not have the ability of retrieval enhancement generation, and the query result generation methods with and without the ability of retrieval enhancement generation are different.

[0094] Specifically, the step of "each candidate node performs retrieval generation based on its own deployed retrieval enhancement scheme, the target query request, and the vector library to obtain the query results of the multiple candidate nodes" in step S330-2 may specifically include: Step B1: When the retrieval enhancement scheme deployed by the candidate node itself has the ability of retrieval enhancement generation, retrieve in the vector library according to the query description of the target query request, and generate the query results of the candidate node according to the large language model; Step B2: When the retrieval enhancement scheme deployed by the candidate node itself does not have the ability of retrieval enhancement generation, perform metadata retrieval in the vector library according to the keywords in the target query request to obtain the query results of the candidate node.

[0095] That is to say, if the candidate node has the ability of retrieval enhancement generation, it can use the large model to generate query results that match the target query request based on the retrieved data (digital objects); if the candidate node does not have the ability of retrieval enhancement generation, the metadata of the digital object retrieved based on the keywords is used as the query result of the candidate node.

[0096] By adopting the technical solution of the embodiment of the present application, each candidate node can perform retrieval generation according to the retrieval enhancement scheme deployed by itself to obtain corresponding query results. Thus, the proxy node can generate the final query result according to the query results of each candidate node, so that the final query result can cover a wider data source, reduce query omissions caused by the data limitation of a single node, and improve the comprehensiveness and accuracy of the results.

[0097] Combined with the above embodiments, in one implementation manner, the embodiment of the present application further provides a data retrieval enhancement generation method for a data space. In this method, for the step of "the proxy node generates a final query result according to the query results of the multiple candidate nodes" in the above step S340, it may specifically include sub-steps S340-1 to step S340-3:

[0098] Step S340-1: Match the query result of each candidate node with the query request, and sort the query results of the candidate nodes in descending order according to the matching degree to obtain a second sorting result.

[0099] Step S340-2: Use the query results of the first M candidate nodes in the second sorting result as the target query result, where M is a positive integer greater than 0.

[0100] Step S340-3: Generate a final query result according to the target query result.

[0101] In the embodiment of the present application, considering that the query results of the candidate nodes may not conform to the query intention of the query request, in order to obtain a more accurate query result, by matching the query result of each candidate node with the query request (for example, calculating the semantic similarity between the query intention and the query ability of the node), the query results that conform to the query intention are selected to generate the final query result.

[0102] Among them, the query results of the first M candidate nodes in the second sorting result can be understood as the query results of M candidate nodes whose matching degree between the query result of the candidate node and the query request is greater than or equal to the target matching degree threshold. For example, for the query results of 4 candidate nodes, the matching degrees between the query results of 4 candidate nodes (candidate node 1, candidate node 2, candidate node 3) and the query request are calculated as: 0.9, 0.4, 0.8. Then, the query results of the candidate nodes are sorted in descending order according to the matching degree, that is, 0.9, 0.8, 0.4. If the target matching degree threshold is 0.8, the query results of candidate node 1 and candidate node 3 can be used as the target query results.

[0103] By adopting the technical solution of the embodiment of the present application, the proxy node integrates the query results of multiple candidate nodes that match the query request, making the generated final query result more in line with the query intention. At the same time, the final query result can cover a wider range of data sources, reducing query omissions caused by the data limitations of a single node and improving the comprehensiveness and accuracy of the results.

[0104] Combined with the above embodiments, in one implementation, the embodiment of the present application also provides a data retrieval enhancement generation method for the data space. In this method, in order to improve the credibility of the generated result, the source of the query result is traced. Therefore, the identifier of the associated digital object is included in the query result, and the associated node is included in the final query result.

[0105] Specifically, in step B1 above, "retrieving in the vector library according to the query description of the target query request and generating the query result of the candidate node according to the large model" can specifically include: retrieving in the vector library according to the query description of the target query request to obtain the first target digital object, and generating the query result of the candidate node through the large model according to the first target digital object and the identifier of the first target digital object.

[0106] In step B1 above, "performing metadata retrieval in the vector library according to the keywords in the target query request to obtain the query result of the candidate node" can specifically include: performing metadata retrieval in the vector library according to the keywords in the target query request to obtain the second target digital object, and obtaining the query result of the candidate node according to the metadata of the second target digital object and the identifier of the second target digital object.

[0107] In step S340-3 above, "generating the final query result according to the target query result" can specifically include: generating the final query result according to the target query result and the candidate node corresponding to the target query result.

[0108] In the embodiment of the present application, when the candidate node performs retrieval and generation, for the retrieved digital object used to generate the query result, the identifier of the digital object is also used as content and carried in the query result of the candidate node, so that the data can be traced based on the digital object identifier in the query result later; and only the identifier of the digital object is added to the query result, that is, the content can be traced and the control of the data content can be achieved. Similarly, when the proxy node generates the final query result, the node corresponding to the filtered target query result is also carried in the final query result, so that the data can be traced based on the node information in the final query result later.

[0109] Exemplarily, Table 1 shows the schematic result of the traceability relationship of the generated content. For example, it can be determined that the final query result is generated based on the query results fed back by Node 1, Node 3, and Node 4, and the query result 1 of Node 1 is generated based on Digital Object 1 and Digital Object 2, the query result 2 of Node 3 is generated based on Digital Object 5 and Digital Object 7, and the query result 3 of Node 4 is generated based on Digital Object 9 and Digital Object 11.

[0110] Table 1 is the traceability relationship of the query results

[0111]

[0112] By adopting the technical solution implemented in this application, the digital object identifier and node information associated with the query result are added to the query result, and the traceability of the retrieval enhancement generation based on the digital object and the enhancement generation process of the retrieved content based on the node is carried out, so as to realize the data traceability of the multi-level data generation in the entire retrieval process.

[0113] Next, a specific embodiment is combined to illustrate the data retrieval enhancement generation method in the embodiments of this application. Refer to Figure 4 as shown Figure 4 is the step flowchart of another data retrieval enhancement generation method provided by the embodiments of this application. This method includes the following steps S410 to S470:

[0114] Step S410: The node generates the ability summary of the node according to the retrieval enhancement scheme deployed by itself, and constructs a vector library. The ability summary describes the query content and support fields provided by the node in the form of natural language, and the vector library represents the data space of the node.

[0115] Step S420: Receive a query request in the form of natural language through the proxy node, and determine the query intention according to the query request.

[0116] Step S430: The proxy node determines the query capabilities of each node according to the registered ability summary and node basic information, matches the query intention with the query capabilities of each node, and sorts the nodes in descending order of the matching degree to obtain the first sorting result; the first N nodes in the first sorting result are used as candidate nodes, where N is a positive integer greater than 0.

[0117] Step S440: The proxy node processes the query request into a target query request and sends it to the multiple candidate nodes. The target query request includes a query description and keywords in the form of natural language.

[0118] Step S450: When the retrieval enhancement solution deployed on the candidate node itself has the ability of retrieval-enhanced generation, retrieve in the vector library according to the query description of the target query request to obtain a first target digital object, and generate a query result of the candidate node through the large model according to the first target digital object and the identifier of the first target digital object.

[0119] Step S460: When the retrieval enhancement solution deployed on the candidate node itself does not have the ability of retrieval-enhanced generation, perform metadata retrieval in the vector library according to the keywords in the target query request to obtain a second target digital object, and obtain the query result of the candidate node according to the metadata of the second target digital object and the identifier of the second target digital object.

[0120] Step S470: Match the query result of each candidate node with the query request, and sort the query results of the candidate nodes in descending order according to the matching degree to obtain a second sorting result; use the query results of the top M candidate nodes in the second sorting result as the target query result, where M is a positive integer greater than 0; generate a final query result according to the target query result and the candidate node corresponding to the target query result.

[0121] In the embodiment of the present application, two-level content generation is implemented by setting proxy nodes and nodes in the data space of the digital network. Since the final query result is generated by the proxy node according to the query results of multiple candidate nodes, the transmission of raw data between nodes is avoided, ensuring the data security of each node in the data space of the digital network, and at the same time ensuring the accuracy of the final query result; moreover, the traceability of the retrieval-enhanced generation based on digital objects and the enhanced generation process of node retrieval content is realized, achieving data traceability of multi-level data generation in the entire retrieval process. Therefore, a retrieval-enhanced generation across nodes in the data space of the digital network is realized, improving the accuracy of retrieval tasks in the data space of the digital network.

[0122] The embodiment of the present application also provides a data retrieval enhancement generation device for a data space, as shown in Figure 5 shown Figure 5 is a schematic structural diagram of a data retrieval enhancement generation device for a data space provided by the embodiment of the present application. The device includes:

[0123] An intent determination module 510, configured to receive a query request in natural language form and determine a query intent according to the query request;

[0124] A node matching module 520, configured to match the query intent with the query capabilities of each node, and determine multiple candidate nodes according to the matching degree, where the node represents a data source in the data network space, and the query capability represents the query content and supported fields provided by the node;

[0125] A retrieval generation module 530, configured to send the query request to the multiple candidate nodes for retrieval generation to obtain query results of the multiple candidate nodes;

[0126] A query generation module 540, configured to generate a final query result according to the query results of the multiple candidate nodes.

[0127] In an alternative embodiment, the apparatus further includes:

[0128] An abstract generation module, configured to generate an ability abstract of the node according to the retrieval enhancement scheme deployed by the node itself, where the ability abstract describes the query content and supported fields provided by the node in a natural language form;

[0129] A registration module, configured to register the ability abstract and node basic information to the proxy node;

[0130] The node matching module is specifically configured to: the proxy node determines the query capabilities of each node according to the registered ability abstract and node basic information, and matches the query intent with the query capabilities of each node.

[0131] In an alternative embodiment, the apparatus further includes:

[0132] A construction module, configured to construct a vector library according to the retrieval enhancement scheme deployed by the node itself, where the vector library represents the data space of the node;

[0133] The retrieval generation module is specifically configured to: the proxy node processes the query request into a target query request and sends it to the multiple candidate nodes, where the target query request includes a query description and keywords in a natural language form; each candidate node performs retrieval generation based on the retrieval enhancement scheme deployed by itself, the target query request, and the vector library to obtain query results of the multiple candidate nodes.

[0134] In an alternative embodiment, the retrieval generation module includes:

[0135] A first generation sub-module, configured to, when the retrieval enhancement scheme deployed by the candidate node itself has the retrieval enhancement generation ability, perform retrieval in the vector library according to the query description of the target query request, and generate a query result of the candidate node according to a large model;

[0136] A second generation sub-module, configured to perform metadata retrieval on the vector library according to the keywords in the target query request to obtain a query result of the candidate node when the retrieval enhancement solution deployed by the candidate node itself does not have the ability to generate retrieval enhancement.

[0137] In an optional embodiment, the first generation sub-module is specifically configured to: perform retrieval on the vector library according to the query description of the target query request to obtain a first target digital object, and generate a query result of the candidate node by the large model according to the first target digital object and the identifier of the first target digital object;

[0138] The second generation sub-module is specifically configured to perform metadata retrieval on the vector library according to the keywords in the target query request to obtain a second target digital object, and obtain a query result of the candidate node according to the metadata of the second target digital object and the identifier of the second target digital object.

[0139] In an optional embodiment, the node matching module includes:

[0140] A first matching module, configured to match the query intention with the query capabilities of each node, and sort the nodes in descending order of the matching degree to obtain a first sorting result;

[0141] A first sorting module, configured to use the first N nodes in the first sorting result as candidate nodes, where N is a positive integer greater than 0.

[0142] In an optional embodiment, the query generation module includes:

[0143] A second matching module, configured to match the query result of each candidate node with the query request, and sort the query results of the candidate nodes in descending order of the matching degree to obtain a second sorting result;

[0144] A second sorting module, configured to use the query results of the first M candidate nodes in the second sorting result as the target query results, where M is a positive integer greater than 0;

[0145] A third generation sub-module, configured to generate a final query result according to the target query result.

[0146] In an optional embodiment, the third generation sub-module is specifically configured to generate a final query result according to the target query result and the candidate node corresponding to the target query result.

[0147] The embodiment of the present application also provides an electronic device, referring to Figure 6 , Figure 6It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 6 shown, the electronic device 600 includes: a memory 610 and a processor 620. The memory 610 and the processor 620 are communicatively connected via a bus. A computer program is stored in the memory 610, and this computer program can run on the processor 620, thereby implementing the steps of the data retrieval enhanced generation method for the data space described in the embodiments of the present application.

[0148] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data retrieval enhanced generation method for the data space described in the embodiments of the present application are implemented.

[0149] An embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the data retrieval enhanced generation method for the data space described in the embodiments of the present application are implemented.

[0150] Each embodiment in this specification is described in a progressive manner. Each embodiment focuses on the differences from other embodiments. For the same or similar parts among the embodiments, reference can be made to each other.

[0151] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods and devices according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the processes and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal devices generate a device for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0152] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing terminal device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0153] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable terminal device provide for implementing the process Figure 1 one process or multiple processes and / or blocks Figure 1 steps for the functions specified in one block or multiple blocks.

[0154] Although the preferred embodiments of the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0155] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or terminal device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or terminal device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or terminal device comprising the element.

[0156] The above has introduced in detail a data retrieval enhancement generation method, device and equipment for a data space provided by the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A data retrieval enhancement generation method for a data space, characterized in that, Applied to the retrieval enhanced generation system of the digital networking data space, the system includes a proxy node and multiple nodes, and the method includes: Receiving a query request in the form of natural language through the proxy node, and determining the query intention according to the query request; Matching the query intention with the query capabilities of each node, and determining multiple candidate nodes according to the matching degree. The node represents a data source in the digital networking data space, and the query capability represents the query content and supported fields provided by the node; Sending the query request to the multiple candidate nodes for retrieval generation to obtain query results of the multiple candidate nodes, including: the proxy node processes the query request into a target query request and sends it to the multiple candidate nodes. The target query request includes a query description and keywords in the form of natural language; when the retrieval enhancement scheme deployed on the candidate node itself has the ability of retrieval enhanced generation, retrieving in the vector library according to the query description of the target query request, and generating the query result of the candidate node according to the large model; when the retrieval enhancement scheme deployed on the candidate node itself does not have the ability of retrieval enhanced generation, performing metadata retrieval in the vector library according to the keywords in the target query request to obtain the query result of the candidate node; wherein, the vector library represents the data space of the node, and the query result of the candidate node contains the identifier of the associated digital object, and the associated digital object represents the digital object used to generate the query result of the candidate node; The proxy node generates a final query result according to the query results of the multiple candidate nodes, including: matching the query result of each candidate node with the query request, and sorting the query results of the candidate nodes in descending order according to the matching degree to obtain a second sorting result; taking the query results of the first M candidate nodes in the second sorting result as the target query result, where M is a positive integer greater than 0; generating a final query result according to the target query result; wherein, the final query result contains an associated node, and the associated node represents the node corresponding to the target query result.

2. The data retrieval enhancement generation method for a data space according to claim 1, wherein The method further includes: The node generates an ability summary according to the retrieval enhancement scheme deployed on itself, and the ability summary describes the query content and supported fields provided by the node in the form of natural language; Registering the ability summary and node basic information to the proxy node; Matching the query intention with the query capabilities of each node, including: The proxy node determines the query capabilities of each node according to the registered ability summary and node basic information, and matches the query intention with the query capabilities of each node.

3. The data retrieval enhancement generation method for a data space according to claim 1 or 2, characterized in that, The method further includes: The node constructs a vector library according to the retrieval enhancement scheme deployed on itself, and the vector library represents the data space of the node.

4. The data retrieval enhancement generation method for a data space according to claim 1, wherein Retrieving in the vector library according to the query description of the target query request, and generating the query result of the candidate node according to the large model, including: Retrieve in the vector library according to the query description of the target query request to obtain a first target digital object, and use the large model to generate a query result of candidate nodes based on the first target digital object and the identifier of the first target digital object; Perform metadata retrieval in the vector library according to the keywords in the target query request to obtain a query result of candidate nodes, including: Perform metadata retrieval in the vector library according to the keywords in the target query request to obtain a second target digital object, and obtain a query result of candidate nodes according to the metadata of the second target digital object and the identifier of the second target digital object.

5. The data retrieval enhancement generation method for a data space according to claim 1, wherein Match the query intention with the query capabilities of each node, and determine multiple candidate nodes according to the matching degree, including: Match the query intention with the query capabilities of each node, and sort the nodes in descending order according to the matching degree to obtain a first sorting result; Take the first N nodes in the first sorting result as candidate nodes, where N is a positive integer greater than 0.

6. The data retrieval enhancement generation method for a data space according to claim 1, wherein Generate a final query result according to the target query result, including: Generate a final query result according to the target query result and the candidate nodes corresponding to the target query result.

7. A data retrieval enhancement generation device for a data space, characterized in that, The device includes: An intention determination module, configured to receive a query request in natural language form and determine a query intention according to the query request; A node matching module, configured to match the query intention with the query capabilities of each node, and determine multiple candidate nodes according to the matching degree, where the node represents a data source in the digital networking data space, and the query capability represents the query content and supported fields provided by the node; A retrieval and generation module, configured to send the query request to the multiple candidate nodes for retrieval and generation to obtain query results of the multiple candidate nodes, including: a proxy node processes the query request into a target query request and sends it to the multiple candidate nodes, and the target query request includes a query description and keywords in natural language form; when the retrieval enhancement scheme deployed on the candidate node itself has the ability of retrieval enhancement generation, retrieve in the vector library according to the query description of the target query request, and generate a query result of the candidate node according to the large model; when the retrieval enhancement scheme deployed on the candidate node itself does not have the ability of retrieval enhancement generation, perform metadata retrieval in the vector library according to the keywords in the target query request to obtain a query result of the candidate node; where the vector library represents the data space of the node, and the query result of the candidate node includes the identifier of the associated digital object, and the associated digital object represents the digital object used to generate the query result of the candidate node; A query generation module, configured to generate a final query result according to query results of the multiple candidate nodes, including: matching the query result of each candidate node with the query request, and sorting the query results of the candidate nodes in descending order according to the matching degree to obtain a second sorting result; taking the query results of the first M candidate nodes in the second sorting result as target query results, where M is a positive integer greater than 0; generating a final query result according to the target query results; wherein, the final query result includes associated nodes, and the associated nodes represent the nodes corresponding to the target query results.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the data retrieval enhancement generation method for a data space according to any one of claims 1-6 are implemented.

Images