Data reasoning method based on distributed system and related equipment

Abstract

The invention provides a data reasoning method based on a distributed system and related equipment. The distributed system comprises a plurality of nodes, and different data fragments are stored in the nodes respectively. The method comprises the following steps: acquiring target data selected by a plurality of nodes from respective retrieved data sets; the data set retrieved by any node contains data matched with the query information input by the user in the data fragments stored in the node; determining matching degrees between the multiple pieces of target data and the query information, and determining a matching degree threshold value from multiple matching degrees corresponding to the multiple pieces of target data; the number of data included in a plurality of data sets retrieved by the plurality of nodes and having the matching degree with the query information higher than a matching degree threshold is greater than or equal to a preset screening number; and obtaining reasoning data screened from respective data sets by a plurality of nodes, wherein the matching degree between the reasoning data and the query information is higher than the matching degree threshold value, and based on the reasoning data, assisting the LLM model to execute a reasoning task.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of data reasoning technology, and in particular, to a data reasoning method based on a distributed system and related equipment. Background Art

[0002] Retrieval Augmented Generation (RAG) is a technology that combines information retrieval and text generation. It aims to use external knowledge bases to improve the generation quality of large-scale language models (LLMs), successfully alleviating problems such as "hallucinations", lack of knowledge in specific fields, and outdated information. RAG usually includes two stages: retrieval and generation: first, find some data related to the query information entered by the user from a large amount of data, and then use the retrieved data to assist the pre-trained LLM model in reasoning to generate accurate query results.

[0003] Generally speaking, the above data retrieval stage can be implemented on a single machine. However, when the data scale is very large and a single machine cannot accommodate all the data, it is particularly important to implement efficient data retrieval based on a distributed system. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide a data reasoning method and related devices based on a distributed system.

[0005] In a first aspect, this specification provides a data reasoning method based on a distributed system, wherein the distributed system includes multiple nodes, each of which stores a different data shard; the method includes:

[0006] Obtaining target data selected by the multiple nodes from the data sets retrieved by each node; wherein the data set retrieved by any node contains a number of data matching the query information input by the user in the data slice stored in the node;

[0007] Determine the matching degree between the multiple target data selected by the multiple nodes and the query information, and determine a matching degree threshold from the multiple matching degrees corresponding to the multiple target data; wherein the number of data with a matching degree higher than the matching degree threshold contained in the multiple data sets retrieved by the multiple nodes and the query information is greater than or equal to a preset screening number;

[0008] Acquire the inference data filtered out by the multiple nodes from their respective data sets, the inference data having a matching degree with the query information higher than the matching degree threshold, and aggregate the acquired inference data filtered out by the multiple nodes into an inference data set;

[0009] Based on the reasoning data set, the auxiliary LLM model performs the reasoning task and generates a query result corresponding to the query information.

[0010] In a second aspect, the present specification provides a data reasoning device based on a distributed system, wherein the distributed system includes a plurality of nodes, each of the plurality of nodes stores a different data shard; the device includes:

[0011] An acquisition unit, configured to acquire target data selected by the plurality of nodes from the data sets retrieved by the plurality of nodes; wherein the data set retrieved by any node includes a number of data matching the query information input by the user in the data slice stored in the node;

[0012] a determination unit, configured to determine a matching degree between the plurality of target data selected by the plurality of nodes and the query information, and determine a matching degree threshold from the plurality of matching degrees corresponding to the plurality of target data; wherein the number of data whose matching degree with the query information is higher than the matching degree threshold and included in the plurality of data sets retrieved by the plurality of nodes is greater than or equal to a preset screening number;

[0013] A data aggregation unit, configured to obtain the inference data filtered out by the multiple nodes from their respective data sets, the inference data having a matching degree with the query information higher than the matching degree threshold, and aggregate the obtained inference data filtered out by the multiple nodes into an inference data set;

[0014] The model reasoning unit is used to assist the LLM model in performing reasoning tasks based on the reasoning data set and generate query results corresponding to the query information.

[0015] Accordingly, this specification also provides a computing device, comprising: a memory and a processor; the memory stores computer programs / instructions that can be executed by the processor; when the processor executes the computer program / instructions, it executes the data reasoning method based on the distributed system described in the first aspect above.

[0016] Accordingly, the present specification also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the data reasoning method based on a distributed system as described in the first aspect above is executed.

[0017] Accordingly, the present specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the data reasoning method based on a distributed system as described in the first aspect above is executed.

[0018] In summary, multiple nodes in a distributed system can retrieve data sets that match the query information input by the user from their respective stored data slices, and can further select target data from their respective retrieved data sets. Then, the matching degree between the multiple target data selected by the multiple nodes and the query information is determined, and the matching degree threshold for further filtering out part of the data from each data set is determined from the multiple matching degrees corresponding to the multiple target data. Based on this, each node can further filter out a small amount of data whose matching degree with the query information is higher than the matching degree threshold from the large amount of data contained in the data set retrieved by each node. Subsequently, only a small amount of data with a high matching degree filtered out by each node needs to be obtained and summarized, and the LLM model is assisted to perform reasoning tasks based on the small amount of data. In this way, under the premise of ensuring data retrieval efficiency and LLM model generation quality, unnecessary data transmission and calculation with a low matching degree are avoided, which greatly reduces the amount of data transmission and calculation in the distributed system, thereby reducing the calculation and communication overhead. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of a system architecture provided by an exemplary embodiment;

[0020] Figure 2 It is a flowchart of a data reasoning method based on a distributed system provided by an exemplary embodiment;

[0021] Figure 3 It is a structural schematic diagram of a data reasoning device based on a distributed system provided by an exemplary embodiment;

[0022] Figure 4 It is a structural diagram of a computing device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0023] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0024] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0025] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0026] Retrieval enhancement generation technology usually includes two stages: retrieval and generation. In the retrieval stage, it is necessary to find some data related to the query information entered by the user from a large amount of data. However, when the data scale is very large and a single machine cannot accommodate all the data, it is necessary to implement data retrieval based on a distributed system.

[0027] Among them, the distributed system includes multiple nodes, and the multiple nodes can store large-scale data sets in a distributed manner. For example, the large-scale data set can be divided into multiple data slices, and the multiple nodes can store different data slices respectively. In an illustrated embodiment, multiple nodes can retrieve a number of data related to the query information input by the user from the data slices stored by each node by running the corresponding search algorithm. Furthermore, through a distributed union operation, a number of data retrieved by multiple nodes can be aggregated to one node (such as a master node), and then the node can assist the pre-trained LLM model to perform corresponding reasoning tasks based on the aggregated multiple data, and output query results corresponding to the query information input by the user.

[0028] In an illustrated embodiment, the search algorithm may be an approximate nearest neighbor search (ANNS) algorithm, and multiple nodes may build a corresponding index structure for their respective stored data slices by running the ANNS algorithm, and then quickly search for a preset number of nearest neighbor data that best matches the query information input by the user from their respective stored data slices based on the index structure, i.e., top-k nearest neighbor data, where k is the preset number and k is an integer greater than 1. It should be understood that when faced with large-scale data sets, the traditional exact nearest neighbor search (ENNS) may not meet the actual needs of users, because its computational cost will rise sharply with the increase in data volume and dimension. ANNS significantly improves the retrieval speed by allowing a certain degree of approximation, while still maintaining a high accuracy in most cases, making it an effective tool for processing big data and high-dimensional data.

[0029] As mentioned above, distributed technology can effectively solve the problem of data storage and retrieval efficiency when facing large-scale data sets. However, when the amount of data retrieved by each node is large (for example, k is equal to millions or even tens of millions), a large amount of data needs to be transmitted and calculated when aggregating the large amount of data retrieved by each node, which incurs a large amount of computing and communication overhead and cannot meet actual usage requirements.

[0030] Based on this, this specification provides a technical solution to further filter out some data with a higher degree of matching from the data sets that match the query information retrieved by multiple nodes, and summarize the filtered data to assist the LLM model in performing reasoning tasks, thereby greatly reducing the amount of data transmission and calculation in the distributed system, thereby reducing the overall computing and communication overhead.

[0031] In implementation, this specification can be applied to a distributed system, which includes multiple nodes, each of which stores different data slices. In an illustrated embodiment, this specification can obtain target data selected by multiple nodes from their respective retrieved data sets. Among them, the data set retrieved by any node contains several data that match the query information input by the user in the data slice stored by the node. Further, the matching degree between the multiple target data selected by the multiple nodes and the query information can be determined, and the matching degree threshold can be determined from the multiple matching degrees corresponding to the multiple target data. Among them, the number of data with a matching degree higher than the matching degree threshold between the query information contained in the multiple data sets retrieved by the multiple nodes is greater than or equal to the preset screening number. Further, the inference data with a matching degree higher than the matching degree threshold between the query information and the multiple nodes screened from their respective data sets can be obtained, and the inference data screened by the multiple nodes obtained can be aggregated into an inference data set. Further, this specification can assist the LLM model in performing the inference task based on the inference data set to generate a query result corresponding to the query information.

[0032] In the above technical scheme, multiple nodes of the distributed system can retrieve data sets that match the query information input by the user from the data slices stored in each node, and can further select target data from the data sets retrieved by each node. Then, the matching degree between the multiple target data selected by the multiple nodes and the query information is determined, and the matching degree threshold for further filtering out part of the data from each data set is determined from the multiple matching degrees corresponding to the multiple target data. Based on this, each node can further filter out a small amount of data whose matching degree with the query information is higher than the matching degree threshold from the large amount of data contained in the data set retrieved by each node. Subsequently, only a small amount of data with a high matching degree filtered out by each node needs to be obtained and summarized, and the LLM model is assisted to perform the reasoning task based on the small amount of data. In this way, under the premise of ensuring the efficiency of data retrieval and the quality of LLM model generation, unnecessary data transmission and calculation with a low matching degree are avoided, which greatly reduces the amount of data transmission and calculation in the distributed system, thereby reducing the calculation and communication overhead.

[0033] See also Figure 1 , Figure 1 This is a schematic diagram of a system architecture provided by an exemplary embodiment. One or more embodiments provided in this specification may be Figure 1 The system architecture shown in the figure or a similar system architecture is specifically implemented. Figure 1 As shown, the distributed system 10 may include multiple nodes, such as node 100a, node 100b, node 100c and node 100d, etc. Communication connections can be established between the multiple nodes in any possible manner, and this specification does not make any specific limitations on this.

[0034] In an illustrated embodiment, the nodes 100a, 100b, 100c, and 100d in the distributed system 10 can be used to store large-scale data sets in a distributed manner. Exemplarily, the large-scale data set can be divided into multiple data slices, and different data slices can be stored in the nodes 100a, 100b, 100c, and 100d, respectively. Exemplarily, the data set can be evenly divided into multiple data slices, and the multiple data slices can be allocated to multiple nodes for storage. Alternatively, if the hardware conditions of the multiple nodes themselves are different, the data set can also be divided into multiple data slices of different sizes, and the larger data slices (i.e., the data slices with a larger amount of data) can be allocated to the nodes with better hardware conditions for storage, and this specification does not specifically limit this.

[0035] The above-mentioned data set may be a data set related to a specific field, or a data set related to multiple fields, such as the medical field, the financial field, the film and television entertainment field, etc. This specification does not make any specific limitations on this.

[0036] In addition, it should be noted that this specification does not specifically limit the data type in the data set. In an illustrated embodiment, the data contained in the data set may be a document, or image data, audio and video data, etc., and this specification does not specifically limit this.

[0037] In an illustrated embodiment, the nodes 100a, 100b, 100c, and 100d in the distributed system 10 may obtain the query information input by the user. Exemplarily, the query information input by the user may be received by a master node (e.g., node 100a) of multiple nodes, and the received query information may be sent to slave nodes (e.g., node 100b, node 100c, and node 100d), which is not specifically limited in this specification.

[0038] In an illustrated embodiment, the query information may be a query text input by a user through an input device (such as a keyboard or a touch screen, etc.) on a user terminal. For example, the query text may be "Who is the winner of the Nobel Prize in Physics in 1943?" For example, the query text may also be "What medicine can I take for a cold and fever?" For example, the query text may also be "What are the major traffic accidents that occurred in City A in the past two years?", etc., which are not specifically limited in this specification.

[0039] Furthermore, after obtaining the query information input by the user, the nodes 100a, 100b, 100c and 100d can respectively retrieve data sets matching the query information from their respective stored data slices. The data set retrieved by any node can include several data matching the query information in the data slice stored by the node.

[0040] Furthermore, after each node has retrieved a data set that matches the query information, each node can further filter out some data that has a high degree of match with the query information from the large amount of data contained in its data set. This part of data can be called inference data, which can be used to assist the LLM model in performing inference tasks.

[0041] In an illustrated embodiment, each node may first select target data from each retrieved data set. Then, the matching degree between the multiple target data selected by the multiple nodes and the query information is determined, and a matching degree threshold for further screening the inference data from the multiple data sets retrieved by the multiple nodes is determined from the multiple matching degrees corresponding to the multiple target data. For details, please refer to the following Figure 2 The description of the corresponding embodiments will not be repeated here.

[0042] Furthermore, after determining the matching degree threshold, each node may further filter out inference data having a matching degree with the query information higher than the matching degree threshold from the retrieved data set.

[0043] Furthermore, through the distributed merge operation, the reasoning data further filtered out by multiple nodes from their respective data sets can be obtained, and all the obtained reasoning data can be aggregated into one reasoning data set. Exemplarily, the slave nodes (such as node 100b, node 100c and node 100d) can send the reasoning data further filtered out from the data set retrieved by themselves to the master node (such as node 100a), and accordingly, the master node can obtain the reasoning data sent by all the slave nodes, and the master node itself can also further filter out the reasoning data from the data set retrieved by it, and finally, the master node can aggregate the reasoning data filtered out by all the nodes to obtain the reasoning data set.

[0044] Furthermore, the LLM model can be assisted to perform reasoning tasks based on the summarized reasoning data set to generate query results corresponding to the query information. In an illustrated embodiment, the master node can be equipped with a pre-trained LLM model. After the master node summarizes the above reasoning data set, it can assist the LLM model to perform reasoning tasks based on the reasoning data set. For details, please refer to the following Figure 2 The description of the corresponding embodiments will not be repeated here.

[0045] Furthermore, the distributed system 10 may return the query results generated by the LLM model to the user terminal, and accordingly, the output device (such as a display screen) on the user terminal may output the query results to the user, and so on, which is not specifically limited in this specification.

[0046] As described above, when implementing retrieval enhancement generation, the multiple nodes in the distributed system 10 will not directly aggregate all data sets after retrieving data sets that match the query information input by the user from their respective stored data shards. Instead, each node can further filter out a small amount of data whose matching degree with the query information is higher than the matching degree threshold from the large amount of data contained in the retrieved data set. Subsequently, it is only necessary to obtain and aggregate a small amount of data with a high matching degree filtered out by each node, and assist the LLM model in performing reasoning tasks based on this small amount of data. In this way, while ensuring data retrieval efficiency and LLM model generation quality, the present specification avoids unnecessary data transmission and calculation with a low matching degree, greatly reduces the amount of data transmission and calculation in the distributed system, and thereby reduces computing and communication overhead.

[0047] It should be noted that each node in the above-mentioned distributed system 10 can be a desktop computer, a server, or a server cluster composed of multiple servers, etc., which has the above-mentioned functions, and this specification does not make specific limitations on this. In addition, it should be noted that Figure 1 The system architecture shown is for illustrative purposes only. In some possible implementations, the distributed system 10 may include more or fewer devices than those shown in the figure, and this specification does not specifically limit this.

[0048] See also Figure 2 , Figure 2 is a flowchart of a data reasoning method based on a distributed system provided by an exemplary embodiment. The method can be applied to Figure 1 The distributed system shown in FIG. 1 includes multiple nodes, each of which stores different data shards. Figure 2 As shown, the method may specifically include the following steps S201 to S204.

[0049] Step S201, obtaining target data selected by multiple nodes from their respective retrieved data sets; wherein the data set retrieved by any node contains a number of data matching the query information input by the user in the data slice stored in the node.

[0050] In an illustrated embodiment, after obtaining the query information input by the user, multiple nodes can respectively retrieve data sets matching the query information from their respective stored data slices. The data set retrieved by any node can include several data matching the query information in the data slice stored by the node.

[0051] It should be noted that this specification does not specifically limit the specific implementation method of data retrieval.

[0052] In an illustrated embodiment, multiple nodes can implement data retrieval by running an approximate nearest neighbor search algorithm to obtain a nearest neighbor data set that matches the query information. Accordingly, the nearest neighbor data set retrieved by any of the multiple nodes through the approximate nearest neighbor search algorithm may include a preset number of nearest neighbor data with the highest matching degree with the query information in the data slice stored by the node, that is, top-k nearest neighbor data, where k is the preset number and k is an integer greater than 1.

[0053] Among them, the approximate nearest neighbor search algorithm usually includes two stages: index construction and data search. Accordingly, when each node implements data retrieval through the approximate nearest neighbor search algorithm, it can first build a corresponding index structure for each stored data shard, and then quickly search for the top-k nearest neighbor data that best matches the query information entered by the user from each stored data shard based on the index structure, thereby obtaining a nearest neighbor data set.

[0054] In one illustrated embodiment, the index construction method adopted by the approximate nearest neighbor search algorithm may include any one of the following methods: a hashing method (Locality Sensitive Hashing, LSH), a neighbor graph method (Hierarchical Navigable Small World Graphs, HNSW), a similarity search library (Facebook AI Similarity Search, FAISS) method, etc., and this specification does not specifically limit this.

[0055] Exemplarily, the data retrieval process of the approximate nearest neighbor search algorithm will be explained below by taking the hash method as an example.

[0056] First, we need to define a series of hash function clusters, each of which can be composed of multiple hash functions. These hash functions are designed to make similar data in the data set (such as movies of the same type or medical records of the same department, etc.) more likely to be mapped to the same bucket, while dissimilar data are more likely to be mapped to different buckets.

[0057] Furthermore, each data in the data set is hashed by a hash function cluster to calculate a hash value combination corresponding to each data. Then, several data with the same calculated hash value combination can be allocated to the same bucket.

[0058] Furthermore, the query information input by the user is also hashed through the hash function cluster to calculate a hash value combination corresponding to the query information. Then, a bucket with the same hash value combination is searched in multiple buckets, and multiple related data are retrieved from the bucket. Furthermore, a preset number of data with the highest matching degree can be screened out from the multiple data as the top-k nearest neighbor data.

[0059] In an illustrated embodiment, the degree of match between the query information and each data can be represented by the distance between the vector corresponding to the query information and the vector corresponding to each data; wherein, the closer the distance between the vectors, the higher the degree of match. Exemplarily, the above distance can be Euclidean distance and cosine distance, etc., which is not specifically limited in this specification.

[0060] Alternatively, in some possible implementations, each node may also implement data retrieval through any other possible search algorithm except the above-mentioned approximate nearest neighbor search algorithm, and this specification does not make any specific limitation on this.

[0061] Furthermore, after each node has retrieved a data set that matches the query information, each node can further filter out some data that has a higher degree of match with the query information from the data contained in its data set. This part of data can be called inference data, which can be used to assist the LLM model in performing inference tasks.

[0062] In an illustrated embodiment, a matching degree threshold for further filtering inference data from multiple data sets retrieved from multiple nodes may be determined first according to a preconfigured filtering rule.

[0063] In one illustrated embodiment, the screening rule may include a preset screening number. Accordingly, when determining the matching degree threshold, it is necessary to ensure that the number of data contained in multiple data sets retrieved by multiple nodes and having a matching degree higher than the matching degree threshold with the query information is greater than the screening number.

[0064] The following will take the screening rule including the screening quantity as an example to explain the implementation method of determining the matching degree threshold.

[0065] First, each node can select at least one target data from the data set retrieved by each node. In an illustrated embodiment, the target data can be data randomly selected by the node from all the data included in the data set; or, the target data can also be data selected sequentially by the node from all the data included in the data set according to a preset step length, etc., which is not specifically limited in this specification.

[0066] Step S202, determining the degree of match between the multiple target data selected by the multiple nodes and the query information, and determining a matching degree threshold from the multiple matching degrees corresponding to the multiple target data; wherein the number of data contained in the multiple data sets retrieved by the multiple nodes and having a matching degree between the query information higher than the matching degree threshold is greater than or equal to a preset screening number.

[0067] Furthermore, the matching degree between the multiple target data selected by the multiple nodes and the query information can be determined. Furthermore, a matching degree threshold can be determined from the multiple matching degrees corresponding to the multiple target data according to a preset screening number. The number of data with a matching degree higher than the matching degree threshold contained in the multiple data sets retrieved by the multiple nodes and with the query information is greater than or equal to the preset screening number.

[0068] Specifically, when determining a matching degree threshold from multiple matching degrees corresponding to multiple target data, it can specifically include: first sorting the multiple matching degrees corresponding to the multiple target data from high to low, and based on the sorted order, traversing from the first matching degree; for the target matching degree currently traversed, counting the number of data contained in multiple data sets whose matching degree with the query information is higher than the target matching degree; and determining whether the counted number of data is greater than or equal to the preset screening number. If so, the target matching degree can be determined as the matching degree threshold and the traversal is terminated; if not, continue to traverse to the next matching degree, and so on.

[0069] In an illustrated embodiment, if when traversing to the last matching degree, the number of data in the multiple data sets obtained by statistics and having a matching degree between the query information higher than the last matching degree is less than the preset screening number, then multiple nodes can further select new target data from their respective retrieved data sets. Among them, the new matching degree corresponding to the new target data selected by the multiple nodes is lower than the last matching degree. Accordingly, multiple new target data further selected by the multiple nodes can be obtained, and the above-mentioned traversal process is performed for the multiple new matching degrees corresponding to the multiple new target data, and so on, until the number of data in the multiple data sets obtained by statistics and having a matching degree between the query information higher than the new matching degree is greater than or equal to the preset screening number, and the new matching degree is determined as the matching degree threshold.

[0070] Alternatively, in an illustrated embodiment, if when traversing to the last matching degree, the number of data in the multiple data sets obtained by counting the matching degree between the query information and the last matching degree is less than the preset screening number, the matching degree threshold can be directly determined as negative infinity. When the matching degree threshold is negative infinity, it is equivalent to that no data screening is performed subsequently, and all data contained in the data sets retrieved by each node meet the matching degree requirements, and this specification does not make specific limitations on this.

[0071] In an illustrated embodiment, the multiple nodes include a master node and a slave node. The master node can select target data from the data set retrieved by itself and determine the degree of match between the target data and the query information. In addition, after each slave node selects the target data from its own data set and determines the degree of match between the target data and the query information, the slave node can send the degree of match to the master node. Accordingly, the master node can receive the degrees of match sent by all slave nodes, and then determine the degree of match threshold from the multiple degrees of match obtained according to the above steps.

[0072] Step S203, obtaining the inference data whose matching degree with the query information is higher than the matching degree threshold and which is screened out by the multiple nodes from their respective data sets, and aggregating the obtained inference data screened out by the multiple nodes into an inference data set.

[0073] Further, after determining the matching degree threshold, each node can filter out the inference data whose matching degree with the query information is higher than the matching degree threshold from the respective retrieved data sets. Exemplarily, each node can delete the data whose matching degree is lower than the matching degree threshold from the data set, thereby obtaining a new data set, and correspondingly, the new data set only contains the inference data whose matching degree is higher than the matching degree threshold.

[0074] In an illustrated embodiment, the master node may send the determined matching degree threshold to the slave node, and accordingly, the slave node may receive the matching degree threshold, and filter out the inference data whose matching degree with the query information is higher than the matching degree threshold from the data set retrieved by itself, and send the filtered inference data (i.e., the above-mentioned new data set) to the master node. Accordingly, the master node may obtain the inference data sent by all the slave nodes. And, the master node may filter out the inference data whose matching degree with the query information is higher than the matching degree threshold from the data set retrieved by it.

[0075] Furthermore, the master node can merge and summarize all the acquired inference data to obtain an inference data set.

[0076] Step S204: Based on the reasoning data set, assist the LLM model in performing the reasoning task to generate a query result corresponding to the query information.

[0077] Furthermore, the LLM model can be assisted in performing reasoning tasks based on the summarized reasoning data set to generate query results corresponding to the query information input by the user. In an illustrated embodiment, a pre-trained LLM model can be mounted in the master node. After the master node obtains the above-mentioned reasoning data set through summary, the LLM model can be assisted in performing reasoning tasks based on the reasoning data set to generate query results corresponding to the query information input by the user.

[0078] It should be noted that this specification does not specifically limit the specific implementation method of assisting the LLM model in performing reasoning tasks based on the reasoning data set.

[0079] In one illustrated embodiment, a prompt word can be constructed based on the query information input by the user and the reasoning data set, and the prompt word can be input into a pre-trained LLM model. The LLM model performs a corresponding reasoning task based on the prompt word to generate a query result corresponding to the query information.

[0080] In an illustrated embodiment, a preset number of candidate data, such as top-k candidate data, with the highest matching degree with the query information input by the user may be further selected from the multiple inference data included in the inference data set. Furthermore, a prompt word may be constructed based on the query information and the top-k candidate data, and the prompt word may be input into a pre-trained LLM model, and the LLM model may perform a corresponding inference task based on the prompt word to generate a query result corresponding to the query information, etc. This specification does not specifically limit this.

[0081] In an illustrated embodiment, the above-mentioned preset screening number can be greater than or equal to the preset number of candidate data, that is, the screening number can be greater than or equal to k, which means that it is necessary to ensure that the inference data set contains at least the number of data finally input into the LLM model.

[0082] Next, we will combine Figure 2 The method flow shown in the figure illustrates the data reasoning method based on the distributed system provided in this specification through several examples. Specifically, the data reasoning method may include the following steps.

[0083] Step 1: Data sharding.

[0084] Divide a large-scale data set into multiple data slices. Exemplarily, a large-scale data set (denoted as X) can be evenly divided into P data slices, that is, X = {X1, X2, ..., XP}, where the pth data slice Xp can be assigned to node p for processing, and correspondingly, process p running in node p can perform data retrieval for the data slice Xp. Wherein, P is an integer greater than 1, and p is an integer greater than or equal to 1 and less than or equal to P.

[0085] Step 2: Index building

[0086] Each node constructs a corresponding ANNS index structure for the data slices stored in each node. For example, a process p running in a node p can construct an ANNS index structure for a data slice Xp.

[0087] Step 3: Data search

[0088] Based on the constructed ANNS index structure, each node searches for data in its own stored data slices to obtain the top-k nearest neighbor data that matches the query information input by the user. Exemplarily, process p in node p can convert the acquired query information into a d-dimensional query vector q, and based on the constructed index structure, search for data in the data slice Xp to obtain the top-k nearest neighbor data with the shortest distance (i.e., the highest matching degree, and the distance is used to represent the matching degree in the following steps). Exemplarily, process p can sort the searched top-k nearest neighbor data from small to large according to the distance between them and the query vector q to obtain an array Ap. That is, the array Ap contains k elements, each of which is a nearest neighbor data, and the first element (for example, denoted as A[1]) is the nearest neighbor data with the smallest distance to the query vector q among the k nearest neighbor data.

[0089] Step 4: Data sampling

[0090] Each node randomly selects several target nearest neighbor data from the top-k nearest neighbor data it searches out, and obtains the degree of match between the several target nearest neighbor data and the query information. Exemplarily, process p in node p randomly selects s ​​target nearest neighbor data from the k nearest neighbor data contained in its array Ap (s can be much smaller than k, for example, k equals 20, s equals 2, or for example, k equals 100, s equals 2). Then, process p obtains the distances between the s target nearest neighbor data and the query vector q respectively, and obtains s distances corresponding to the s target nearest neighbor data. Then, these s distances are sorted from small to large to obtain an array Sp. That is, the array Sp contains s elements, each element is a distance corresponding to the target nearest neighbor data, and the first element (for example, denoted as Sp[1]) is the smallest distance among the s distances.

[0091] Then, the processes on all nodes can perform distributed merge operations on all arrays Sp (i.e., P arrays Sp) to form a new array S, which contains |S| elements, and these |S| elements are also sorted from small to large according to distance. If there are no duplicate values ​​in the arrays Sp of each process, then the array S contains a total of P×s elements, i.e., |S|=P×s.

[0092] Step 5: Data bucketing

[0093] It is not difficult to understand that the s elements (i.e., s distances) contained in each array Sp can divide the array Ap into s+1 distance intervals. Correspondingly, the |S| elements contained in the merged array S can divide the array {Ap} into |S|+1 distance intervals. Among them, the array {Ap} represents the sum of all arrays Ap in all processes, containing P×k elements.

[0094] Assume that the array Sp contains two elements, Sp[1] = distance 10, Sp[2] = distance 20, then the s+1 distance intervals may include the three distance intervals (-∞, 10], (10, 20], (20, ∞) in sequence.

[0095] Assume that P is equal to 2, and the array S contains 4 elements, namely S[1] = distance 3, S[2] = distance 10, S[3] = distance 12, S[4] = distance 20, then |S|+1 distance intervals can include the five distance intervals (-∞, 3], (3, 10], (10, 12], (12, 20], (20, ∞) in sequence.

[0096] Based on this, process p can determine which distance interval of |S|+1 distance intervals the distance between the k nearest neighbor data in its array Ap and the query vector q belongs to, and obtain the array Cp by counting. Among them, array Cp contains |S|+1 elements corresponding to |S|+1 distance intervals, and the element Cp[i] in array Cp is used to record the number of nearest neighbor data contained in array Ap that belongs to the i-th distance interval. Among them, i is an integer greater than or equal to 1 and less than or equal to |S|+1.

[0097] Then, the processes on all nodes can perform distributed accumulation for all arrays Cp (i.e., P arrays Cp), that is, add the elements at corresponding positions in all arrays Cp to obtain a new array C. The new array C also contains |S|+1 elements corresponding to |S|+1 distance intervals, and the element C[j] in the new array C is used to record the number of nearest neighbor data contained in the array {Ap} belonging to the jth distance interval. Where j is an integer greater than or equal to 1 and less than or equal to |S|+1.

[0098] For example, it is still assumed that P is equal to 2, and the array S contains 4 elements, namely S[1] = distance 3, S[2] = distance 10, S[3] = distance 12, S[4] = distance 20, and |S|+1 distance intervals can include the five distance intervals (-∞, 3], (3, 10], (10, 12], (12, 20], and (20, ∞) in sequence. In addition, assuming that k is equal to 20, that is, {Ap} contains a total of 40 nearest neighbor data. According to the distances between these 40 nearest neighbor data and the query vector q, the final statistically obtained array C may be [2, 8 , 12, 15, 3], that is, among these 40 nearest neighbor data, the number of nearest neighbor data with a distance from the query vector q in the interval (-∞, 3] is equal to 2, the number of nearest neighbor data with a distance from the query vector q in the interval (3, 10] is equal to 8, the number of nearest neighbor data with a distance from the query vector q in the interval (10, 12] is equal to 12, the number of nearest neighbor data with a distance from the query vector q in the interval (12, 20] is equal to 15, and the number of nearest neighbor data with a distance from the query vector q in the interval (20, ∞) is equal to 3.

[0099] Step 6: Threshold calculation

[0100] According to array C, accumulation is started from element C[1] therein until the accumulated value is not less than a preset number (eg, k) for the first time. Assuming that this condition is met at C[j], the distance threshold (equivalent to the matching threshold) is determined to be S[j].

[0101] Still taking the example in step 5 above as an example, the preset number k=20, when accumulated to C[3], 2+8+12=22, the accumulated value is greater than 20, that is, the number of nearest neighbor data contained in the array {Ap} whose distance is less than or equal to S[3]=12 is greater than 20, then S[3]=12 can be determined as the distance threshold.

[0102] For example, assume that the final statistical array C is [1, 2, 1, 2, 34]. Since the values ​​of C[1], C[2], C[3] and C[4] are all very small, the accumulated value is greater than 20 until it is accumulated to C[5], 1+2+1+2+34=40.

[0103] In this case, the distance threshold can be determined as positive infinity (that is, the matching degree threshold is negative infinity), which is equivalent to no subsequent data screening, and the k nearest neighbor data obtained by querying each node all meet the requirements.

[0104] Alternatively, in this case, each node can also select two new target nearest neighbor data from the array Ap after S[4] = distance 20, and then merge them to obtain a new array S'. Exemplarily, the new array S' can also contain 4 elements, for example, S'[1] = distance 21, S'[2] = distance 25, S'[3] = distance 30, S'[4] = distance 35. Correspondingly, |S|+1 distance intervals may include the five distance intervals (-∞, 21], (21, 25], (25, 30], (30, 35], (35, ∞) in sequence. Correspondingly, assuming that the new array C' obtained by the final statistics is [1, 24, 8, 2, 5], according to the new array C', starting from the element C'[1] therein, when accumulated to C'[2], 1+24=25, the accumulated value is greater than 20, that is, the number of nearest neighbor data contained in the array {Ap} whose distance is less than or equal to S'[2]=25 is greater than 20, then S'[2]=25 can be determined as the distance threshold.

[0105] For example, assume that the new array C' obtained by the final statistics is [8, 2, 1, 4, 25]. Since the values ​​of C'[1], C'[2], C'[3] and C'[4] in the new array C' are also relatively small, until the accumulation to C'[5], 8+2+1+4+25=40, the accumulated value is greater than 20. Then, after S'[4]=distance 35, each node can select 2 new target nearest neighbor data from the array Ap, and then merge them to obtain a new array S", which can include S"[1]=distance 37, S"[2]=distance 45, S"[3]=distance 50, S"[4]=distance 55, and so on. This manual does not make any specific limitations on this.

[0106] Step 7: Data screening

[0107] Process p deletes the nearest neighbor data whose distance to the query vector q is less than the distance threshold from the array Ap and generates a new array Ap'.

[0108] Step 8: Data merging

[0109] Through the distributed merge operation, the new arrays Ap' obtained by all processes are merged to obtain the array {Ap'}. Then, the final top-k candidate nearest neighbor data can be further selected from the array {Ap'}, such as the top-k candidate nearest neighbor data with the closest distance. Then, a prompt word is constructed based on the query vector q and the top-k candidate nearest neighbor data, and the prompt word is input into the pre-trained LLM model, which performs the corresponding reasoning task based on the prompt word to generate a query result corresponding to the query information.

[0110] In summary, multiple nodes in a distributed system can retrieve data sets that match the query information input by the user from their respective stored data slices, and can further select target data from their respective retrieved data sets. Then, the matching degree between the multiple target data selected by the multiple nodes and the query information is determined, and the matching degree threshold for further filtering out part of the data from each data set is determined from the multiple matching degrees corresponding to the multiple target data. Based on this, each node can further filter out a small amount of data whose matching degree with the query information is higher than the matching degree threshold from the large amount of data contained in the data set retrieved by each node. Subsequently, only a small amount of data with a high matching degree filtered out by each node needs to be obtained and summarized, and the LLM model is assisted to perform reasoning tasks based on the small amount of data. In this way, under the premise of ensuring data retrieval efficiency and LLM model generation quality, unnecessary data transmission and calculation with a low matching degree are avoided, which greatly reduces the amount of data transmission and calculation in the distributed system, thereby reducing the calculation and communication overhead.

[0111] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides a data reasoning device based on a distributed system. Figure 3 , Figure 3 is a schematic diagram of a data reasoning device based on a distributed system provided by an exemplary embodiment. The device 30 can be applied to Figure 1 The distributed system shown in FIG. 1 includes multiple nodes, each of which stores different data shards. Figure 3 As shown, the device 30 comprises:

[0112] The acquisition unit 301 is used to acquire the target data selected by the multiple nodes from the data sets retrieved by each node; wherein the data set retrieved by any node includes a number of data matching the query information input by the user in the data slice stored in the node;

[0113] The determination unit 302 is used to determine the matching degree between the multiple target data selected by the multiple nodes and the query information, and determine the matching degree threshold from the multiple matching degrees corresponding to the multiple target data; wherein the number of data with a matching degree higher than the matching degree threshold contained in the multiple data sets retrieved by the multiple nodes and the query information is greater than or equal to a preset screening number;

[0114] A data aggregation unit 303 is used to obtain the inference data whose matching degree with the query information is higher than the matching degree threshold and which is filtered out by the multiple nodes from their respective data sets, and aggregate the obtained inference data filtered out by the multiple nodes into an inference data set;

[0115] The model reasoning unit 304 is used to assist the LLM model in performing reasoning tasks based on the reasoning data set, and generate a query result corresponding to the query information.

[0116] In one illustrated embodiment, the data set retrieved by any node is a nearest neighbor data set matching the query information retrieved from the data shards stored by the node by running the approximate nearest neighbor search algorithm ANNS on the node, and the nearest neighbor data set includes a preset number of nearest neighbor data with the highest degree of match with the query information in the data shards stored at the node.

[0117] In an illustrated embodiment, the target data is data randomly selected from all data included in the data set; or, the target data is data selected sequentially from all data included in the data set according to a preset step size.

[0118] In an illustrated embodiment, the determining unit 302 is specifically configured to:

[0119] Sort the multiple matching degrees corresponding to the multiple target data from high to low, and traverse from the first matching degree based on the sorted order;

[0120] For the currently traversed target matching degree, counting the number of data contained in the multiple data sets whose matching degree with the query information is higher than the target matching degree; and

[0121] Determine whether the number of data counted is greater than or equal to the screening number. If so, determine the target matching degree as the matching degree threshold and end the traversal; if not, continue to traverse to the next matching degree.

[0122] In an illustrated embodiment, the determining unit 302 is specifically configured to:

[0123] If, when traversing to the last matching degree, the number of data in the multiple data sets whose matching degree with the query information is higher than the last matching degree obtained by counting is less than the screening number, then further obtaining new target data selected by the multiple nodes from the respective retrieved data sets; wherein the new matching degree corresponding to the new target data selected by the multiple nodes is lower than the last matching degree;

[0124] And so on, until the number of data in the multiple data sets whose matching degree with the query information is higher than the new matching degree is obtained and is greater than or equal to the screening number, the new matching degree is determined as the matching degree threshold.

[0125] In an illustrated embodiment, the determining unit 302 is specifically configured to:

[0126] If, when traversing to the last matching degree, the number of data in the multiple data sets whose matching degree with the query information is higher than the last matching degree obtained by counting is less than the screening number, the matching degree threshold is determined to be negative infinity.

[0127] In an illustrated embodiment, the plurality of nodes include a master node and a slave node; the data aggregation unit 303 is specifically configured to:

[0128] The master node sends the determined matching degree threshold to the slave node, so that the slave node selects inference data having a matching degree with the query information higher than the matching degree threshold from the retrieved data set, and sends the inference data to the master node; and

[0129] The master node selects inference data whose matching degree with the query information is higher than the matching degree threshold from the retrieved data set.

[0130] In an illustrated embodiment, the model reasoning unit 304 is specifically configured to:

[0131] Selecting a preset number of candidate data with the highest matching degree with the query information from the multiple inference data included in the inference data set;

[0132] A prompt word is constructed based on the query information and the candidate data, and the prompt word is input into the LLM model, and the LLM model performs an inference task based on the prompt word to generate a query result corresponding to the query information.

[0133] In an illustrated embodiment, the degree of matching between the query information and each data is represented by the distance between a vector corresponding to the query information and a vector corresponding to each data; wherein, the closer the distance between the vectors, the higher the degree of matching.

[0134] The implementation process of the functions and effects of each unit in the above-mentioned device 30 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 30 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0135] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0136] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0137] Corresponding to the above method embodiment, the embodiment of this specification also provides a computing device. Figure 4 , Figure 4 It is a structural diagram of a computing device provided by an exemplary embodiment. Figure 4 The computing device shown may be the above Figure 1 The computing device in the distributed system shown in the figure includes multiple nodes, each of which stores different data shards. Figure 4 As shown, the computing device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Figure 4As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be executed by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the data reasoning method based on the distributed system in the embodiment of this specification, including: obtaining target data selected by multiple nodes from the data sets retrieved by each node; wherein the data set retrieved by any node contains a number of data matching the query information input by the user in the data shard stored by the node; determining the matching degree between the multiple target data selected by the multiple nodes and the query information, and determining the matching degree threshold from the multiple matching degrees corresponding to the multiple target data; wherein the number of data with a matching degree between the query information and the multiple data sets retrieved by the multiple nodes and higher than the matching degree threshold is greater than or equal to the preset screening number; obtaining the inference data with a matching degree between the query information and the multiple nodes screened from their respective data sets and higher than the matching degree threshold, and aggregating the obtained inference data screened by the multiple nodes into an inference data set; based on the inference data set, assisting the LLM model to perform the inference task and generate the query result corresponding to the query information, etc. For a detailed description of the various steps of the above-mentioned data reasoning method based on the distributed system, please refer to the previous content, which will not be repeated here.

[0138] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored. When these computer programs are executed by a processor, they execute the various steps of the data reasoning method based on a distributed system in the embodiment of this specification. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0139] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0140] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0141] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0143] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0144] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0145] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. A data reasoning method based on a distributed system, characterized in that: The distributed system includes a plurality of nodes, each of which stores a different data shard; the method includes: Obtaining target data selected by the multiple nodes from the data sets retrieved by each node; wherein the data set retrieved by any node contains a number of data matching the query information input by the user in the data slice stored in the node; Determine the matching degree between the multiple target data selected by the multiple nodes and the query information, and determine a matching degree threshold from the multiple matching degrees corresponding to the multiple target data; wherein the number of data with a matching degree higher than the matching degree threshold contained in the multiple data sets retrieved by the multiple nodes and the query information is greater than or equal to a preset screening number; Acquire the inference data filtered out by the multiple nodes from their respective data sets, the inference data having a matching degree with the query information higher than the matching degree threshold, and aggregate the acquired inference data filtered out by the multiple nodes into an inference data set; Based on the reasoning data set, the auxiliary LLM model performs the reasoning task and generates a query result corresponding to the query information.

2. The method according to claim 1, characterized in that The data set retrieved by any node is the nearest neighbor data set matching the query information retrieved from the data shards stored by the node by running the approximate nearest neighbor search algorithm ANNS on the node. The nearest neighbor data set includes a preset number of nearest neighbor data with the highest matching degree with the query information in the data shards stored by the node.

3. The method according to claim 1, characterized in that: The target data is data randomly selected from all the data included in the data set; or, the target data is data selected sequentially from all the data included in the data set according to a preset step size.

4. The method according to claim 1, characterized in that: The step of determining a matching degree threshold from a plurality of matching degrees corresponding to the plurality of target data includes: Sort the multiple matching degrees corresponding to the multiple target data from high to low, and traverse from the first matching degree based on the sorted order; For the currently traversed target matching degree, counting the number of data contained in the multiple data sets whose matching degree with the query information is higher than the target matching degree; and Determine whether the number of data counted is greater than or equal to the screening number. If so, determine the target matching degree as the matching degree threshold and end the traversal; if not, continue to traverse to the next matching degree.

5. The method according to claim 4, characterized in that The method further comprises: If, when traversing to the last matching degree, the number of data in the multiple data sets whose matching degree with the query information is higher than the last matching degree obtained by counting is less than the screening number, then further obtaining new target data selected by the multiple nodes from the respective retrieved data sets; wherein the new matching degree corresponding to the new target data selected by the multiple nodes is lower than the last matching degree; And so on, until the number of data in the multiple data sets whose matching degree with the query information is higher than the new matching degree is obtained and is greater than or equal to the screening number, the new matching degree is determined as the matching degree threshold.

6. The method according to claim 4, characterized in that The method further comprises: If, when traversing to the last matching degree, the number of data in the multiple data sets whose matching degree with the query information is higher than the last matching degree obtained by counting is less than the screening number, the matching degree threshold is determined to be negative infinity.

7. The method according to claim 1, characterized in that The multiple nodes include a master node and a slave node; the obtaining of the inference data filtered out by the multiple nodes from their respective data sets and having a matching degree with the query information higher than the matching degree threshold includes: The master node sends the determined matching degree threshold to the slave node, so that the slave node selects inference data having a matching degree with the query information higher than the matching degree threshold from the retrieved data set, and sends the inference data to the master node; and The master node selects inference data whose matching degree with the query information is higher than the matching degree threshold from the retrieved data set.

8. The method according to claim 1, characterized in that The assisting LLM model to perform the reasoning task based on the reasoning data set to generate a query result corresponding to the query information includes: Selecting a preset number of candidate data with the highest matching degree with the query information from the multiple inference data included in the inference data set; A prompt word is constructed based on the query information and the candidate data, and the prompt word is input into the LLM model, and the LLM model performs an inference task based on the prompt word to generate a query result corresponding to the query information.

9. The method according to any one of claims 1 to 8, characterized in that: The degree of matching between the query information and each data is represented by the distance between the vector corresponding to the query information and the vector corresponding to each data; wherein, the closer the distance between the vectors, the higher the degree of matching.

10. A data reasoning device based on a distributed system, characterized in that: The distributed system includes a plurality of nodes, each of which stores a different data fragment; the device includes: An acquisition unit, configured to acquire target data selected by the plurality of nodes from the data sets retrieved by the plurality of nodes; wherein the data set retrieved by any node includes a number of data matching the query information input by the user in the data slice stored in the node; a determination unit, configured to determine a matching degree between the plurality of target data selected by the plurality of nodes and the query information, and determine a matching degree threshold from the plurality of matching degrees corresponding to the plurality of target data; wherein the number of data whose matching degree with the query information is higher than the matching degree threshold and included in the plurality of data sets retrieved by the plurality of nodes is greater than or equal to a preset screening number; A data aggregation unit, configured to obtain the inference data filtered out by the multiple nodes from their respective data sets, the inference data having a matching degree with the query information higher than the matching degree threshold, and aggregate the obtained inference data filtered out by the multiple nodes into an inference data set; The model reasoning unit is used to assist the LLM model in performing reasoning tasks based on the reasoning data set and generate query results corresponding to the query information.

11. A computing device, characterized in that: include: Memory and processor; The memory has stored thereon computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, it performs the method according to any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that: A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

13. A computer program product, characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images