Data processing method and system

Abstract

The invention provides a data processing method and system. The method comprises the following steps: obtaining a target problem, and obtaining a fact data set; searching from a knowledge graph to obtain first knowledge information related to the target question, and screening out key fact data from the fact data set based on the first knowledge information; second knowledge information related to the target question and the key fact data is retrieved from the knowledge graph; and providing the target question, the key fact data and the second knowledge information to a large model so as to generate answer information corresponding to the target question through the large model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a data processing method and system. Background Art

[0002] With the development of artificial intelligence technology, big models have been widely used in various fields. For example, in customer service and consulting scenarios, in order to quickly answer questions raised by users, the questions can be input into the big model, and the answer to the question can be generated with the help of the reasoning and generation capabilities of the big model.

[0003] At present, when using a large model to answer a target question, the target question and the fact data set are often directly provided to the large model so that the large model can generate the answer to the target question with the fact data set as a reference. In order to enable the large model to deal with a variety of problems, the data in the fact data set provided to the large model is often rich. However, in actual applications, it is found that the answers generated by the large model are not ideal.

[0004] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0005] This specification provides a data processing method and system, which can use the professional knowledge in the knowledge graph to filter out key fact data from the fact data set, and assist the large model to reason based on the key fact data to generate answers to questions, thereby improving the problem of unsatisfactory answers from the large model.

[0006] In a first aspect, the present specification provides a data processing method, comprising: obtaining a target question and obtaining a set of factual data; retrieving first knowledge information related to the target question from a knowledge graph, and filtering out key factual data from the factual data set based on the first knowledge information; retrieving second knowledge information related to both the target question and the key factual data from the knowledge graph; and providing the target question, the key factual data and the second knowledge information to a large model to generate answer information corresponding to the target question through the large model.

[0007] In a second aspect, the present specification also provides a data processing system, comprising: at least one storage medium storing at least one instruction set for processing data; and at least one processor, communicatively connected to the at least one storage medium, wherein when the test system of the data processing system is running, the at least one processor reads the at least one instruction set, and according to the instructions of the at least one instruction set: obtains a target problem and obtains a set of fact data; retrieves first knowledge information related to the target problem from a knowledge graph, and filters out key fact data from the set of fact data based on the first knowledge information; retrieves second knowledge information related to both the target problem and the key fact data from the knowledge graph; and provides the target problem, the key fact data and the second knowledge information to a large model to generate answer information corresponding to the target problem through the large model.

[0008] In a third aspect, the present specification also provides a computer-readable non-volatile storage medium, wherein the computer-readable non-volatile storage medium stores at least one instruction set, and when the at least one instruction set is executed by at least one processor, the data processing method provided in the first aspect is implemented.

[0009] Other functions of the data processing method provided in this specification will be partially listed in the following description. The creative aspects of the data processing method provided in this specification can be fully explained by practicing or using the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 A schematic diagram of an application scenario of a data processing method provided according to an embodiment of this specification is shown;

[0012] Figure 2 A hardware structure diagram of a computing system provided according to an embodiment of this specification is shown;

[0013] Figure 3 A flowchart of a data processing method provided according to an embodiment of this specification is shown;

[0014] Figure 4 A schematic diagram of the structure of a triplet provided according to an embodiment of this specification is shown;

[0015] Figure 5 A schematic diagram of a knowledge graph provided according to an embodiment of this specification is shown;

[0016] Figure 6 A schematic diagram of triple clustering in a knowledge graph provided according to an embodiment of this specification is shown; and

[0017] Figure 7 A schematic diagram of a two-stage search in a data processing method provided according to an embodiment of this specification is shown. DETAILED DESCRIPTION

[0018] The following description provides specific application scenarios and requirements of this specification, with the purpose of enabling those skilled in the art to make and use the contents of this specification. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this specification. Therefore, this specification is not limited to the embodiments shown, but to the widest scope consistent with the claims.

[0019] The terms used herein are only used for the purpose of describing specific example embodiments and are not restrictive. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a", "an" and "the" may also include plural forms. When used in this specification, the terms "include", "comprise" and / or "contain" mean that the associated integers, steps, operations, elements and / or components exist, but do not exclude the existence of one or more other features, integers, steps, operations, elements, components and / or groups or that other features, integers, steps, operations, elements, components and / or groups may be added in the system / method.

[0020] In view of the following description, these and other features of the present specification, as well as the operation and function of the related elements of the structure, and the economy of the combination and manufacture of the parts can be significantly improved. Reference is made to the accompanying drawings, all of which form a part of this specification. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this specification. It should also be understood that the drawings are not drawn to scale.

[0021] The flowcharts used in this specification illustrate the operations implemented by the system according to some embodiments in this specification. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. On the contrary, the operations may be implemented in reverse order or simultaneously. In addition, one or more other operations may be added to the flowchart. One or more operations may be removed from the flowchart.

[0022] For the convenience of description, the terms that will appear in the following text of this specification are first explained.

[0023] Term 1: Schema. In the data field, schema refers to a summary description of any other way to organize and structure data. The schema defines how data is organized, the relationships between data, and the format and type of data. Specifically, the schema can provide a structured framework for data. When building a knowledge graph, the schema defines the entity types in the knowledge graph, the relationship types between entities, and the attribute types.

[0024] Term 2: Triple. Triple is a simple and efficient structure for representing information in computer science, especially in the fields of semantic web, knowledge graph and natural language processing (NLP). A triple usually contains three parts: subject (first knowledge element), predicate (relationship), object (second knowledge element). Triples can be used to describe the relationship or attributes between things. This structure is very suitable for representing knowledge and facts because they can capture the relationship between entities in the smallest unit.

[0025] The following is an introduction to the application scenarios of this manual.

[0026] The technical solution provided in this specification is applicable to the scenario of using a large model to answer questions related to financial analysis. In this scenario, the target question is a question related to financial analysis, and the factual data set includes multiple financial-related factual data. The knowledge graph can include multiple financial knowledge information, and the first knowledge information and the second knowledge information retrieved from the knowledge graph are both financial knowledge information.

[0027] This specification provides a data processing method that can be executed by a corresponding data processing system. The data processing system can first obtain a target question and a set of factual data, retrieve the first knowledge information related to the target question from the knowledge graph, and filter the factual data set based on the first knowledge information to obtain key factual data. Then, the second knowledge information is retrieved from the knowledge graph based on the target question and the key factual data. Finally, based on the target question, the key factual data and the second knowledge information, the answer information corresponding to the target question is generated by the large model. Since the answer information generated by the large model refers to the relevant knowledge in the knowledge graph, the generated answer information is more in line with the target question, which can effectively reduce the probability of hallucination problems when answering questions in the large model.

[0028] It should be noted that the above scenario of using a large model to answer questions related to financial analysis is only one of the multiple usage scenarios provided in this specification. The data processing method provided in this specification can be applied not only to the scenario of using a large model to answer questions related to financial analysis, but also to all scenarios of answering questions based on a large model, such as intelligent customer service scenarios, intelligent assistant scenarios, etc. Those skilled in the art should understand that when the data processing method provided in this specification is applied to other usage scenarios, its implementation method and technical effects are similar.

[0029] Figure 1 FIG. 1 is a schematic diagram showing an application scenario of a data processing method provided according to an embodiment of this specification. Figure 1 As shown, the application scenario 100 may include a data processing system 11, a knowledge graph 12 and a large model 13.

[0030] The data processing system 11 can receive the target question and obtain the fact data set, and obtain the key fact data and the second knowledge information based on the target question, the fact data set and the knowledge graph 12, based on the two-stage search of the knowledge graph. Then, the data processing system 11 provides the target question, the key fact data and the second knowledge information to the big model 13, so that the big model 13 generates the answer information corresponding to the target question.

[0031] In some embodiments, the data processing method may be executed on the data processing system 11. In this case, the data processing system 11 may store data or instructions for executing the data processing method described in this specification, and may execute or be used to execute the data or instructions. In some embodiments, the data processing system 11 may include a hardware device with a data information processing function and a necessary program required to drive the hardware device to work.

[0032] The data processing system 11 may correspond to a single computing device or a computing cluster composed of multiple computing devices.

[0033] In some embodiments, the system architecture of the data processing system 11 may further include a client and a server. In this case, the data processing system 11 may include a terminal device and a server. The client is deployed on the terminal device, and the server is deployed on the server. The data processing method may be executed by the terminal device alone, by the server alone, or by the terminal device and the server in collaboration.

[0034] In some embodiments, the terminal device may include a mobile device, a tablet computer, a laptop computer, a built-in device of a motor vehicle, or the like, or any combination thereof. In some embodiments, the mobile device may include a smart home device, a smart mobile device, a virtual reality device, an augmented reality device, or the like, or any combination thereof. In some embodiments, the smart home device may include a smart TV, a desktop computer, etc., or any combination thereof. In some embodiments, the smart mobile device may include a smart phone, a personal digital assistant, a gaming device, a navigation device, etc., or any combination thereof. In some embodiments, the virtual reality device or the augmented reality device may include a virtual reality helmet, virtual reality glasses, a virtual reality patch, an augmented reality helmet, augmented reality glasses, an augmented reality patch, or the like, or any combination thereof. For example, the virtual reality device or the augmented reality device may include Google Glass, a head-mounted display, VR, etc. In some embodiments, the built-in device in the motor vehicle may include an onboard computer, an onboard TV, etc.

[0035] In some embodiments, the knowledge graph 12 may be stored in an entity that executes the data processing method, or may be stored in an independent database. For example, when the data processing method is executed by a terminal device, the knowledge graph 12 may be stored in the terminal device or the server. When the knowledge graph 12 is stored in the server, the electronic device may access the server through the network to retrieve the knowledge graph 12; or, when the knowledge graph 12 is stored in the memory of the terminal device, the terminal device may directly retrieve the knowledge graph 12 in the memory.

[0036] In some embodiments, when the data processing method is executed by a terminal device, the terminal device may be installed with one or more application programs (APPs). The APP can provide the ability and interface to receive target questions and obtain fact data sets. The APP includes but is not limited to: web browser APP programs, search APP programs, chat APP programs, shopping APP programs, video APP programs, financial management APP programs, instant messaging tools, email clients, social platform software, etc. In some embodiments, a target APP may be installed on the terminal device. The terminal device can receive the target question and obtain the fact data set through the target APP. In some embodiments, the target APP can execute the data processing method described in this specification in response to receiving the target question and obtaining the fact data set. The fact data set can be obtained through the interactive interface of the APP, or it can be obtained from the target database. The target database is used to store fact data in a specific field.

[0037] In some embodiments, the large model 13 may be integrated into the data processing system 11, or deployed on a device external to the data processing system 11. When the large model 13 is located in an external device of the data processing system 11, the data processing system 11 may interact with the application programming interface (API) provided by the large model through a network to send data to the large model and receive data fed back by the large model. As an example, the large model 13 may be a large language model (LLM).

[0038] It should be understood that Figure 1 The number of data processing systems 11, knowledge graphs 12, and large models 13 in the embodiment is only for illustration. Any number of data processing systems 11, knowledge graphs 12, and large models 13 may be provided according to implementation requirements.

[0039] Figure 2 200 is a hardware structure diagram of a computing system provided according to an embodiment of the present specification. Figure 1 The data processing system 11 in the embodiment executes the data processing method described in this specification.

[0040] like Figure 2 As shown, the computing system 200 may include at least one storage medium 230 and at least one processor 220. In some embodiments, the computing system 200 may also include a communication port 250 and an internal communication bus 210. The computing system 200 may also include an I / O component 260.

[0041] The internal communication bus 210 can connect different system components. For example, the internal communication bus 210 can connect the storage medium 230, the processor 220, the communication port 250, and the I / O component 260, etc.

[0042] I / O components 260 support input / output between computing system 200 and other components.

[0043] The communication port 250 is used for data communication between the computing system 200 and the outside world. For example, the communication port 250 can be used for data communication between the computing system 200 and the network 140. The communication port 250 can be a wired communication port or a wireless communication port.

[0044] The storage medium 230 may include a data storage device. The data storage device may be a non-temporary storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 232, a read-only storage medium (ROM) 234, or a random access storage medium (RAM) 235. The storage medium 230 also includes at least one instruction set stored in the data storage device. The instruction set may include computer program code, and the computer program code may include a program, a routine, an object, a component, a data structure, a process, a module, and the like.

[0045] At least one processor 220 may be in communication with at least one storage medium 230. When the computing system 200 is running, at least one processor 220 reads the at least one instruction set and executes the data processing method provided in this specification according to the instructions of the at least one instruction set. The processor 220 may execute the steps included in the data processing method. The processor 220 may be in the form of one or more processors. In some embodiments, the processor 220 may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof.

[0046] For illustration purposes only, the computing system 200 in the figure shows only one processor 220. However, it should be noted that the computing system 200 in this specification may also include multiple processors, and therefore, the operations and / or method steps disclosed in this specification may be performed by one processor or by multiple processors in combination. For example, if the processor 220 of the computing system 200 is described in this specification as performing step A and step B, it should be understood that step A and step B may also be performed jointly or separately by two different processors 220 (e.g., the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0047] Figure 3 A flow chart of a data processing method provided according to an embodiment of the present specification is shown. As before, the data processing system 11 can execute the data processing method of the present specification.

[0048] like Figure 3 As shown, the data processing method may include:

[0049] S310: Obtain a target question and obtain a fact data set.

[0050] In some embodiments, the fact data set may be a data set related to the field involved in the target problem. For example, when the target problem is related to financial analysis, the field involved in the target problem is the financial field. The fact data set may include financial research report data, financial professional knowledge data, financial-related news data, and operating data of financial entities related to the target problem.

[0051] It should be noted that the fact data in the fact data set may be related to the target problem or may not be related to the target problem.

[0052] In some embodiments, the target question may be a question that the data processing system receives directly from the user. For example, the data processing system may receive a question directly input by the user through an interface provided by the target APP. The fact data set may be input by the user in the interface of the target APP, or may be obtained from a target database. The target database is used to store fact data in a specific field.

[0053] S320: Retrieve first knowledge information related to the target problem from the knowledge graph, and filter out key fact data from the fact data set based on the first knowledge information.

[0054] In some embodiments, the knowledge graph is a knowledge graph related to the field involved in the target problem. As an example, when the target problem is related to financial analysis, the knowledge graph is a knowledge graph in the financial field.

[0055] The knowledge graph can be pre-constructed based on knowledge data in a specific field. This specification does not limit the construction process of the knowledge graph. The following is an example of a possible construction method.

[0056] In some embodiments, when constructing a knowledge graph for a field, a knowledge dataset corresponding to the field can be prepared first. For example, in the financial field, based on prompt engineering, examples can be constructed, tasks can be decomposed, and other methods can be used to filter out texts that meet the requirements from financial research report data. Then, the filtered texts are annotated with respect to reasoning links and reasoning results to obtain a knowledge dataset with clear logical links.

[0057] Then, through LLM, the finance-related rules in the knowledge dataset are classified, and corresponding schemas and examples are provided for the rule classification (such as the hierarchical relationship and comparative relationship between two pieces of financial knowledge).

[0058] Next, based on the schema and examples, the hint engineering method of the Large Language Model (LLM) can be used to extract triples, and the extracted triples can be manually verified.

[0059] Finally, the triples extracted and verified through the above steps are integrated to construct a structured financial logic graph, which is the knowledge graph in the financial field and can clearly represent various logical relationships and causal links in the financial field.

[0060] Figure 4 A schematic diagram of the structure of a triplet provided according to an embodiment of the present specification is shown.

[0061] In some embodiments, each triple includes a first knowledge element, a second knowledge element, and a relationship between the first knowledge element and the second knowledge element. Figure 4 , the relationship between the first knowledge element and the second knowledge element is represented by an arrow.

[0062] In some embodiments, the first knowledge element and the second knowledge element may both be entities. For example, the first knowledge element and the second knowledge element may both be one of the following entities: personnel, company name, stock name, fund name, product name, professional terminology, etc. The relationship between the first knowledge element and the second knowledge element may include a subordinate relationship, a hierarchical relationship, a logical relationship, a comparative relationship, etc.

[0063] For example, refer to Figure 4 , assuming that the first knowledge element is person A, the second knowledge element is company B, and the relationship is belongs to, then the triple means "person A belongs to company B".

[0064] In some embodiments, the first knowledge element may be an entity, the second knowledge element may be an attribute value, and the relationship between the two represents an attribute type. In this case, the second knowledge element may be an attribute value of the first knowledge element.

[0065] For example, refer to Figure 4 , assuming that the first knowledge element is company A, the relationship between the first knowledge element and the second knowledge element is used to represent the attribute "market value", and the second knowledge element is the value of the attribute "market value". In this case, the triplet represents "the market value of company A is XXX".

[0066] In some embodiments, the first knowledge element and the second knowledge element in the triplet correspond to a type respectively. For example, the type may include an individual, an organization, a product, a stock type, a fund type, a term or an attribute value, etc. For example, referring to the above example, when the triplet represents "personnel A belongs to company B", the type of the first knowledge element may be "individual" and the type of the second knowledge element may be "organization". When the triplet represents "the market value of company A is XXX", the type of the first knowledge element may be "organization" and the type of the second knowledge element may be "attribute value".

[0067] Figure 5 A schematic diagram of a knowledge graph provided according to an embodiment of the present specification is shown.

[0068] In some embodiments, reference Figure 5 ,The knowledge graph includes multiple triples, among which at least two triples can be connected in series to represent a logical relationship or a causal link.,Taking the financial field as an example, these triples can represent various logical relationships and causal,links in the financial field.

[0069] In some embodiments, each triple in the knowledge graph is divided into M clusters, and each triple in each cluster satisfies a preset similarity condition. The preset similarity condition includes at least one of the following: the type of the first knowledge element in each triple is the same; the type of the second knowledge element in each triple is the same; or the type of the relationship between the first knowledge element and the second knowledge element in each triple is the same.

[0070] Figure 6 A schematic diagram of triple clustering in a knowledge graph provided according to an embodiment of the present specification is shown.

[0071] In some embodiments, it is assumed that the preset similarity condition includes: the type of the first knowledge element in each triple is the same, and the type of the second knowledge element in each triple is the same. The data processing system processes the data according to the preset similarity condition. Figure 5 By clustering the triples in the knowledge graph shown in Figure 6 The clustering results are shown. Figure 6 , each triple in the recognition graph is divided into 4 clusters, including "Cluster 1", "Cluster 2", "Cluster 3" and Cluster 4". In Cluster 1, the type of the first knowledge element is "Type A" and the type of the second knowledge element is "Type B". In Cluster 2, the type of the first knowledge element is "Type A" and the type of the second knowledge element is "Type C". In Cluster 3, the type of the first knowledge element is "Type B" and the type of the second knowledge element is "Type D". In Cluster 4, the type of the first knowledge element is "Type D" and the type of the second knowledge element is "Type C".

[0072] In this embodiment, by clustering the triples in the knowledge graph according to the type of knowledge element, the types of knowledge elements of the triples in the same cluster in the knowledge graph are the same, while the types of knowledge elements of the triples in different clusters are different. In subsequent retrieval, the data processing system can accurately distinguish the first knowledge elements and / or second knowledge elements that have high text similarity but belong to different types based on clustering, thereby improving the accuracy of the retrieval and making the answer information generated based on the retrieval results more accurate.

[0073] Figure 7 A schematic diagram of a two-stage search in a data processing method provided according to an embodiment of this specification is shown.

[0074] In some embodiments, reference Figure 7 The data processing system can obtain key factual data and second knowledge information based on the target question and the factual data set, based on the two-stage retrieval of the knowledge graph, and provide the target question, key factual data and second knowledge information to the big model to instruct the big model to generate answer information corresponding to the target question.

[0075] In some embodiments, the data processing system retrieves the first knowledge information related to the target problem from the knowledge graph as the first stage retrieval. The data processing system retrieves the second knowledge information related to both the target problem and the key factual data from the knowledge graph as the second stage retrieval. Among them, the first knowledge information includes: K1 first triples in the knowledge graph, the knowledge elements in the first triples are related to the target problem, and K1 is an integer greater than or equal to 1. The second knowledge information includes: K2 second triples in the knowledge graph, the knowledge elements in the second triples are related to both the target problem and the key factual data, and K2 is an integer greater than or equal to 1.

[0076] In some embodiments, the data processing system may search the knowledge graph through a preset searcher. When the preset searcher searches the knowledge graph according to the search information, the distribution of the retrieved triples in the M clusters is related to the relevance between the search information and each cluster.

[0077] Based on this, the preset retriever can be configured as follows: when the correlation between the search information received by the preset retriever and the first cluster is greater than the correlation between the search information and the second cluster, the number of triples output by the preset retriever that fall into the first cluster is greater than the number that fall into the second cluster, and the first cluster and the second cluster are any two different clusters among the M clusters. That is, when the data processing system searches through the preset retriever, if the correlation between a cluster and the search information is higher, the number of triples obtained from the cluster is greater.

[0078] In this embodiment, the data processing system can effectively reduce a large number of redundant triples in the search results by clustering the triples and obtaining a corresponding number of triples from each cluster according to the relevance between the search information and each cluster. Since the triples obtained by the search are more representative, the answer information generated according to the search results is more accurate.

[0079] In some embodiments, the retrieval process of the preset retriever includes:

[0080] (1) The data processing system determines the relevance between the retrieval information and the M clusters.

[0081] (2) The data processing system determines the number of triplets to be output corresponding to the M clusters respectively according to the relevance between the search information and the M clusters respectively, and the number K of triplets to be output by the preset retriever.

[0082] (3) The data processing system obtains and outputs a corresponding number of triplets from the M clusters according to the number of triplets to be output corresponding to the M clusters.

[0083] In some embodiments, the data processing system may adopt the following method when determining the relevance between the search information and the M clusters respectively: according to the received search information, retrieve N reference triples from the knowledge graph and the similarity between the N reference triples and the search information. The data processing system determines the relevance between the search information and the M clusters respectively according to the distribution of the N reference triples in the M clusters and the similarity between the N reference triples and the search information.

[0084] In some embodiments, the data processing system may calculate the similarity between the retrieved information and each triple in the knowledge graph, and determine the N triples with the highest similarity as reference triples.

[0085] In some embodiments, when determining the relevance between the search information and the M clusters, the data processing system may determine, for any i-th cluster among the M clusters: P of the N reference triples falling into the i-th cluster; i reference triples, based on P i The average similarity is determined by the similarity between the reference triples and the retrieval information, and the average similarity is used as the correlation between the retrieval information and the i-th cluster, where P i is an integer greater than or equal to 0.

[0086] As an example, assuming that N is 100, among the 100 reference triples, 10 triples belong to cluster 1 (P1=10). For cluster 1, the data processing system can obtain the average value of the similarities between the 10 reference triples belonging to cluster 1 and the search information as the relevance between the search information and the first cluster.

[0087] In some embodiments, the data processing system may normalize the correlations between the search information and the M clusters, and use the normalized values ​​as weights of the M clusters. And the data processing system determines the number of triples to be output corresponding to each cluster in the M clusters based on the product of the weight of the cluster and K.

[0088] As an example, assume that the number of triples to be output by the preset retriever is K, which is 10, and there are 4 clusters (M is 4), the weight corresponding to cluster 1 is 0.2, the weight corresponding to cluster 2 is 0.3, the weight corresponding to cluster 3 is 0.4, and the weight corresponding to cluster 4 is 0.1. Then the number of triples to be output corresponding to cluster 1 is 0.2*10=2, the number of triples to be output corresponding to cluster 2 is 0.3*10=3, the number of triples to be output corresponding to cluster 3 is 0.4*10=4, and the number of triples to be output corresponding to cluster 4 is 0.1*10=1.

[0089] In some embodiments, the M clusters respectively correspond to cluster features. When determining the relevance between the search information and the M clusters respectively, the data processing system may adopt the following method: based on the received search information and the cluster features corresponding to the M clusters respectively, determine the relevance between the search information and the M clusters.

[0090] In some embodiments, the cluster feature is used to represent the knowledge elements involved in the triples in the cluster. The cluster features corresponding to each of the M clusters may be preset. The correlation between the search information and the M clusters may be the similarity between the search information and each cluster feature.

[0091] In some embodiments, when determining the triples to be output corresponding to each cluster, the data processing system may adopt the following method: for any i-th cluster among the M clusters, the reference triples that fall into the i-th cluster among the N reference triples form a target set. If the number of triples in the target set is P i Greater than or equal to the number of triplets to be output corresponding to the i-th cluster k i , then select the top k with the highest similarity from the target set i triplets as the triplets to be output for the i-th cluster. If the number of triplets in the target set is P i Less than the number of triplets to be output corresponding to the i-th cluster k i, then based on the retrieval information, re-retrieve k from the i-th cluster i The triples are taken as the triples to be output for the i-th cluster.

[0092] As an example, assume that N is 100, the number of triples to be output by the preset retriever is K, there are 4 clusters in total (M is 4), the number of triples to be output corresponding to cluster 1 is k1=2, the number of triples to be output corresponding to cluster 2 is k2=3, the number of triples to be output corresponding to cluster 3 is k3=4, and the number of triples to be output corresponding to cluster 4 is k4=1. If 10 of the N reference triples retrieved from the knowledge graph fall into cluster 1, that is, the number of triples in the target set corresponding to cluster 1 is P1=10. Since P1 is greater than k1, the data processing system can select the first two triples with the highest similarity from the triples in the target set corresponding to cluster 1 as the triples to be output from cluster 1.

[0093] If 3 of the N reference triples retrieved from the knowledge graph belong to cluster 3, then the number of triples in the target set corresponding to cluster 3, P3, is 3. Since P3 is less than k3, the data processing system can select the top 4 triples with the highest similarity from the triples corresponding to cluster 3 in the knowledge graph as the triples to be output for cluster 3 based on the retrieval information.

[0094] In some other embodiments, the data processing system determines the number k of triples to be output corresponding to the i-th cluster. i , based on the retrieval information, select the top k with the highest similarity from the i-th cluster in the knowledge graph i The triples are taken as the triples to be output for the i-th cluster.

[0095] As an example, the number of triples K to be output by the preset retriever is 10, there are 4 clusters in total (M is 4), the number of triples to be output corresponding to cluster 1 is k1=2, the number of triples to be output corresponding to cluster 2 is k2=3, the number of triples to be output corresponding to cluster 3 is k3=4, and the number of triples to be output corresponding to cluster 4 is k4=1. The data processing system can select the first two triples with the highest similarity from cluster 1 in the knowledge graph as the triples to be output for cluster 1 according to the retrieval information; select the first three triples with the highest similarity from cluster 2 in the knowledge graph as the triples to be output for cluster 2; select the first four triples with the highest similarity from cluster 3 in the knowledge graph as the triples to be output for cluster 3; and select the 1 triple with the highest similarity from cluster 4 in the knowledge graph as the triple to be output for cluster 4.

[0096] In some embodiments, the data processing system retrieves first knowledge information related to the target problem from the knowledge graph, which may include: the data processing system generates first retrieval information based on the target problem, inputs the first retrieval information into a preset retriever to retrieve K1 first triples from the knowledge graph through the preset retriever.

[0097] In some embodiments, the first search information may be a keyword related to the target question, or a query instruction generated based on the target question. The data processing system inputs the first search information into a preset searcher, and the process of retrieving K1 first triples from the knowledge graph through the preset searcher is the first stage search process. The first stage search process can be expressed by formula 1:

[0098] K1=Retriever(Q)→(Formula 1)

[0099] Where Retriever is a preset retriever, Q is a target question, and arrows are used to indicate execution. This formula indicates that the first retrieval information is generated based on Q, and the preset retriever is executed to perform retrieval based on the first retrieval information to obtain K1 first triples.

[0100] It should be noted that the retrieval process of the preset retriever can refer to the above example and will not be elaborated here.

[0101] In some embodiments, filtering out key fact data from a fact data set based on the first knowledge information includes: the data processing system provides the target question, the first knowledge information, and the fact data set to the big model, so as to filter out the key fact data from the fact data set through the big model. The above process can be expressed by Formula 2:

[0102] F select =LLM(Q, K1, F, prompt) → (Formula 2)

[0103] Among them, F is the fact data set, F select is the key fact data, LLM is the large model, Q is the target question, K1 is the first knowledge information (i.e., K1 first triples), and prompt represents the large model guidance instruction generated according to Q, K1 and F.

[0104] As an example, the data processing system can generate a large model guidance instruction based on Q, K1 and F based on the following template: "You are an expert in the financial field. You need to filter out key factual data related to the target problem from the factual data set based on the target problem, knowledge information and factual data set I provide. Target problem:' <q>'; Knowledge information:' <k1>';Facts:' <f>'."in," <q>”、" <k1>"and" <f>" is a placeholder used to refer to the real target problem, real knowledge information, and real fact data set filled in during actual application. After receiving the big model guidance instruction, the big model can execute the guidance instruction and output the key fact data obtained by reasoning.

[0105] S330: Retrieve second knowledge information related to the target question and key factual data from the knowledge graph.

[0106] In some embodiments, the data processing system will be able to generate second retrieval information based on the target question and key factual data, and input the second retrieval information into a preset retriever to retrieve K2 second triples from the knowledge graph through the preset retriever.

[0107] In some embodiments, the second search information may be keywords related to the target question and key factual data, or query instructions generated based on the target question and key factual data. The data processing system inputs the second search information into a preset retriever, and the process of retrieving K2 second triples from the knowledge graph through the preset retriever is the second stage search process. The second stage search process can be expressed by formula three:

[0108] K2=Retriever(Q,F select )→(Formula 3)

[0109] Retriever is the preset retriever, Q is the target question, and F select The key facts are shown in Figure 1. The arrows are used to indicate execution. select Generate second search information, and execute a preset retriever to perform a search based on the second search information to obtain K2 second triples.

[0110] It should be noted that the retrieval process of the preset retriever can refer to the example in S320, which will not be described in detail here.

[0111] S340: Provide the target question, key fact data and second knowledge information to the big model to generate answer information corresponding to the target question through the big model.

[0112] In some embodiments, the data processing system may provide the target question, key factual data, and second knowledge information to the big model, so as to generate answer information corresponding to the target question through the big model. The above process can be expressed by formula 4:

[0113] A=LLM(Q,F select , K2, prompt) → (Formula 4)

[0114] Among them, A is the answer information corresponding to the target question generated by the large model, LLM is the large model, and F select is the key fact data, Q is the target question, K2 is the second knowledge information (i.e., K2 second triples), and prompt means the answer based on Q, K2, and F select Generated large model boot instructions.

[0115] As an example, the data processing system can be based on Q, K2 and F select , based on the following template, generate a large model guidance instruction: "You are an expert in the financial field. You need to analyze and answer the target question based on the target question, knowledge information and key factual data I provide. Target question:' <q>'; Knowledge information:' <k2>'Key facts:' <F select >'. "Among them, " <q>”、" <k2>"and" <F select >" is a placeholder used to refer to the real target question, real knowledge data, and real key fact data set filled in during actual application. After receiving the big model guidance instruction, the big model can execute the guidance instruction and output the answer information corresponding to the target question obtained by reasoning.

[0116] In some embodiments, after generating answer information corresponding to the target question, the data processing system can also obtain the score corresponding to the answer information; and when the score is greater than a preset threshold, a target sample is generated based on the target question, key factual data, second knowledge information, and answer information, and the target sample is added to a sample library, and the data in the sample library is used to fine-tune the large model.

[0117] In some embodiments, the data processing system can generate a scoring task based on the answer information corresponding to the target question and present the scoring task to the operator. When the data processing system receives the score given by the operator based on the feedback of the scoring task, if the score is greater than a preset threshold, it can be determined that the answer information corresponding to the scoring task has good quality. The data processing system can generate corresponding target samples based on these good quality answer information and the corresponding target question, key fact data, and second knowledge information, and add the target samples to the sample library.

[0118] The data processing system can use the target samples in the sample library to fine-tune the large model to improve the quality of the answer information corresponding to the target question generated by the large model. The data format of the target sample can refer to Formula 5:

[0119] M(I,X)=Y→(Formula 5)

[0120] Wherein, M represents a large model (LLM), I represents an instruction for instructing the large model to perform fine-tuning, X represents the input of the large model, X may include a target question, key factual data, and second knowledge information, and Y represents answer information.

[0121] In this embodiment, the data processing system generates corresponding target samples through answer information with good quality and corresponding target questions, key fact data, and second knowledge information, and uses the target samples to fine-tune the large model. The data processing method provided in this specification can be applied to large models with insufficient reasoning capabilities. By fine-tuning large models with insufficient reasoning capabilities through target samples, these large models can maintain a high accuracy when generating answer information corresponding to the target questions.

[0122] In summary, in the data processing method and system provided in this specification, according to the acquired target question and fact data set, the first knowledge information related to the target question is retrieved from the knowledge graph, and the key fact data is screened out from the fact data set based on the first knowledge information. Then, the second knowledge information related to both the target question and the key fact data is retrieved from the knowledge graph. Finally, the target question, the key fact data and the second knowledge information are provided to the large model to generate the answer information corresponding to the target question through the large model. Through the pre-constructed knowledge graph, the first knowledge information in the knowledge graph is used to screen out the key fact data from the fact data set when answering the target question, which can avoid the interference of irrelevant fact data in the fact data set on the reasoning of the large model, thereby improving the accuracy of the answer information generated by the large model. Furthermore, while providing the target question and key fact data to the large model, the second knowledge information obtained from the knowledge graph is also provided to the large model, so that the large model has the knowledge required to answer the target question, thereby making the answer information generated by the large model more accurate, further improving the accuracy and practicality of the answer information.

[0123] On the other hand, the present specification provides a computer-readable non-transitory storage medium, storing at least one instruction set for image recognition. When the at least one instruction set is executed by the processor, the at least one instruction set instructs the processor to implement the steps of the data processing method described in the present specification. In some possible implementations, various aspects of the present specification can also be implemented in the form of a program product, which includes a program code. When the program product is run on the computing system 200, the program code is used to enable the computing system 200 to perform the steps of the data processing method described in the present specification. The program product for implementing the above method can use a portable compact disk read-only memory (CD-ROM) to include program code and can be run on the computing system 200. However, the program product of the present specification is not limited to this. In the present specification, the readable storage medium can be any tangible medium containing or storing a program, which can be used by the instruction execution system or used in combination with it. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media include: an electrical connection with one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above. Program code for performing the operations of the present specification may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the computing system 200, partially on the computing system 200, as a stand-alone software package, partially on the computing system 200 and partially on a remote computing device, or entirely on a remote computing device.

[0124] The above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require a specific order or a continuous order to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0125] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented only by way of example and may not be limiting. Although not explicitly stated herein, those skilled in the art will appreciate that this specification requires various reasonable changes, improvements and modifications to the embodiments. These changes, improvements and modifications are intended to be proposed by this specification and are within the spirit and scope of the exemplary embodiments of this specification.

[0126] In addition, certain terms in this specification have been used to describe embodiments of this specification. For example, "one embodiment", "embodiments" and / or "some embodiments" mean that a particular feature, structure or characteristic described in conjunction with the embodiment may be included in at least one embodiment of this specification. Therefore, it can be emphasized and should be understood that two or more references to "an embodiment" or "one embodiment" or "an alternative embodiment" in various parts of this specification do not necessarily refer to the same embodiment. In addition, particular features, structures or characteristics may be appropriately combined in one or more embodiments of this specification.

[0127] It should be understood that in the foregoing description of the embodiments of this specification, in order to help understand a feature and for the purpose of simplifying this specification, this specification combines various features in a single embodiment, figure or its description. However, this does not mean that the combination of these features is necessary. When reading this specification, it is entirely possible for a person skilled in the art to mark out some of the devices as separate embodiments. In other words, the embodiments in this specification can also be understood as the integration of multiple secondary embodiments. And the content of each secondary embodiment is also valid when it is less than all the features of a single aforementioned disclosed embodiment.

[0128] Each patent, patent application, publication of patent applications, and other materials, such as articles, books, specifications, publications, documents, articles, etc., cited herein, except to the extent that it is inconsistent or conflicting with this document or that has a limiting effect on the broadest scope of the claims, may be incorporated herein by reference and used for all purposes now or hereafter associated with this document. In addition, in the event of any inconsistency or conflict between the description, definition, and / or use of a term in any material and the description, definition, and / or use of a term in this document, the term in this document shall prevail.

[0129] Finally, it should be understood that the embodiments of the application disclosed herein are explanations of the principles of the embodiments of this specification. Other modified embodiments are also within the scope of this specification. Therefore, the embodiments disclosed in this specification are only used as examples and not as limitations. Those skilled in the art can adopt alternative configurations according to the embodiments in this specification to implement the applications in this specification. Therefore, the embodiments of this specification are not limited to the embodiments accurately described in the application. < / q> < / q> < / f> < / q> < / f> < / q>

Claims

1. A data processing method, comprising: Get the target problem and obtain a set of factual data; Retrieving first knowledge information related to the target problem from the knowledge graph, and filtering out key fact data from the fact data set based on the first knowledge information; Retrieving second knowledge information related to the target question and the key factual data from the knowledge graph; as well as The target question, the key fact data and the second knowledge information are provided to the big model so as to generate answer information corresponding to the target question through the big model.

2. The method according to claim 1, wherein: The knowledge graph includes a plurality of triples, each triple including a first knowledge element, a second knowledge element, and a relationship between the first knowledge element and the second knowledge element; The first knowledge information includes: K1 first triples in the knowledge graph, the knowledge elements in the first triples are related to the target problem, and K1 is an integer greater than or equal to 1; The second knowledge information includes: K2 second triples in the knowledge graph, the knowledge elements in the second triples are related to the target question and the key factual data, and K2 is an integer greater than or equal to 1.

3. The method according to claim 2, wherein: Each triple in the knowledge graph is divided into M clusters, and each triple in each cluster satisfies a preset similarity condition; The retrieving first knowledge information related to the target question from the knowledge graph includes: generating first retrieval information based on the target question, inputting the first retrieval information into a preset retriever to retrieve the K1 first triples from the knowledge graph through the preset retriever; The retrieving second knowledge information related to the target question and the key fact data from the knowledge graph includes: generating second retrieval information based on the target question and the key fact data, and inputting the second retrieval information into the preset retriever to retrieve the K2 second triples from the knowledge graph through the preset retriever; In which, the preset retriever is configured as follows: when the correlation between the retrieval information received by the preset retriever and the first cluster is greater than the correlation between the retrieval information and the second cluster, the number of triples output by the preset retriever that fall into the first cluster is greater than the number that fall into the second cluster, and the first cluster and the second cluster are any two different clusters among the M clusters.

4. The method according to claim 3, wherein: The retrieval process of the preset retriever includes: According to the received search information, N reference triples and similarities between the N reference triples and the search information are retrieved from the knowledge graph; Determine, according to the distribution of the N reference triples in the M clusters and the similarity between the N reference triples and the search information, the relevance between the search information and the M clusters respectively; Determining the number of triplets to be output corresponding to the M clusters respectively according to the relevance between the search information and the M clusters respectively, and the number K of triplets to be output by the preset searcher; and According to the numbers of triples to be output corresponding to the M clusters, corresponding numbers of triples are obtained from the M clusters and output.

5. The method according to claim 4, wherein: The determining, according to the distribution of the N reference triples in the M clusters and the similarity between the N reference triples and the search information, the relevance between the search information and the M clusters respectively includes: For any i-th cluster among the M clusters, determine P that falls into the i-th cluster from the N reference triples i reference triples, based on the P i The similarities between the reference triples and the search information are used to determine the average similarity, and the average similarity is used as the correlation between the search information and the i-th cluster, wherein the P i is an integer greater than or equal to 0.

6. The method according to claim 4, wherein: Determining the number of triplets to be output corresponding to the M clusters respectively according to the relevance between the search information and the M clusters respectively, and the number K of triplets to be output by the preset retriever, comprises: Normalizing the correlations between the search information and the M clusters respectively, and using the normalized values ​​as weights of the M clusters; and For each cluster in the M clusters, based on the product of the weight of the cluster and K, the number of to-be-output triplets corresponding to the cluster is determined.

7. The method according to claim 4, wherein: According to the number of triples to be output corresponding to the M clusters, respectively obtaining and outputting corresponding numbers of triples from the M clusters, comprises: For any i-th cluster among the M clusters, reference triples in the N reference triples that fall into the i-th cluster form a target set, and perform any one of the following: If the number of triples in the target set is P i Greater than or equal to the number k of triplets to be output corresponding to the i-th cluster i , then select the top k with the highest similarity from the target set i triplets as the triplets to be output for the i-th cluster; or If the number of triples in the target set is P i Less than the number k of triplets to be output corresponding to the i-th cluster i , then based on the retrieval information, k are retrieved again from the i-th cluster i The triples are used as the triples to be output by the i-th cluster.

8. The method according to claim 3, wherein: The M clusters respectively correspond to cluster features, and the retrieval process of the preset retriever includes: Determine, based on the received search information and the cluster features corresponding to each of the M clusters, the relevance between the search information and the M clusters; Determining the number of triplets to be output corresponding to the M clusters respectively according to the relevance between the search information and the M clusters respectively, and the number K of triplets to be output by the preset searcher; and According to the numbers of triples to be output corresponding to the M clusters, corresponding numbers of triples are obtained from the M clusters and output.

9. The method according to claim 3, wherein: The preset similarity condition includes at least one of the following: The first knowledge element in each triple is of the same type; The types of the second knowledge elements in each triple are the same; or The type of relationship between the first knowledge element and the second knowledge element in each triple is the same.

10. The method according to claim 1, wherein: The step of selecting key fact data from the fact data set based on the first knowledge information includes: The target question, the first knowledge information and the fact data set are provided to a large model, so as to filter out the key fact data from the fact data set through the large model.

11. The method according to claim 1, wherein: After generating answer information corresponding to the target question, the method further includes: Obtaining a score corresponding to the answer information; and When the score is greater than a preset threshold, a target sample is generated according to the target question, the key factual data, the second knowledge information, and the answer information, and the target sample is added to a sample library. The data in the sample library is used to fine-tune the large model.

12. A data processing system comprising: at least one storage medium storing at least one instruction set for processing data; as well as At least one processor is communicatively connected to the at least one storage medium, wherein when the data processing system is running, the at least one processor reads the at least one instruction set and implements the method as described in claims 1-11 according to the instructions of the at least one instruction set.

Images