Knowledge retrieval method based on causal reasoning, terminal and storage medium
By constructing a structural causal graph in knowledge retrieval and using the attention layer of causal reasoning to interact between documents, the causal semantic feature vector is generated, and the problem of difficult to take into account in the existing technology is solved, and efficient and accurate knowledge retrieval is achieved.
Patent Information
- Application Number
- CN202510240540.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-03
- Publication Date
- 2025-06-20
AI Technical Summary
Existing knowledge retrieval methods are difficult to balance between improving search accuracy and speed, and ignore semantic interactions between different documents, resulting in inaccurate sorting of documents containing error information.
By constructing a structural causal graph, the attention layer based on causal reasoning is used to interact between documents and between documents and with causal graph nodes, a word-level causal semantic feature vector is generated, and the recalled documents are sorted through similarity matching.
It realizes the acceleration of the search process while improving the search accuracy, reduces the document scores containing error information, and improves the accuracy of the search results.
Smart Images
Figure CN120179798A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of deep learning, and in particular to a knowledge retrieval method based on causal reasoning, as well as a computer terminal and a computer-readable storage medium applying the method. Background Art
[0002] Existing knowledge retrieval mainly recalls relevant documents through a document recall model according to a query input by a user, then encodes the query and the documents respectively by using two encoders, or splices the query and the recalled documents and then uses one encoder for semantic encoding, and finally calculates the scores of the recalled documents by using similarity functions such as cosine for sorting.
[0003] However, the above methods have some defects. First, encoding the query and the documents separately by using two encoders abandons the information interaction between the query and the documents. Although the retrieval speed is relatively fast, a certain retrieval accuracy is lost. Second, the method of splicing the query and the recalled documents and then using one encoder for semantic encoding requires splicing the query and each recalled document. Although the retrieval accuracy is high, the retrieval speed is too slow. Finally, whether it is the method of separate encoding or the method of encoding after splicing, the semantic interaction between different documents is ignored, and the information between different documents cannot be used for interaction, reducing the score ranking of documents containing incorrect information. Summary of the Invention
[0004] To solve the technical problems existing in the prior art, the present invention takes into account both retrieval accuracy and retrieval speed through the interaction between documents and between documents and causal graph nodes.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] The present invention discloses a knowledge retrieval method based on causal reasoning, including the following steps:
[0007] S1. Input the user query into a document recall model to generate a candidate retrieval document set;
[0008] S2. Based on a large model and a preset prompt template, construct a structural causal graph for the candidate retrieval document set;
[0009] S3. Perform embedding processing on each recalled document in the candidate retrieval document set to generate a word-level embedding vector sequence;
[0010] S4. According to the embedding vector sequence, perform causal semantic encoding on each document by using an attention layer based on causal reasoning, so as to generate a word-level causal semantic feature vector of the document;
[0011] S5. Perform an averaging operation on the word-level causal semantic feature vectors of each document to generate the causal semantic feature vector of the document;
[0012] S6. Replace the recalled documents with the user query, and obtain the causal semantic feature vector of the user query in the same way as in steps S3 to S5;
[0013] S7. Perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query, and sort each recalled document according to the similarity result.
[0014] As a further improvement of the above solution, step S4 includes the following specific steps:
[0015] S41. Use the attention layer based on causal reasoning to perform causal semantic encoding on the recalled documents, and the calculation formula is as follows:
[0016]
[0017] In the formula, l ∈ [1, L], where L is the number of attention layers; represents the l-th layer attention layer feature vector of the t-th node in the structural causal graph; N(t) represents the set of neighbor nodes of the t-th node, represents the initial feature vector of the t-th node; represents the (l - 1)-th layer attention feature vector of the j-th node, represents the initial feature vector of the j-th node; represents the l-th layer attention layer feature vector of the j-th word in document i, i ∈ [1, n], where n is the length of the candidate retrieval document set; V(t) represents the set of documents related to the t-th node, represents the (l - 1)-th layer attention layer feature vector of the z-th word in document p, represents the (l - 1)-th layer attention layer feature vector of the r-th word in document i; r, z ∈ [1, m], where m represents the length of the embedding vector sequence; f(·) and g(·) represent activation functions; α(·) represents the attention function; C(i) represents the set of nodes related to document i, represents the (l - 1)-th layer attention layer feature vector of the k-th node among them; e ij represents the embedding vector of the j-th word in document i; represents the input feature vector of the first layer attention layer of the j-th word in document i;
[0018] S42. Use a feed-forward neural network to perform further semantic encoding on the encoding result obtained in step S42, and the calculation formula is as follows:
[0019]
[0020] In the formula, FFN l (·) represents the l-th layer feedforward neural network; represents the l-th layer causal semantic feature vector of the j-th word in the i-th document;
[0021] S43. Repeat steps S41 - S42 for L times to obtain the word-level causal semantic feature vector of document i
[0022] As a further improvement of the above solution, in step S5, the expression formula of the causal semantic feature vector of the document is:
[0023]
[0024] In the formula, cdh i represents the causal semantic feature vector of the document; Mean(·) represents the average operation.
[0025] As a further improvement of the above solution, in step S7, a cosine similarity function is used to perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query, and the calculation formula is as follows:
[0026] similarity i =Cosine(queryh,cdh i )
[0027] In the formula, similarity i represents the similarity between the causal semantic feature vector of document i and the causal semantic feature vector of the user query; queryh represents the causal semantic feature vector of the user query; Cosine(·) represents the cosine similarity function.
[0028] As a further improvement of the above solution, the attention function adopts a multi-head attention mechanism; the average operation adopts an arithmetic average.
[0029] As a further improvement of the above solution, in step S7, all the recalled documents are sorted in the order of similarity from high to low.
[0030] As a further improvement of the above solution, in step S2, the specific process of constructing the structural causal graph includes:
[0031] Based on the large model, perform causal relationship recognition on the candidate retrieval document set, extract key variables and their causal relationships, and generate an adjacency matrix and a node source list; the adjacency matrix is an N×N matrix representing the direct causal relationship between nodes; the node source list records all the document numbers where each node appears.
[0032] As a further improvement of the above solution, in step S1, the expression formula for generating the candidate retrieval document set D is:
[0033] D = Recall(query)
[0034] In the formula, Recall represents the document recall model; query represents the user query; D = {cd1, cd2,..., cd n}, cd i represents the i-th document recalled according to the user query query; n is the length of the candidate retrieval document set;
[0035] In step S3, the expression formula for generating the embedding vector sequence is:
[0036] E i = Embedding(cd i )
[0037] In the formula, cd i = {cd i1 , cd i2 ,..., cd iJ} represents the text sequence of document i, cd ij represents the j-th word of document i, J represents the length of the text sequence; Embedding(·) represents the word semantic embedding operation; E i = {e i1 , e i2 ,..., e im} represents the embedding vector sequence after being mapped by the Embedding embedding layer, m represents the length of the embedding vector sequence, and e ij represents the embedding vector of the j-th word of document i.
[0038] The present invention also discloses a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the knowledge retrieval method based on causal reasoning as described above are implemented.
[0039] The present invention also discloses a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the knowledge retrieval method based on causal reasoning as described above are implemented.
[0040] Compared with the prior art, the beneficial effects of the present invention are:
[0041] First, the present invention uses a large model to construct a structural causal graph for the recalled documents, and then uses an attention layer based on causal reasoning to perform information interaction between the structural causal graph and the documents, realizing implicit interaction between different documents and reducing the scores of documents containing incorrect information. In addition, the present invention uses the interaction between the query and the structural causal graph of the recalled documents to perform semantic interaction between the query and the documents, obtaining query semantic feature vectors and document semantic feature vectors containing rich information, and improving the retrieval accuracy as much as possible on the premise of adding a small amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] Figure 1 It is a flowchart of the knowledge retrieval method based on causal reasoning in Embodiment 1 of the present invention.
[0043] Figure 2 It is a flowchart of document recall and causal semantic encoding in Embodiment 1 of the present invention.
[0044] Figure 3 It is a flowchart of causal semantic encoding for the user query query in Embodiment 1 of the present invention.
[0045] Figure 4 It is a schematic structural diagram of a computer terminal in Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0046] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.
[0047] Embodiment 1
[0048] Please refer to Figures 1 to 3 , this embodiment provides a knowledge retrieval method based on causal reasoning, including the following steps:
[0049] S1. Input the user query into the document recall model to generate a candidate retrieval document set.
[0050] In step S1, the expression formula for generating the candidate retrieval document set D is:
[0051] D = Recall(query)
[0052] In the formula, Recall represents the document recall model; query represents the user query; D = {cd1, cd2,..., cd n}, cd iDenote the i-th document recalled according to the user query query, and n represents the length of the candidate retrieval document set.
[0053] S2. Based on the large model and the preset prompt template, construct a structural causal graph for the candidate retrieval document set.
[0054] In this embodiment, the content of the prompt template in Markdown format can be as shown in Table 1 below. It should be noted that the cell division in Table 1 reflects the primary and secondary hierarchical relationship of the prompt template. In actual operation, all the content can be directly formed into a document and input to the large model.
[0055] Table 1: Content of the prompt template
[0056]
[0057]
[0058] The structural causal graph can be understood as follows: In the structural causal graph, each node is a binary tuple composed of a word and the document to which the word belongs. If there is a causal relationship between the j-th word in the i-th document and the j-th word in the p-th document, then connect the node represented by the j-th word in the i-th document and the node represented by the j-th word in the p-th document to construct an edge.
[0059] S3. Perform embedding processing on each recalled document in the candidate retrieval document set, map the words to embedding vectors, and generate a word-level embedding vector sequence.
[0060] In step S3, the expression formula for generating the embedding vector sequence is:
[0061] E i =Embedding(cd i )
[0062] where cd i ={cd i1 ,cd i2 ,…,cd iJ} represents the text sequence of document i, cd ij represents the j-th word in document i, J represents the length of the text sequence; Embedding(·) represents the word semantic embedding operation; E i ={e i1 ,e i2 ,…,e im} represents the embedding vector sequence after being mapped by the Embedding embedding layer, m represents the length of the embedding vector sequence, and e ij represents the embedding vector of the j-th word in document i.
[0063] S4. Based on the sequence of embedding vectors, use the attention layer based on causal reasoning to perform causal semantic encoding on each document, thereby generating a word-level causal semantic feature vector for the document.
[0064] Step S4 includes the following specific steps:
[0065] S41. Use the attention layer based on causal reasoning to perform causal semantic encoding on the recalled documents. The calculation formula is as follows:
[0066]
[0067]
[0068] In the formula, l ∈ [1, L], where L is the number of attention layers; represents the l-th layer attention layer feature vector of the t-th node in the structural causal graph; N(t) represents the set of neighbor nodes of the t-th node, represents the initial feature vector of the t-th node; represents the (l - 1)-th layer attention feature vector of the j-th node, represents the initial feature vector of the j-th node; represents the l-th layer attention layer feature vector of the j-th word in document i, where i ∈ [1, n], and n is the length of the candidate retrieval document set; V(t) represents the set of documents related to the t-th node, represents the (l - 1)-th layer attention layer feature vector of the z-th word in document p, represents the (l - 1)-th layer attention layer feature vector of the r-th word in document i; r, z ∈ [1, m], where m represents the length of the embedding vector sequence; f(·) and g(·) represent activation functions; α(·) represents the attention function; C(i) represents the set of nodes related to document i, represents the (l - 1)-th layer attention layer feature vector of the k-th node among them; e ij represents the embedding vector of the j-th word in document i; represents the input feature vector of the j-th word in document i at the first layer of the attention layer.
[0069] S42. Use a feedforward neural network to perform further semantic encoding on the encoding result obtained in step S42. The calculation formula is as follows:
[0070]
[0071] In the formula, FFN(·) represents the l-th layer feedforward neural network; represents the causal semantic feature vector of the j-th word in the i-th document.
[0072] S43. Repeat steps S41 - S42 for L times to obtain the word - level causal semantic feature vector of document i
[0073] S5. Perform an averaging operation on the word - level causal semantic feature vectors of each document to generate the causal semantic feature vector of the document.
[0074] In step S5, the expression formula for the causal semantic feature vector of the document is:
[0075]
[0076] In the formula, cdh i represents the causal semantic feature vector of the document; Mean(·) represents the averaging operation.
[0077] S6. Replace the recalled documents with the user query, and refer to the method of steps S3 - S5 to obtain the causal semantic feature vector of the user query.
[0078] S7. Perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query, and sort each recalled document according to the similarity result.
[0079] In step S7, use the cosine similarity function to perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query. The calculation formula is as follows:
[0080] similarity i =Cosine(queryh,cdh i )
[0081] In the formula, similarity i represents the similarity between the causal semantic feature vector of document i and the causal semantic feature vector of the user query; queryh represents the causal semantic feature vector of the user query; Cosine(·) represents the cosine similarity function.
[0082] After calculating the similarities corresponding to each document, sort all the recalled documents in descending order of similarity.
[0083] Embodiment 2
[0084] This embodiment provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the knowledge retrieval method based on causal reasoning as described in Embodiment 1.
[0085] As Figure 4As shown in the figure, the computer terminal provided in this embodiment includes: at least one processor 101 and a memory 102 connected to the at least one processor 101. In this embodiment, the specific connection medium between the processor 101 and the memory 102 is not limited. Figure 4 In the figure, it is taken as an example that the processor 101 and the memory 102 are connected through a bus 100. The bus 100 is represented by a thick line in Figure 4 the figure. The connection manners between other components are only illustrative and not restrictive. The bus 100 can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 4 it is only represented by a thick line in the figure, but it does not mean that there is only one bus or one type of bus. Alternatively, the processor 101 can also be called a controller, and the name is not limited.
[0086] In this embodiment, the memory 102 stores instructions executable by the at least one processor 101. By executing the instructions stored in the memory 102, the at least one processor 101 can execute the foregoing method.
[0087] Among them, the processor 101 is the control center of the device, and can connect various parts of the entire control device through various interfaces and lines. By running or executing the instructions stored in the memory 102 and calling the data stored in the memory 102, various functions of the device and process data, so as to monitor the device as a whole.
[0088] In a possible design, the processor 101 may include one or more processing units. The processor 101 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the foregoing modem processor may not be integrated into the processor 101. In some embodiments, the processor 101 and the memory 102 can be implemented on the same chip, and in some embodiments, they can also be separately implemented on independent chips.
[0089] The processor 101 can be a general-purpose processor, such as a central processing unit (CPU), a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, and can implement or execute the various methods, steps and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the knowledge retrieval method based on causal reasoning disclosed in conjunction with Embodiment 1 can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor 101.
[0090] The memory 102, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The memory 102 may include at least one type of storage medium. For example, it may include flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disc, and so on. The memory 102 is any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 102 in this embodiment may also be a circuit or any other device capable of implementing a storage function, for storing program instructions and / or data.
[0091] By programming the design of the processor 101, the code corresponding to the knowledge retrieval method based on causal reasoning introduced in the foregoing embodiments can be solidified into the chip, so that the chip can execute Figure 1 the steps of the knowledge retrieval method based on causal reasoning in the illustrated embodiments. How to program the design of the processor 101 is a well-known technology to those skilled in the art and will not be elaborated herein.
[0092] Embodiment 3
[0093] This embodiment provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the steps of the knowledge retrieval method based on causal reasoning as described in Embodiment 1.
[0094] The computer-readable storage medium may include flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the storage medium may be an internal storage unit of the computer device, such as the hard disk or memory of the computer device. In other embodiments, the storage medium may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc. Of course, the storage medium may also include both the internal storage unit and the external storage device of the computer device. In this embodiment, the memory is generally used to store the operating system and various application software installed in the computer device. In addition, the memory may also be used to temporarily store various data that have been output or will be output.
[0095] As described above, only the preferred specific embodiments of the present invention are provided, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, according to the technical solution and inventive concept of the present invention, makes equivalent substitutions or changes, and should be covered by the protection scope of the present invention.
Claims
1. A knowledge retrieval method based on causal reasoning, characterized in that: The following steps are involved: S1. Input the user query into the document recall model to generate a candidate retrieval document set; S2. constructing a structural causal graph for the candidate retrieval document set based on the large model and the preset prompt template; S3. Embed each recalled document in the candidate retrieval document set to generate a word-level embedding vector sequence; S4. According to the embedding vector sequence, causal semantic encoding is performed on each document using an attention layer based on causal reasoning, thereby generating a word-level causal semantic feature vector of the document; S5. perform an average operation on the word-level causal semantic feature vectors of each document to generate a causal semantic feature vector of the document; S6. Replace the recalled document with the user query, and obtain the causal semantic feature vector of the user query by referring to steps S3 to S5; S7. Perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query, and sort each recalled document according to the similarity result.
2. The knowledge retrieval method based on causal reasoning according to claim 1 is characterized in that: Step S4 includes the following specific steps: S41. Use the attention layer based on causal reasoning to encode the recalled documents causally. The calculation formula is as follows: Where l∈[1,L], L is the number of attention layers; represents the l-th attention layer feature vector of the t-th node in the structural causal graph; N(t) represents the set of neighbor nodes of the t-th node, represents the initialization feature vector of the tth node; represents the l-1th layer attention feature vector of the jth node, represents the initialization feature vector of the jth node; represents the l-th attention layer feature vector of the j-th word in document i, i∈[1,n], n is the length of the candidate retrieval document set; V(t) represents the document set related to the t-th node, represents the l-1th attention layer feature vector of the zth word in document p, represents the l-1th attention layer feature vector of the rth word in document i; r,z∈[1,m], m represents the length of the embedding vector sequence; f(·) and g(·) represent activation functions; α(·) represents the attention function; C(i) represents the set of nodes related to document i, represents the l-1th attention layer feature vector of the kth node; e ij Represents the embedding vector of the jth word in document i; The input feature vector of the first attention layer representing the jth word in document i; S42. Use a feedforward neural network to further semantically encode the encoding result obtained in step S42, and the calculation formula is as follows: In the formula, FFN l (·) represents the lth layer of feedforward neural network; Represents the l-th level causal semantic feature vector of the j-th word in the i-th document; S43. Repeat steps S41 to S42 L times to obtain the word-level causal semantic feature vector of document i 3. The knowledge retrieval method based on causal reasoning according to claim 2 is characterized in that: In step S5, the expression formula of the causal semantic feature vector of the document is: In the formula, cdh i represents the causal semantic feature vector of the document; Mean(·) represents the average operation.
4. The knowledge retrieval method based on causal reasoning according to claim 3 is characterized in that: In step S7, a cosine similarity function is used to perform similarity matching on the causal semantic feature vector of the document and the causal semantic feature vector of the user query, and the calculation formula is as follows: similarity i =Cosine(queryh,cdh i ) In the formula, similarity i represents the similarity between the causal semantic feature vector of document i and the causal semantic feature vector of the user query; queryh represents the causal semantic feature vector of the user query; Cosine(·) represents the cosine similarity function.
5. The knowledge retrieval method based on causal reasoning according to claim 4 is characterized in that: In step S7, all recalled documents are sorted in descending order of similarity.
6. The knowledge retrieval method based on causal reasoning according to claim 3 is characterized in that: The attention function adopts a multi-head attention mechanism; the averaging operation adopts arithmetic averaging.
7. The knowledge retrieval method based on causal reasoning according to claim 1 is characterized in that: In step S2, the specific process of constructing the structural causal graph includes: Based on the large model, causal relationships are identified for the candidate retrieval document set, key variables and their causal relationships are extracted, and an adjacency matrix and a node source list are generated; the adjacency matrix is an N×N matrix, which represents the direct causal relationship between nodes; the node source list records all document numbers that appear in each node.
8. The knowledge retrieval method based on causal reasoning according to claim 1 is characterized in that: In step S1, the expression formula for generating the candidate retrieval document set D is: D=Recall(query) In the formula, Recall represents the document recall model; query represents the user query; D = {cd1, cd2, …, cd n }, cd i represents the i-th document recalled according to the user query; n is the length of the candidate retrieval document set; In step S3, the expression formula for generating the embedding vector sequence is: E i =Embedding(cd i ) In the formula, cd i ={cd i1 ,cd i2 ,…,cd iJ } represents the text sequence of document i, cd ij Represents the jth word in document i, and J represents the length of the text sequence; Embedding(·) represents the word semantic embedding operation; E i ={e i1 ,e i2 ,…,e im } represents the embedded vector sequence after being mapped by the Embedding layer, m represents the length of the embedded vector sequence, e ij Represents the embedding vector of the jth word in document i.
9. A computer terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the processor implements the steps of the knowledge retrieval method based on causal reasoning as described in any one of claims 1 to 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the knowledge retrieval method based on causal reasoning as described in any one of claims 1 to 8 are implemented.
Citation Information
Cited By
Policy document-oriented retrieval system and method
CN122087086A