An entity retrieval method and device based on knowledge injection

Abstract

The knowledge injection-based entity retrieval method provided in the application comprises the following steps: retrieving according to a query text in an entity pool to obtain a first candidate entity sequence, a second candidate entity sequence and a third candidate entity sequence; the first candidate entity sequence and the second candidate entity sequence are obtained according to semantic retrieval, and the third candidate entity sequence is obtained according to lexical retrieval; in the case that the number of the same candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to a first threshold value, inputting the query text, the first candidate entity sequence and the concept entry of each candidate entity in the first candidate entity sequence into a large language model for rearrangement to obtain a first target entity sequence; wherein the concept entry is obtained according to a knowledge base; and outputting the first target entity sequence. The application uses a high-computing-cost retrieval path only in the case of a naming gap, thereby reducing the consumption of computing power caused by the introduction of the knowledge base.

Description

Technical Field

[0001] This application relates to the field of intelligent education, and in particular to a knowledge-injection-based entity retrieval method and apparatus. Background Technology

[0002] A retrieval pipeline refers to the process of breaking down an information retrieval task into multiple ordered and combinable processing stages, where data flows through each stage like a factory production line, ultimately outputting retrieval results. Retrieval pipelines typically improve the ranking quality of retrieval results by introducing external knowledge; common techniques include training-based knowledge augmentation and Retrieval-Augmented Generation (RAG) technology.

[0003] In training-based knowledge augmentation techniques, external knowledge graph signals are fused into a cross-encoder at the embedding layer to update model parameters. Examples include distilling the knowledge meta-graph into the cross-encoder or fusing structured knowledge graph signals into the sorter. This method requires computational resources to retrain the model and adjust its parameters, and cannot be used plug-and-play with already deployed models.

[0004] Retrieval enhancement generation technology combines an external knowledge base with a large language model (LLM). By "rearrangement instead of retraining", it saves the computational cost of model retraining and can be used in a plug-and-play manner with the deployed large language model rearranger. However, this technology shifts the computational cost to the inference side used to perform the rearrangement, which increases the computational cost on the inference side.

[0005] Therefore, when introducing a knowledge base into a retrieval pipeline, it is urgent to find a solution that can both plug and play the deployed model and save on computing power. Summary of the Invention

[0006] This application provides a knowledge injection-based entity retrieval method and apparatus, which reduces the computational power consumption when introducing a knowledge base in the retrieval pipeline and can be used plug-and-play with deployed large language model reorderers.

[0007] In a first aspect, this application provides a knowledge-injection-based entity retrieval method. This method can be executed by a computing device, or by a component of the computing device, such as a processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the functions of the computing device. The method provided in the first aspect includes: obtaining query text; retrieving from an entity pool based on the query text to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence; obtaining the first and second candidate entity sequences based on semantic retrieval, wherein the lengths of the first and second candidate entity sequences may be the same or different; obtaining the third candidate entity sequence based on lexical retrieval, wherein the lengths of the second and third candidate entity sequences are the same; if the number of identical candidate entities contained in the second and third candidate entity sequences is less than or equal to a first threshold, inputting the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence into a large language model for rearrangement to obtain a first target entity sequence; wherein the concept entries are the output first target entity sequence obtained from a knowledge base.

[0008] The entity retrieval method based on knowledge injection provided in this application determines whether a naming gap exists based on the overlap between lexical retrieval and semantic retrieval. Only when a naming gap exists is the semantic retrieval result (first candidate entity sequence) reordered using a large language model based on knowledge injection, which has high computational cost. This reduces the computational cost of introducing a knowledge base while ensuring retrieval accuracy, and it can be used plug-and-play with the already deployed large language model.

[0009] The naming gap refers to the mismatch between the literal expression of the query text and the canonical names of candidate entities in the entity pool. The entity pool stores the canonical names of entities. When the canonical name does not appear in the query text, or when the query text and the canonical name only have a few overlapping words, lexical retrieval will fail. In this case, semantics and an imported knowledge base are needed to improve the accuracy of the retrieval.

[0010] Furthermore, the knowledge (concept entries) injected in this embodiment only increases the amount of input text for the large language model, without changing the user's query text. Therefore, it eliminates the need to retrain the large language model, does not change its parameters, and does not alter the candidate entities in the initial retrieval results (first candidate entity sequence). This allows for plug-and-play application of the deployed large language model reorderer. This method enables the large language model to reorder the first candidate entity sequence based on the query text and the injected knowledge, improving retrieval accuracy.

[0011] As one implementation method, the top N candidate entities are selected from the candidate entity sequence after the large language model is rearranged to obtain the first target entity sequence, and the first target entity sequence is output to the user. N is less than or equal to the length of the first candidate entity sequence, which is the number of entities contained in the first candidate entity sequence.

[0012] In one implementation, if the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is greater than a first threshold, the first candidate entity sequence is output as the final retrieval result.

[0013] In the above implementation, when the number of identical candidate entities contained in the second and third candidate entity sequences exceeds the first threshold, it is determined that there is no naming gap in this retrieval. In this case, a semantic retrieval path with low computational cost can meet the requirements for retrieval accuracy. Therefore, in this situation, the first candidate entity sequence is directly output as the final retrieval result; or, the first M candidate entities in the first candidate entity sequence are selected to obtain the second target entity sequence, and the second target entity sequence is output to the user.

[0014] As one implementation method, this application embodiment determines which specific task to allocate additional computing power to based on task-level computing power allocation. The tasks include three types: reachability, sorting, and inference. Before retrieving the entity pool based on the query text to obtain the first candidate entity sequence, the second candidate entity sequence, and the third candidate entity sequence, this application embodiment performs a retrieval based on the labeled text in the entity pool. The result of the computing power allocation decision based on the retrieval quality, latency cost, call cost / resource cost, and GPU / RAM cost of each task is inference.

[0015] In the above implementation, only when the result of task-level computing power allocation is inference is the decision made on whether to execute a high-computational-cost rearrangement path based on the existence of a naming gap, thereby allocating computing power to the most needed inference task and further saving computing power consumption. In the embodiments of this application, the inference task refers to the rearrangement process of the initial retrieval results.

[0016] As one implementation method, the method provided in this application can be applied to educational scenarios. In educational scenarios, the entities in the entity pool are knowledge points or courses. Concept entries include at least a specification name, definition, and example, and may also include one or more of hierarchical relationships, prerequisite relationships, or mutual exclusion relationships.

[0017] As one implementation method, the first target entity sequence includes evidence sources for each entity, which include specification names, definitions, and examples. Before outputting the first target entity sequence, the method further includes: verifying the evidence sources for each entity in the first target entity sequence against a knowledge base, deleting entities whose evidence sources do not belong to the knowledge base, or lowering the order of entities whose evidence sources do not belong to the knowledge base in the first target entity sequence. In this embodiment, the knowledge base is the source of real knowledge. Verifying the evidence sources through the knowledge base ensures the authenticity of the injected knowledge, thereby ensuring the accuracy and reliability of the retrieval results.

[0018] Secondly, embodiments of this application provide a knowledge injection-based rearrangement device, which includes an acquisition module, a retrieval module, a rearrangement module, and an output module. The acquisition module acquires query text; the retrieval module retrieves entity data from an entity pool based on the query text to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence; the first and second candidate entity sequences are obtained through semantic retrieval, and the third candidate entity sequence is obtained through lexical retrieval, with the second and third candidate entity sequences having the same length; the rearrangement module, when the number of identical candidate entities in the second and third candidate entity sequences is less than or equal to a first threshold, inputs the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence into a large language model for rearrangement to obtain a first target entity sequence; the concept entries are obtained from a knowledge base; and the output module outputs the first target entity sequence.

[0019] Thirdly, embodiments of this application provide a computing device including a processor coupled to a memory. The processor is used to store instructions, and when the instructions are executed by the processor, the computing device cluster executes the method described in the first aspect or any possible implementation of the first aspect.

[0020] Fourthly, embodiments of this application provide a computer-readable storage medium having instructions stored thereon, which, when executed, cause a computer to perform the method described in the first aspect or any possible implementation thereof.

[0021] Fifthly, embodiments of this application provide a computer program product that includes instructions that, when executed, cause a computer to implement the method described in the first aspect or any possible implementation of the first aspect.

[0022] It is understood that the beneficial effects achieved by any of the knowledge injection-based rearrangement devices, computing devices, computer-readable storage media, or computer program products provided above can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0023] Figure 1 This is the system architecture diagram provided in this application;

[0024] Figure 2 This is a flowchart of a knowledge injection-based entity retrieval method provided in this application;

[0025] Figure 3 This is a flowchart of another entity retrieval method based on knowledge injection provided in this application;

[0026] Figure 4 This is a functional block diagram of the knowledge injection-based entity retrieval device provided in this application;

[0027] Figure 5 This is an example diagram of the computer device provided in this application. Detailed Implementation

[0028] The embodiments of the present invention will be described in detail below with reference to examples. However, those skilled in the art will understand that the following examples are only for illustrating the present invention and should not be regarded as limiting the scope of the present invention.

[0029] First, some of the terms used in the embodiments of this application are introduced to facilitate understanding of the technical solutions by those skilled in the art.

[0030] Task-level computing power allocation: The retrieval pipeline is divided into recall, ranking, and reasoning tasks. At the task level, decisions are made on whether to allocate additional computing power to recall, ranking, or reasoning.

[0031] Semantic retrieval: Semantic retrieval is a technology in the field of information retrieval that achieves accurate retrieval by understanding the deep semantics of queries and documents rather than simple keyword matching.

[0032] Lexical retrieval: This method only processes the surface level of text, words, and characters, without understanding the meaning of sentences. It relies solely on matching textual characters, words, word frequency, word segmentation, etc.

[0033] To make the technical solution of this application clearer and easier to understand, the following is in conjunction with the appendix. Figure 1 The system architecture of the embodiments of this application will be described.

[0034] The system architecture of this application embodiment includes a first retrieval unit, a second retrieval unit, a gating module, a large language model reordering module, and an output module. The first retrieval unit retrieves query text using a first retrieval method. From the retrieved candidate entity sequence, it selects a first preset number of candidate entities ranked high to form a first candidate entity sequence, which is used for reordering or directly outputting the final retrieval result. It then selects a second preset number of candidate entities ranked high to form a second candidate entity sequence, and uses this sequence, along with a third candidate entity sequence output by the second retrieval unit, to determine the retrieval path. The second retrieval unit retrieves query text according to the second retrieval method and outputs a second preset number of third candidate entity sequences. Both the first and second retrieval units perform retrieval within an entity pool. The gating module determines the retrieval path based on the relationship between the number of identical candidate entities in the second and third candidate entity sequences and a gating threshold. The retrieval path includes a high-computational-cost large language model reordering retrieval path and a low-computational-cost retrieval path that does not require reordering. When reordering using the large language model, knowledge from a knowledge base is simultaneously injected, and this knowledge is input into the large language model in a standardized way of concept entries to obtain a highly accurate reordering result. The search results acquisition module is used to select the top number of candidate entities from the rearranged results or low-computation-cost search results as the final search results, and output the search results to the interface for users to view.

[0035] Based on the system architecture provided in this application, embodiments of this application provide an entity retrieval method based on knowledge injection. See also Figure 2 The knowledge injection-based entity retrieval method provided in the embodiments of this application will be described in detail below.

[0036] S201, retrieve the query text.

[0037] The query text can be entered by the user according to their actual query needs. For example, the query text is text entered by a student to query knowledge points or courses. This text can be one or more of the following: exercises, examples, problem explanations, or other document fragments related to the subject. In an educational context, the query text can be called the linking text, and the purpose of the query is to link to the target knowledge point or course. The user can be a student or a teacher, and can enter it through a student client or a teacher client.

[0038] Users can input queries via voice or image. After receiving the user's input, the system recognizes the voice or image to obtain the corresponding query text. Furthermore, the query request may consist of at least two of the following: text, semantics, or images. Upon receiving the query request, the system converts the content of the query request into the corresponding query text for retrieval.

[0039] After obtaining the query text, it can be preprocessed. For example, when the query text is used to represent the knowledge points or courses being queried, preprocessing operations include cleaning, formula preservation, unification of full-width and half-width characters, and subject recognition.

[0040] S202, based on the query text, search the entity pool to obtain the first candidate entity sequence, the second candidate entity sequence, and the third candidate entity sequence.

[0041] The entity pool includes multiple entities, which can be knowledge points or courses, or standardized entities such as standard terms, skill tags, question types, competency indicators, legal provisions, medical entities, product categories, patent classification numbers, and enterprise codes. Entity names in the entity pool use standardized naming conventions, and the fields in the entity pool must include at least an entity identifier and a standardized name.

[0042] The entity pool is searched according to the first retrieval method. From the retrieved results, a first preset number of candidate entities ranked high are selected as the first candidate entity sequence, and a second preset number of candidate entities ranked high are selected as the second candidate entity sequence. The first candidate entity sequence provides the final output target entity sequence, and the second candidate entity sequence is used to determine whether the first candidate entity sequence needs to be rearranged. The first and second preset numbers can be the same or different.

[0043] The entity pool is searched according to the second retrieval method. From the retrieved results, a second predetermined number of candidate entities with the highest ranking are selected as the third candidate entity sequence. The first and second retrieval methods are different; for example, the first retrieval method is semantic retrieval, while the second is lexical retrieval. The third and second candidate entity sequences are used to determine whether the first candidate entity sequence needs to be rearranged.

[0044] S203, determine whether the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to the first threshold. If yes, proceed to step S204; otherwise, proceed to step S205.

[0045] The first retrieval method is semantic retrieval, and the second is lexical retrieval. When the number of identical candidate entities contained in the second and third candidate entity sequences is less than or equal to the first threshold, it is determined that a naming gap exists in this retrieval. This means that, from the perspective of textual expression, there is a significant difference between the query text and the standard name of the corresponding entity; there is little or no overlap in words. In this case, routing to the high-computational-cost rearrangement path in S204 is required. This involves the large language model using injected knowledge to perform rearrangement to address the naming gap and improve retrieval accuracy. When the number of identical candidate entities contained in the second and third candidate entity sequences exceeds the first threshold, it is determined that no naming gap exists in this retrieval. In this case, a low-computational-cost retrieval path can meet the accuracy requirements; for example, the results of semantic retrieval can satisfy the accuracy requirements.

[0046] The first threshold is a gating signal. Its value is independent of labeled samples and is a cost-quality adjustment parameter that can be calculated during the inference phase. A larger first threshold results in more queries being routed to high-cost paths, higher coverage gain, and higher computational cost. The first threshold can be adjusted according to cost-quality objectives, specifically based on one or more of the following: call rate, injection gain, gating hit rate, and latency.

[0047] S204, input the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence into the large language model for rearrangement to obtain the first target entity sequence.

[0048] This step is a computationally expensive approach that uses injected knowledge to improve the rearrangement accuracy of the large language model. The injected knowledge comes from an expert knowledge base and consists entirely of true knowledge. The concept entries for each candidate entity input into the large language model are derived from the knowledge base. Each concept entry includes at least a canonical name, definition, and example, and may include one or more of the following: hypernym / hypernym, precondition, or mutual exclusion. For a candidate entity, the injected knowledge is its concept entry.

[0049] The large language model reorders the candidate entities in the first candidate entity sequence based on the injected knowledge and the query text to obtain a reordered candidate entity sequence. This reordered sequence can be directly output to the user as the first target entity sequence, or the top N candidate entities can be selected from the reordered sequence to obtain the first target entity sequence, which is then output to the user. Therefore, the first target entity sequence can have the same length as the first candidate entity sequence, or they can be different. "Same length" means that the number of entities in the first target entity sequence is the same as the number of entities in the first candidate entity sequence. The large language model can use a scoring method, reading the probability of positive tags as the relevance score, or it can use a list method to reorder the candidate entities in the first candidate entity sequence.

[0050] When a naming gap exists in the search, the recall rate of lexical retrieval will be significantly reduced. Although the recall rate of semantic retrieval is better than that of lexical retrieval, it will also be affected by the naming gap. In this case, the accuracy of semantic retrieval is limited, and a knowledge base needs to be introduced to reorder the results of semantic retrieval in order to improve the accuracy of the search results.

[0051] As one implementation method, to improve the authenticity of the injected knowledge, the large language model outputs the evidence source for each entity when outputting the first target entity sequence. The knowledge base verifies the evidence source for each entity, and based on the verification results, deletes entities in the first target entity sequence whose evidence source does not belong to the knowledge base, or lowers the order of entities whose evidence source does not belong to the knowledge base in the first target entity sequence.

[0052] As one implementation method, consistency checks can be performed during evidence source verification, and entities that do not meet the consistency check criteria can be removed from the first target entity sequence or their order reduced. For example, if the first target sequence output by the large language model includes one or more names, definitions, or examples, consistency checks can be performed on these names, definitions, or examples to ensure the reliability of the injected knowledge. Furthermore, erroneous or irrelevant text of equal length not only does not generate gain but also reduces the relevance of the candidate entities anchored by the definition, and its damage may outweigh the impact of random noise. Therefore, consistency checks can be performed on one or more names, definitions, or examples in the first target sequence, and entities that do not meet the consistency check criteria in the first target entity sequence can be removed or their order reduced to reject fluent but factually incorrect definitions generated by generative expansion.

[0053] As one implementation method, when rearranging the first candidate entity sequence, the large language model can impose candidate entity identifier constraints, ensuring that the injected knowledge is only used for closed-set sorting without creating new candidate entities. Furthermore, the first target sequence output by the large language model can include one or more of the following: preconditions, hierarchical relationships, and mutual exclusion relationships. A graph structure check is performed on one or more of these relationships, deleting entities from the first target entity sequence that do not meet the graph structure check criteria, or lowering the order of entities that do not meet the graph structure check criteria in the first target entity sequence.

[0054] When outputting the first target entity sequence, the large language model can also output one or more of the following: entity identifier, canonical name, initial recall score (the recall score of the first candidate target entity), reordering, and gating decision, to facilitate full-process auditing or verification. The gating decision includes one or more of the following: a first preset quantity, a second preset quantity, and a first threshold.

[0055] In the candidate entity sequence after rearrangement of the large language model, the top N candidate entities are selected to obtain the first target entity sequence, which is then output to the user. N is less than or equal to the length of the first candidate entity sequence, which is the number of entities contained in the first candidate entity sequence.

[0056] S205, output the first target entity sequence / the second target entity sequence.

[0057] After obtaining the target entity sequence, the target entity sequence is output to the user for viewing. The second target entity sequence can be either the first candidate entity sequence or the first M candidate entities selected from the first candidate entity sequence, where M is less than the number of candidate entities in the first candidate entity sequence.

[0058] For example, in an educational setting, when a student's query text is one or more of the following: exercises, examples, problem explanations, or other document fragments related to the subject, the system outputs the corresponding knowledge points or courses to link the student's needs to the relevant entities. When outputting the knowledge points or courses, the system can simultaneously output one or more of the following: name, definition, examples, explanation links, hierarchical relationships, prerequisite relationships, and mutual exclusion relationships, enabling students to quickly grasp the corresponding knowledge points or course content.

[0059] In the knowledge injection-based entity retrieval method provided in the above embodiments, the existence of a naming gap is determined based on the overlap between lexical retrieval and semantic retrieval. If a naming gap exists, a high-computational-cost large language model path based on knowledge injection is used to reorder the results. If no naming gap exists, a low-computational-cost semantic retrieval path is used to obtain the retrieval results. Thus, in the retrieval pipeline, while ensuring retrieval accuracy, the consumption of computing power by introducing external knowledge is reduced, and the deployed large language model can be used plug-and-play.

[0060] Furthermore, in the knowledge-injection-based entity retrieval method provided in the above embodiments, the injected knowledge (concept entries) only increases the amount of input text for the large language model, without changing the user's query text. Therefore, it eliminates the need for retraining the large language model, does not change its parameters, and does not alter the candidate entities in the initial retrieval results (first candidate entity sequence). It can be used plug-and-play with already deployed large language model reorderers. This method allows the large language model to reorder the first candidate entity sequence based on the query text and the injected knowledge, improving retrieval accuracy.

[0061] In pipelined retrieval, related technologies typically operate on a query-by-query basis. During the retrieval process for each query, decisions are made regarding whether to allocate additional computing power to recall, ranking, or inference tasks. When computing power is limited, performing decision-making operations on each query increases computational cost.

[0062] As one implementation method, this application's embodiments divide tasks into three types: reachable, sorted, and inference. Task-driven decisions are made regarding the allocation of additional computing power, locating tasks within one of these categories. This allocates additional computing power to the most demanding tasks, and after specific task-driven execution, it can be applied to the processing of multiple query requests. Figure 2 In the entity retrieval method based on knowledge injection shown, after obtaining the query text in step S201, it can be determined whether the result of task-level computing power allocation is inference. If so, subsequent steps are executed; otherwise, semantic retrieval is used directly to obtain the final target entity sequence. In other words, only when the result of task-level computing power allocation is inference is the decision made on whether to execute a high-computation-cost rearrangement path by determining whether a naming gap exists, thereby allocating computing power to the most needed inference task.

[0063] As one implementation method, task-level computing power allocation can be achieved by running probes on a small number of labeled queries. Specifically, three probes can be run to search the entity pool using sample query text and make decisions based on the search results. The first probe reads the recall upper bound of the semantic retrieval at a preset depth (a third preset number of candidate entity sequences). If the recall upper bound is much lower than the total hit (e.g., the recall upper bound is less than the second threshold), it indicates that many candidate entities are not in the candidate set, and no downstream ranking method can retrieve these candidate entities. In this case, the second probe scales the recall number. If the recall upper bound increases with the recall number, it indicates that there is still reachability margin, which is considered reachability-limited, and the result of task-level computing power allocation is reachable. The third probe fixes the candidate entity sequence and changes the ranking algorithm. If the algorithm changes and a fourth preset number of candidate entities are found, it is considered ranking-restricted, and the task-level computing power allocation result is ranking. The third probe fixes the candidate entity sequence and reduces the fields contained in the entities in the entity pool. When the fields are reduced to include entity identifier and canonical name, if the similarity between the ranking result after reduction and the ranking result before reduction is greater than a third threshold, the ranking result is driven by the knowledge base and is considered reasoning-restricted. It must be rearranged by a large language model with reasoning capabilities. In this case, the task-level computing power allocation result is reasoning. Among them, reachability restriction means that the target entity has not entered the candidate set; ranking restriction means that the target entity is already in the candidate set, but ranked low; reasoning restriction means that the candidate entities in the entity pool do not expose the knowledge required for ranking, and the knowledge base is needed to improve the accuracy of retrieval.

[0064] As one implementation method, given task-level samples, action sets, and cost constraints, the resource consumption of each task is calculated, and the result of task-level computing power allocation is obtained through utility function and objective function.

[0065] Let the sample set of experiment t be... , It is labeled text, a set of task-level actions.

[0066] , , and These represent the actions corresponding to the three tasks: reachability, sorting, and reasoning. The utility function formula is:

[0067]

[0068] in, Indicates that action a is in Search quality on This represents the time delay cost of action a on task t. This represents the cost / resource cost of calling action a (e.g., the cost of calling is GPU, and the resource cost is the cost of calling an external large model). This represents the video memory cost / memory cost of action 'a'. , , , It is the weighting coefficient.

[0069] After obtaining the utility value of each action 'a' based on the utility function, the task-level computing power allocation result is obtained based on the objective function. The formula for the objective function is:

[0070]

[0071] The optimal value of action 'a' is selected based on the objective function. This value indicates that the result of task-level computing power allocation is one of recall, sorting, or inference.

[0072] When the result of task-level computing power allocation is reachability, query expansion, hybrid recall, increasing the length of candidate entity sequences, or upgrading the search engine can be employed. When the result of task-level computing power allocation is ranking, rearranging within candidate entity sequences of the same depth (with equal length) can be used. When the result of task-level computing power allocation is ranking, low-cost recalled candidate entity sequences can be retained, and a large language model with reasoning capabilities can be used to rearrange the candidate entity sequences, while simultaneously injecting knowledge to improve the accuracy of the rearrangement. For example, using... Figure 2 The diagram illustrates a knowledge-injection-based entity retrieval method for performing the rearrangement process.

[0073] See Figure 3 This application provides an entity retrieval method based on injection, which includes the following steps:

[0074] S301, retrieve the query text.

[0075] S302, based on the query text, a search is performed in the entity pool to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence.

[0076] Steps S302 and S303 are similar to S201 and S202 in the previous embodiments, and will not be described in detail again.

[0077] S303, if the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to the first threshold, the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence are input into the large language model for rearrangement to obtain the first target entity sequence.

[0078] When the number of identical candidate entities contained in the second and third candidate entity sequences is less than or equal to the first threshold, a naming gap is identified in the retrieval, and the process enters the high-computing-cost path. In the high-computing-cost path, injected knowledge is used to improve the reordering accuracy of the large language model. The large language model reorders the candidate entities in the first candidate entity sequence based on the injected knowledge and the query text to obtain a reordered candidate entity sequence. This reordered candidate entity sequence can be directly output to the user as the first target entity sequence, or the top N candidate entities can be selected from the reordered candidate entity sequence to obtain the first target entity sequence, which is then output to the user.

[0079] S304, output the first target entity sequence.

[0080] In the knowledge injection-based entity retrieval method provided in this application embodiment, the existence of a naming gap is determined based on the overlap between lexical retrieval and semantic retrieval. Only when a naming gap exists is the high-computing-cost knowledge injection-based reordering path used to re-process the semantic retrieval results (first candidate entity sequence), thereby reducing the consumption of computing power by introducing a knowledge base while ensuring retrieval accuracy.

[0081] As one implementation method, in this application embodiment, if the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is greater than a first threshold, it is determined that there is no naming gap in this retrieval, and at this time, a semantic retrieval path with low computing power cost is used to obtain the retrieval results.

[0082] This application provides an entity retrieval device 40 based on knowledge injection, see [link to relevant documentation]. Figure 4 The device includes an acquisition module 41, a retrieval module 42, a rearrangement module 43, and an output module 44. The functions of each module are described below.

[0083] Module 41 is used to retrieve the query text.

[0084] The retrieval module 42 is used to perform retrieval in the entity pool based on the query text to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence. The first and second candidate entity sequences are obtained based on semantic retrieval, and the third candidate entity sequence is obtained based on lexical retrieval. The second and third candidate entity sequences have the same length.

[0085] The rearrangement module 43 is used to rearrange the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence into a large language model to obtain the first target entity sequence when the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to a first threshold.

[0086] Output module 44 is used to output the first target entity sequence.

[0087] In one implementation, the output module is also used to output the first candidate entity sequence if the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is greater than a first threshold.

[0088] As one implementation, the entity retrieval device based on knowledge injection provided in this application further includes a task-level computing power allocation module, used to perform retrieval in the entity pool based on the labeled text, and to obtain the computing power allocation result as one of reachability, ranking, and inference based on the retrieval quality, latency cost, call cost / resource cost, and GPU / RAM cost of each task. When the computing power allocation result is inference, the function of the reordering module can be executed.

[0089] As one implementation method, the entity retrieval device based on knowledge injection provided in this application embodiment further includes a verification module, which verifies the evidence source of each entity in the first target entity sequence according to the knowledge base, deletes entities in the first target entity sequence whose evidence source does not belong to the knowledge base, or lowers the order of entities in the first target entity sequence whose evidence source does not belong to the knowledge base in the first target entity sequence.

[0090] It should be understood that the above Figure 4 The entity retrieval device 400 based on knowledge injection is illustrated using the above-described division of functional modules as an example. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the entity retrieval device 400 based on knowledge injection provided in the above embodiments and the method embodiments share the same concept; its specific implementation process is detailed in the method embodiments and will not be repeated here.

[0091] Figure 5 This is a schematic diagram of the structure of a computing device provided in this application. Figure 5 As shown, the computing device 50 includes a processor 51, a bus 52, a memory 53, a communication interface 54, and a memory unit 55 (also referred to as a main memory unit). The processor 50, memory 53, memory unit 55, and communication interface 54 are connected via the bus 52.

[0092] This application also provides a computer program product containing instructions. This computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product is run on at least one computing device, it causes the at least one computing device to perform the data generation method described above.

[0093] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct a computing device to perform the above-described data generation method.

[0094] This application also provides a chip, including a processor. The processor is used to retrieve and execute instructions stored in a memory, causing a communication device equipped with the chip to perform the aforementioned data generation method.

[0095] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A knowledge-injection-based entity retrieval method, characterized in that, include: Get the query text; Based on the query text, a search is performed in the entity pool to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence; the first candidate entity sequence and the second candidate entity sequence are obtained based on semantic retrieval, and the third candidate entity sequence is obtained based on lexical retrieval; the second candidate entity sequence and the third candidate entity sequence have the same length. If the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to the first threshold, the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence are input into the large language model for rearrangement to obtain the first target entity sequence. Output the sequence of the first target entity.

2. The method according to claim 1, characterized in that, The method further includes: If the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is greater than a first threshold, a second target entity sequence is output; the second target entity sequence is the same as the first candidate entity sequence.

3. The method according to claim 1 or 2, characterized in that, The task includes reachability, sorting, and reasoning; prior to retrieving from the entity pool based on the query text to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence, the method further includes: The entity pool is searched based on the labeled text, and the result of the computing power allocation is inferred based on the retrieval quality, latency cost, call cost / resource cost, and video memory cost / memory cost of each task.

4. The method according to any one of claims 1-3, characterized in that, The entities in the entity pool are knowledge points or courses.

5. The method according to claim 4, characterized in that, The concept entries include the standard name, definition, and examples.

6. The method according to claim 5, characterized in that, The first target entity sequence includes evidence sources for each entity, and the evidence sources include canonical names, definitions, and examples; Before outputting the first target entity sequence, the method further includes: The evidence source for each entity in the first target entity sequence is verified based on the knowledge base. Entities in the first target entity sequence whose evidence source does not belong to the knowledge base are deleted, or the order of entities in the first target entity sequence whose evidence source does not belong to the knowledge base is reduced in the first target entity sequence.

7. A knowledge-injection-based rearrangement device, characterized in that, include: The retrieval module is used to retrieve the query text; The retrieval module is used to retrieve the entity pool based on the query text to obtain a first candidate entity sequence, a second candidate entity sequence, and a third candidate entity sequence; the first candidate entity sequence and the second candidate entity sequence are obtained based on semantic retrieval, and the third candidate entity sequence is obtained based on lexical retrieval; the second candidate entity sequence and the third candidate entity sequence have the same length. The rearrangement module is used to rearrange the query text, the first candidate entity sequence, and the concept entries of each candidate entity in the first candidate entity sequence into a large language model to obtain a first target entity sequence when the number of identical candidate entities contained in the second candidate entity sequence and the third candidate entity sequence is less than or equal to a first threshold; the concept entries are obtained from a knowledge base. The output module is used to output the first target entity sequence.

8. A computing device, characterized in that, The device includes a processor coupled to a memory, the processor being used to store instructions, and when the instructions are executed by the processor, the computing device performs the method of any one of claims 1-7.

9. A computer-readable storage medium having instructions stored thereon, characterized in that, When the instructions are executed, the computer performs the method according to any one of claims 1-7.

10. A computer program product, the computer program product comprising instructions, characterized in that, When the instructions are executed, they cause the computer to perform the method of any one of claims 1-7.

Images