Method for resource retrieval, question answering method and corresponding apparatuses

Abstract

Embodiments of the present application disclose a resource retrieval method, a question and answer method and corresponding devices, and relate to the technical field of artificial intelligence. Input information is obtained, and a first feature representation of the input information is obtained by encoding the input information; first retrieval is performed in a resource library based on the first feature representation of the input information and first feature representations of resources in the resource library to obtain candidate resources; entity information extracted from the candidate resources is obtained, and second feature representations of the candidate resources are obtained by using the extracted entity information; second retrieval is performed in the candidate resources based on the first feature representation of the input information and the second feature representations of the candidate resources to obtain a reference resource as a retrieval result for the input information. The present application can improve the accuracy of resource retrieval, and further improve the accuracy of question and answer based on resource retrieval.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method for resource retrieval, a question-answering method, and a corresponding device. Background Technology

[0002] With the explosive growth of multimodal data such as images, text, and videos, the diversification of content formats helps people better perceive and understand the world around them. In the field of artificial intelligence, resource retrieval and question-answering methods based on resource retrieval are becoming increasingly intelligent. In addition to retrieving resources within the same modality, cross-modal retrieval has also been proposed. Taking image-text retrieval as an example, after a user enters text, relevant image resources are retrieved from the resource library to return to the user, or relevant image resources are retrieved from the resource library as reference resources and used to generate an answer to return to the user.

[0003] Traditional implementations often employ dual-tower encoders to encode the input information and resources in the resource repository separately, then calculate the similarity between the input information and the resources to obtain the reference resource. However, this approach suffers from low retrieval accuracy, resulting in low accuracy in obtaining the answer, especially in cross-modal retrieval scenarios. Summary of the Invention

[0004] In view of this, this application provides a resource retrieval method, a question-and-answer method, and a corresponding apparatus to improve the accuracy of resource retrieval and, consequently, the accuracy of question-and-answer.

[0005] This application provides the following solution:

[0006] Firstly, a method for resource retrieval is provided, the method comprising:

[0007] Obtain input information and encode the input information to obtain a first feature representation of the input information;

[0008] Based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources;

[0009] Obtain entity information extracted from the candidate resources, and use the extracted entity information to obtain the second feature representation of each candidate resource;

[0010] Based on the first feature representation of the input information and the second feature representation of each candidate resource, a second search is performed on each candidate resource to obtain a reference resource as the search result for the input information.

[0011] According to one achievable method in this application embodiment, obtaining entity information extracted from the candidate resources includes:

[0012] Entity extraction is performed on the candidate resources to obtain one or more entities;

[0013] The triple information of the entity is obtained using a knowledge graph, and the triple information includes the entity and the relationship between the entities.

[0014] According to one achievable method in an embodiment of this application, obtaining the second feature representation of each candidate resource using the extracted entity information includes:

[0015] The extracted entity information is encoded to obtain the feature representation of each entity information;

[0016] Using the feature representations of each entity information, the entity information associated with each candidate resource is determined, and the entity feature representation is obtained using the feature representations of the determined entity information;

[0017] By using the entity feature representation and the encoding results of each candidate resource, a second resource representation of each candidate resource is obtained.

[0018] According to one achievable method in the embodiments of this application, the entity information associated with each candidate resource is determined by utilizing the feature representation of each entity information, and the entity feature representation is obtained by utilizing the feature representation of the determined entity information, including:

[0019] Determine the average feature representation corresponding to the first feature representation of each reference resource;

[0020] By utilizing the similarity between the average feature representation and the feature representation of each entity information, the feature representation of the entity information whose corresponding similarity meets the preset similarity requirements is selected;

[0021] The average feature representation is added to the feature representation of the selected entity information, and the resulting feature representation is encoded to obtain the entity feature representation.

[0022] According to one achievable method in the embodiments of this application, the second resource representation of each candidate resource is obtained by utilizing the entity feature representation and the encoding result of each candidate resource:

[0023] The input information is concatenated with each candidate resource and encoded to obtain the third feature representation of each candidate resource. The third feature representation of each candidate resource is then concatenated with the entity feature representation to obtain the second feature representation of each candidate resource.

[0024] According to one achievable method in an embodiment of this application, based on the first feature representation of the input information and the second feature representation of each candidate resource, a second retrieval is performed on each candidate resource to obtain reference resources including:

[0025] Multi-hop retrieval is performed in each candidate resource, wherein each hop retrieval includes: searching in the remaining candidate resources using the currently obtained inference path, adding the retrieved resources to the inference path and removing the retrieved resources from the candidate resources, and executing the next hop retrieval until the preset stop retrieval condition is met.

[0026] Use the resources obtained from each hop as reference resources;

[0027] The process of retrieving from the remaining candidate resources using the currently obtained inference path includes: obtaining a query matrix using the feature representation of the currently obtained inference path; using each remaining candidate resource as a key matrix and a value matrix; performing cross-attention processing on each remaining candidate resource; and selecting the candidate resource with the highest cross-attention score from the remaining candidate resources as the retrieved resource.

[0028] According to one achievable method in an embodiment of this application, the preset retrieval stop condition includes:

[0029] The updated reference resource feature representation is input into a preset classifier. If the classifier outputs a classification result indicating that the search has stopped, then the preset search stop condition is determined to be met.

[0030] Secondly, a question-and-answer method is provided, the method comprising:

[0031] Obtain the problem text, and use the problem text as input information to obtain reference resources using the method described in the first aspect;

[0032] Using the reference resources, generate an answer text for the question text.

[0033] Thirdly, a method for training a resource retrieval model is provided, the method comprising:

[0034] Acquire training data including multiple first training samples, wherein the first training samples include input samples and retrieval result samples corresponding to the input samples in the resource library;

[0035] A resource retrieval model is trained using the training data. The resource retrieval model includes an encoding network and a decoding network. The encoding network encodes the input sample to obtain a first feature representation of the input sample. Based on the first feature representation of the input sample and the first feature representations of each resource in the resource database, a first retrieval is performed in the resource database to obtain candidate resources. Entity information is extracted from the candidate resources, and the extracted entity information is used to obtain a second feature representation of each candidate resource. The decoding network, based on the first feature representation of the input sample and the second feature representations of each candidate resource, performs a second retrieval in each candidate resource to obtain reference resources as retrieval results for the input sample.

[0036] The training objective includes minimizing the difference between the retrieval results obtained by the decoding network for the input sample and the retrieval result samples corresponding to the input sample.

[0037] Fourthly, a method for training a question-answering model is provided, the method comprising:

[0038] Obtain training data including a second training sample, wherein the second training sample includes question samples and answer samples labeled with question samples;

[0039] A question-answering model is trained using the training data. The question-answering model includes an encoding network and a decoding network. The encoding network encodes the question sample to obtain a first feature representation of the question sample. Based on the first feature representation of the question sample and the first feature representations of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources. Entity information extracted from the candidate resources is used to obtain a second feature representation of each candidate resource. The decoding network performs a second retrieval in each candidate resource based on the first feature representation of the question sample and the second feature representations of each candidate resource to obtain reference resources. The reference resources are used to generate an answer text for the question sample.

[0040] The training objective is to minimize the difference between the answer text obtained by the decoding network for the question sample and the answer sample corresponding to the question sample.

[0041] Fifthly, a resource retrieval device is provided, the device comprising:

[0042] The input acquisition unit is configured to acquire input information;

[0043] A retrieval processing unit is configured to input the input information into a resource retrieval model and obtain retrieval results obtained by the resource retrieval model for the input information; the resource retrieval model includes an encoding network and a decoding network.

[0044] The encoding network encodes the input information to obtain a first feature representation of the input information; based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources; entity information extracted from the candidate resources is obtained, and the extracted entity information is used to obtain a second feature representation of each candidate resource;

[0045] The decoding network performs a second retrieval on each candidate resource based on the first feature representation of the input information and the second feature representation of each candidate resource, and obtains a reference resource as the retrieval result for the input information.

[0046] Sixthly, a question-and-answer device is provided, the device comprising:

[0047] The issue retrieval unit is configured to retrieve the issue text;

[0048] The answer acquisition unit is configured to input the question text into a question-answering model and acquire the answer text generated by the question-answering model for the question text; the question-answering model includes a decoding network and an encoding network.

[0049] The decoding network encodes the question text to obtain a first feature representation of the question text; based on the first feature representation of the question text and the first feature representation of each resource in the resource library, a first search is performed in the resource library to obtain candidate resources; entity information extracted from the candidate resources is obtained, and the extracted entity information is used to obtain a second feature representation of each candidate resource;

[0050] The decoding network performs a second retrieval in each candidate resource based on a first feature representation of the question text and a second feature representation of each candidate resource to obtain reference resources; and uses the reference resources to generate an answer text for the question text.

[0051] According to a seventh aspect, a computer-readable storage medium is provided that stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in any one of the first to fourth aspects.

[0052] According to the eighth aspect, an electronic device is provided, comprising:

[0053] One or more processors; and

[0054] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any one of the first to fourth aspects.

[0055] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0056] 1) This application proposes an entity-centered retrieval method. First, candidate resources are obtained by using the first feature representation of the input information and the first feature representation of each resource in the resource database. Then, the entity information extracted from the candidate resources is used to perform a more refined retrieval in each candidate resource, thereby making full use of the possible themes expressed by the candidate resources and improving the accuracy of resource retrieval.

[0057] 2) In this application, knowledge graphs are used to annotate the triple information of entities in candidate resources, thereby better expressing the semantics and themes of candidate resources. Based on these entities, a structured intermediate representation, namely entity feature representation, is established, and this representation is used to realize the connection between candidate resources and input information, thereby improving the accuracy of retrieval.

[0058] 3) This application adopts a multi-hop retrieval method. In each hop, the candidate resource with the highest cross-attention score is selected from the candidate resources, and the classifier is used to automatically determine the number of retrieval hops, which has stronger generalization performance and can achieve better retrieval results.

[0059] 4) This application employs an end-to-end question-answering model for reference resource retrieval and answer generation, enabling the answer generation stage to utilize information from the retrieval stage, thereby improving the accuracy of the generated answers.

[0060] Of course, any product implementing this application does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a system architecture diagram applicable to the embodiments of this application;

[0063] Figure 2 A flowchart of a resource retrieval method provided in an embodiment of this application;

[0064] Figure 3 A schematic diagram illustrating the principle structure of the resource retrieval model provided in the embodiments of this application;

[0065] Figure 4 A flowchart of the question-and-answer method provided in the embodiments of this application;

[0066] Figure 5 A schematic diagram illustrating the principle structure of the question-answering model provided in the embodiments of this application;

[0067] Figure 6 A flowchart illustrating the method for training a resource retrieval model provided in this application embodiment;

[0068] Figure 7 A flowchart illustrating the method for training a question-answering model provided in this application embodiment;

[0069] Figure 8A schematic block diagram of a resource retrieval device provided in the embodiments of this application;

[0070] Figure 9 A schematic block diagram of a question-and-answer device provided in an embodiment of this application;

[0071] Figure 10 A schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0072] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0073] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0074] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0075] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0076] To facilitate understanding of this application, a brief description of the system architecture to which this application applies will be given first. Figure 1 An exemplary system architecture that can be applied to embodiments of this application is shown, such as Figure 1 As shown, the system architecture may include a first model training device that establishes a resource retrieval model offline and a resource retrieval device that performs resource retrieval on input information online; or, the system architecture may include a second model training device that establishes a question-answering model offline and a question-answering device that answers question text online. Figure 1 Both of these cases are shown in the text.

[0077] In this embodiment, after acquiring the training data composed of the first training samples, the first model training device can train the resource retrieval model using the method provided in this application.

[0078] After acquiring the training data composed of the second training samples, the second model training device can train the question-answering model using the method provided in the embodiments of this application.

[0079] The resource retrieval device uses an established resource retrieval model to process input information in a resource database and obtain retrieval results. This resource database can be a single-modal database or a multi-modal database. For example, the database may contain both text and images.

[0080] The question-answering device uses an established question-answering model to generate answer text from the question text.

[0081] The first model training device, the second model training device, the resource retrieval device, and the question-answering device can be configured as independent servers, or they can be configured on the same server or server group, or they can be configured on independent servers or the same cloud server. A cloud server, also known as a cloud computing server or cloud host, is a host product in the cloud computing service system, designed to address the shortcomings of traditional physical hosts and Virtual Private Servers (VPs) services, such as high management difficulty and weak service scalability. The first model training device, the second model training device, the resource retrieval device, and the question-answering device can also be configured on computer terminals with strong computing capabilities.

[0082] It should be noted that, in addition to online searching and answer generation, the aforementioned resource retrieval device and question-and-answer device can also perform offline searching and answer generation.

[0083] It should be understood that Figure 1 The number of the first model training device, second model training device, resource retrieval device, question answering device, resource retrieval model, question answering model, and resource library in the diagram is merely illustrative. Depending on the implementation requirements, any number of these devices can be used.

[0084] It should be noted that the terms "first," "second," etc., used in this disclosure do not imply any limitation in size, order, or quantity; they are merely used to distinguish between two types of training samples. For example, "first training sample" and "second training sample" distinguish between two types of training samples. Similarly, "first feature representation" and "second feature representation" distinguish between two types of feature representations. And so on.

[0085] Figure 2 This is a flowchart of a resource retrieval method provided in an embodiment of this application. This method can be performed by... Figure 1 The resource retrieval device in the system shown is executed. For example... Figure 2 As shown, the method may include the following steps:

[0086] Step 202: Obtain input information and encode the input information to obtain the first feature representation of the input information.

[0087] Step 204: Based on the first feature representation of the input information and the first feature representation of each resource in the resource library, perform a first search in the resource library to obtain candidate resources.

[0088] Step 206: Obtain entity information extracted from candidate resources, and use the extracted entity information to obtain the second feature representation of each candidate resource.

[0089] Step 208: Based on the first feature representation of the input information and the second feature representation of each candidate resource, perform a second search in each candidate resource to obtain the reference resource as the search result for the input information.

[0090] As can be seen from the above process, this application proposes an entity-centered retrieval method. First, it uses the first feature representation of the input information and the first feature representation of each resource in the resource database to initially retrieve candidate resources in the resource database. Then, it uses the entity information extracted from the candidate resources to perform a more refined retrieval in each candidate resource, thereby making full use of the possible themes expressed by the candidate resources and improving the accuracy of resource retrieval.

[0091] The following is a detailed description of each step in the above process. First, step 202, namely "acquiring input information and encoding the input information to obtain a first feature representation of the input information", will be described in detail with reference to the embodiment.

[0092] The source of input information can vary depending on the application scenario. For example, users can enter text (characters, words, sentences, etc.) into a search engine. This text can be directly entered into the text box or obtained through voice recognition. Alternatively, users can enter images into the search engine. Furthermore, users can also enter text or images into other applications that embed search functionality.

[0093] The resource retrieval implemented in this application can be achieved using an end-to-end resource retrieval model. This resource retrieval model may include an encoding network and a decoding network.

[0094] like Figure 3As shown, the encoding of the input information in this step can be implemented by the first encoding module in the encoding network. The first encoding module can adopt a pre-trained language model such as BERT (Bidirectional Encoder Representation from Transformers), XLNet, or GPT (Generative Pre-Training). After encoding by the first encoding module, the first feature representation of the input information is obtained.

[0095] The following describes in detail step 204, namely, "based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources", with reference to the embodiments.

[0096] The resource library involved in this application embodiment may contain resources of one modality or resources of multiple modalities. For example, it may contain text or images, or both. The resources in the resource library may be of the same modality as the input information or may be of a different modality. Alternatively, the resources in the resource library may be partially of the same modality as the input information.

[0097] Each resource in the resource repository can be encoded separately to obtain its first feature representation. Different models can be used to encode resources of different modalities. For example, pre-trained language models such as BERT, XLNet, and GPT can be used for text resources. Similarly, neural network models such as AlexNet and VGG (Visual Geometry Group Network) can be used for image resources.

[0098] To improve the efficiency of the first retrieval, as a preferred implementation, each resource in the resource database can be pre-encoded to obtain a first feature representation of each resource, and then these first feature representations can be stored. During resource retrieval, the pre-stored first feature representations of each resource can be directly used.

[0099] In this step, the similarity between the first feature representation of the input information and the first feature representation of each resource in the resource library can be calculated, and candidate resources whose corresponding similarity meets the preset first similarity requirement can be selected. For example, the N resources with the highest corresponding similarity can be selected as candidate resources, where N is a preset positive integer. Another example is selecting resources whose corresponding similarity is greater than or equal to the preset first similarity threshold as candidate resources.

[0100] The first retrieval performed in this step is based on the similarity between the first feature representation of the input information and the first feature representation of the resource. This approach has limited understanding of the resource, especially when the modalities of the input information and the resource are inconsistent. The first feature representations for the two modalities are generated independently, lacking semantic connection in the expression of features, thus resulting in low retrieval accuracy. Therefore, in this embodiment, the first retrieval performed in the resource database can be regarded as a coarse retrieval, and the resources obtained from the first retrieval are used as candidate resources.

[0101] The following describes in detail step 206, namely "obtaining entity information extracted from candidate resources and using the extracted entity information to obtain the second feature representation of each candidate resource", with reference to the embodiments.

[0102] For resources such as text or images, the entity information contained therein can largely reflect the theme of the resource and greatly aid in the semantic understanding of each resource. Therefore, in this embodiment, entity information is extracted from candidate resources. The extracted entity information is used to better express the semantics of the resources and to better establish the association between resources across modalities.

[0103] One possible approach is to extract one or more entities from the candidate resources, and the aforementioned entity information is the extracted entity.

[0104] As another feasible approach, entity extraction is performed on candidate resources to obtain more than one entity. Then, a knowledge graph is used to obtain the triple information of each entity, which includes the entity and the relationships between them. In other words, the entity information described above is the entity's triple information.

[0105] This application does not impose any restrictions on the entity extraction method; any existing entity extraction method can be used to extract entities from candidate resources. A resource may contain one entity or multiple entities.

[0106] If a resource contains only one entity, a triple corresponding to that entity can be randomly selected from the knowledge graph. For example, if an image resource contains entity e1, a triple (e1, r1, e2) can be randomly selected from the knowledge graph, where e2 is an entity that has a relation r1 with e1. The triple information (e1, r1, e2) for entity e1 is recorded.

[0107] If a resource contains multiple entities, and two of these entities have a relationship in the knowledge graph, then the triples corresponding to these two entities are obtained using these two entities and their relationship in the knowledge graph. If an entity has no relationship with other entities in the knowledge graph, then a triple corresponding to that entity is randomly selected from the knowledge graph. For example, an image resource contains entities e1, e2, and e3. e1 and e2 have a relationship r1 in the knowledge graph, so the triple information for entities e1 and e2 is recorded as (e1, r1, e2). e3 has no relationship with e1, and e3 also has no relationship with e2 in the knowledge graph. Therefore, to ensure entity coverage, a triple for e3 is randomly selected from the knowledge graph as (e3, r2, e4), and the triple information for entity e3 is recorded as (e3, r2, e4).

[0108] When obtaining the second feature representation of each candidate resource using the extracted entity information, various methods can be used to integrate the extracted entity information into the feature representation of each candidate resource to obtain the aforementioned second feature representation. For example, the entity information of the candidate resource can be encoded and then concatenated with the first feature representation of the candidate resource. However, a more preferred implementation method is provided in this application embodiment, such as... Figure 3 As shown, this implementation can be executed by the second encoding module in the encoding network. Specifically, the following steps S2 to S6 can be performed:

[0109] Step S2: Encode the extracted entity information to obtain the feature representation of each entity information.

[0110] In this step, the second encoding module can use a multi-layer Transformer network to encode the information of each entity and obtain the feature representation of each entity.

[0111] Step S4: Utilize the feature representations of each entity information to determine the entity information associated with each candidate resource, and use the feature representations of the determined entity information to obtain entity feature representations.

[0112] In this embodiment, the feature representations of all entity information can be used as entity feature representations. However, preferably, further filtering can be performed on each entity information to determine the entity information associated with each candidate resource.

[0113] For example, the average feature representation corresponding to the first feature representation of each reference resource can be determined. Using the similarity between this average feature representation and the feature representations of each entity information, the feature representation of the entity information whose similarity meets a preset second similarity requirement can be selected. For example, the feature representation of the entity information with the highest similarity can be selected. This average feature representation is then added to the feature representation of the selected entity information, and the resulting feature representation is encoded (e.g., using a multi-layer Transformer network) to obtain the entity feature representation.

[0114] Step S6: Using entity feature representation and the encoding results of each candidate resource, obtain the second resource representation of each candidate resource.

[0115] One possible approach is to concatenate the entity feature representation with the first feature representation of each candidate resource to obtain the second resource representation of each candidate resource.

[0116] As another preferred implementation, the input information can be concatenated with each candidate resource and encoded to obtain the third feature representation of each candidate resource. The third feature representation of each candidate resource can then be concatenated with the entity feature representation to obtain the second feature representation of each candidate resource.

[0117] Assuming there are N candidate resources, the input information X is concatenated with each of the N candidate resources to obtain N concatenation results. If the concatenation result is multimodal (i.e., the input information and candidate resources are of different modalities), a multimodal pre-trained model such as OFA (One-For-All) is used to encode the concatenation result. If the concatenation result is unimodal, a pre-trained model corresponding to the specific modality is used for encoding.

[0118] The third feature representations of the N candidate resources obtained by encoding are then concatenated with the entity feature representations to obtain the second feature representations of each candidate resource.

[0119] The following describes in detail step 208, namely, "based on the first feature representation of the input information and the second feature representation of each candidate resource, a second retrieval is performed in each candidate resource to obtain a reference resource as the retrieval result for the input information," with reference to the embodiments.

[0120] If this resource retrieval is only used to retrieve resources related to the user's input information, then in this step, the similarity between the first feature representation of the input information and the second feature representation of each candidate resource can be used to select candidate resources whose similarity meets a preset third similarity threshold. For example, the top M candidate resources in terms of similarity can be selected, or candidate resources whose similarity is greater than or equal to a preset second similarity threshold can be selected. The selected candidate resources are called reference resources, and reference resources can be used as the retrieval results for the input information.

[0121] Besides the more common methods mentioned above, multi-hop retrieval can be used. Multi-hop retrieval is applicable to search-oriented scenarios and is particularly suitable for question-and-answer scenarios based on resource retrieval. Typically, after a user asks a question, information extracted from a single resource is insufficient to accurately answer it; information needs to be gathered from multiple resources and reasoned through to provide an accurate answer. Therefore, multi-hop retrieval is necessary, with each retrieval leading to a resource, thus forming a "chain of evidence."

[0122] The multi-hop retrieval approach is as follows: the initial inference path p is the input information X, and the initial candidate resources are all candidate resources; each hop retrieval includes the following processing:

[0123] The inference path p is used to search the remaining candidate resources (initially, the remaining candidate resources are all candidate resources). The searched resources are added to the inference path p and removed from the candidate resources. Then, the next hop search is performed until the preset stop search condition is met.

[0124] The candidate resources contained in the final reasoning path p are the reference resources. In a retrieval-oriented scenario, these reference resources can serve as retrieval results. In a question-answering scenario, these reference resources serve as "evidence" to generate the answer.

[0125] In each hop, when searching among the remaining candidate resources using inference path p, one possible approach is to utilize the similarity between the retrieval feature representation of the current hop and the second feature representation of the remaining candidate resources to retrieve the candidate resource with the highest similarity. The retrieval feature representation of the current hop can be the feature representation of the current inference path p.

[0126] As another possible approach, such as Figure 3 As shown, during the multi-hop retrieval process of the decoding network, when retrieving from the remaining candidate resources using the inference path p, a crossattention mechanism can be adopted. The query matrix Q is obtained by using the feature representation of the current inference path. Each remaining candidate resource is used as the key matrix K and the value matrix V. After crossattention processing, the candidate resource with the highest crossattention score is selected from the remaining candidate resources.

[0127] The aforementioned preset retrieval stopping condition could be, for example, that the number of candidate resources contained in the inference path p reaches a preset threshold. Besides this method, since different input information corresponds to different retrieval results, and the number of reference resources required when generating questions varies, a gating mechanism can be used to determine whether to stop the retrieval. Specifically, a classifier can be used, such as a sigmoid classifier for binary classification. The input to this classifier is the feature representation of the inference path p, and the classification result is either continue retrieval or stop retrieval. If the classifier's classification result is continue retrieval, then the next hop retrieval continues; if the classifier's classification result is stop retrieval, then the multi-hop retrieval ends, and the candidate resources contained in the currently obtained inference path p are used as reference resources.

[0128] In scenarios where resource retrieval is the purpose, the obtained reference resources can be returned to the user as search results. For example, in an image-text search scenario, the user enters text keywords, and the method and flow described above in the embodiments of this application are used to retrieve relevant images from a resource library containing images as search results.

[0129] The above search process can also be applied to question-and-answer scenarios. The process applied to question-and-answer scenarios can be as follows: Figure 4 As shown, the steps include:

[0130] Step 402: Obtain the problem text.

[0131] Users can enter questions into the intelligent question-and-answer system. Users can directly enter the question text or enter the question by voice, and then the question text will be obtained through voice recognition.

[0132] Step 404: Execute the following steps with the question text as input: Figure 2 The process shown in the diagram yields reference resources.

[0133] This process can be found in [reference]. Figure 2 The relevant details in the illustrated embodiments will not be repeated here.

[0134] Step 406: Using reference resources, generate answer text for the question text.

[0135] In this step, the first feature representation of the question text and the second feature representation of the reference resource can be used for decoding to predict the answer text. Furthermore, when predicting each token in the answer text, the previously predicted tokens can be combined with the aforementioned decoding process.

[0136] In this question-answering scenario, the question-answering model can be in Figure 3 Based on the retrieval model shown, a prediction module is added to the decoding network, such as... Figure 5As shown in the diagram, the prediction module uses the reference resources obtained by the retrieval module to predict the answer text. After cross-attention processing using the first feature representation of the question text and the second feature representation of the reference resources in the decoding network, the resulting vector representation and the feature representations of each token already predicted by the prediction network are input into the prediction network, which then predicts the next token in the answer text. Figure 5 As can be seen from the embodiments, an end-to-end question-and-answer model is used for reference resource retrieval and answer generation, so that the answer generation stage can utilize the information from the retrieval stage, thereby improving the accuracy of the generated answer.

[0137] In order to facilitate cross-attention processing using the first feature representation of the problem text and the second feature representation of the reference resource in the decoding network, the second feature representation of the input candidate resource can be masked, retaining only the features of the reference resource and the corresponding entity information.

[0138] This question-answering method can be applied to various question-answering scenarios. Here, we will only take one scenario as an example:

[0139] With the continuous development and popularization of internet technology and computer terminals, more and more users are using online meetings for communication. During these meetings, information is input in multiple ways, including text, audio, and images, ultimately generating meeting minutes. Users can generate questions based on these meeting minutes. To generate more accurate answers, the method described in this application embodiment can be used: the multimodal resources (text, images, etc.) in the meeting minutes are treated as a resource library. One or more reference resources are retrieved from this library, and the answers generated from these reference resources are recorded and returned to the user.

[0140] Figure 6 This is a flowchart of a method for training a resource retrieval model provided in an embodiment of this application. This method can be... Figure 1 The training of the first model in the system architecture shown is then transferred to execution. For example... Figure 6 As shown, the method may include the following steps:

[0141] Step 602: Obtain training data including multiple first training samples, wherein the first training samples include input samples and the retrieval result samples corresponding to the input samples in the resource library.

[0142] In this embodiment of the application, when training the retrieval model, an input sample can be constructed, and the retrieval result sample corresponding to the input sample in the resource library can be labeled, that is, label one or more resources in the resource library as retrieval result samples.

[0143] Step 604: Train a resource retrieval model using training data. The resource retrieval model includes an encoding network and a decoding network. The encoding network encodes the input sample to obtain a first feature representation of the input sample. Based on the first feature representation of the input sample and the first feature representations of each resource in the resource database, a first retrieval is performed in the resource database to obtain candidate resources. Entity information is extracted from the candidate resources, and the extracted entity information is used to obtain a second feature representation of each candidate resource. The decoding network performs a second retrieval in each candidate resource based on the first feature representation of the input sample and the second feature representations of each candidate resource to obtain reference resources as retrieval results for the input sample. The training objective includes minimizing the difference between the retrieval results obtained by the decoding network for the input sample and the retrieval result samples corresponding to the input sample.

[0144] The structure and principle of the resource retrieval model can be described as follows: Figure 3 As shown, the first encoding module in the encoding network can employ a pre-trained language model such as BERT, XLNet, or GPT. It encodes the input samples to obtain the first feature representation of the input information.

[0145] Each resource in the resource repository can be encoded separately to obtain its first feature representation. Different models can be used to encode resources of different modalities. For example, pre-trained language models such as BERT, XLNet, and GPT can be used for text resources. Similarly, neural network models such as AlexNet and VGG can be used for image resources.

[0146] When obtaining candidate resources through the initial search in the resource library, the similarity between the first feature representation of the input sample and the first feature representation of each resource in the resource library can be calculated, and candidate resources whose corresponding similarity meets the preset first similarity requirement can be selected. For example, the N resources with the highest corresponding similarity can be selected as candidate resources, where N is a preset positive integer. Another example is selecting resources whose corresponding similarity is greater than or equal to a preset first similarity threshold as candidate resources.

[0147] One possible approach is to extract one or more entities from the candidate resources, and the aforementioned entity information is the extracted entity.

[0148] As another feasible approach, entity extraction is performed on candidate resources to obtain more than one entity. Then, a knowledge graph is used to obtain the triple information of each entity, which includes the entity and the relationships between them. In other words, the entity information described above is the entity's triple information.

[0149] If a resource contains only one entity, then a triple corresponding to that entity can be randomly selected from the knowledge graph.

[0150] If a resource contains multiple entities, and two of these entities have a relationship in the knowledge graph, then the triples corresponding to these two entities are obtained using these two entities and their relationship in the knowledge graph. If an entity has no relationship with other entities in the knowledge graph, then the triple corresponding to that entity is randomly selected from the knowledge graph.

[0151] In one preferred embodiment, when obtaining the second feature representation of each candidate resource using the extracted entity information, the second encoding module can encode each extracted entity information to obtain the feature representation of each entity information; then, using the feature representation of each entity information, the entity information associated with each candidate resource is determined, and the entity feature representation is obtained using the determined entity information feature representation; using the entity feature representation and the encoding result of each candidate resource, the second resource representation of each candidate resource is obtained.

[0152] As one feasible approach, when obtaining the second resource representation of each candidate resource by utilizing entity feature representation and the encoding results of each candidate resource, the input sample can be concatenated with each candidate resource and encoded to obtain the third feature representation of each candidate resource. The third feature representation of each candidate resource can then be concatenated with the entity feature representation to obtain the second feature representation of each candidate resource.

[0153] During the second retrieval among candidate resources, the decoding network can utilize the similarity between the first feature representation of the input sample and the second feature representation of each candidate resource to select candidate resources whose similarity meets a preset third similarity threshold. For example, it can select the top M candidate resources in terms of similarity, or select candidate resources whose similarity is greater than or equal to the preset second similarity threshold. The selected candidate resources are called reference resources, and reference resources can be used as the retrieval results of the input sample.

[0154] However, as a more preferred approach, the decoding network can employ a multi-hop retrieval method. The idea behind multi-hop retrieval is: the initial inference path p is the input sample X, and the initial candidate resources are all candidate resources; each hop retrieval includes the following processing:

[0155] The remaining candidate resources are searched using inference path p. The searched resources are added to inference path p and removed from the candidate resources. Then the next hop search is performed until the preset stop search condition is met.

[0156] The candidate resources contained in the final inference path p are the reference resources.

[0157] In each hop, when searching among the remaining candidate resources using inference path p, one possible approach is to utilize the similarity between the retrieval feature representation of the current hop and the second feature representation of the remaining candidate resources to retrieve the candidate resource with the highest similarity. The retrieval feature representation of the current hop can be the feature representation of the current inference path p.

[0158] As another possible approach, such as Figure 3 As shown, during the multi-hop retrieval process of the decoding network, when retrieving from the remaining candidate resources using the inference path p, a crossattention mechanism can be adopted. The query matrix Q is obtained by using the feature representation of the current inference path. Each remaining candidate resource is used as the key matrix K and the value matrix V. After cross-attention processing, the candidate resource with the highest cross-attention score is selected from the remaining candidate resources.

[0159] The aforementioned preset retrieval stopping condition could be, for example, that the number of candidate resources contained in the inference path p reaches a preset threshold. Besides this method, a gating mechanism can be used to determine whether to stop the retrieval. Specifically, a classifier, such as a sigmoid classifier, can be used for binary classification. The input to this classifier is the feature representation of the inference path p, and the classification result is either continue retrieval or stop retrieval. If the classifier's classification result is continue retrieval, then the next hop retrieval continues; if the classifier's classification result is stop retrieval, then the multi-hop retrieval ends, and the candidate resources contained in the currently obtained inference path p are used as reference resources.

[0160] The training objective of the resource retrieval model is to minimize the difference between the retrieval results obtained by the decoding network for the input sample and the corresponding retrieval result samples. In other words, it aims to make the retrieval results obtained by the decoding network as consistent as possible with the retrieval result samples. In this embodiment, a loss function can be constructed based on the above training objective. In each iteration, the parameters of the retrieval model are updated using methods such as gradient descent, based on the value of the loss function, until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0161] Figure 7 This is a flowchart of a method for training a question-answering model provided in an embodiment of this application. The method can be described by… Figure 1 The second model training device in the system architecture shown is executed. For example... Figure 7 As shown, the method may include the following steps:

[0162] Step 702: Obtain training data including the second training samples, which include question samples and answer samples labeled with the question samples.

[0163] In this embodiment of the application, when training the question-answering model, some question-answer samples and answer samples corresponding to question samples can be constructed. These can be constructed manually, or they can be question-answer pairs with high user feedback or evaluation from the historical question-answering logs of the question-answering model, or other methods, which will not be listed one by one.

[0164] Step 704: Train a question-answering model using training data. The question-answering model includes an encoding network and a decoding network. The encoding network encodes the question sample to obtain a first feature representation of the question sample. Based on the first feature representation of the question sample and the first feature representations of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources. The entity information extracted from the candidate resources is used to obtain a second feature representation of each candidate resource. Based on the first feature representation of the question sample and the second feature representations of each candidate resource, the decoding network performs a second retrieval in each candidate resource to obtain reference resources. The reference resources are used to generate answer text for the question sample. The training objective includes minimizing the difference between the answer text for the question sample obtained by the decoding network and the answer sample corresponding to the question sample.

[0165] The structure and principles of the question-answering model can be described as follows: Figure 5 As shown in the figure, the specific details can be found in the relevant descriptions in the method embodiments, and will not be repeated here.

[0166] The training objective of a question-answering model is to minimize the difference between the answer text obtained by the decoding network for a question sample and the corresponding answer sample; in other words, to make the answer text obtained by the decoding network as consistent as possible with the answer sample. In this embodiment, a loss function can be constructed based on the above training objective. In each iteration, the parameters of the question-answering model are updated using methods such as gradient descent, based on the value of the loss function, until a preset training termination condition is met. The training termination condition may include, for example, the value of the loss function being less than or equal to a preset loss function threshold, or the number of iterations reaching a preset threshold.

[0167] The embodiments of this application have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0168] According to another embodiment, a resource retrieval device is provided. Figure 8 A schematic block diagram of a resource retrieval apparatus according to one embodiment is shown. Figure 8 As shown, the device 800 includes an input acquisition unit 801 and a retrieval processing unit 802, and may further include an entity extraction unit 803. The main functions of each component are as follows:

[0169] The input acquisition unit 801 is configured to acquire input information.

[0170] The retrieval processing unit 802 is configured to input input information into a resource retrieval model and obtain retrieval results obtained by the resource retrieval model for the input information; the resource retrieval model includes an encoding network and a decoding network.

[0171] The encoding network encodes the input information to obtain a first feature representation of the input information; based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources; entity information extracted from the candidate resources is obtained, and the extracted entity information is used to obtain a second feature representation of each candidate resource.

[0172] The decoding network performs a second retrieval among the candidate resources based on the first feature representation of the input information and the second feature representation of each candidate resource, and obtains the reference resource as the retrieval result for the input information.

[0173] As one possible approach, the encoding network calculates the similarity between the first feature representation of the input information and the first feature representation of each resource in the resource library, and selects the N resources with the highest corresponding similarity as candidate resources, where N is a preset positive integer. Alternatively, it selects resources with a corresponding similarity greater than or equal to a preset similarity threshold as candidate resources.

[0174] As one possible implementation method, the entity extraction unit 803 is configured to extract entities from candidate resources to obtain more than one entity; and to obtain triple information of the entity using a knowledge graph, the triple information including the entity and the relationship between the entities.

[0175] As one more specific implementation, the entity extraction unit 803 can query the relationship between one or more entities in the knowledge graph; if two entities among the one or more entities have a relationship in the knowledge graph, then the three-tuples corresponding to the two entities are obtained using the two entities and their relationship in the knowledge graph; if one entity among the one or more entities has no relationship with other entities in the knowledge graph, then the three-tuples corresponding to that entity are randomly selected in the knowledge graph.

[0176] As one possible approach, when the encoding network obtains the second feature representation of each candidate resource using the extracted entity information, it can encode each extracted entity information to obtain the feature representation of each entity information; use the feature representation of each entity information to determine the entity information associated with each candidate resource, and use the determined feature representation of the entity information to obtain the entity feature representation; and use the entity feature representation and the encoding result of each candidate resource to obtain the second resource representation of each candidate resource.

[0177] As a more specific implementation, the coding network can determine the average feature representation corresponding to the first feature representation of each reference resource; use the similarity between the average feature representation and the feature representation of each entity information to select the feature representation of the entity information whose similarity meets the preset similarity requirements; add the average feature representation to the feature representation of the selected entity information, encode the resulting feature representation, and obtain the entity feature representation.

[0178] As one possible approach, when the encoding network obtains the second resource representation of each candidate resource by utilizing the entity feature representation and the encoding results of each candidate resource, it can concatenate the input information with each candidate resource separately and encode them to obtain the third feature representation of each candidate resource. Then, it can concatenate the third feature representation of each candidate resource with the entity feature representation to obtain the second feature representation of each candidate resource.

[0179] As one possible approach, the decoding network can perform multi-hop searches among candidate resources during the second retrieval. Each hop search includes: searching the remaining candidate resources using the currently obtained inference path, adding the retrieved resources to the inference path and removing the retrieved resources from the candidate resources, and executing the next hop search until a preset stopping retrieval condition is met. Then, the resources obtained from each hop search are used as reference resources.

[0180] The process of retrieving from the remaining candidate resources using the currently obtained inference path includes: obtaining a query matrix using the feature representation of the currently obtained inference path; using each remaining candidate resource as a key matrix and a value matrix; performing cross-attention processing on each remaining candidate resource; and selecting the candidate resource with the highest cross-attention score from the remaining candidate resources as the retrieved resource.

[0181] As one possible approach, the decoding network can input the updated reference resource feature representation into a preset classifier. If the classifier outputs a classification result indicating that the search has stopped, then the preset search stop condition is determined to be met.

[0182] According to another embodiment, a question-and-answer device is provided. Figure 9 A schematic block diagram of a question-and-answer device according to one embodiment is shown. Figure 9As shown, the device 900 includes a question acquisition unit 901 and an answer acquisition unit 902. The main functions of each component are as follows:

[0183] Problem acquisition unit 901 is configured to acquire problem text.

[0184] The answer acquisition unit 902 is configured to input the question text into the question answering model and acquire the answer text generated by the question answering model for the question text; the question answering model includes a decoding network and an encoding network.

[0185] The decoding network encodes the question text to obtain the first feature representation of the question text; based on the first feature representation of the question text and the first feature representation of each resource in the resource library, a first search is performed in the resource library to obtain candidate resources; entity information extracted from the candidate resources is obtained, and the extracted entity information is used to obtain the second feature representation of each candidate resource.

[0186] The decoding network performs a second retrieval among the candidate resources based on the first feature representation of the question text and the second feature representation of each candidate resource to obtain reference resources; and uses the reference resources to generate the answer text for the question text.

[0187] As one possible approach, the decoding network uses a first feature representation of the question text and a second feature representation of the reference resource to perform decoding processing to predict the answer text of the question text; wherein, when predicting each element of the answer text, the already predicted elements are further combined for decoding processing.

[0188] This application also provides an apparatus for training a resource retrieval model, including a first sample acquisition unit and a first model training unit. The main functions of each component are as follows:

[0189] The first sample acquisition unit is configured to acquire training data including multiple first training samples, wherein the first training samples include input samples and retrieval result samples corresponding to the input samples in the resource library.

[0190] The first model training unit is configured to train a resource retrieval model using training data. The resource retrieval model includes an encoding network and a decoding network. The encoding network encodes the input sample to obtain a first feature representation of the input sample. Based on the first feature representation of the input sample and the first feature representations of each resource in the resource database, a first retrieval is performed in the resource database to obtain candidate resources. Entity information is extracted from the candidate resources, and the extracted entity information is used to obtain a second feature representation of each candidate resource. The decoding network performs a second retrieval in each candidate resource based on the first feature representation of the input sample and the second feature representations of each candidate resource to obtain reference resources as retrieval results for the input sample. The training objective includes minimizing the difference between the retrieval results obtained by the decoding network for the input sample and the corresponding retrieval result samples.

[0191] This application also provides an apparatus for training a question-answering model, including a second sample acquisition unit and a second model training unit. The main functions of each component are as follows:

[0192] The second sample acquisition unit is configured to acquire training data including a second training sample, which includes question samples and answer samples labeled with the question samples.

[0193] The second model training unit trains a question-answering model using training data. The question-answering model includes an encoding network and a decoding network. The encoding network encodes the question sample to obtain a first feature representation of the question sample. Based on the first feature representation of the question sample and the first feature representations of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources. The entity information extracted from the candidate resources is used to obtain a second feature representation of each candidate resource. Based on the first feature representation of the question sample and the second feature representations of each candidate resource, the decoding network performs a second retrieval in each candidate resource to obtain reference resources. The reference resources are used to generate answer text for the question sample. The training objective includes minimizing the difference between the answer text for the question sample obtained by the decoding network and the answer sample corresponding to the question sample.

[0194] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0195] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation portals are provided for users to choose to authorize or refuse.

[0196] In addition, embodiments of this application also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0197] And an electronic device, comprising:

[0198] One or more processors; and

[0199] A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method described in any of the foregoing method embodiments.

[0200] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing method embodiments.

[0201] in, Figure 10An exemplary architecture of an electronic device is shown, which may include a processor 1010, a video display adapter 1011, a disk drive 1012, an input / output interface 1013, a network interface 1014, and a memory 1020. The processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020 can communicate with each other via a communication bus 1030.

[0202] The processor 1010 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.

[0203] The memory 1020 can be implemented in the form of ROM (Read-Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system 1021 for controlling the operation of the electronic device 1000, and the basic input / output system (BIOS) 1022 for controlling the low-level operations of the electronic device 1000. Additionally, it can store a web browser 1023, a data storage management system 1024, and a resource retrieval device / question-answering device 1025, etc. The aforementioned resource retrieval device / question-answering device 1025 can be the application program that specifically implements the aforementioned steps in this embodiment. In summary, when implementing the technical solution provided in this application through software or firmware, the relevant program code is stored in the memory 1020 and executed by the processor 1010.

[0204] Input / output interface 1013 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0205] The network interface 1014 is used to connect the communication module (not shown in the figure) to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0206] Bus 1030 includes a pathway for transmitting information between various components of the device (e.g., processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, and memory 1020).

[0207] It should be noted that although the above-described device only shows the processor 1010, video display adapter 1011, disk drive 1012, input / output interface 1013, network interface 1014, memory 1020, bus 1030, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the solution of this application, and does not necessarily include all the components shown in the figures.

[0208] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer program product. This computer program product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0209] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for resource retrieval, characterized in that, The method includes: Obtain input information and encode the input information to obtain a first feature representation of the input information; Based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources; Obtain entity information extracted from the candidate resources, and use the extracted entity information to obtain the second feature representation of each candidate resource; Based on the first feature representation of the input information and the second feature representation of each candidate resource, a second search is performed on each candidate resource to obtain a reference resource as the search result for the input information. The step of obtaining the second feature representation of each candidate resource using the extracted entity information includes: The input information is concatenated with each candidate resource and then encoded to obtain the third feature representation of each candidate resource; The extracted entity information is encoded to obtain entity feature representations; The third feature representation of each candidate resource and the entity feature representation are concatenated to obtain the second feature representation of each candidate resource.

2. The method according to claim 1, characterized in that, Obtaining entity information extracted from the candidate resources includes: Entity extraction is performed on the candidate resources to obtain one or more entities; The triple information of the entity is obtained using a knowledge graph, and the triple information includes the entity and the relationship between the entities.

3. The method according to claim 1, characterized in that, The extracted entity information is encoded to obtain entity feature representations, including: The extracted entity information is encoded to obtain the feature representation of each entity information; By utilizing the feature representations of each entity information, the entity information associated with each candidate resource is determined, and the entity feature representation is obtained by utilizing the feature representations of the determined entity information.

4. The method according to claim 3, characterized in that, Using the feature representations of each entity information, the entity information associated with each candidate resource is determined, and the entity feature representation is obtained using the feature representations of the determined entity information, including: Determine the average feature representation corresponding to the first feature representation of each candidate resource; By utilizing the similarity between the average feature representation and the feature representation of each entity information, the feature representation of the entity information whose corresponding similarity meets the preset similarity requirements is selected; The average feature representation is added to the feature representation of the selected entity information, and the resulting feature representation is encoded to obtain the entity feature representation.

5. The method according to any one of claims 1 to 4, characterized in that, Based on the first feature representation of the input information and the second feature representation of each candidate resource, a second retrieval is performed on each candidate resource to obtain reference resources including: Multi-hop retrieval is performed on each candidate resource, wherein each hop retrieval includes: searching the remaining candidate resources using the currently obtained inference path, adding the retrieved resources to the inference path and removing the retrieved resources from the candidate resources, and executing the next hop retrieval until the preset stop retrieval condition is met. Use the resources obtained from each hop as reference resources; The process of retrieving from the remaining candidate resources using the currently obtained inference path includes: obtaining a query matrix using the feature representation of the currently obtained inference path; treating each remaining candidate resource as a key matrix and a value matrix; performing cross-attention processing on each remaining candidate resource; and selecting the candidate resource with the highest cross-attention score from the remaining candidate resources as the retrieved resource.

6. The method according to claim 5, characterized in that, The preset stop search conditions include: The updated reference resource feature representation is input into a preset classifier. If the classifier outputs a classification result indicating that the search has stopped, then the preset search stop condition is determined to be met.

7. A question-and-answer method, characterized in that, The method includes: Obtain the problem text, and use the problem text as input information to obtain the reference resource using the method described in any one of claims 1 to 6; Using the reference resources, generate an answer text for the question text.

8. A method for training a resource retrieval model, characterized in that, The method includes: Acquire training data including multiple first training samples, wherein the first training samples include input samples and retrieval result samples corresponding to the input samples in the resource library; A resource retrieval model is trained using the training data. The resource retrieval model includes an encoding network and a decoding network. The encoding network encodes the input sample to obtain a first feature representation of the input sample. Based on the first feature representation of the input sample and the first feature representations of each resource in the resource database, a first retrieval is performed in the resource database to obtain candidate resources. Entity information is extracted from the candidate resources, and the extracted entity information is encoded to obtain an entity feature representation. The entity feature representation and the third feature representation of each candidate resource are concatenated to obtain a second feature representation of each candidate resource. The third feature representation is obtained by concatenating the input sample with each candidate resource and then encoding it. The decoding network performs a second retrieval in each candidate resource based on the first feature representation of the input sample and the second feature representations of each candidate resource to obtain reference resources as retrieval results for the input sample. The training objective includes minimizing the difference between the retrieval results obtained by the decoding network for the input sample and the retrieval result samples corresponding to the input sample.

9. A method for training a question-answering model, characterized in that, The method includes: Obtain training data including a second training sample, wherein the second training sample includes question samples and answer samples labeled with question samples; A question-answering model is trained using the training data. The question-answering model includes an encoding network and a decoding network. The encoding network encodes the question sample to obtain a first feature representation of the question sample. Based on the first feature representation of the question sample and the first feature representations of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources. Entity information extracted from the candidate resources is encoded to obtain entity feature representations. The entity feature representations and the third feature representations of each candidate resource are concatenated to obtain a second feature representation of each candidate resource. The third feature representation is obtained by concatenating the question sample with each candidate resource and then encoding it. The decoding network performs a second retrieval in each candidate resource based on the first feature representation of the question sample and the second feature representations of each candidate resource to obtain reference resources. Using the reference resources, an answer text for the question sample is generated. The training objective is to minimize the difference between the answer text obtained by the decoding network for the question sample and the answer sample corresponding to the question sample.

10. A resource retrieval device, characterized in that, The device includes: The input acquisition unit is configured to acquire input information; A retrieval processing unit is configured to input the input information into a resource retrieval model and obtain retrieval results obtained by the resource retrieval model for the input information; the resource retrieval model includes an encoding network and a decoding network. The encoding network encodes the input information to obtain a first feature representation of the input information; based on the first feature representation of the input information and the first feature representation of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources; entity information is extracted from the candidate resources, the extracted entity information is encoded to obtain an entity feature representation, and the entity feature representation and the third feature representation of each candidate resource are concatenated to obtain a second feature representation of each candidate resource, wherein the third feature representation is obtained by concatenating the input information with each candidate resource and then encoding it; The decoding network performs a second retrieval on each candidate resource based on the first feature representation of the input information and the second feature representation of each candidate resource, and obtains a reference resource as the retrieval result for the input information.

11. A question-and-answer device, characterized in that, The device includes: The issue retrieval unit is configured to retrieve the issue text; The answer acquisition unit is configured to input the question text into a question-answering model and acquire the answer text generated by the question-answering model for the question text; the question-answering model includes a decoding network and an encoding network. The decoding network encodes the question text to obtain a first feature representation of the question text; based on the first feature representation of the question text and the first feature representations of each resource in the resource library, a first retrieval is performed in the resource library to obtain candidate resources; entity information is extracted from the candidate resources, the extracted entity information is encoded to obtain entity feature representations, and the entity feature representations and the third feature representations of each candidate resource are concatenated to obtain the second feature representations of each candidate resource; the third feature representations are obtained by concatenating the question text and each candidate resource and then encoding them. The decoding network performs a second retrieval in each candidate resource based on a first feature representation of the question text and a second feature representation of each candidate resource to obtain reference resources; and uses the reference resources to generate an answer text for the question text.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 9.

13. An electronic device, characterized in that, include: One or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the method according to any one of claims 1 to 9.

Images