A retrieval method, apparatus, device and readable storage medium

Abstract

Embodiments of the present application disclose a retrieval method, device and equipment and a readable storage medium. The method comprises: obtaining a target text for retrieval; determining a retrieval result based on the target text, a knowledge text and a pre-trained language model, the knowledge text being determined based on the target text and used to assist the target text in retrieval, and the pre-trained language model comprising a first part network and a second part network. Since the input of the second part network is determined based on the output of the first part network and the knowledge text, the second part network without an input layer can directly utilize the knowledge contained in the knowledge text, thereby improving the knowledge enhancement effect of the pre-trained language model.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to a retrieval method, apparatus, device, and readable storage medium. Background Technology

[0002] Pre-training technology for deep learning models has become a key technology for the generalization of artificial intelligence and its application in business. It can be understood as using a general deep learning network and massive amounts of data to pre-train the deep learning network on a large scale to obtain a pre-trained model.

[0003] Because of the large scale of deep learning networks and the large amount of data used to train them, the pre-trained models often possess some general knowledge, which can then be used to process other tasks.

[0004] In recent years, pre-training techniques for deep learning models have been widely applied in various fields of artificial intelligence, such as computer vision, natural language processing, and search recommendation. However, their development has been most rapid in the field of natural language processing. Pre-trained models for natural language are called pre-trained language models. They often require a network structure capable of fully acquiring contextual information from the corpus, such as a recurrent neural network (RNN). They also need to be pre-trained on different natural language tasks based on massive amounts of corpus data. The pre-trained model can be saved and has universal input capabilities, allowing it to be used for other tasks.

[0005] However, pre-trained language models also have certain drawbacks. The most prominent drawback is that pre-trained language models can only learn the knowledge from the training data and do not possess any knowledge itself, which affects the accuracy of the output results of the pre-trained language models.

[0006] Therefore, when using pre-trained language models to handle certain complex tasks, it is often necessary to enhance the knowledge content of the pre-trained language models.

[0007] Currently, the method to increase the knowledge of pre-trained language models is to acquire additional knowledge related to the search content input by the user when using the pre-trained language model to process the search task, and input this knowledge and the search content input by the user together into the input layer of the pre-trained language model, so that the pre-trained language model can use this knowledge to complete the search task.

[0008] However, since this method inputs additional knowledge into the input layer of the pre-trained language model, other layers in the pre-trained language model can only utilize the knowledge processing results of the previous layer or multiple layers, resulting in poor knowledge enhancement of the pre-trained language model. Summary of the Invention

[0009] This application provides a retrieval method, apparatus, device, and readable storage medium. The method enables the second part of the network, which is not an input layer, to directly utilize the knowledge contained in the knowledge text, thereby improving the knowledge enhancement effect of the pre-trained language model.

[0010] The first aspect of this application provides a retrieval method, including:

[0011] The process involves obtaining target text for retrieval, which is derived from the user's query input in the search engine. When the user's query is in text format, the target text can be the user's query content. When the user's query is in the form of images, audio, or video, the target text can be converted from the user's query content. The retrieval results are determined based on the target text, knowledge text, and a pre-trained language model. The knowledge text is determined based on the target text and is used to assist in the retrieval process. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text.

[0012] Knowledge text can be understood as text that is input into a pre-trained language model along with the target text during the retrieval process, so that the pre-trained language model can refer to it when determining the retrieval results. It can also be understood as text that is input into a pre-trained language model along with the target text and is used to supplement the knowledge of the pre-trained language model. For example, the target text is the name of the movie that actor A has starred in, while the knowledge text can be the name of the film and television works that actor A has starred in, as well as the names of the actors he has worked with, etc.

[0013] The second part of the network is a non-input layer network. Specifically, the second part of the network can be a single-layer network or a multi-layer network. When the second part is a multi-layer network, it can also be called a sub-model. When the second part of the network is a single-layer network, it can be a hidden layer or an output layer.

[0014] Since the input of the second part of the network is determined based on the output of the first part of the network and the knowledge text, the second part of the network can directly utilize the knowledge in the knowledge text, rather than using the knowledge processed by other layers such as the input layer. This enables knowledge backtracking and improves the knowledge supplementation effect of the second part of the network, thereby further improving the knowledge enhancement effect and thus improving the accuracy of retrieval.

[0015] As one possible approach, the input to the second part of the network is the output of the first part of the network and the knowledge text.

[0016] In this feasible approach, the output of the first part of the network and the knowledge text are directly used as the input of the second part of the network to supplement the knowledge of the second part of the network.

[0017] As one possible approach, there are multiple knowledge texts. The input of the second part of the network is determined based on the output of the first part of the network and some knowledge texts from the multiple knowledge texts. The partial knowledge texts are the knowledge texts from the multiple knowledge texts whose relevance to the output of the first part of the network meets a first requirement. The relevance of the knowledge texts to the output of the first part of the network can be calculated by the similarity between the text vector of the knowledge text and the vector of the output of the first part of the network. The first requirement can specifically be a threshold of relevance.

[0018] When there are multiple knowledge texts, not all of them are input into the second part of the network. Instead, based on the output of the first part of the network, knowledge texts that meet the first requirement of relevance to the output of the first part of the network are selected from the multiple knowledge texts. The input of the second part of the network is determined based on the selected knowledge texts and the output of the first part of the network. This allows for the selective extraction of knowledge in the second part of the network, avoiding the input of knowledge texts with low relevance to the second part of the network, which would result in a large amount of computation and fail to achieve the effect of knowledge enhancement.

[0019] As one feasible approach, determining knowledge text based on target text includes:

[0020] Based on one or more of the target text, target sentences, and keywords, multiple candidate texts are selected as knowledge texts from a text set. The text set is a collection of candidate texts, and the target sentences and keywords are extracted from the target text. The candidate texts can be any text and knowledge texts, and specifically, they can be any text that may be related to the target text. For example, if the target text is the name of a movie that actor A has starred in, then the candidate texts can be any text related to actor A. Specifically, the target text can be decomposed into multiple target sentences using the sentence extraction method in natural language processing, and keywords can be parsed from the target sentences using the keyword extraction method in natural language processing.

[0021] In this implementation, the target text is divided into more granular segments, resulting in target sentences and keywords. This allows for the acquisition of more knowledge text based on the target text, target sentences, and keywords, thus providing more comprehensive knowledge for training the language model.

[0022] As one possible approach, the text set is represented by a node graph, where candidate texts in the text set are represented by nodes in the node graph, and two nodes connected by edges in the node graph correspond to two candidate texts in the text set whose relevance meets the second requirement.

[0023] Using a node graph to identify the text set makes the relationships between candidate texts in the text set clearer and more explicit, making it easier to select knowledge texts from them.

[0024] As an achievable approach, candidate texts in the text set are extracted from web pages; two candidate texts in the text set that meet the second requirement of relevance correspond to two candidate texts that are hyperlinked on the web page; the existence of a hyperlink between two candidate texts can be understood as clicking on one candidate text on one web page, which will jump to another candidate text on another web page; a candidate text can be hyperlinked to one candidate text or to multiple candidate texts.

[0025] Extracting candidate text from web pages ensures the diversity and sufficiency of candidate texts; furthermore, selecting two candidate texts that meet the second requirement of relevance through hyperlinks ensures that the selected candidate texts have a strong relevance to each other.

[0026] As an achievable method, selecting multiple candidate texts as knowledge texts from a text set based on one or more of the target text, target statement, and keywords includes: retrieving multiple knowledge texts from the text set based on the target text, target statement, and keywords, where the multiple knowledge texts include a first candidate text, a second candidate text, a third candidate text, and a fourth candidate text. The first candidate text is obtained from the text set based on the target text, the second candidate text is obtained from the text set based on the target statement, the third candidate text is obtained from the text set based on the keywords, and the fourth candidate text includes candidate texts represented by nodes on the target connection paths in the node graph. The target connection paths include connection paths between nodes representing the first candidate text and nodes representing the second candidate text, and / or connection paths between nodes representing the second candidate text and nodes representing the third candidate text.

[0027] In the node graph, if the node corresponding to the first candidate text and the node corresponding to the second candidate text are connected by one or more nodes, then it can be considered that there is a connection path between the node corresponding to the first candidate text and the node corresponding to the second candidate text. For the same node corresponding to the first candidate text and the node corresponding to the second candidate text, there can be one connection path or multiple connection paths between them.

[0028] In this feasible approach, not only are the first, second, and third candidate texts treated as knowledge texts, but also the candidate texts represented by the nodes on the connection paths between the nodes of the first, second, and third candidate texts are treated as knowledge texts. This effectively increases the number of knowledge texts related to the target text, thereby providing more comprehensive knowledge for training the language model.

[0029] As an achievable approach, determining retrieval results based on target text, knowledge text, and a pre-trained language model includes: for each knowledge text among multiple knowledge texts, calculating the semantic vector of each knowledge text in its respective modality based on its text vector and the mapping matrix corresponding to the modality to which it belongs. The modality to which each knowledge text belongs is one of multiple modalities, used to represent the level of abstraction of the knowledge text. The number of modalities can be set according to actual needs. Specifically, each modality in the multiple modalities corresponds to a mapping matrix. For each knowledge text, multiplying its text vector by the mapping matrix corresponding to its modality yields the semantic vector of the knowledge text in its respective modality. The retrieval results are determined based on the text vector of the target text, the semantic vectors of multiple knowledge texts in their respective modalities, and the pre-trained language model. The input to the second part of the network is determined based on the vector output by the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0030] First, the modal to which each of the multiple knowledge texts belongs is determined. Then, the semantic vector of each knowledge text in its respective modal is obtained based on the mapping matrix of each modal. Finally, based on the semantic vector of each knowledge text in its respective modal, since the modality is used to represent the level of abstraction of the knowledge text, this embodiment realizes the division of knowledge texts according to their level of abstraction, effectively increasing the granularity of knowledge sources and providing a foundation for supplementing different knowledge for pre-trained language models. On this basis, the input of the second part of the network is determined based on the output vector of the first part of the network and the semantic vectors of the multiple knowledge texts in their respective modalities, so that the second part of the network can obtain knowledge at different levels of abstraction, thereby improving the knowledge supplementation effect.

[0031] As one possible approach, multiple modalities include a first mode, a second mode, and a third mode;

[0032] The knowledge text belonging to the first modality includes: the first candidate text and the fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph through less than a first number of nodes; the knowledge text belonging to the second modality includes: the third candidate text and the fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph through less than a second number of nodes; the knowledge text belonging to the third modality includes: other knowledge texts among the multiple knowledge texts besides the knowledge texts belonging to the first modality and the knowledge texts belonging to the third modality.

[0033] The first and second quantities can be set according to actual needs, and this application embodiment does not specifically limit them; taking the first quantity as an example, when the first quantity is 2, the fourth candidate text whose relevance to the first candidate text meets the sixth requirement refers to the candidate text corresponding to the first-order neighbor node of the node corresponding to the first candidate text; when the first quantity is 3, the fourth candidate text whose relevance to the first candidate text meets the sixth requirement refers to the candidate text corresponding to the second-order neighbor node of the node corresponding to the first candidate text.

[0034] Since the first candidate text is determined based on the target text, the second candidate text is determined based on the target statement, and the third candidate text is determined based on keywords, and the target text, target statement, and candidate text have different levels of abstraction, the first candidate text and the fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph by less than a first number of nodes are classified as the first modality. The third candidate text and the fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph by less than a second number of nodes are classified as the second modality. All other knowledge texts among the multiple knowledge texts, except for those belonging to the first modality and those belonging to the third modality, are classified as the third modality, which can achieve a reasonable modality classification of all knowledge texts.

[0035] Furthermore, calculating the relevance between the first and fourth candidate texts by the number of nodes between them, and calculating the relevance between the first and fourth candidate texts by the number of nodes between them, is more accurate and convenient.

[0036] As one feasible approach, the relevance of multiple knowledge texts to the output of the first part of the network is calculated based on the vector of the output of the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0037] The semantic vector of a knowledge text in its modality is calculated based on the text vector of the knowledge text and the mapping matrix corresponding to the modality to which the knowledge text belongs. Therefore, the degree of abstraction of the knowledge text is considered in the process of calculating the semantic vector of the knowledge text in its modality. Based on this, the correlation between the knowledge text calculated based on the output vector of the first part of the network and the semantic vector of the knowledge text in its modality is more accurate than the correlation between the knowledge text calculated based on the output vector of the first part of the network and the text vector of the knowledge text.

[0038] As a feasible approach, knowledge texts can be words, sentences, paragraphs, or chapters.

[0039] This approach offers a variety of options for the forms of knowledge text.

[0040] A second aspect of this application provides a retrieval apparatus, comprising: an acquisition unit for acquiring target text for retrieval; determining retrieval results based on the target text, knowledge text, and a pre-trained language model, wherein the knowledge text is determined based on the target text and is used to assist in the retrieval of the target text; and the pre-trained language model includes a first part network and a second part network, wherein the input of the second part network is determined based on the output of the first part network and the knowledge text.

[0041] As one possible approach, there are multiple knowledge texts; the input of the second part of the network is determined based on the output of the first part of the network and some knowledge texts from the multiple knowledge texts. The partial knowledge texts are the knowledge texts from the multiple knowledge texts whose relevance to the output of the first part of the network meets the first requirement.

[0042] As an achievable approach, a unit is defined for selecting multiple candidate texts as knowledge texts from a text set based on one or more of the target text, target statement, and keywords. The text set is a collection of candidate texts, and the target statement and keywords are extracted from the target text.

[0043] As one possible approach, the text set is represented by a node graph, where candidate texts in the text set are represented by nodes in the node graph, and two nodes connected by edges in the node graph correspond to two candidate texts in the text set whose relevance meets the second requirement.

[0044] As one possible approach, candidate texts in the text set are extracted from web pages; two candidate texts in the text set that meet the second relevance requirement correspond to two candidate texts that have hyperlinks in the web pages.

[0045] As one feasible approach, a unit is defined for retrieving multiple knowledge texts from a text set based on target text, target statement, and keywords. These multiple knowledge texts include a first candidate text, a second candidate text, a third candidate text, and a fourth candidate text. The first candidate text is obtained from the text set based on the target text, the second candidate text is obtained from the text set based on the target statement, the third candidate text is obtained from the text set based on keywords, and the fourth candidate text includes candidate texts represented by nodes on target connection paths in a node graph. The target connection paths include connection paths between nodes representing the first candidate text and nodes representing the second candidate text, and / or connection paths between nodes representing the second candidate text and nodes representing the third candidate text.

[0046] As an achievable retrieval unit, it is used to calculate the semantic vector of each knowledge text in its respective modality for each knowledge text among multiple knowledge texts, based on the text vector of each knowledge text and the mapping matrix corresponding to the modality to which each knowledge text belongs. The modality to which each knowledge text belongs is one of multiple modalities, and the modality is used to represent the level of abstraction of the knowledge text. The retrieval result is determined based on the text vector of the target text, the semantic vectors of multiple knowledge texts in their respective modalities, and the pre-trained language model. The input of the second part of the network is determined based on the vector output by the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0047] As one possible approach, the multiple modalities include a first modality, a second modality, and a third modality; the knowledge text belonging to the first modality includes: a first candidate text and a fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph through a number less than a first number of nodes; the knowledge text belonging to the second modality includes: a third candidate text and a fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph through a number less than a second number of nodes; the knowledge text belonging to the third modality includes: other knowledge texts among the multiple knowledge texts besides the knowledge texts belonging to the first modality and the knowledge texts belonging to the third modality.

[0048] As one feasible approach, the relevance of multiple knowledge texts to the output of the first part of the network is calculated based on the vector of the output of the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0049] As a feasible approach, knowledge texts can be words, sentences, paragraphs, or chapters.

[0050] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the second aspect of the embodiments of this application.

[0051] A third aspect of this application provides a computer device, including: one or more processors and a memory; wherein the memory stores computer-readable instructions; the one or more processors read the computer-readable instructions to cause a training device to implement the method as described in any implementation of the first aspect.

[0052] A fourth aspect of this application provides a computer-readable storage medium including computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in any implementation of the first aspect.

[0053] A fifth aspect of this application provides a chip including one or more processors. Part or all of the processors are configured to read and execute a computer program stored in a memory to perform the methods in any possible implementation of the first aspect described above.

[0054] Optionally, the chip includes a memory, which is connected to the processor via a circuit or wire. Further optionally, the chip also includes a communication interface, to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed. The processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input / output interface.

[0055] In some implementations, some of the processors may implement some steps of the above method through dedicated hardware. For example, the processing of neural network models may be implemented by a dedicated neural network processor or graphics processor.

[0056] The method provided in this application embodiment can be implemented by a single chip or by multiple chips working together.

[0057] A sixth aspect of this application provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the method described in any of the implementations of the first aspect. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the retrieval system architecture in an embodiment of this application;

[0059] Figure 2 This is a schematic diagram of one embodiment of the retrieval method in this application;

[0060] Figure 3 This is a schematic diagram of a knowledge partitioning tree in an embodiment of this application;

[0061] Figure 4 This is a schematic diagram of the first embodiment of the node diagram in this application;

[0062] Figure 5 This is a schematic diagram of one embodiment of determining knowledge text in this application.

[0063] Figure 6 This is a schematic diagram of a second embodiment of the node diagram in this application;

[0064] Figure 7 This is a schematic diagram illustrating an embodiment of determining search results in this application.

[0065] Figure 8 This is a schematic diagram of the third embodiment of the node diagram in this application;

[0066] Figure 9 This is a schematic diagram illustrating the calculation of semantic vectors in an embodiment of this application;

[0067] Figure 10 A schematic diagram of an embodiment that supplements the knowledge presented in this application;

[0068] Figure 11 This is a schematic diagram illustrating an embodiment of the retrieval process in this application.

[0069] Figure 12 This is a schematic diagram of an embodiment of the retrieval device in this application;

[0070] Figure 13 This is a schematic diagram of a computer device provided in an embodiment of this application. Detailed Implementation

[0071] This application provides a retrieval method, apparatus, device, and readable storage medium. The method inputs the acquired knowledge into a non-input layer to avoid the fact that other layers in the pre-trained language model, other than the input layer, cannot directly utilize the knowledge, resulting in poor knowledge enhancement of the pre-trained language model, thereby improving the knowledge enhancement effect of the pre-trained language model.

[0072] The embodiments of this application can be applied to Figure 1 The retrieval system 100 includes a text retrieval module 101, a knowledge base 102, and a pre-trained language model 103. The retrieval system 100 can take many forms, such as a search engine on a webpage or a question-and-answer system for a robot.

[0073] Knowledge base 102 stores a wide variety of knowledge texts.

[0074] The text retrieval module 101 is used to retrieve knowledge text related to the text input by the user from the knowledge base 102; wherein, the text retrieval module 101 can be a semantic retrieval device or a non-semantic retrieval device.

[0075] The pre-trained language model 103 is used to perform retrieval based on knowledge text and user-input text to obtain retrieval results.

[0076] In this context, retrieval can be understood as starting from a user's specific information needs, using certain methods and technical means, and based on certain clues and rules, to find information related to those needs from a specific set of information.

[0077] A pre-trained language model 103 is typically a neural network, which usually includes an input layer, an output layer, and one or more hidden layers between the input and output layers. Therefore, a pre-trained language model 103 typically includes an input layer, an output layer, and one or more hidden layers between the input and output layers.

[0078] Based on this, the knowledge text retrieved by the text retrieval module 101 can be input not only into the input layer, but also into the hidden layer or the output layer. If the knowledge text is directly input into the input layer, the hidden layer and the output layer can only use the processing results of the knowledge text by the input layer, but cannot directly use the knowledge text. That is, the knowledge text cannot be used to directly supplement the knowledge of the hidden layer or the output layer, which may affect the effect of knowledge enhancement and thus affect the retrieval results.

[0079] To address this, this application provides a retrieval method that inputs knowledge text into a hidden layer or an output layer, enabling the hidden layer or output layer to directly utilize the knowledge text instead of processing it through other layers of the network. This achieves knowledge supplementation of the hidden layer or output layer directly through the knowledge text, thereby improving the knowledge enhancement effect.

[0080] The retrieval method provided in the embodiments of this application will be described in detail below.

[0081] like Figure 2 As shown, this application provides an embodiment of a retrieval method, which includes:

[0082] Step 201: Obtain the target text for retrieval.

[0083] The target text is derived from the query content entered by the user in the search engine; when the user's query content is in text form, the target text can be the user's query content; when the user's query content is in the form of images, audio, or video, the target text can be converted from the user's query content.

[0084] Step 202: Determine knowledge text based on target text. Knowledge text is used to assist target text in retrieval.

[0085] Knowledge text can be understood as text that is input into a pre-trained language model along with the target text during the retrieval process, so that the pre-trained language model can refer to it when determining the retrieval results. It can also be understood as text that is input into a pre-trained language model along with the target text and is used to supplement the knowledge of the pre-trained language model. For example, the target text is the name of the movie that actor A has starred in, while the knowledge text can be the name of the film and television works that actor A has starred in, as well as the names of the actors he has worked with, etc.

[0086] Among them, knowledge text can be words, sentences, paragraphs or chapters, and a chapter can be composed of multiple paragraphs.

[0087] The role of knowledge text is to assist in retrieval. In other words, knowledge text is used to supplement the knowledge of pre-trained language models to enhance their knowledge content.

[0088] based on Figure 1 As can be seen from the relevant description, knowledge text can be determined from knowledge base 102 through text retrieval module 101. Knowledge base 102 will be referred to as text set below.

[0089] Specifically, texts that are highly relevant to the target text can be selected from the text set as knowledge texts.

[0090] The process of determining knowledge text based on target text will be described in detail below.

[0091] The number of knowledge texts determined based on the target text can be one or more, and this application embodiment does not specifically limit this; in order to improve the effect of knowledge enhancement, multiple knowledge texts are usually determined.

[0092] Step 203: Determine the retrieval results based on the target text, knowledge text, and pre-trained language model.

[0093] The knowledge text is determined based on the target text and is used to assist in the retrieval of the target text. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text.

[0094] For example, the input to the second part of the network is the output of the first part of the network and the knowledge text.

[0095] In this embodiment, since the input of the second part of the network is determined based on the output of the first part of the network and the knowledge text, the knowledge text is input to the second part of the network, and the second part of the network is a non-input layer network.

[0096] Specifically, the second part of the network can be a single-layer network or a multi-layer network; when the second part is a multi-layer network, it can also be called a sub-model; when the second part is a single-layer network, it can be a hidden layer or an output layer.

[0097] Similarly, the first part of the network can be a single-layer network or a multi-layer network; when the first part is a multi-layer network, it can also be called a sub-model; when the first part is a single-layer network, it can be a hidden layer or an input layer.

[0098] For example, when the second part of the network is a hidden layer, the first part of the network can be either an input layer or a hidden layer; when the second part of the network is an output layer, the first part of the network can be either an input layer or a hidden layer.

[0099] It should be noted that the process of determining the input of the second part of the network based on the output of the first part of the network and the knowledge text can be understood as a knowledge fusion process. There are various methods for determining the input of the second part of the network based on the output of the first part of the network and the knowledge text. This application does not limit the specific methods. For example, if the knowledge text is represented by a text vector and the output of the first part of the network is a vector, then a new vector can be calculated based on the text vector of the knowledge text and the vector of the output of the first part of the network, and the new vector can be used as the input of the second part of the network.

[0100] For example, the text vector of the knowledge text can be concatenated with the vector output by the first part of the network to obtain a new vector.

[0101] The text vectors of knowledge text can be generated through vector models, which may include TF-IDF (Term Frequency-inverse Document Frequency) models, word2vec (Word to vector) models, and BERT (Bidirectional Encoder Representations from Transformers) models. TF-IDF is a commonly used weighting technique for information retrieval and data mining, word2vec is a word-to-vector mapping technique, and the BERT model is a pre-trained language model that can be used to obtain text representations, which can then be used for downstream tasks such as text classification, question answering systems, and sentiment analysis.

[0102] In addition, the target text is usually input into the input layer of a pre-trained language model.

[0103] Based on the relevant explanation of step 202, it can be known that there can be multiple knowledge texts. When there are multiple knowledge texts, all knowledge texts can be input into the second part of the network, or some knowledge texts can be selected from multiple knowledge texts and input into the second part of the network.

[0104] As one possible approach, there are multiple knowledge texts; correspondingly, the input of the second part of the network is determined based on the output of the first part of the network and some knowledge texts among the multiple knowledge texts, wherein the partial knowledge texts are the knowledge texts among the multiple knowledge texts whose relevance to the output of the first part of the network meets the first requirement.

[0105] The relevance of the knowledge text to the output of the first part of the network can be calculated by the similarity between the text vector of the knowledge text and the output vector of the first part of the network.

[0106] The first requirement can be a relevance threshold; specifically, multiple knowledge texts with a relevance greater than the threshold to the output of the first part of the network can be selected as the aforementioned partial knowledge texts.

[0107] For example, 100 knowledge texts are identified through step 202, 5 knowledge texts are selected from these 100 knowledge texts based on the output of the first part of the network, and the input of the second part of the network is determined based on the output of the first part of the network for these 5 knowledge texts.

[0108] It should be noted that when there are multiple knowledge texts, multiple knowledge texts or a portion of multiple knowledge texts can be input into the first part of the network.

[0109] In this embodiment, when there are multiple knowledge texts, not all knowledge texts are input into the second part of the network. Instead, based on the output of the first part of the network, knowledge texts whose relevance to the output of the first part of the network meets the first requirement are selected from the multiple knowledge texts. The input of the second part of the network is determined based on the selected knowledge texts and the output of the first part of the network. This achieves targeted extraction of knowledge at the second part of the network, avoiding the input of knowledge texts with low relevance to the second part of the network, which would result in a large amount of computation and fail to achieve the effect of knowledge enhancement.

[0110] In this embodiment, since the input of the second part of the network is determined based on the output of the first part of the network and the knowledge text, the second part of the network can directly utilize the knowledge in the knowledge text, rather than the knowledge processed by other layers such as the input layer. This enables knowledge backtracking and improves the knowledge supplementation effect of the second part of the network, thereby further improving the knowledge enhancement effect and thus improving the accuracy of retrieval.

[0111] As another possible approach, the output of the first part of the network is determined based on the knowledge text; specifically, when the first part of the network is the input layer, the output of the first part of the network is determined based on the knowledge text and the target text.

[0112] In this embodiment, since the output of the first part of the network is determined based on the knowledge text and the target text, and the input of the second part of the network is determined based on the output of the first part of the network and the knowledge text, this embodiment of the application supplements different knowledge to different layers of the network based on the knowledge text, rather than supplementing the knowledge in the knowledge text to one layer of the network all at once. In this way, the multi-layer network of the pre-trained language model can reasonably acquire the required knowledge, so that the pre-trained language model can adaptively acquire the required knowledge at different stages, thereby improving the effect of knowledge supplementation.

[0113] The process of determining knowledge text based on target text is described in detail below.

[0114] As one possible approach, step 202 includes:

[0115] Based on one or more of the target text, target statement, and keywords, multiple candidate texts are selected as knowledge texts from a text set. The text set is a collection of candidate texts, and the target statement and keywords are extracted from the target text.

[0116] Candidate texts can be any text and knowledge texts. Specifically, candidate texts can be any text that may be related to the target text. For example, if the target text is the name of a movie that actor A has starred in, then candidate texts can be any text related to actor A.

[0117] The target statement and keywords are presented in the following ways.

[0118] Specifically, the target text can be decomposed into multiple target sentences using sentence extraction methods in natural language processing. In this embodiment, the target sentences can be short sentences. The sentence extraction methods are not limited to syntactic analysis methods and the TextRank algorithm. Keywords can then be parsed from the target sentences using keyword extraction methods in natural language processing. The keyword extraction methods are not limited to using the Latent Dirichlet Allocation (LDA) model or the Best Match (BM) model for keyword extraction.

[0119] In this embodiment, the target text is divided into more granular segments to obtain target sentences and keywords, which enables the acquisition of more knowledge text based on the target text, target sentences, and keywords, thereby providing more comprehensive knowledge for training the language model.

[0120] To facilitate searching, after extracting the target statement and keywords, a knowledge partitioning tree can be constructed based on the target text, target statement, and keywords.

[0121] Knowledge partitioning trees can be like Figure 3As shown, the knowledge partitioning tree includes nodes that can be divided into 3 layers of nodes with a depth of 2. These 3 layers of nodes are numbered as layer 0, layer 1, and layer 2, respectively. Layer 0, layer 1, and layer 2 correspond to the target text node layer, the target statement node layer, and the keyword node layer, respectively.

[0122] The node corresponding to the keyword can also be called the leaf node, and the node corresponding to the target statement can also be called the root node.

[0123] A text set is a collection of candidate texts; that is, a text set includes a large number of candidate texts.

[0124] There are various ways to represent a text set, and this application does not specifically limit this method.

[0125] For example, the text set is represented by a node graph, and the candidate texts in the text set are represented by nodes in the node graph. Two nodes connected by edges in the node graph correspond to two candidate texts in the text set whose relevance meets the second requirement.

[0126] like Figure 4 As shown, Figure 4 An example of a node graph representing a set of texts is shown, in which each candidate text corresponds to one node, and two candidate texts that meet the second relevance requirement are connected by an edge.

[0127] It is understandable that two candidate texts that meet the second requirement of relevance usually refer to two candidate texts with high relevance, and there can be multiple conditions for judging high relevance.

[0128] For example, the candidate texts in the text set are extracted from web pages. In this case, the two candidate texts in the text set that meet the second requirement of relevance correspond to the two candidate texts that have hyperlinks in the web pages.

[0129] The hyperlink between the two candidate texts can be understood as follows: clicking on one candidate text on one webpage will redirect to the other candidate text on another webpage.

[0130] A candidate text can be hyperlinked to another candidate text or to multiple candidate texts.

[0131] There are several ways to extract candidate text from web pages. For example, you can obtain as many web pages as possible and then break them down into paragraphs, with each paragraph being considered a candidate text.

[0132] In this embodiment, since the candidate texts in the text set are extracted from web pages, the node graph representing the text set can also be called a web page hyperlink graph.

[0133] As can be seen from the foregoing explanation, there are multiple methods for selecting multiple candidate texts from a text set as knowledge texts. The following section introduces one such method based on the node graph described above.

[0134] As one possible approach, step 202 may include:

[0135] The search is performed on the text set based on the target text, target statement and keywords to obtain multiple knowledge texts, including first candidate text, second candidate text, third candidate text and fourth candidate text.

[0136] The first candidate text is obtained from the text set based on the target text, the second candidate text is obtained from the text set based on the target statement, the third candidate text is obtained from the text set based on the keywords, and the fourth candidate text includes candidate text represented by nodes on the target connection path in the node graph. The target connection path includes the connection path between the node representing the first candidate text and the node representing the second candidate text, and / or the connection path between the node representing the second candidate text and the node representing the third candidate text.

[0137] Specifically, the first candidate text can be understood as the candidate text obtained by searching the text set using only the target text and whose relevance to the target text meets the third requirement; the second candidate text can be understood as the candidate text obtained by searching the text set using only the target sentence and whose relevance to the target text meets the fourth requirement; and the third candidate text can be understood as the candidate text obtained by searching the text set using only the keywords and whose relevance to the target text meets the fifth requirement.

[0138] Accordingly, such as Figure 5 As shown, step 202 can specifically include:

[0139] Step 301: Based on the target text, target statement, and keywords, search the text set to obtain the first candidate text, the second candidate text, and the third candidate text. The first candidate text is the candidate text in the text set whose relevance to the target text meets the third requirement. The second candidate text is the candidate text in the text set whose relevance to the target statement meets the fourth requirement. The third candidate text is the candidate text in the text set whose relevance to the keywords meets the fifth requirement.

[0140] Taking the first candidate text as an example, to obtain the first candidate text, it is necessary to first calculate the relevance between each text in the text set and the target text. There are various methods to calculate the relevance. For example, a language model can be used to represent the target text, target sentences and keywords as semantic vectors, and a language model can be used to represent each text page in the text set as a semantic vector. Then, the relevance between each text in the text set and the target text can be calculated based on the similarity between the semantic vectors.

[0141] The method for obtaining the second candidate text is the same as the method for obtaining the third candidate text; please refer to the above explanation for details.

[0142] For example, it can be based on Figure 3 The knowledge partitioning tree shown Figure 4 The node graph shown above performs the retrieval operation; specifically, it can be traversed. Figure 3 Each node in the knowledge partitioning tree shown is in Figure 4 Search the node graph shown to obtain the results. Figure 3 The nodes whose relevance meets the fourth requirement are the candidate texts represented by the nodes, which include the first candidate text, the second candidate text, and the third candidate text.

[0143] It is understandable that after retrieving the nodes of the first, second, and third candidate texts, these nodes can be marked.

[0144] Specifically, it can be used Figure 3 The knowledge partitioning tree shown is labeled with the level number of the nodes, such as... Figure 4 As shown, for nodes utilizing the target text in layer 0 nodes, from Figure 4 Nodes retrieved from the middle are marked with 0; similarly, nodes that utilize the target text in the first-level nodes are... Figure 4 Nodes retrieved from the middle are marked with 1; similarly, nodes that utilize the target text in the second-level nodes are also marked with 1. Figure 4 The nodes retrieved are labeled with a value of 2.

[0145] In addition, this embodiment also uses circular dashed boxes to mark the nodes of the first candidate text, the second candidate text, and the third candidate text, and uses the color of the circular dashed boxes to mark the nodes of the first candidate text, the second candidate text, and the third candidate text; specifically, the circular dashed box corresponding to the first candidate text is the darkest color, and the circular dashed box corresponding to the third candidate text is the lightest color.

[0146] Step 302: Based on the first candidate text, the second candidate text, the third candidate text, and the node graph, determine multiple knowledge texts. The multiple knowledge texts include the first candidate text, the second candidate text, the third candidate text, and the fourth candidate text. The fourth candidate text includes candidate texts corresponding to nodes on the target connection path in the node graph. The target connection path includes the connection path between the node corresponding to the first candidate text and the node corresponding to the second candidate text, and / or the connection path between the node corresponding to the second candidate text and the node corresponding to the third candidate text.

[0147] It should be noted that in the node graph, if the node corresponding to the first candidate text and the node corresponding to the second candidate text are connected by one or more nodes, then it can be considered that there is a connection path between the node corresponding to the first candidate text and the node corresponding to the second candidate text.

[0148] For the nodes corresponding to the first candidate text and the nodes corresponding to the second candidate text, there can be one connection path or multiple connection paths between them.

[0149] The connection path between the node corresponding to the second candidate text and the node corresponding to the third candidate text is similar to the connection path between the node corresponding to the first candidate text and the node corresponding to the second candidate text. For details, please refer to the above description to understand the connection path between the node corresponding to the second candidate text and the node corresponding to the third candidate text.

[0150] Specifically, from Figure 4 After determining the nodes of the first candidate text, the nodes of the second candidate text, and the nodes of the third candidate text, in Figure 4 The process involves finding the connection paths between the nodes corresponding to each first candidate text and the nodes corresponding to each second candidate text, as well as the connection paths between the nodes corresponding to each second candidate text and the nodes corresponding to each third candidate text; then, the nodes on these two connection paths are determined, such as... Figure 6 As shown, Figure 6 The solid nodes, excluding the nodes of the first, second, and third candidate texts (shown in circular dashed boxes), are the nodes on the two connection paths mentioned above. Finally, the candidate texts represented by the nodes on the two connection paths mentioned above are taken as the fourth candidate text.

[0151] In this way, the first candidate text, the second candidate text, the third candidate text, and the fourth candidate text constitute multiple knowledge texts, and the nodes of these multiple knowledge texts are located at... Figure 6 The node graph forms a knowledge domain.

[0152] In this embodiment, not only are the first candidate text, the second candidate text, and the third candidate text regarded as knowledge text, but also the candidate text represented by the nodes on the connection path between the nodes of the first candidate text, the nodes of the second candidate text, and the nodes of the third candidate text are regarded as knowledge text. This effectively increases the number of knowledge texts related to the target text, thereby providing more sufficient knowledge for training the language model.

[0153] It should be noted that there are multiple methods for determining search results, and this application does not specifically limit the methods used in the embodiments.

[0154] As a feasible approach, such as Figure 7As shown, step 203 includes:

[0155] Step 401: Determine the modality to which each knowledge text belongs from multiple modalities. The modality is used to represent the level of abstraction of the knowledge text.

[0156] It is understandable that knowledge texts may be determined based on target texts, target sentences, and keywords, so different knowledge texts have different levels of abstraction; for example, a knowledge text consisting of a phrase is usually more abstract than a knowledge text consisting of a paragraph.

[0157] Therefore, in this embodiment, knowledge texts are modally segmented to determine their level of abstraction; different knowledge texts belonging to the same modality have the same level of abstraction.

[0158] Specifically, multiple modalities can be pre-set. The number of modalities can be set according to actual needs. For example, three, four or even more modalities can be pre-set.

[0159] Then, the modality to which each knowledge text belongs is determined from multiple modalities. It should be noted that there are various methods for determining the modality to which each knowledge text belongs from multiple modalities, and this application embodiment does not specifically limit this method.

[0160] For example, based on the above embodiments, this embodiment can determine the modality to which multiple knowledge texts belong based on the node graph.

[0161] Specifically, as an achievable approach, multiple modalities include a first modality, a second modality, and a third modality.

[0162] The first candidate text and the fourth candidate text whose relevance to the first candidate text meets the sixth requirement belong to the first modality. The third candidate text and the fourth candidate text whose relevance to the third candidate text meets the seventh requirement belong to the second modality. Among the multiple knowledge texts, the knowledge texts that do not belong to the first modality and the knowledge texts that belong to the third modality belong to the third modality.

[0163] It is understandable that, since the first candidate text is determined based on the target text, the first candidate text and the fourth candidate text whose relevance to the first candidate text meets the sixth requirement can be considered as a level of abstraction. Therefore, this embodiment divides the first candidate text and the fourth candidate text whose relevance to the first candidate text meets the sixth requirement into a first modality. Furthermore, since the content of the target text is more specific and detailed, the first modality can be called a concrete modality. Correspondingly, the nodes of the first candidate text and the nodes of the fourth candidate text whose relevance to the first candidate text meets the sixth requirement can also be referred to as... Figure 6 The region formed by the middle belongs to the specific state of the knowledge domain.

[0164] Similarly, since the third candidate text is determined based on keywords, the third candidate text and the fourth candidate text whose relevance to the third candidate text meets the seventh requirement can be regarded as a level of abstraction. Therefore, this embodiment divides the third candidate text and the fourth candidate text whose relevance to the third candidate text meets the seventh requirement into the second modality.

[0165] Understandably, keywords are more abstract than the target text, so the second modality can be called the abstract modality; correspondingly, the nodes of the third candidate text and the nodes of the fourth candidate text whose relevance to the third candidate text meets the sixth requirement can also be called the abstract modality. Figure 6 The region formed by this belongs to the abstract state of the knowledge domain.

[0166] The knowledge texts in the multiple knowledge texts, excluding those belonging to the first and third modalities, are mainly composed of second candidate texts and knowledge texts highly relevant to the second candidate texts. The second candidate texts are determined based on the target statement. Therefore, this embodiment classifies the knowledge texts in the multiple knowledge texts, excluding those belonging to the first and third modalities, into the third modal. Furthermore, since the target statement is more abstract than the target text but more concrete than the keywords, the second modal can be called an intermediate state, which is a modality between the concrete and abstract states. Correspondingly, the nodes of the knowledge texts in the multiple knowledge texts, excluding those belonging to the first and third modalities, can also be referred to as... Figure 6 The region formed in the middle belongs to the intermediate state of the knowledge domain.

[0167] In this embodiment, the relevance between the first candidate text and the fourth candidate text can be determined by... Figure 6 The positional relationship between the nodes of the first candidate text and the nodes of the fourth candidate text in the node graph shown is determined.

[0168] As an achievable approach, the knowledge text belonging to the first modality (i.e., the fourth candidate text whose relevance to the first candidate text meets the sixth requirement) includes: the first candidate text and the fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph through less than a first number of nodes;

[0169] Knowledge text belonging to the second modality (i.e., the fourth candidate text whose relevance to the third candidate text meets the seventh requirement) includes: the third candidate text and the fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph through less than the second number of nodes; knowledge text belonging to the third modality (i.e., the fourth candidate text whose relevance to the third candidate text meets the seventh requirement) includes: other knowledge texts among the multiple knowledge texts besides the knowledge texts belonging to the first modality and the knowledge texts belonging to the third modality.

[0170] It should be noted that the first quantity and the second quantity can be set according to actual needs, and this application embodiment does not impose specific limitations on them.

[0171] Taking the first quantity as an example, when the first quantity is 2, the fourth candidate text whose relevance to the first candidate text meets the sixth requirement refers to the candidate text corresponding to the first-order neighbor node of the node corresponding to the first candidate text; when the first quantity is 3, the fourth candidate text whose relevance to the first candidate text meets the sixth requirement refers to the candidate text corresponding to the second-order neighbor node of the node corresponding to the first candidate text.

[0172] The following is combined Figure 6 and Figure 8 Step 401 provides a further explanation of the above process.

[0173] Specifically, firstly Figure 6 Starting from the first candidate node (marked with 0), determine the first-order fourth candidate nodes adjacent to the first candidate node; the first candidate node and the first-order fourth candidate nodes adjacent to the first candidate node belong to the first mode, and, in Figure 8 The first and fourth candidate nodes belonging to the first mode are marked with 0.

[0174] Then with Figure 6 Starting from the third candidate node (labeled with 2), determine the first-order fourth candidate nodes adjacent to the third candidate node; the third candidate node and the first-order fourth candidate nodes adjacent to the third candidate node belong to the third mode, and, in Figure 8 The third and fourth candidate nodes belonging to the first mode are marked with 2.

[0175] like Figure 8 As shown, the second candidate node and the fourth candidate node, except for the fourth candidate nodes belonging to the first mode and the third mode, are the fourth preferred nodes belonging to the second mode, and are marked with 1.

[0176] It should be noted that the above embodiment determines the nodes belonging to the first mode, the third mode, and the second mode in sequence. In addition, the nodes belonging to the first mode, the third mode, and the second mode can also be determined in other orders. For example, the nodes belonging to the first mode, the second mode, and the third mode can be determined in sequence, or the nodes belonging to the first mode, the third mode, and the second mode can be determined in sequence.

[0177] Step 402: For each knowledge text among multiple knowledge texts, calculate the semantic vector of each knowledge text in its respective modality based on the text vector of each knowledge text and the mapping matrix corresponding to the modality to which each knowledge text belongs. The modality to which each knowledge text belongs is one of multiple modalities, and the modality is used to represent the level of abstraction of the knowledge text.

[0178] Specifically, each modality in the multiple modalities corresponds to a mapping matrix; for each knowledge text, the text vector of the knowledge text is multiplied by the mapping matrix corresponding to the modality to which the knowledge text belongs, to obtain the semantic vector of the knowledge text in the modality to which it belongs.

[0179] For example, such as Figure 9 As shown, multiple modalities include a first modality, a second modality, and a third modality. The text vector of the knowledge text belonging to the first modality is multiplied by the mapping matrix corresponding to the first modality to obtain the semantic vector of the knowledge text belonging to the first modality; the text vector of the knowledge text belonging to the second modality is multiplied by the mapping matrix corresponding to the second modality to obtain the semantic vector of the knowledge text belonging to the second modality; and the text vector of the knowledge text belonging to the third modality is multiplied by the mapping matrix corresponding to the third modality to obtain the semantic vector of the knowledge text belonging to the third modality.

[0180] Step 403: Determine the retrieval results based on the text vector of the target text, the semantic vectors of multiple knowledge texts in their respective modalities, and the pre-trained language model.

[0181] The input to the second part of the network is determined based on the output vector of the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0182] In this embodiment, the modal to which each of the multiple knowledge texts belongs is first determined. Then, the semantic vector of each knowledge text in its respective modality is obtained based on the mapping matrix of each modality. Finally, based on the semantic vector of each knowledge text in its respective modality, since the modality is used to represent the level of abstraction of the knowledge text, this embodiment realizes the division of knowledge texts according to their level of abstraction, effectively increasing the granularity of knowledge sources and providing a foundation for supplementing different knowledge to the pre-trained language model. On this basis, the input of the second part of the network is determined based on the output vector of the first part of the network and the semantic vectors of the multiple knowledge texts in their respective modalities, so that the second part of the network can obtain knowledge at different levels of abstraction, thereby improving the knowledge supplementation effect.

[0183] Furthermore, as can be seen from the foregoing description, the input of the second part of the network is determined based on the output of the first part of the network and some knowledge texts from multiple knowledge texts. The partial knowledge texts are the knowledge texts from multiple knowledge texts whose relevance to the output of the first part of the network meets the first requirement.

[0184] It should be noted that there are multiple methods for calculating the correlation between the knowledge text and the output of the first part of the network; for example, the aforementioned vector model can be used to generate text vectors of the knowledge text, and the correlation between the knowledge text and the output vector of the first part of the network can be calculated based on the text vectors of the knowledge text and the output vector of the first part of the network.

[0185] In addition, based on the semantic vector of the knowledge text in its respective modality obtained in the above embodiments, the correlation between multiple knowledge texts and the output of the first part of the network can also be calculated based on the vector of the output of the first part of the network and the semantic vector of multiple knowledge texts in their respective modalities.

[0186] Based on the semantic vector of the knowledge text in its modality obtained in the above embodiments, the following combines... Figure 10 The process of supplementing knowledge in a pre-trained language model is explained.

[0187] like Figure 10 As shown, in step 402, semantic vectors of multiple knowledge texts in their respective modalities are obtained. Based on the semantic vectors of multiple knowledge texts in their respective modalities, a first knowledge extraction is performed, and the extracted knowledge is fused with the output of the previous layer network to obtain the input of the first part of the network, thus completing the knowledge supplementation of the first part of the network. Similarly, based on the semantic vectors of multiple knowledge texts in their respective modalities, a second knowledge extraction is performed, and the extracted knowledge is fused with the output of the first part of the network to obtain the input of the second part of the network, thus completing the knowledge supplementation of the second part of the network.

[0188] In summary, the retrieval process using the retrieval method provided in the embodiments of this application is as follows.

[0189] like Figure 11 As shown, the user inputs the target text into the question-answering system; the system constructs a knowledge partitioning tree based on the target text, then builds a webpage hyperlink graph based on the knowledge partitioning tree, and determines the knowledge domain from the hyperlink graph; subsequently, the system partitions the knowledge domain into modalities and updates the representation of the knowledge domain in different modalities (i.e., updates the semantic vector of the knowledge text represented by each node in the knowledge domain in the corresponding modality); based on this, for the pre-trained language model, knowledge backtracking and knowledge fusion are performed sequentially based on the representation of the knowledge domain in different modalities, thereby supplementing the knowledge of the pre-trained language model, and finally the pre-trained language model outputs the answer.

[0190] Please see Figure 12 This application also provides a retrieval device, including:

[0191] The acquisition unit 501 is used to acquire the target text for retrieval; and to determine the retrieval results based on the target text, knowledge text, and pre-trained language model. The knowledge text is determined based on the target text and is used to assist the target text in retrieval. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text.

[0192] As one possible approach, there are multiple knowledge texts; the input of the second part of the network is determined based on the output of the first part of the network and some knowledge texts from the multiple knowledge texts. The partial knowledge texts are the knowledge texts from the multiple knowledge texts whose relevance to the output of the first part of the network meets the first requirement.

[0193] As an achievable approach, a unit is defined for selecting multiple candidate texts as knowledge texts from a text set based on one or more of the target text, target statement, and keywords. The text set is a collection of candidate texts, and the target statement and keywords are extracted from the target text.

[0194] As one possible approach, the text set is represented by a node graph, where candidate texts in the text set are represented by nodes in the node graph, and two nodes connected by edges in the node graph correspond to two candidate texts in the text set whose relevance meets the second requirement.

[0195] As one possible approach, candidate texts in the text set are extracted from web pages; two candidate texts in the text set that meet the second relevance requirement correspond to two candidate texts that have hyperlinks in the web pages.

[0196] As one feasible approach, a unit is defined for retrieving multiple knowledge texts from a text set based on target text, target statement, and keywords. These multiple knowledge texts include a first candidate text, a second candidate text, a third candidate text, and a fourth candidate text. The first candidate text is obtained from the text set based on the target text, the second candidate text is obtained from the text set based on the target statement, the third candidate text is obtained from the text set based on keywords, and the fourth candidate text includes candidate texts represented by nodes on target connection paths in a node graph. The target connection paths include connection paths between nodes representing the first candidate text and nodes representing the second candidate text, and / or connection paths between nodes representing the second candidate text and nodes representing the third candidate text.

[0197] As an achievable method, retrieval unit 502 is used to calculate the semantic vector of each knowledge text in its respective modality for each knowledge text among multiple knowledge texts, based on the text vector of each knowledge text and the mapping matrix corresponding to the modality to which each knowledge text belongs. The modality to which each knowledge text belongs is one of multiple modalities, and the modality is used to represent the level of abstraction of the knowledge text. The retrieval result is determined based on the text vector of the target text, the semantic vectors of multiple knowledge texts in their respective modalities, and the pre-trained language model. The input of the second part of the network is determined based on the vector output by the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0198] As one possible approach, the multiple modalities include a first modality, a second modality, and a third modality; the knowledge text belonging to the first modality includes: a first candidate text and a fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph through a number less than a first number of nodes; the knowledge text belonging to the second modality includes: a third candidate text and a fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph through a number less than a second number of nodes; the knowledge text belonging to the third modality includes: other knowledge texts among the multiple knowledge texts besides the knowledge texts belonging to the first modality and the knowledge texts belonging to the third modality.

[0199] As one feasible approach, the relevance of multiple knowledge texts to the output of the first part of the network is calculated based on the vector of the output of the first part of the network and the semantic vectors of multiple knowledge texts in their respective modalities.

[0200] As a feasible approach, knowledge texts can be words, sentences, paragraphs, or chapters.

[0201] For details on the implementation, related explanations, and technical effects of each of the above units, please refer to the description of the retrieval method in the foregoing embodiments.

[0202] This application also provides an embodiment of a computer device, which may be a server. Please refer to [link to relevant documentation]. Figure 13 , Figure 13 This is a schematic diagram of a computer device provided in an embodiment of this application, used to implement... Figure 12 In accordance with the function of the retrieval device in the corresponding embodiment, specifically, the computer device 1800 is implemented by one or more servers. The computer device 1800 can vary significantly due to different configurations or performance, and may include one or more central processing units (CPUs) 1822 (e.g., one or more processors) and memory 1832, and one or more storage media 1830 (e.g., one or more mass storage devices) for storing application programs 1842 or data 1844. The memory 1832 and storage media 1830 can be temporary or persistent storage. The program stored in the storage media 1830 may include one or more modules (not shown in the figure), each module may include a series of instruction operations on the computer device. Furthermore, the CPU 1822 may be configured to communicate with the storage media 1830 and execute the series of instruction operations in the storage media 1830 on the computer device 1800.

[0203] The computer device 1800 may also include one or more power supplies 1826, one or more wired or wireless network interfaces 1850, one or more input / output interfaces 1858, and / or one or more operating systems 1841, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0204] In this embodiment, the central processing unit 1822 is used to execute... Figure 12 The retrieval method executed by the retrieval device in the corresponding embodiment. Specifically, the central processing unit 1822 can be used for:

[0205] Obtain the target text used for retrieval;

[0206] The retrieval results are determined based on the target text, knowledge text, and a pre-trained language model. The knowledge text is determined based on the target text and is used to assist in the retrieval of the target text. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text.

[0207] This application also provides a chip including one or more processors. Part or all of the processors are used to read and execute computer programs stored in a memory to perform the methods of the foregoing embodiments.

[0208] Optionally, the chip includes a memory, which is connected to the processor via a circuit or wire. Further optionally, the chip also includes a communication interface, to which the processor is connected. The communication interface is used to receive data and / or information that needs to be processed. The processor obtains the data and / or information from the communication interface, processes the data and / or information, and outputs the processing result through the communication interface. The communication interface can be an input / output interface.

[0209] In some implementations, some of the processors may implement some steps of the above method through dedicated hardware. For example, the processing of neural network models may be implemented by a dedicated neural network processor or graphics processor.

[0210] The method provided in this application embodiment can be implemented by a single chip or by multiple chips working together.

[0211] This application also provides a computer storage medium for storing computer software instructions for use with the aforementioned computer device, including instructions for executing programs designed for the computer device.

[0212] The computer device can be as described above. Figure 12 The retrieval device described.

[0213] This application also provides a computer program product, which includes computer software instructions that can be loaded by a processor to implement the processes in the methods shown in the foregoing embodiments.

[0214] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0215] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0216] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0217] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0218] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A retrieval method, characterized in that, include: Obtain the target text used for retrieval; The retrieval results are determined based on the target text, knowledge text, and pre-trained language model. The knowledge text is determined based on the target text and is used to assist the target text in retrieval. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text. Wherein, the second part of the network is a hidden layer of the neural network, and the first part of the network is an input layer or a hidden layer of the neural network; or, the second part of the network is an output layer of the neural network, and the first part of the network is an input layer or a hidden layer of the neural network.

2. The retrieval method according to claim 1, characterized in that, The number of knowledge texts is multiple; The input to the second part of the network is determined based on the output of the first part of the network and some knowledge texts from multiple knowledge texts. The partial knowledge texts are those knowledge texts from the multiple knowledge texts whose relevance to the output of the first part of the network meets the first requirement.

3. The retrieval method according to claim 2, characterized in that, The determination of knowledge text based on the target text includes: Based on one or more of the target text, target statement, and keywords, multiple candidate texts are selected as knowledge texts from a text set, where the text set is a collection of candidate texts, and the target statement and keywords are extracted from the target text.

4. The retrieval method according to claim 3, characterized in that, The text set is represented by a node graph. The candidate texts in the text set are represented by nodes in the node graph. Two nodes connected by an edge in the node graph correspond to two candidate texts in the text set whose relevance meets the second requirement.

5. The retrieval method according to claim 4, characterized in that, The candidate texts in the text set are extracted from web pages; The two candidate texts in the text set that meet the second requirement in terms of relevance correspond to the two candidate texts in the webpage that have hyperlinks.

6. The retrieval method according to claim 4 or 5, characterized in that, The step of selecting multiple candidate texts as knowledge texts from a text set based on one or more of the target text, target statement, and keywords includes: Based on the target text, the target statement, and the keywords, a search is conducted in the text set to obtain multiple knowledge texts, including a first candidate text, a second candidate text, a third candidate text, and a fourth candidate text. The first candidate text is obtained from the text set based on the target text, the second candidate text is obtained from the text set based on the target statement, the third candidate text is obtained from the text set based on the keywords, and the fourth candidate text includes candidate texts represented by nodes on the target connection path in the node graph. The target connection path includes the connection path between the node representing the first candidate text and the node representing the second candidate text, and / or the connection path between the node representing the second candidate text and the node representing the third candidate text.

7. The retrieval method according to claim 6, characterized in that, The determination of retrieval results based on the target text, the knowledge text, and the pre-trained language model includes: For each of the plurality of knowledge texts, based on the text vector of each knowledge text and the mapping matrix corresponding to the modality to which each knowledge text belongs, the semantic vector of each knowledge text in the modality to which it belongs is calculated. The modality to which each knowledge text belongs is one of a plurality of modalities, and the modality is used to represent the level of abstraction of the knowledge text. The retrieval results are determined based on the text vector of the target text, the semantic vectors of the multiple knowledge texts in their respective modalities, and the pre-trained language model. The input of the second part of the network is determined based on the vector output by the first part of the network and the semantic vectors of the multiple knowledge texts in their respective modalities.

8. The retrieval method according to claim 7, characterized in that, The multiple modes include a first mode, a second mode, and a third mode; The knowledge text belonging to the first modality includes: the first candidate text and the fourth candidate text represented by nodes connected to the node representing the first candidate text in the node graph by less than a first number of nodes; The knowledge text belonging to the second modality includes: the third candidate text and the fourth candidate text represented by nodes connected to the node representing the third candidate text in the node graph by less than a second number of nodes; The knowledge text belonging to the third modality includes: other knowledge texts among the plurality of knowledge texts besides the knowledge texts belonging to the first modality and the knowledge texts belonging to the third modality.

9. The retrieval method according to claim 7 or 8, characterized in that, The relevance of the multiple knowledge texts to the output of the first part of the network is calculated based on the vector of the output of the first part of the network and the semantic vector of the multiple knowledge texts in their respective modalities.

10. The retrieval method according to any one of claims 1 to 5, characterized in that, The knowledge text refers to words, sentences, paragraphs, or chapters.

11. A retrieval device, characterized in that, include: The acquisition unit is used to acquire the target text for retrieval; A retrieval unit is used to determine retrieval results based on the target text, knowledge text, and a pre-trained language model. The knowledge text is determined based on the target text and is used to assist the target text in retrieval. The pre-trained language model includes a first part network and a second part network. The input of the second part network is determined based on the output of the first part network and the knowledge text. Wherein, the second part of the network is a hidden layer of the neural network, and the first part of the network is an input layer or a hidden layer of the neural network; or, the second part of the network is an output layer of the neural network, and the first part of the network is an input layer or a hidden layer of the neural network.

12. A computer device, characterized in that, The feature is that it includes: one or more processors and a memory; wherein the memory stores computer-readable instructions; The one or more processors read the computer-readable instructions to cause the training device to implement the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, It includes computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 10.

Images