Method, device and equipment for training correlation identification model, correlation identification method

Abstract

The application discloses a kind of correlation identification model training method, correlation identification method and equipment, belong to computer technical field.Method includes: control neural network model determines the predicted correlation of two sample texts in sample text pair;Sample at least one from first text and second text corresponding to any sample text pair in knowledge graph, the first text corresponding to any sample text is relevant to any sample text, and the second text corresponding to any sample text pair is irrelevant to any sample text;Based on at least one sample text corresponding first text and second text at least one, and the correlation of labeling and predicted correlation, train neural network model to obtain correlation identification model.Correlation identification model can not only identify the correlation between two sample texts, but also learn the correlation between sample text and first text and / or the irrelevant relationship between sample text and second text, improve the generalization ability.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer technology, and in particular, to a correlation identification model training method, a correlation identification method, and a device. BACKGROUND

[0002] In the field of computer technology, a variety of tasks involve correlation identification. For example, in a search task, correlation identification is mainly used to identify search results related to search terms from candidate results, and in a question and answer task, correlation identification is mainly used to identify answers related to questions from candidate answers. Generally, a correlation identification model can be trained to use the correlation identification model to perform correlation identification.

[0003] In related technologies, a sample text pair can be obtained, and the sample text pair includes two sample texts. On the one hand, by labeling the sample text pair, the labeled correlation between the two sample texts in the sample text pair is obtained. On the other hand, the sample text pair is input into a neural network model, and the neural network model determines the predicted correlation between the two sample texts in the sample text pair. The neural network model is trained using the predicted correlation and the labeled correlation to obtain a correlation identification model.

[0004] The correlation identification model trained by the above technology has poor generalization ability, resulting in low accuracy of the correlation identification model and affecting the accuracy of the correlation identification result. SUMMARY

[0005] The present application provides a correlation identification model training method, a correlation identification method, and a device, which can be used to solve the problems in related technologies. The technical solution includes the following contents.

[0006] On the one hand, a correlation identification model training method is provided, and the method includes:

[0007] obtaining a sample text pair and a labeled correlation between two sample texts in the sample text pair;

[0008] controlling a neural network model to determine a predicted correlation between the two sample texts in the sample text pair;

[0009] obtaining a knowledge graph, the knowledge graph including a plurality of nodes, and the nodes representing base texts;

[0010] For any one of the sample text pairs, at least one of the first text and the second text corresponding to the any one of the sample text pairs is sampled from the knowledge graph, the first text corresponding to the any one of the sample text pairs is a basic text related to the any one of the sample texts, and the second text corresponding to the any one of the sample text pairs is a basic text unrelated to the any one of the sample texts;

[0011] Based on at least one of the first text and the second text corresponding to at least one of the sample text pairs, and the labeled relevance and the predicted relevance, the neural network model is trained to obtain a relevance identification model.

[0012] In another aspect, a relevance identification method is provided, the method comprising:

[0013] Obtaining a target text pair and a relevance identification model, the relevance identification model being trained according to the training method of the relevance identification model described above;

[0014] Inputting the target text pair into the relevance identification model to determine text features of two target texts in the target text pair by the relevance identification model;

[0015] Fusing the text features of the two target texts in the target text pair to obtain a target fusion feature;

[0016] Determining the relevance of the two target texts in the target text pair based on the target fusion feature.

[0017] In another aspect, a training device of a relevance identification model is provided, the device comprising:

[0018] An obtaining module for obtaining a sample text pair and a labeled relevance of two sample texts in the sample text pair;

[0019] A determining module for controlling a neural network model to determine a predicted relevance of the two sample texts in the sample text pair;

[0020] The obtaining module is further configured to obtain a knowledge graph, the knowledge graph comprising a plurality of nodes, the nodes representing basic texts;

[0021] A sampling module for, for any one of the sample text pairs, sampling at least one of the first text and the second text corresponding to the any one of the sample text pairs from the knowledge graph, the first text corresponding to the any one of the sample text pairs being a basic text related to the any one of the sample texts, and the second text corresponding to the any one of the sample text pairs being a basic text unrelated to the any one of the sample texts;

[0022] The training module is configured to train the neural network model based on at least one of the first text and the second text corresponding to each sample text pair in the sample text pair, the annotated relevance, and the predicted relevance, to obtain a relevance identification model.

[0023] In another aspect, a relevance identification device is provided, and the device comprises:

[0024] The obtaining module is configured to obtain a target text pair and a relevance identification model, the relevance identification model being trained according to the training method of the relevance identification model described above;

[0025] The determining module is configured to input the target text pair into the relevance identification model, and determine text features of two target texts in the target text pair based on the relevance identification model.

[0026] The fusion module is configured to fuse the text features of the two target texts in the target text pair to obtain a target fusion feature.

[0027] The determining module is further configured to determine the relevance of the two target texts in the target text pair based on the target fusion feature.

[0028] In another aspect, an electronic device is provided, and the electronic device comprises a processor and a memory, the memory storing at least one computer program, the at least one computer program being loaded and executed by the processor to enable the electronic device to implement the training method of the relevance identification model or the relevance identification method described above.

[0029] In another aspect, a computer readable storage medium is also provided, and the computer readable storage medium stores at least one computer program, the at least one computer program being loaded and executed by a processor to enable an electronic device to implement the training method of the relevance identification model or the relevance identification method described above.

[0030] In another aspect, a computer program or computer program product is also provided, and the computer program or computer program product stores at least one computer program, the at least one computer program being loaded and executed by a processor to enable an electronic device to implement the training method of the relevance identification model or the relevance identification method described above.

[0031] The technical solutions provided in the present application bring at least the following beneficial effects:

[0032] The technical scheme provided in the application samples at least one of the first text and the second text corresponding to any one of the sample text pairs from the knowledge graph, and the first text is related to the sample text, and the second text is not related to the sample text. Based on the at least one of the first text and the second text, the prediction relevance and the annotation relevance of the two sample texts in the sample text pair are combined to train the relevance identification model, so that the relevance identification model can not only identify the relevance between the two sample texts, but also learn the relevant relationship between the sample text and the first text and / or the irrelevant relationship between the sample text and the second text, improve the robustness and generalization ability of the relevance identification model, and thus improve the accuracy of the relevance identification model. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0034] Figure 1 is an implementation environment schematic diagram of a relevance identification model training method or a relevance identification method provided by the embodiments of the present application;

[0035] Figure 2 is a flowchart of a relevance identification model training method provided by the embodiments of the present application;

[0036] Figure 3 is a schematic diagram of a text relationship provided by the embodiments of the present application;

[0037] Figure 4 is a schematic diagram of model training provided by the embodiments of the present application;

[0038] Figure 5 is a flowchart of a relevance identification method provided by the embodiments of the present application;

[0039] Figure 6 is a structural schematic diagram of a relevance identification model training device provided by the embodiments of the present application;

[0040] Figure 7 is a structural schematic diagram of a relevance identification device provided by the embodiments of the present application;

[0041] Figure 8 is a structural schematic diagram of a terminal device provided by the embodiments of the present application;

[0042] Figure 9 is a structural schematic diagram of a server provided by the embodiments of the present application. DETAILED DESCRIPTION

[0043] In order to make the purposes, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0044] Figure 1 is an implementation environment diagram of a relatedness recognition model training method or a relatedness recognition method provided by an embodiment of the present application, as shown in the figure, the implementation environment includes a terminal device 101 and a server 102. Wherein, the relatedness recognition model training method or the relatedness recognition method in the embodiment of the present application can be executed by the terminal device 101, or can be executed by the server 102, or can be executed by the terminal device 101 and the server 102 together. Figure 1

[0045] The terminal device 101 can be a smart phone, a game console, a desktop computer, a tablet computer, a laptop computer, a smart television, a smart vehicle device, a smart voice interaction device, a smart home appliance, etc. The server 102 can be a server, or a server cluster composed of multiple servers, or any one of a cloud computing platform and a virtualization center, which is not limited in the embodiment of the present application. The server 102 can be connected with the terminal device 101 through a wired network or a wireless network. The server 102 can have functions of data processing, data storage and data transceiving, which is not limited in the embodiment of the present application. The number of the terminal device 101 and the server 102 is not limited, which can be one or more.

[0046] In the field of computer technology, relatedness recognition is a common technology, which is usually performed by using a relatedness recognition model. Generally, a relatedness recognition model can be obtained by using sample text, and the relatedness recognition model can be used to perform relatedness recognition on a target text pair.

[0047] In the related technology, on the one hand, the labeled relatedness between two sample texts in a sample text pair is obtained by manual labeling. On the other hand, the predicted relatedness between two sample texts in a sample text pair is determined by using a neural network model. The neural network model is trained by using the predicted relatedness and the labeled relatedness to obtain a relatedness recognition model, but the generalization ability of the relatedness recognition model is poor and the accuracy is low.

[0048] In order to improve the generalization ability of the relatedness recognition model, an embodiment of the present application provides a relatedness recognition model training method, which can be applied in the above implementation environment. Taking a flowchart of a relatedness recognition model training method provided by an embodiment of the present application as shown in the figure, the method can be executed by Figure 2 Figure 1 ​​The terminal device 101 or the server 102 in the method can perform the method, or the terminal device 101 and the server 102 jointly perform the method. For ease of description, the terminal device 101 or the server 102 that performs the training method of the correlation identification model in the embodiment of the present application is referred to as an electronic device, and the method can be performed by the electronic device. As shown in FIG. 10, the method includes the following steps 201 to 205. Figure 2

[0049] In step 201, a sample text pair and a label correlation of two sample texts in the sample text pair are obtained.

[0050] The number of the sample text pair in the embodiment of the present application is at least one, and the content and processing manner of each sample text pair are the same. For ease of description, the embodiment of the present application is described from the perspective of one sample text pair below.

[0051] The sample text pair includes two sample texts. For different technical fields, the text types of the two sample texts are also different. In a general way, the two sample texts are texts whose correlation needs to be determined.

[0052] For example, in the question and answer technical field, it is often necessary to determine the correlation between a question text and an answer text. When the correlation between the question text and the answer text is high, the answer text can be used as a reply text of the question text; when the correlation between the question text and the answer text is low, the answer text cannot be used as the reply text of the question text. At this time, the two sample texts in the sample text pair are the question text and the answer text respectively. That is, the text type of one sample text in the sample text pair is the question text, and the text type of the other sample text is the answer text.

[0053] For another example, in the search technical field, it is often necessary to determine the correlation between a search word and a candidate result (i.e., a candidate text). When the correlation between the search word and the candidate result is high, the candidate result can be used as a search result corresponding to the search word; when the correlation between the search word and the candidate result is low, the candidate result cannot be used as the search result corresponding to the search word. At this time, the two sample texts in the sample text pair are the search word and the candidate result respectively. That is, the text type of one sample text in the sample text pair is the search word, and the text type of the other sample text is the candidate result.

[0054] ​For example, in the field of knowledge graph, it is often necessary to determine the relevance between two item names. When the relevance between two item names is high, an edge is built between the two item names to represent the relationship between the two item names by using the edge; when the relevance between two item names is low, it indicates that the probability of no relationship between the two item names is high, and therefore, an edge cannot be built between the two item names. Taking two item names "sunlight rose grape" and "rose" as an example, since "sunlight rose" is a variety of "grape", and "grape" and "rose" are two varieties, it can be determined that the relevance between the two item names is low, and an edge cannot be built between the two item names. At this time, the two sample texts in the sample text pair are both item names. That is, the text types of the two sample texts in the sample text pair are both item names.

[0055] In the embodiment of the present application, the sample text pair can be obtained in any way. For example, the sample text pair can be constructed manually, for example, the question text and the answer text are constructed manually, and the question text and the answer text are taken as the sample text pair. Alternatively, a large amount of text can be grabbed from the network, and the sample text pair can be constructed by screening the text, for example, a large amount of entity names are grabbed from the network, which include not only item names such as apples and clothes, but also non-item names such as threads and performances. The item names are screened from the entity names, and each two item names are taken as a sample text pair.

[0056] The relevance of the two sample texts in the sample text pair can be determined manually, and the two sample texts are labeled based on the determination result to obtain the labeled relevance of the two sample texts. That is, the labeled relevance of the two sample texts is used to represent whether the two sample texts are relevant. If the two sample texts are relevant, the labeled relevance of the two sample texts is the first labeled data. If the two sample texts are not relevant, the labeled relevance of the two sample texts is the second labeled data, and the first labeled data and the second labeled data are two different data. The present application does not limit the value of the first labeled data and the second labeled data, and the first labeled data is 1 and the second labeled data is 0, for example.

[0057] In step 202, the control neural network model determines the predicted relevance of the two sample texts in the sample text pair.

[0058] In the embodiments of the present application, two sample texts in a sample text pair are input into a neural network model, and the neural network model is controlled to process the two sample texts, so that the predicted relevance of the two sample texts is obtained and output. The predicted relevance of the two sample texts is used to represent the possibility of the relevance of the two sample texts. The greater the predicted relevance of the two sample texts, the greater the possibility of the relevance of the two sample texts, that is, the predicted relevance of the two sample texts is directly proportional to the possibility of the relevance of the two sample texts. Optionally, the predicted relevance of the two sample texts is data greater than or equal to 0.5 and less than or equal to 1.

[0059] The neural network model in the embodiments of the present application can include, but is not limited to, encoding layers, feature fusion layers, classification layers and the like. Optionally, the encoding layers, the feature fusion layers and the classification layers are sequentially connected in series. In addition, the size and structure of each network layer are not limited in the embodiments of the present application. For example, the encoding layer is any one of a pre-training model, a BILSTM (a kind of long short-term memory network), a convolutional neural network (CNN) and the like, the feature fusion layer is any one of a Transformer (a kind of deep learning model based on self-attention mechanism), a multilayer perceptron (MLP), a CNN and the like, and the classification layer is an MLP and the like.

[0060] In a possible implementation, step 202 includes steps 2021 to 2023 as shown below.

[0061] In step 2021, a sample text pair is input into a neural network model, and the neural network model determines the text features of two sample texts in the sample text pair.

[0062] After the two sample texts in the sample text pair are input into the neural network model, for any sample text, the encoding layer encodes the sample text to obtain the text features of the sample text.

[0063] Optionally, the sample text includes at least one character. The encoding layer includes a mapping relationship between the characters and word vectors, based on which each character included in the sample text can be converted into a corresponding word vector. A vector obtained by splicing the word vectors is used as the text features of the sample text. Alternatively, for any character, the encoding layer can also convert the position of the character in the sample text into a position vector, and splice the word vector and the position vector of the character to obtain a spliced vector of the character. A vector obtained by splicing the spliced vectors of the characters is used as the text features of the sample text.

[0064] In a possible implementation, the neural network model comprises one encoding layer, which can determine the text features of the two sample texts in the sample text pair according to the manner of determining the text features of the sample texts described above. Alternatively, the neural network model comprises two encoding layers, and the parameters of the two encoding layers are shared, that is, the parameters of the two encoding layers are the same. One of the two encoding layers is used to determine the text feature of one sample text in the sample text pair, and the other encoding layer is used to determine the text feature of the other sample text in the sample text pair. Each encoding layer can determine the text feature of the corresponding sample text according to the manner of determining the text features of the sample texts described above.

[0065] In step 2022, the text features of the two sample texts in the sample text pair are fused to obtain sample fusion features.

[0066] After the encoding layer determines the text features of the two sample texts in the sample text pair, the feature fusion layer fuses the text features of the two sample texts to obtain sample fusion features. Since the structure and size of the feature fusion layer are not limited in the embodiments of the present application, the manner of fusing the text features of the two sample texts is also not limited in the embodiments of the present application.

[0067] By fusing the text features of the two sample texts in the sample text pair, the relationship between the text features of the two sample texts is mined, so that the sample fusion features can reflect the correlation between the two sample texts, thereby providing a basis for determining the prediction correlation of the two sample texts subsequently.

[0068] In step 2023, the prediction correlation of the two sample texts in the sample text pair is determined based on the sample fusion features.

[0069] In the embodiments of the present application, the sample fusion features are input into the classification layer. Since the sample fusion features can reflect the correlation between the two sample texts, the classification layer can map the sample fusion features into the prediction correlation of the two sample texts in the sample text pair.

[0070] In step 203, a knowledge graph is obtained, and the knowledge graph comprises a plurality of nodes, and each node represents a basic text.

[0071] The knowledge graph can be constructed and obtained by an artificial construction manner, or can be obtained by crawling from a network. The number of knowledge graphs is at least one. When the number of knowledge graphs is at least two, the at least two knowledge graphs can be merged into one knowledge graph, or the merging can not be performed.

[0072] The knowledge graph includes multiple nodes, and there can be an edge between any two nodes or no edge. When there is an edge between any two nodes, it indicates that there is a certain relationship between the two nodes. Therefore, the edge in the knowledge graph is used to represent the relationship between the two nodes. Any node corresponds to a basic text. The embodiments of the present application do not limit the basic text. Exemplarily, the basic text is an object name.

[0073] In step 204, for any sample text in the sample text pair, at least one of the first text and the second text corresponding to any sample text is sampled from the knowledge graph.

[0074] Among them, the first text corresponding to any sample text is the basic text related to any sample text, and the second text corresponding to any sample text is the basic text unrelated to any sample text.

[0075] For any sample text in the sample text pair, the basic texts corresponding to the nodes in the knowledge graph can be divided into two categories: one category is the basic text related to the sample text, and the other category is the basic text unrelated to the sample text. At least one basic text can be sampled from the basic text related to the sample text to obtain the first text corresponding to the sample text, and at least one basic text can be sampled from the basic text unrelated to the sample text to obtain the second text corresponding to the sample text.

[0076] It should be noted that for another sample text in the sample text pair other than any sample text, the above-mentioned manner can also be used to sample the first text and the second text corresponding to the other sample text from the knowledge graph, and the implementation principles are similar, which will not be described here.

[0077] For example, the sample text pair includes two sample texts, "tomato" and "fresh tomato 500g". For "tomato", there is a knowledge graph, and the basic texts corresponding to the nodes in the knowledge graph include "tomato", "tomato", "tomato sauce", "cucumber", and "potato". Among them, "tomato" and "tomato" are basic texts related to "tomato", and "tomato sauce", "cucumber", and "potato" are basic texts unrelated to "tomato". Then, the first text corresponding to "tomato" can be sampled from the knowledge graph: "tomato", and the second text corresponding to "tomato" can be sampled from the knowledge graph: "tomato sauce", "cucumber", and "potato".

[0078] For "500g fresh tomatoes", there is a knowledge graph, and the basic texts corresponding to the nodes in the knowledge graph include "1kg sandy core tomatoes", "half jin special grade large tomatoes", "rich tomato sauce", "500g fresh cucumbers", and "1kg potatoes". Among them, "1kg sandy core tomatoes" and "half jin special grade large tomatoes" are basic texts related to "500g fresh tomatoes", and "rich tomato sauce", "500g fresh cucumbers", and "1kg potatoes" are basic texts unrelated to "500g fresh tomatoes". Therefore, the first text corresponding to "500g fresh tomatoes" can be sampled from the knowledge graph as "1kg sandy core tomatoes" and "half jin special grade large tomatoes", and the second text corresponding to "500g fresh tomatoes" can be sampled from the knowledge graph as "rich tomato sauce", "500g fresh cucumbers", and "1kg potatoes".

[0079] As mentioned above, the edges in the knowledge graph are used to represent the relationship between the two nodes at both ends of the edge. Based on the relationship between the two nodes at both ends of the edge represented by the edge, the first text and the second text corresponding to the sample text can be sampled from the knowledge graph. In one possible implementation, one possible implementation of step 204 at least includes the following cases A1 and A2.

[0080] In case A1, the knowledge graph further includes a plurality of edges, and the edges represent the semantic relationship between the basic texts corresponding to the two nodes at both ends of the edge. Sampling the first text corresponding to any sample text from the knowledge graph includes: determining a target node in the knowledge graph, and the basic text corresponding to the target node is any sample text; determining at least one first edge in the knowledge graph, and one end node of the first edge is the target node, and the semantics of any sample text contains or is equivalent to the semantics of the basic text corresponding to the other end node of the first edge; sampling the first text corresponding to any sample text from the basic text corresponding to the other end node of each first edge.

[0081] In the embodiments of the present application, the edges between any two nodes in the knowledge graph are used to represent the semantic relationship between the basic texts corresponding to the nodes. Among them, the semantic relationship between two basic texts includes a semantic equivalence relationship, that is, two basic texts belong to synonyms, for example, the semantic relationship between "tomatoes" and "tomatoes" is a semantic equivalence relationship.

[0082] The semantic relationship between two basic texts also includes a semantic inclusion relationship, that is, the semantics of one basic text contains the semantics of another basic text, that is, the two basic texts are hypernyms and hyponyms, respectively, and the semantics of the hypernym contains the semantics of the hyponym. For example, "fruit" is the hypernym of "tomatoes", and the semantics of "fruit" contains the semantics of "tomatoes".

[0083] Any node in the knowledge graph corresponds to a basic text. If a basic text is the same as any sample text in a sample text pair, the node corresponding to the basic text is determined as a target node. At this time, the edges of all target nodes in the knowledge graph can be determined, and the edge of the target node refers to an edge with one end node being the target node.

[0084] Since the edge represents the semantic relationship between the basic texts corresponding to the two end nodes of the edge, the edge of the target node can represent the semantic relationship between the basic text corresponding to the target node (i.e., one end node of the edge) and the basic text corresponding to the other end node of the edge, and the semantic relationship is one of the following three relationships.

[0085] Relationship A1: The semantic of the basic text corresponding to the target node is the same as the semantic of the basic text corresponding to the other end node of the edge, that is, the basic text corresponding to the target node is a synonym of the basic text corresponding to the other end node of the edge.

[0086] Relationship A2: The semantic of the basic text corresponding to the target node contains the semantic of the basic text corresponding to the other end node of the edge, that is, the basic text corresponding to the target node is a hypernym of the basic text corresponding to the other end node of the edge.

[0087] Relationship A3: The semantic of the basic text corresponding to the other end node of the edge contains the semantic of the basic text corresponding to the target node, that is, the basic text corresponding to the target node is a hyponym of the basic text corresponding to the other end node of the edge.

[0088] For any edge of a target node, if the semantic relationship represented by the edge is relationship A1 or relationship A2, the edge is taken as a first edge. In this way, all first edges in the knowledge graph can be determined. Then, at least one basic text is randomly sampled from the basic texts corresponding to the other end nodes of all first edges, to obtain a first text corresponding to any sample text. That is, the basic text corresponding to the first edge one end node (i.e., the target node) is any sample text, and the basic text corresponding to the first edge other end node can be a first text corresponding to any sample text.

[0089] Through case A1, a basic text related to any sample text in a sample text pair can be sampled from the knowledge graph to obtain a first text corresponding to the sample text, and the sample text is a synonym or hypernym of the first text corresponding to the sample text. By determining the first text corresponding to the sample text, a basis is provided for subsequently training a relevance recognition model using the first text corresponding to the sample text.

[0090] In case A2, the knowledge graph further includes a plurality of edges, and the edges represent semantic relationships between the basic texts corresponding to the two end nodes. The second text corresponding to any sample text is sampled from the knowledge graph, including: determining a target node in the knowledge graph, and the basic text corresponding to the target node being the any sample text; determining at least one first edge in the knowledge graph, and one end node of the first edge being the target node, and the semantic of the any sample text containing or being equivalent to the semantic of the basic text corresponding to the other end node of the first edge; and sampling the second text corresponding to the any sample text from the basic text corresponding to the first node, the first node being a node in the knowledge graph other than the two end nodes of the first edge.

[0091] In the embodiment of the present application, after all the first edges in the knowledge graph are determined, all the nodes in the knowledge graph other than the two end nodes of the first edge are determined as the first node. At least one basic text is randomly sampled from the basic text corresponding to the first node to obtain the second text corresponding to any sample text in the sample text pair. The manner of determining the first edge is described in detail in case A1, and will not be described here.

[0092] Through case A2, a basic text unrelated to any sample text in the sample text pair can be sampled from the knowledge graph to obtain the second text corresponding to the sample text, and the sample text can be a hyponym of the second text or a word without semantic relationship. By determining the second text corresponding to the sample text, a basis is provided for subsequently training a relevance recognition model using the second text corresponding to the sample text.

[0093] The above describes the case where the edge represents the semantic relationship between the basic texts corresponding to the two end nodes. In another possible implementation, the edge can represent the category relationship between the basic texts corresponding to the two end nodes. At this time, one possible implementation of step 204 at least includes cases B1 and B2 as follows.

[0094] Case B1, the knowledge graph further includes a plurality of edges, and the edges represent category relationships between the basic texts corresponding to the two end nodes. The first text corresponding to any sample text is sampled from the knowledge graph, including: determining a target node in the knowledge graph, and the basic text corresponding to the target node being the any sample text; determining at least one second edge in the knowledge graph, and one end node of the second edge being the target node, and the category of the any sample text being the same as the category of the basic text corresponding to the other end node of the second edge; and sampling the first text corresponding to the any sample text from the basic text corresponding to the other end node of each second edge.

[0095] In the embodiments of the present application, the edge between any two nodes of the knowledge graph is used to represent the category relationship between the basic texts corresponding to the nodes. The category relationship between two basic texts includes the same category relationship, i.e., the two basic texts correspond to the same category, for example, “sand pulp tomatoes 1 kg” and “special grade large tomatoes half catty” both correspond to the “tomato” category. The semantic relationship between two basic texts also includes the different category relationship, i.e., the two basic texts correspond to different categories, for example, “sand pulp tomatoes 1 kg” and “fresh cucumbers 500 g” correspond to different categories, wherein “sand pulp tomatoes 1 kg” corresponds to the “tomato” category, and “fresh cucumbers 500 g” corresponds to the “cucumber” category.

[0096] Any node in the knowledge graph corresponds to a basic text. If a certain basic text is the same as any sample text in the sample text pair, the node corresponding to the basic text is determined as the target node. At this time, the edges of all target nodes in the knowledge graph can be determined, and the edge of the target node refers to the edge with one end node being the target node.

[0097] Since the edge represents the category relationship between the basic texts corresponding to the two end nodes of the edge, the edge of the target node can represent the category relationship between the basic text corresponding to the target node (i.e., one end node of the edge) and the basic text corresponding to the other end node of the edge. The category relationship is one of the following two relationships.

[0098] Relationship B1: The category of the basic text corresponding to the target node is the same as the category of the basic text corresponding to the other end node of the edge, i.e., the basic text corresponding to the target node and the basic text corresponding to the other end node of the edge belong to the same category of text.

[0099] Relationship B2: The category of the basic text corresponding to the target node is different from the category of the basic text corresponding to the other end node of the edge, i.e., the basic text corresponding to the target node and the basic text corresponding to the other end node of the edge belong to different categories of text.

[0100] For any edge of a target node, if the category relationship represented by the edge is relationship B1, the edge is taken as a second edge. In this way, all second edges in the knowledge graph can be determined. Then, at least one basic text is randomly sampled from the basic text corresponding to the other end node of all second edges, to obtain the first text corresponding to any sample text pair. That is, the basic text corresponding to the one end node of the second edge (i.e., the target node) is any sample text, and the basic text corresponding to the other end node of the second edge can be the first text corresponding to any sample text.

[0101] In case B1, a basic text related to any one of the sample text pairs can be sampled from the knowledge graph to obtain the first text corresponding to the sample text, and the sample text and the first text corresponding thereto belong to the same category of text. By determining the first text corresponding to the sample text, a basis is provided for subsequent training of the relevance identification model using the first text corresponding to the sample text.

[0102] In case B2, the knowledge graph further includes a plurality of edges, and the edges represent the category relationship between the basic texts corresponding to the two end nodes. The second text corresponding to any one of the sample text pairs is sampled from the knowledge graph, including: determining a target node in the knowledge graph, and the basic text corresponding to the target node is any one of the sample texts; determining at least one second edge in the knowledge graph, and one end node of the second edge is the target node, and the category of any one of the sample texts is the same as the category of the basic text corresponding to the other end node of the second edge; and sampling the second text corresponding to any one of the sample text pairs from the basic text corresponding to the second node, the second node being a node in the knowledge graph other than the two nodes at the ends of the second edge.

[0103] In the embodiments of the present application, after all the second edges in the knowledge graph are determined, all the nodes in the knowledge graph other than the two nodes at the ends of the second edge can be determined as the second nodes. At least one basic text is randomly sampled from the basic text corresponding to the second node to obtain the second text corresponding to any one of the sample text pairs. The manner of determining the second edge is described in detail in case B1, and will not be repeated here.

[0104] In case B2, a basic text unrelated to any one of the sample text pairs can be sampled from the knowledge graph to obtain the second text corresponding to the sample text, and the sample text and the second text corresponding thereto belong to different categories of text. By determining the second text corresponding to the sample text, a basis is provided for subsequent training of the relevance identification model using the second text corresponding to the sample text.

[0105] In step 205, the neural network model is trained based on at least one of the first text and the second text corresponding to any one of the sample text pairs and the labeled relevance and the predicted relevance to obtain a relevance identification model.

[0106] In the embodiment of the present application, the loss of the neural network model can be calculated based on at least one of the first text and the second text corresponding to at least one of the sample text pairs and the annotated relevance and the predicted relevance. Based on the loss of the neural network model, the neural network model is trained by using the gradient update manner to obtain the trained neural network model. If the trained neural network model meets the training end condition, the trained neural network model is taken as the relevance identification model. If the trained neural network model does not meet the training end condition, the trained neural network model is taken as the neural network model for the next training, and the neural network model is trained again according to the manner of steps 201 to 205 until the training end condition is met, and the relevance identification model is obtained.

[0107] The embodiment of the present application does not limit the training end condition. Illustratively, the training end condition is met when the number of times of training of the neural network model reaches a set number of times (such as 500 times). Alternatively, the training end condition is met when the difference between the loss of the neural network model calculated in the present training and the loss of the neural network model calculated in the last training is less than a set threshold.

[0108] In a possible implementation, step 205 includes steps 2051 to 2053.

[0109] In step 2051, the first loss is determined based on the annotated relevance and the predicted relevance.

[0110] In the embodiment of the present application, the first loss can be calculated based on the cross-entropy loss function and the annotated relevance and the predicted relevance. The embodiment of the present application does not limit the form of the cross-entropy loss function. Illustratively, the logarithm of the predicted relevance is calculated, and the product of the logarithm of the annotated relevance and the logarithm of the predicted relevance is taken as the first loss.

[0111] The first loss is used to measure the difference between the annotated relevance and the predicted relevance. In the embodiment of the present application, the smaller the difference between the annotated relevance and the predicted relevance, the higher the accuracy of the neural network model. Therefore, when the neural network model is trained by using the gradient update manner, the gradient descent manner is adopted for the first loss to train the neural network model, so that the predicted relevance output by the neural network model is closer and closer to the annotated relevance, and the accuracy of the neural network model is improved.

[0112] In step 2052, for any one of the sample texts, the second loss corresponding to any one of the sample text pairs is determined based on at least one of the first text and the second text corresponding to any one of the sample text pairs.

[0113] For any one of the sample text pairs, the second loss corresponding to the sample text can be calculated using the sample text and the first text corresponding thereto, and the calculation manner can be seen from the following case C1. Alternatively, the second loss corresponding to the sample text can be calculated using the sample text and the second text corresponding thereto, and the calculation manner can be seen from the following case C2. Alternatively, the second loss corresponding to the sample text can be calculated using the sample text, the first text corresponding thereto, and the second text corresponding thereto, and the calculation manner can be seen from the following case C3.

[0114] In the embodiments of the present application, step 2052 includes the following cases C1 to C3.

[0115] Case C1, the first similarity between any one of the sample texts and the corresponding first text is determined, and the second loss corresponding to any one of the sample texts is determined based on the first similarity.

[0116] As mentioned above, any one of the sample texts needs to be input into the neural network model, and the sample text is encoded by the encoding layer of the neural network model to obtain the text feature of the sample text. Based on the same principle, the first text corresponding to any one of the sample texts needs to be input into the neural network model, and the first text corresponding to the sample text is encoded by the encoding layer of the neural network model to obtain the text feature of the first text corresponding to the sample text. The principle of encoding the sample text and encoding the first text corresponding to the sample text is similar, and will not be described here.

[0117] Then, the text feature of the sample text and the text feature of the first text corresponding to the sample text are used to calculate the similarity to obtain the first similarity. Alternatively, the cosine similarity between the text feature of the sample text and the text feature of the first text corresponding to the sample text is calculated, and the cosine similarity or the difference between the reference data (such as 1) and the cosine similarity is taken as the first similarity.

[0118] In one possible implementation, based on the KL divergence (full name: Kullback-Leibler divergence, also known as relative entropy), on the one hand, the KL divergence from the text feature of the sample text to the text feature of the first text corresponding to the sample text is calculated, and on the other hand, the KL divergence from the text feature of the first text corresponding to the sample text to the text feature of the sample text is calculated. The sum of the two KL divergences is taken as the first similarity. In the embodiments of the present application, the calculation manner of the KL divergence will not be described here.

[0119] Alternatively, the first similarity can be represented as the following formula (1).

[0120]

[0121] in, Characterizing the sample text q and the first text q corresponding to the sample text * The first similarity between them. Characterizing textual features E from sample text q Text features of the first text corresponding to the sample text The KL divergence. Text features representing the first text corresponding to the sample text Text features E of the sample text q The KL divergence.

[0122] In this embodiment, a mapping relationship between similarity and a second loss can be established. After determining the first similarity, the second loss can be determined based on this mapping relationship. For example, the mapping relationship between similarity and the second loss is set as follows: when the similarity is [0, 0.5], the second loss is 0; when the similarity is (0.5, 1], the second loss is 1. When the first similarity is determined to be 0.38, the second loss can be determined to be 0 based on this mapping relationship.

[0123] In case C1, a first similarity is determined between the sample text and its corresponding first text, and this first similarity is used to determine a second loss, which is then used to train the neural network model. Since the first similarity reflects the distance between the sample text and its corresponding first text, and a smaller distance indicates a stronger correlation, the neural network model can learn the correlation between the sample text and its corresponding first text. During the training process using the second loss, the neural network model can narrow the distance between the sample text and its corresponding first text, thereby improving the accuracy of the neural network model.

[0124] Case C2 involves determining the second similarity between any sample text and its corresponding second text, and then determining the second loss for any sample text based on this second similarity. The method for determining the second similarity in Case C2 is similar to the method for determining the first similarity in Case C1, and the method for determining the second loss in Case C2 is also similar to the method for determining the second loss in Case C1; therefore, it will not be elaborated further here.

[0125] In case C2, the second similarity between the sample text and the corresponding second text is determined, and the second loss is determined based on the second similarity, so as to train the neural network model based on the second loss. Since the second similarity can reflect the distance between the sample text and the corresponding second text, and the greater the distance, the less relevant the sample text and the corresponding second text are, the neural network model can learn the irrelevant relationship between the sample text and the corresponding second text, and in the process of training the neural network model based on the second loss, the neural network model can push away the distance between the sample text and the corresponding second text, thereby improving the accuracy of the neural network model.

[0126] In case C3, the first similarity between any sample text and the corresponding first text is determined, the second similarity between any sample text and the corresponding second text is determined, and the second loss corresponding to any sample text is determined based on the first similarity and the second similarity. In case C3, the determination of the first similarity and the second similarity is similar to the determination of the first similarity in case C1, which will not be described herein.

[0127] In a possible implementation, a mapping relationship between the first similarity, the second similarity and the second loss can be set, and after the first similarity and the second similarity are determined, the second loss can be determined based on the mapping relationship. For example, when the first similarity is [0, 0.5] and the second similarity is (0.5, 1], the second loss is 0; when the first similarity is (0.5, 1] and the second similarity is [0, 0.5], the second loss is 1. When the first similarity is 0.38 and the second loss is 0.88, the second loss is 0 based on the mapping relationship.

[0128] In another possible implementation, a first operation result with the first similarity as an index can be calculated, and a second operation result with the second similarity as an index can be calculated. The sum of the first operation result and the second operation result is calculated to obtain a sum value. The ratio of the first operation result to the sum value is calculated, the logarithm value of the ratio is calculated, and the negative of the logarithm value is taken as the second loss corresponding to any sample text.

[0129] It should be noted that the number of the first texts corresponding to any sample text is at least one, and similarly, the number of the second texts corresponding to the sample text is also at least one. In the embodiment of the present application, the first similarity between any sample text and each first text corresponding to the sample text can be calculated, and the second similarity between the sample text and each second text corresponding to the sample text can be calculated. The first operation result corresponding to each first similarity and the second operation result corresponding to each second similarity can be calculated. Then, the sum of all first operation results and all second operation results is calculated to obtain a sum value. The ratio of the sum of each first operation result to the sum value is calculated, the logarithm value of the ratio is calculated, and the negative of the logarithm value is taken as the second loss corresponding to any sample text.

[0130] Alternatively, the determination manner of the second loss corresponding to any sample text is shown in the following formula (2).

[0131]

[0132] wherein, L CQ denotes the second loss corresponding to any sample text q, log is a logarithm symbol, ∑ is a summation symbol, and e is a natural constant. Q pos denotes a set formed by at least one first text corresponding to any sample text q, and qi denotes any first text in the set, that is, qi denotes one first text corresponding to any sample text q. sim q,qi denotes the first similarity between the sample text q and the first text qi corresponding to the sample text. Q neg denotes a set formed by at least one second text corresponding to any sample text q, and qj denotes any second text in the set, that is, qj denotes one second text corresponding to any sample text q. sim q,qj denotes the second similarity between the sample text q and the second text qj corresponding to the sample text. τ is a set hyperparameter, and the value of τ is not limited.

[0133] In case C3, the first similarity between the sample text and the corresponding first text, the second similarity between the sample text and the corresponding second text are determined, and the second loss is determined by using the first similarity and the second similarity, so as to train the neural network model by using the second loss. Since the first similarity can reflect the distance between the sample text and the corresponding first text, the smaller the distance, the more relevant the sample text and the corresponding first text, and the second similarity can reflect the distance between the sample text and the corresponding second text, the greater the distance, the more irrelevant the sample text and the corresponding second text, therefore, the neural network model can learn the relevant relationship between the sample text and the corresponding first text, and learn the irrelevant relationship between the sample text and the corresponding second text at the same time. In the process of training the neural network model by using the second loss, the neural network model can shorten the distance between the sample text and the corresponding first text, and push away the distance between the sample text and the corresponding second text, thereby improving the accuracy of the neural network model.

[0134] In step 2053, the neural network model is trained based on the first loss and the second loss corresponding to at least one sample text pair in the sample text pair, to obtain a relevance identification model.

[0135] In the embodiments of the present application, the first loss and the second loss corresponding to at least one sample text pair in the sample text pair can be weighted and summed to obtain the loss of the neural network model, so as to train the neural network model by using the loss of the neural network model, and obtain the relevance identification model.

[0136] For example, the loss of the neural network model is L final = L T + α * L CQ + β * L CD . Wherein, L final characterizes the loss of the neural network model, L T characterizes the first loss, L CQ characterizes the second loss corresponding to one sample text pair in the sample text pair, and L CD characterizes the second loss corresponding to another sample text pair in the sample text pair. α and β are two weight coefficients, which are hyperparameters set in the embodiments of the present application and are not limited in value.

[0137] In the embodiments of the present application, the first loss is calculated according to the predicted relevance and the labeled relevance of the two sample texts in the sample text pair, and the neural network model is trained by using the first loss, so that the neural network model can learn the relevance between the two sample texts in the sample text pair. When determining the predicted relevance of the two sample texts, each sample text is encoded to obtain the text features of each sample text, realizing knowledge fusion in the dimension of each sample text. Then, the text features of the two sample texts are fused to realize the conversion of the knowledge combination of the two sample texts into single knowledge, and the single knowledge is used to represent the knowledge combination, improving the utilization rate and representation ability of the knowledge.

[0138] In addition, the embodiments of the present application also sample the first text related to any sample text and / or the second text unrelated to the sample text from the knowledge graph, calculate the second loss by the sample text, the first text and / or the second text, and train the neural network model by using the second loss, so that the neural network model can also learn the relevant relationship between the sample text and the first text, the irrelevant relationship between the sample text and the second text, realize knowledge comparison, generalization and fusion in an implicit way, and improve the robustness of the neural network model to noisy knowledge. In addition, the knowledge in the knowledge graph is fused in the training stage, which can save the knowledge in the model, improve the training effect, and does not need to construct additional training samples. While keeping the distribution of the sample text set stable, the role of each sample text is fully played, and the generalization ability of the model is improved.

[0139] Please refer to Figure 3 , Figure 3 is a schematic diagram of a text relationship provided by the embodiments of the present application. The sample text pair includes "tomato" and "fresh tomato 500g". The first text corresponding to "tomato" is "tomato", and the second text corresponding to "tomato" is "tomato paste". The first text corresponding to "fresh tomato 500g" is "sand-filled tomato 1kg", and the second text corresponding to "fresh tomato 500g" is "rich tomato paste". When the neural network model is trained by using the above texts, the neural network model can not only learn the relevance between "tomato" and "fresh tomato 500g", but also narrow the distance between "tomato" and "tomato", "fresh tomato 500g" and "sand-filled tomato 1kg", and push away the distance between "tomato" and "tomato paste", "fresh tomato 500g" and "rich tomato paste".

[0140] In this way, the neural network model learns the correlation between two sample texts in an explicit manner, and learns the correlation between the first text corresponding to one sample text and the second text corresponding to another sample text in an implicit manner. For example, the neural network model learns that "tomato" is related to "fresh tomato 500g", and also learns that "tomato" is related to "fresh tomato 500g", and "tomato sauce" is not related to "fresh tomato 500g", thereby improving the generalization ability of the neural network model.

[0141] It can be understood that the embodiment of the present application can also fuse the text features of one sample text in the sample text pair and the text features of the first text corresponding to another sample text to obtain first fusion features, wherein the determination manner of the first fusion features is similar to that of the sample fusion features, and details are not repeated here. Then, the predicted correlation between one sample text and the first text corresponding to another sample text is determined based on the first fusion features, wherein the determination manner of the predicted correlation can be found in the description of step 2023 above, and details are not repeated here.

[0142] In addition, according to the labeled correlation of the two sample texts in the sample text pair, the first text corresponding to one sample text and another sample text is labeled to obtain the labeled correlation between the first text corresponding to one sample text and another sample text (which is the same as the labeled correlation of the two sample texts in the sample text pair). In this way, the time cost, resource cost, etc. required for labeling is reduced, and the training efficiency is improved.

[0143] Then, the third loss is determined based on the predicted correlation and the labeled correlation between one sample text and the first text corresponding to another sample text. The determination manner of the third loss is similar to that of the first loss, and details are not repeated here.

[0144] Similarly, the embodiment of the present application can also fuse the text features of one sample text in the sample text pair and the text features of the second text corresponding to another sample text to obtain second fusion features, wherein the determination manner of the second fusion features is similar to that of the sample fusion features, and details are not repeated here. Then, the predicted correlation between one sample text and the second text corresponding to another sample text is determined based on the second fusion features, wherein the determination manner of the predicted correlation can be found in the description of step 2023 above, and details are not repeated here.

[0145] In addition, according to the annotation correlation of the two sample texts in the sample text pair, a second text corresponding to the one sample text and the other sample text is annotated to obtain an annotation correlation between the second text corresponding to the one sample text and the other sample text (the annotation correlation is opposite to the annotation correlation of the two sample texts in the sample text pair). In this way, the time cost, resource cost and the like required for annotation are reduced, and the training efficiency is improved.

[0146] Then, based on the prediction correlation and the annotation correlation between the second text corresponding to the one sample text and the other sample text, a fourth loss is determined. The determination manner of the fourth loss is similar to the determination manner of the first loss, and will not be described herein again.

[0147] In the embodiments of the present application, the neural network model can be trained based on the first loss and the second loss, in combination with at least one of the third loss and the fourth loss, to obtain a correlation identification model. For example, the first loss, the second loss, the third loss and the fourth loss are weighted and summed to obtain a loss value of the neural network model. Based on the loss value of the neural network model, the neural network model is trained to obtain the correlation identification model.

[0148] The training method of the correlation identification model provided by the embodiments of the present application is described from the perspective of method steps above, and will be described in detail in combination with scenarios. In the scenarios of the embodiments of the present application, the two sample texts in the sample text pair are a search word and a candidate text respectively. Please refer to Figure 4 , Figure 4 is a schematic diagram of model training provided by the embodiments of the present application, and the correlation identification model is obtained by training the neural network model.

[0149] In the embodiments of the present application, after the sample text pair is obtained, on the one hand, based on the semantics of the search word, the first text and the second text corresponding to the search word are sampled from the knowledge graph, and on the other hand, based on the semantics of the candidate text, the first text and the second text corresponding to the candidate text are sampled from the knowledge graph. The search word and its corresponding first text and second text, and the candidate text and its corresponding first text and second text are input into the neural network model.

[0150] The neural network model includes an encoding layer, a feature fusion layer and a classification layer. The encoding layer can perform encoding processing on the search word to obtain the text feature of the search word. Similarly, the text feature of the first text corresponding to the search word, the text feature of the second text corresponding to the search word, the text feature of the candidate text, the text feature of the first text corresponding to the candidate text and the text feature of the second text corresponding to the candidate text can be obtained through the encoding layer.

[0151] The feature fusion layer can fuse the text features of the search word and the text features of the candidate text to obtain sample fusion features. After the sample fusion features are input into the classification layer, the classification layer classifies the sample fusion features to obtain a predicted relevance between the search word and the candidate text. The labeled relevance between the search word and the candidate text can also be obtained, and the first loss is determined through the predicted relevance and the labeled relevance.

[0152] In addition, based on the text features of the search word, the text features of the first text corresponding to the search word, and the text features of the second text corresponding to the search word, the second loss corresponding to the search word can be determined. Based on the text features of the candidate text, the text features of the first text corresponding to the candidate text, and the text features of the second text corresponding to the candidate text, the second loss corresponding to the candidate text can be determined.

[0153] Then, based on the first loss, the second loss corresponding to the search word, and the second loss corresponding to the candidate text, the neural network model is trained to obtain the relevance identification model.

[0154] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions. For example, the sample text and the like involved in the present application are obtained under sufficient authorization.

[0155] In the above method, at least one of the first text and the second text corresponding to any one of the sample text pair is sampled from the knowledge graph, the first text is related to the sample text, and the second text is unrelated to the sample text. Based on at least one of the first text and the second text, in combination with the predicted relevance and the labeled relevance of the two sample texts in the sample text pair, the relevance identification model is trained to make the relevance identification model not only identify the relevance between the two sample texts, but also learn the relevant relationship between the sample text and the first text and / or the irrelevant relationship between the sample text and the second text, thereby improving the robustness and generalization ability of the relevance identification model, and thus improving the accuracy of the relevance identification model.

[0156] The present application also provides a relevance identification method, which can be applied in the above-mentioned implementation environment. Figure 5 The flowchart of the relevance identification method provided by the present application is taken as an example, which can be implemented by the above-mentioned computing device. Figure 1The terminal device 101 or the server 102 in the method can also be jointly executed by the terminal device 101 and the server 102. For ease of description, the terminal device 101 or the server 102 that executes the correlation identification method in the embodiments of the present application is referred to as an electronic device, and the method can be executed by the electronic device. As shown in FIG. 5, the method includes the following steps 501 to 505. Figure 5

[0157] In step 501, a target text pair and a correlation identification model are obtained. The correlation identification model is obtained by training according to the correlation identification model training method related to the present application, and the description of the correlation identification model training method related to the present application can be referred to in the present application. Figure 2 Figure 2

[0158] In the embodiments of the present application, the target text pair includes two target texts. The text types of the two target texts are different in different technical fields. In a general way, the two target texts are texts whose correlation is to be determined. The content of the target text pair is similar to that of the sample text pair, and will not be described herein.

[0159] In the embodiments of the present application, the target text pair can be obtained in any way. For example, in the question and answer technical field, a question text input by a user is taken as one of the target texts in the target text pair, and an answer text is selected from an answer text library as the other target text in the target text pair. In the search technical field, a search word input by a user is taken as one of the target texts in the target text pair, and a candidate text is selected from a candidate text library as the other target text in the target text pair. In the knowledge graph field, any two object names grabbed from the network are taken as the two target texts in the target text pair.

[0160] In step 502, the target text pair is input into the correlation identification model, and the text features of the two target texts in the target text pair are determined by the correlation identification model.

[0161] The correlation identification model in the embodiments of the present application is similar to the model structure and size of the neural network model mentioned above. For example, the correlation identification model includes network layers such as an encoding layer, a feature fusion layer and a classification layer connected in sequence, and the size and structure of each network layer are not limited in the embodiments of the present application.

[0162] After the two target texts in the target text pair are input into the correlation identification model, for any target text, the encoding layer encodes the target text to obtain the text feature of the target text. The determination method of the text feature of the target text is similar to the determination method of the text feature of the sample text described above, and will not be described herein.

[0163] ​​​At step 503, the text features of the two target texts in the target text pair are fused to obtain a target fusion feature.

[0164] When the text features of the two target texts in the target text pair are determined by the encoding layer, the text features of the two target texts are fused by the feature fusion layer to obtain the target fusion feature. The determination manner of the target fusion feature is similar to the determination manner of the sample fusion feature, which will not be described herein.

[0165] By fusing the text features of the two target texts in the target text pair, the relationship between the text features of the two target texts is mined, so that the target fusion feature can reflect the correlation between the two target texts, and provides a basis for determining the correlation of the two target texts subsequently.

[0166] At step 504, the correlation of the two target texts in the target text pair is determined based on the target fusion feature.

[0167] The target fusion feature is input into the classification layer. Since the target fusion feature can reflect the correlation between the two target texts, the classification layer can map the target fusion feature into the correlation of the two target texts in the target text pair. The correlation of the two target texts is used to represent the possibility of the correlation of the two target texts. The greater the correlation of the two target texts, the greater the possibility of the correlation of the two target texts, that is, the correlation of the two target texts is proportional to the possibility of the correlation of the two target texts. Alternatively, the correlation of the two target texts is data greater than or equal to 0.5 and less than or equal to 1.

[0168] After determining the relevance of the two target texts, if the relevance is greater than the relevance threshold, it is determined that the two target texts are relevant, and if the relevance is not greater than the relevance threshold, it is determined that the two target texts are not relevant. Subsequent processing can be performed based on the information of whether the two target texts are relevant. For example, in the field of question and answer technology, if the object input question text is relevant to the answer text, the answer text is taken as the reply text of the question text, so that the terminal device displays the reply text, and if the object input question text is not relevant to the answer text, the answer text cannot be taken as the reply text of the question text. In the field of search technology, if the object input search word is relevant to the candidate text, the candidate text is taken as the search result of the search word, so that the terminal device displays the search result, and if the object input search word is not relevant to the candidate text, the candidate text cannot be taken as the search result of the search word. In the field of knowledge graph, if any two object names grabbed from the network are relevant, an edge is added between the nodes corresponding to the two object names, and if any two object names grabbed from the network are not relevant, an edge cannot be added between the nodes corresponding to the two object names. In this way, it is determined whether an edge can be added between the nodes corresponding to each two object names, so as to obtain a knowledge graph.

[0169] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.) and signals involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data need to comply with relevant laws, regulations and standards of countries and regions. For example, the target text and the like involved in the present application are obtained under sufficient authorization.

[0170] The relevance identification model in the above method has high robustness and generalization ability, and has high accuracy, so the accuracy of the relevance between the two target texts determined based on the relevance identification model is also high.

[0171] Figure 6 As shown in FIG. 6, the device includes: Figure 6 As shown in FIG. 6, the device includes:

[0172] The obtaining module 601 is configured to obtain a sample text pair and a labeled relevance of two sample texts in the sample text pair.

[0173] The determining module 602 is configured to control the neural network model to determine a predicted relevance of the two sample texts in the sample text pair.

[0174] The obtaining module 601 is further configured to obtain a knowledge graph, and the knowledge graph includes a plurality of nodes, and each node represents a basic text.

[0175] The sampling module 603 is configured to sample at least one of the first text and the second text corresponding to any sample text pair from the knowledge graph, the first text corresponding to any sample text pair being a basic text related to the any sample text, and the second text corresponding to any sample text pair being a basic text unrelated to the any sample text.

[0176] The training module 604 is configured to train the neural network model based on at least one of the first text and the second text corresponding to at least one sample text pair in the sample text pairs, the labeled correlation, and the predicted correlation, to obtain the correlation identification model.

[0177] In a possible implementation, the knowledge graph further includes a plurality of edges, and the edges represent semantic relationships between the basic texts corresponding to the two end nodes.

[0178] The sampling module 603 is configured to determine a target node in the knowledge graph, the basic text corresponding to the target node being any sample text; determine at least one first edge in the knowledge graph, one end node of the first edge being the target node, and the semantic of the any sample text containing or being equivalent to the semantic of the basic text corresponding to the other end node of the first edge; and sample the first text corresponding to the any sample text from the basic texts corresponding to the other end nodes of the first edges.

[0179] In a possible implementation, the knowledge graph further includes a plurality of edges, and the edges represent semantic relationships between the basic texts corresponding to the two end nodes.

[0180] The sampling module 603 is configured to determine a target node in the knowledge graph, the basic text corresponding to the target node being any sample text; determine at least one first edge in the knowledge graph, one end node of the first edge being the target node, and the semantic of the any sample text containing or being equivalent to the semantic of the basic text corresponding to the other end node of the first edge; and sample the second text corresponding to the any sample text from the basic text corresponding to the first node, the first node being a node in the knowledge graph other than the two end nodes of the first edges.

[0181] In a possible implementation, the knowledge graph further includes a plurality of edges, and the edges represent category relationships between the basic texts corresponding to the two end nodes.

[0182] The sampling module 603 is configured to determine a target node in the knowledge graph, the basic text corresponding to the target node being any sample text; determine at least one second edge in the knowledge graph, one end node of the second edge being the target node, and the category of the any sample text being the same as the category of the basic text corresponding to the other end node of the second edge; and sample the first text corresponding to the any sample text from the basic texts corresponding to the other end nodes of the second edges.

[0183] In a possible implementation, the knowledge graph further includes a plurality of edges, and the edges represent category relationships between the basic texts corresponding to the two end nodes.

[0184] The sampling module 603 is configured to determine a target node in the knowledge graph, the basic text corresponding to the target node being any sample text; determine at least one second edge in the knowledge graph, one end node of the second edge being the target node, and the category of any sample text being the same as the category of the basic text corresponding to the other end node of the second edge; and sample a second text corresponding to any sample text from the basic text corresponding to the second node, the second node being a node in the knowledge graph other than the two end nodes of each second edge.

[0185] In a possible implementation, the determining module 602 is configured to input a sample text pair into a neural network model, determine text features of the two sample texts in the sample text pair by using the neural network model, fuse the text features of the two sample texts in the sample text pair to obtain sample fusion features, and determine a predicted relevance of the two sample texts in the sample text pair based on the sample fusion features.

[0186] In a possible implementation, the training module 604 is configured to determine a first loss based on the labeled relevance and the predicted relevance, determine a second loss corresponding to any sample text in the sample text pair based on at least one of the first text and the second text corresponding to the sample text, and train the neural network model based on the first loss and the second loss corresponding to at least one sample text pair in the sample text pair to obtain the relevance identification model.

[0187] In a possible implementation, the training module 604 is configured to determine a first similarity between any sample text and the corresponding first text, determine a second similarity between any sample text and the corresponding second text, and determine the second loss corresponding to any sample text based on the first similarity and the second similarity.

[0188] The above apparatus samples at least one of the first text and the second text corresponding to any sample text pair in the sample text pair from the knowledge graph, the first text being relevant to the sample text, and the second text being irrelevant to the sample text. Based on the at least one of the first text and the second text, in combination with the predicted relevance and the labeled relevance of the two sample texts in the sample text pair, the relevance identification model is trained, so that the relevance identification model not only can identify the relevance between the two sample texts, but also learns the relevant relationship between the sample text and the first text and / or the irrelevant relationship between the sample text and the second text, improves the robustness and generalization ability of the relevance identification model, and thus improves the accuracy of the relevance identification model.

[0189] It should be understood that the above Figure 6 The apparatus provided in the above is only exemplified by the division of the above functional modules in realizing its functions, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0190] Figure 7 Fig. 1 shows a structural schematic diagram of a correlation identification apparatus provided by an embodiment of the present application. As shown in Fig. 1, the apparatus comprises: Figure 7 The acquisition module 701 is configured to acquire a target text pair and a correlation identification model, wherein the correlation identification model is obtained by training according to the training method of the correlation identification model mentioned in the above embodiments.

[0191] The determination module 702 is configured to input the target text pair into the correlation identification model, and determine the text features of the two target texts in the target text pair by the correlation identification model.

[0192] The fusion module 703 is configured to fuse the text features of the two target texts in the target text pair to obtain a target fusion feature.

[0193] The determination module 702 is configured to determine the correlation between the two target texts in the target text pair based on the target fusion feature.

[0194] The correlation identification model in the above apparatus has high robustness and generalization ability, and has high accuracy, so the accuracy of the correlation between the two target texts determined based on the correlation identification model is also high.

[0195] It should be understood that the above

[0196] The apparatus provided in the above is only exemplified by the division of the above functional modules in realizing its functions, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process is detailed in the method embodiments, which will not be repeated here. Figure 7

[0197] Fig. 8 shows a structural block diagram of a terminal device 800 provided by an example embodiment of the present application. The terminal device 800 comprises a processor 801 and a memory 802. Figure 8

[0198] ​The processor 801 can include one or more processing cores. The processor 801 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 801 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 801 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 801 can further include an AI (Artificial Intelligence) processor for processing machine learning related computing operations.

[0199] The memory 802 can include at least one computer-readable storage medium, which can be non-transitory. The memory 802 can also include a high-speed random access memory and a nonvolatile memory. In some embodiments, the non-transitory computer-readable storage medium in the memory 802 is used to store at least one computer program for being executed by the processor 801 to implement the training method of the correlation identification model or the correlation identification method provided by the method embodiments in the present application.

[0200] In some embodiments, the terminal device 800 can also optionally include a peripheral device interface 803 and at least one peripheral device. The processor 801, the memory 802 and the peripheral device interface 803 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 803 through a bus, a signal line or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 804, a display screen 805, a camera assembly 806, an audio circuit 807 and a power supply 808.

[0201] The peripheral interface 803 can be used to connect at least one I / O (Input / Output) related peripheral device to the processor 801 and the memory 802. In some embodiments, the processor 801, the memory 802 and the peripheral interface 803 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 801, the memory 802 and the peripheral interface 803 can be implemented on a separate chip or circuit board, and the present embodiments are not limited in this regard.

[0202] The radio frequency circuit 804 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 804 communicates with a communication network and other communication devices through electromagnetic signals. The radio frequency circuit 804 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. The radio frequency circuit 804 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes but is not limited to: the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks, a wireless local area network, etc. In some embodiments, the radio frequency circuit 804 can also include NFC (Near Field Communication) related circuit, and the present application is not limited in this regard.

[0203] The display screen 805 is used to display a UI (User Interface). When the display screen 805 is a touch display screen, the display screen 805 also has the ability to collect touch signals on or above the surface of the display screen 805. The touch signals can be input as control signals to the processor 801 for processing. At this time, the display screen 805 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 805 can be one, arranged on the front panel of the terminal device 800; in some other embodiments, the display screen 805 can be at least two, arranged on different surfaces of the terminal device 800 or in a folding design; in some other embodiments, the display screen 805 can be a flexible display screen, arranged on a curved surface or a folding surface of the terminal device 800. Even, the display screen 805 can also be arranged in an irregular pattern that is not rectangular. The display screen 805 can be made of LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode) and other materials.

[0204] The camera component 806 is configured to capture images or videos. Optionally, the camera component 806 includes a front camera and a rear camera. In some embodiments, the rear camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, a long-focus camera, to realize the background blur function of the main camera and the depth-of-field camera, the panoramic shooting and VR (Virtual Reality) shooting function of the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera component 806 can also include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to the combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0205] The audio circuit 807 can include a microphone and a speaker. The microphone is configured to capture sound waves of a user and an environment, and convert the sound waves into an electrical signal input to the processor 801 for processing, or input to the radio frequency circuit 804 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, which are respectively arranged at different parts of the terminal device 800. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is configured to convert an electrical signal from the processor 801 or the radio frequency circuit 804 into sound waves. The speaker can be a traditional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert an electrical signal into a sound wave audible to humans, but also convert an electrical signal into an inaudible sound wave to humans for ranging purposes. In some embodiments, the audio circuit 807 can also include a headphone jack.

[0206] The power supply 808 is configured to supply power to each component in the terminal device 800. The power supply 808 can be alternating current, direct current, disposable batteries, or rechargeable batteries. When the power supply 808 includes rechargeable batteries, the rechargeable batteries can be wired rechargeable batteries or wireless rechargeable batteries.

[0207] In some embodiments, the terminal device 800 further includes one or more sensors 809. The one or more sensors 809 include, but are not limited to, an acceleration sensor 811, a gyroscope sensor 812, a pressure sensor 813, an optical sensor 814, and a proximity sensor 815.

[0208] The acceleration sensor 811 can detect the acceleration magnitude in three coordinate axes of the coordinate system established by the terminal device 800. For example, the acceleration sensor 811 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 801 can control the display screen 805 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signals collected by the acceleration sensor 811. The acceleration sensor 811 can also be used for game or user motion data collection.

[0209] The gyroscope sensor 812 can detect the body direction and rotation angle of the terminal device 800, and the gyroscope sensor 812 can collect the 3D motion of the user to the terminal device 800 in cooperation with the acceleration sensor 811. The processor 801 can implement the following functions according to the data collected by the gyroscope sensor 812: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0210] The pressure sensor 813 can be arranged on the side frame of the terminal device 800 and / or the lower layer of the display screen 805. When the pressure sensor 813 is arranged on the side frame of the terminal device 800, the user's holding signal to the terminal device 800 can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 801 according to the holding signal collected by the pressure sensor 813. When the pressure sensor 813 is arranged on the lower layer of the display screen 805, the controllable control on the UI interface can be controlled by the processor 801 according to the user's pressure operation to the display screen 805. The controllable control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0211] The optical sensor 814 is used to collect the ambient light intensity. In one embodiment, the processor 801 can control the display brightness of the display screen 805 according to the ambient light intensity collected by the optical sensor 814. Specifically, when the ambient light intensity is high, the display brightness of the display screen 805 is increased; when the ambient light intensity is low, the display brightness of the display screen 805 is decreased. In another embodiment, the processor 801 can also dynamically adjust the shooting parameters of the camera assembly 806 according to the ambient light intensity collected by the optical sensor 814.

[0212] The proximity sensor 815, also known as a distance sensor, is typically located on the front panel of the terminal device 800. The proximity sensor 815 is used to detect the distance between the user and the front of the terminal device 800. In one embodiment, when the proximity sensor 815 detects that the distance between the user and the front of the terminal device 800 is gradually decreasing, the processor 801 controls the display screen 805 to switch from a screen-on state to a screen-off state; when the proximity sensor 815 detects that the distance between the user and the front of the terminal device 800 is gradually increasing, the processor 801 controls the display screen 805 to switch from a screen-off state to a screen-on state.

[0213] Those skilled in the art will understand that Figure 8 The structure shown does not constitute a limitation on the terminal device 800, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0214] Figure 9 This is a schematic diagram of the server structure provided in the embodiments of this application. The server 900 can vary considerably due to different configurations or performance. It may include one or more processors 901 and one or more memories 902. The one or more memories 902 store at least one computer program, which is loaded and executed by the one or more processors 901 to implement the training method or correlation identification method of the correlation identification model provided in the above-described method embodiments. For example, the processor 901 is a CPU. Of course, the server 900 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server 900 may also include other components for implementing device functions, which will not be elaborated here.

[0215] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one computer program, which is loaded and executed by a processor to enable an electronic device to implement the training method or the correlation identification method of any of the above-described correlation identification models.

[0216] Optionally, the aforementioned computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0217] In the example embodiment, a computer program or computer program product is also provided, in which at least one computer program is stored, the at least one computer program is loaded and executed by a processor to enable an electronic device to implement any one of the above correlation identification model training methods or correlation identification methods.

[0218] It should be understood that "multiple" referred to herein means two or more. The "and / or" describes the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally represents that the associated objects before and after it are in an "or" relationship.

[0219] It should be understood that the above-mentioned sequence numbers of the embodiments of the present application are only for description, and do not represent the advantages and disadvantages of the embodiments.

[0220] The above only describes the example embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A training method for a correlation identification model, characterized in that, The method comprises: obtaining a sample text pair and a labeled relevance of two sample texts in the sample text pair; controlling a neural network model to determine a predicted relevance of the two sample texts in the sample text pair; obtaining a knowledge graph, the knowledge graph comprising a plurality of nodes, the nodes representing base texts; for any one sample text in the sample text pair, sampling at least one of a first text and a second text corresponding to the any one sample text pair from the knowledge graph, the first text corresponding to the any one sample text pair being a base text related to the any one sample text, and the second text corresponding to the any one sample text pair being a base text unrelated to the any one sample text; training the neural network model based on at least one of the first text and the second text corresponding to at least one sample text pair in the sample text pair, and the labeled relevance and the predicted relevance, to obtain a relevance identification model; the training of the neural network model based on at least one of the first text and the second text corresponding to at least one sample text pair in the sample text pair, and the labeled relevance and the predicted relevance, to obtain a relevance identification model, comprises: determining a first loss based on the labeled relevance and the predicted relevance; for any one sample text in the sample text pair, determining a second loss corresponding to the any one sample text pair based on at least one of the first text and the second text corresponding to the any one sample text pair; training the neural network model based on the first loss and the second loss corresponding to at least one sample text pair in the sample text pair to obtain a relevance identification model.

2. The method of claim 1, wherein, The knowledge graph further comprises a plurality of edges, the edges representing semantic relationships between base texts corresponding to two end nodes; sampling the first text corresponding to the any one sample text pair from the knowledge graph comprises: determining a target node in the knowledge graph, the base text corresponding to the target node being the any one sample text; determining at least one first edge in the knowledge graph, one end node of the first edge being the target node, and the semantic of the any one sample text containing or being equivalent to the semantic of the base text corresponding to the other end node of the first edge; sampling the first text corresponding to the any one sample text pair from the base text corresponding to the other end node of each first edge.

3. The method of claim 1, wherein, The knowledge graph further comprises a plurality of edges, the edges representing semantic relationships between base texts corresponding to two end nodes; sampling the second text corresponding to the any one sample text pair from the knowledge graph comprises: determining a target node in the knowledge graph, the base text corresponding to the target node being the any one sample text; determining at least one first edge in the knowledge graph, one end node of the first edge being the target node, and the semantic of the any one sample text containing or being equivalent to the semantic of the base text corresponding to the other end node of the first edge; sampling a second text corresponding to the any one sample text from a basic text corresponding to a second node, the second node being a node in the knowledge graph other than the two end nodes of each first edge.

4. The method of claim 1, wherein, The knowledge graph further comprises a plurality of edges, and the edges represent category relationships between basic texts corresponding to the two end nodes. The method comprises: determining a target node in the knowledge graph, wherein the basic text corresponding to the target node is the any one sample text; determining at least one second edge in the knowledge graph, wherein one end node of the second edge is the target node, and the category of the any one sample text is the same as the category of the basic text corresponding to the other end node of the second edge; and sampling the first text corresponding to the any one sample text from the basic text corresponding to the other end node of each second edge.

5. The method of claim 1, wherein, The knowledge graph further comprises a plurality of edges, and the edges represent category relationships between basic texts corresponding to the two end nodes. The method comprises: determining a target node in the knowledge graph, wherein the basic text corresponding to the target node is the any one sample text; and determining at least one second edge in the knowledge graph, wherein one end node of the second edge is the target node, and the category of the any one sample text is the same as the category of the basic text corresponding to the other end node of the second edge; sampling a second text corresponding to the any one sample text from a basic text corresponding to a second node, the second node being a node in the knowledge graph other than the two end nodes of each second edge.

6. The method of claim 1, wherein, The method comprises: inputting the sample text pair into a neural network model to determine text features of the two sample texts in the sample text pair by the neural network model; fusing the text features of the two sample texts in the sample text pair to obtain sample fusion features; determining the predicted relevance of the two sample texts in the sample text pair based on the sample fusion features.

7. The method of claim 1, wherein, The method comprises: determining a first similarity between the any one sample text and the corresponding first text; determining a second similarity between the any one sample text and the corresponding second text; determining the second loss corresponding to the any one sample text based on the first similarity and the second similarity.

8. A correlation identification method characterized by, The method comprises: obtaining a target text pair and a relevance identification model, wherein the relevance identification model is obtained by training according to the training method of the relevance identification model in any one of claims 1 to 7; inputting the target text pair into the relevance identification model to determine text features of the two target texts in the target text pair by the relevance identification model; fusing the text features of the two target texts in the target text pair to obtain target fusion features; and determining the predicted relevance of the two target texts in the target text pair based on the target fusion features. Determine the relevance of the two target texts in the target text pair based on the target fusion feature.

9. An electronic device, comprising: The electronic device includes a processor and a memory, and the memory stores at least one computer program, which is loaded and executed by the processor to enable the electronic device to implement the training method of the relevance identification model according to any one of claims 1 to 7 or implement the relevance identification method according to claim 8.

Images