Multi-granularity text matching model and method based on representation and interaction combination
By designing a multi-grained text matching model based on the combination of representation and interaction, combining global features and local matching features, the balance problem of existing models between global information and semantic focus is solved, and a more efficient text matching effect is achieved.
Patent Information
- Application Number
- CN202411872095.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-18
- Publication Date
- 2025-04-11
AI Technical Summary
Existing representation models can display the global information of the text relatively comprehensively but easily lose semantic focus, while interactive models can grasp the semantic focus well but easily ignore the syntax and other information of the global text.
A multi-grained text matching model based on the combination of representation and interaction is designed. Through the embedding layer, multi-grained feature encoding layer, multi-grained feature representation layer, multi-grained feature interaction layer, feature fusion layer and matching prediction layer, combining global features and local matching features, more comprehensive and deep text semantic information is obtained.
It realizes the integration of representation and interactive models, combining global features and local matching features from a multi-grained perspective, and obtaining more comprehensive and deep text semantic information, improving the accuracy of text matching.
Smart Images
Figure CN119808790A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of information processing technology, and in particular to a multi-granularity text matching model and method based on a combination of representation and interaction. Background Art
[0002] With the advent of the big data era and the continuous development of artificial intelligence, the Internet has become an important way for people to transmit and obtain information in their daily lives. Achieving efficient and accurate text matching has a profound impact on application fields such as document retrieval, intelligent question and answer, and human-computer dialogue. The text matching task is a core task in the field of natural language processing, which involves tasks such as inferring the relationship between two texts. For example, tasks such as machine translation, information retrieval, interpretation recognition, and automatic question and answer can all be abstracted into text matching tasks. Machine translation can be abstracted into matching different language expressions of the same semantic information, information retrieval can be abstracted into matching the relevance between two or more texts, interpretation recognition is to measure whether the semantics expressed by two texts are similar, and automatic question and answer can be converted into matching the relevance between questions and answers. For different tasks, it is necessary to establish corresponding text matching models for the characteristics of the corresponding tasks. How to use a better model to solve the text matching problem has become the focus and challenge of research. The complexity of natural language is the main challenge faced by text matching problems, such as: (1) natural language may have multiple meanings or multiple different words may express the same meaning; (2) there are phrases composed of multiple words, which may have different semantics due to different orders; (3) language-specific features such as grammar, part of speech, and word frequency will also have different effects on the semantic representation of text.
[0003] Traditional text matching methods rely on manually set extraction rules and matching templates, and all use literal similarity to measure the degree of text matching, resulting in very few parameters that the model can learn. When faced with text matching tasks in specific fields, a lot of manual intervention is required, and the model is difficult to be universal. For text matching tasks, there are great limitations to literal matching alone, and semantic matching also needs to be considered. With the use of deep neural networks and their good results in artificial intelligence fields such as image and natural language processing, many scholars have begun to use deep neural networks to handle text matching tasks.
[0004] Compared with traditional text matching methods, deep learning methods use multi-layer neural networks to enable machines to automatically learn text language features, saving the cost of manually setting matching rules, and making the model more suitable for various text matching task scenarios. Deep learning methods map texts to vector space, allowing machines to automatically learn the relationship between texts from text vector representations, and use text representations of different lengths and various feature information of words in sentences or paragraphs to improve the accuracy of text matching. Nowadays, most of the text matching using deep learning methods only considers the matching between words. For a complete sentence, its basic semantic unit does not only contain one word, but there are multiple words forming phrases, and different word orders will also show different semantic information. Therefore, there are still certain challenges in matching the similarity between two texts. Therefore, in order to better grasp the local feature information and global feature information between two texts, it is of great research significance to design an efficient text matching model for a variety of natural language processing tasks. Therefore, a multi-granularity text matching model based on the combination of representation and interaction is designed. Summary of the invention
[0005] The present application provides a multi-granularity text matching model and method based on the combination of representation and interaction to solve the problem that the existing representation model can relatively comprehensively display the global information of the text but easily loses the semantic focus, while the interactive model can grasp the semantic focus well but easily ignores the global syntactic information of the text. The application realizes the fusion of the two semantic matching models, combines global features and local matching features for analysis from a multi-granularity perspective, and obtains more comprehensive and in-depth text semantic information.
[0006] The present application provides a multi-granularity text matching model based on a combination of representation and interaction, including an embedding layer, a multi-granularity feature encoding layer, a multi-granularity feature representation layer, a multi-granularity feature interaction layer, a feature fusion layer, and a matching prediction layer;
[0007] The embedding layer is connected to the input end of the multi-granularity feature encoding layer;
[0008] The output end of the multi-granularity feature encoding layer is respectively connected to the input end of the multi-granularity feature representation layer and the input end of the multi-granularity feature interaction layer;
[0009] The output ends of the multi-granularity feature representation layer and the multi-granularity feature interaction layer are connected to the input end of the feature fusion layer;
[0010] The output end of the feature fusion layer is connected to the matching prediction layer.
[0011] Preferably, the embedding layer uses the trained GloV word vectors to obtain an embedding lookup matrix for all words in the training text, and represents words that do not appear in the GloV word vector library using zero vectors, obtains the word vector representation of each word through the embedding lookup matrix, and then serializes the text into a matrix data form for output.
[0012] Preferably, the multi-granularity feature encoding layer adopts a convolutional neural network and sets different convolution kernel parameters to simulate word granularity information of different lengths in the text and obtain multi-granularity features of the text.
[0013] Preferably, the multi-granularity feature representation layer uses a convolutional neural network to perform feature compression of different granularities on the representation matrix of the text sequence, and then passes the text features compressed at different granularities through a Bi-LSTM network to extract the sequence features of the text, wherein the Bi-LSTM network is a two-layer LSTM network structure, the first layer LSTM analyzes the text in a time sequence from left to right, and the second layer LSTM analyzes the text in a time sequence from right to left, respectively extracting the semantic information of the text from two directions, thereby avoiding the loss of contextual information of the text.
[0014] Preferably, the multi-granularity feature interaction layer adopts an interactive attention mechanism, and under the feature representation of the same granularity, the similarity between the corresponding segments in the two text sequences is calculated to obtain a similarity matrix of the two texts; the normalized weighted sum of the horizontal data in the matrix is the attention weight Att of the document to the query. D2Q , the normalized weighted sum of its longitudinal data is the attention weight Att of the query to the document Q2D , respectively using Att D2Q and Att Q2D The original feature matrix of the query and document is calculated to improve the weight information of important features in the text and obtain a new text feature matrix.
[0015] Preferably, the feature fusion layer uses three methods to compare the overall text representation and the local text representation, and then fuses them together. The output of the text representation of the first text sequence at the k-granularity size after the feature fusion layer is calculated by the following formula:
[0016]
[0017]
[0018] Among them, E represents the element-by-element calculation of the matrix. The overall representation of the text and the local representation of the text are respectively compared by matrix concatenation, matrix subtraction and matrix multiplication. The two representations of the text are compared from multiple angles to fully integrate the multi-granularity representations between the two texts.
[0019] Preferably, the matching prediction layer predicts the logical relationship between the two texts according to the feature vectors of the two texts. The matching prediction layer adopts the same method as the MGTA model to interactively enhance the two feature vectors to capture the logical relationship between the two texts, as shown below:
[0020]
[0021] in, Represents the prediction result, L is multiple fully connected layers, “;” represents the concatenation of vectors, and “⊙” represents element multiplication.
[0022] Preferably, a method of predicting logical relationship classification is used to determine whether two texts are similar or dissimilar in logical relationship, and a cross entropy loss function is used, and the calculation formula is as follows:
[0023]
[0024] Preferably, a method for using a multi-granularity text matching model based on a combination of representation and interaction includes the following steps:
[0025] A. During the model training process, a set of data is input into the left and right network structures of the twin model respectively, and the text data is converted into a text feature representation vector through the word embedding layer of the model;
[0026] B. After that, the output of the embedding layer is transmitted to the multi-granularity feature representation layer and the multi-granularity feature interaction layer. In the multi-granularity feature representation layer, the text features at different granularities are globally represented. At the same time, in the multi-granularity feature interaction layer, two text features at different granularities are interactively calculated to increase the weight of the important part of the text information.
[0027] C. The obtained text representation vector and the text features after the multi-granularity interaction layer are used as input and transmitted to the feature fusion layer, where the multi-granularity text representation features and the text interaction features are fused together, so that the model can capture text information at different levels to a greater extent;
[0028] D. Finally, the generated text final vector representation is input into the matching prediction layer, and the matching result is obtained through calculation in the fully connected layer.
[0029] Beneficial effect: The present invention integrates two semantic matching models, combines global features and local matching features for analysis from a multi-granularity perspective, and obtains more comprehensive and in-depth text semantic information.
[0030] The above description is only an overview of the technical solution of the embodiment of the present application. In order to more clearly understand the technical means of the embodiment of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, a brief introduction will be given below to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0032] Figure 1 It is a schematic diagram of a representation-based text matching model in the prior art;
[0033] Figure 2 It is a schematic diagram of an interactive text matching model in the prior art;
[0034] Figure 3 Schematic diagram of the text matching model of the present invention;
[0035] Figure 4 A schematic diagram of the differences and similarities between phrases;
[0036] Figure 5 This is a schematic diagram of the BiLSTM network structure;
[0037] Figure 6 The workflow of the present invention is as follows;
[0038] Explanation of the accompanying drawings: embedding layer 1, multi-granularity feature encoding layer 2, multi-granularity feature representation layer 3, multi-granularity feature interaction layer 4, feature fusion layer 5, matching prediction layer 6. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by technicians in the technical field to which this application belongs; the terms used in the specification of the application are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification, claims and drawings of this application are intended to cover non-exclusive inclusions.
[0041] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase "embodiments" in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.
[0042] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.
[0043] like Figure 1 As shown in the figure, in the text matching task, the text matching models based on deep learning methods are mainly divided into two categories: representational text matching models and interactive text matching models. In the training process, the representational deep matching model encodes the two text sequences to be matched into semantic representation vectors of the same length in a relatively independent manner, that is, maps them to the same vector space, and then calculates the similarity of the vector representations of the two texts in the final prediction layer of the model. The main advantage of the representational text matching model is that it is easy to train and easy to use in the industry. The core problem is that in the text representation process, only the final semantic vector of the text is extracted, ignoring the important semantics of the words in the text, which is prone to semantic deviation.
[0044] like Figure 2 As shown in the figure, the interactive deep matching model mainly obtains the local similarity features of the text by interactively calculating the local features of the two texts. Unlike the representation model, this type of model allows the text sequences to be matched to interact in the early stage of model training, better grasps the local semantic information of the text, and uses the local matching degree of the text as the main basis for global matching. However, its disadvantage is that it overemphasizes the comparison between words in the text, fails to characterize the global information of the text, and easily loses the overall feature information of the text, such as word order and syntax.
[0045] The representation model can show the global information of the text relatively comprehensively but it is easy to lose the semantic focus, while the interactive model can grasp the semantic focus well but it is easy to ignore the global syntactic information of the text. This chapter integrates these two semantic matching models, analyzes the global features and local matching features from a multi-granular perspective, and obtains more comprehensive and in-depth text semantic information.
[0046] like Figure 3-Figure 4 As shown, the present invention discloses a multi-granularity text matching model based on the combination of representation type and interaction type, characterized in that it includes an embedding layer 1, a multi-granularity feature encoding layer 2, a multi-granularity feature representation layer 3, a multi-granularity feature interaction layer 4, a feature fusion layer 5 and a matching prediction layer 6;
[0047] The embedding layer 1 is connected to the input end of the multi-granularity feature encoding layer 2;
[0048] The output end of the multi-granularity feature encoding layer 2 is respectively connected to the input end of the multi-granularity feature representation layer 3 and the multi-granularity feature interaction layer 4;
[0049] The output ends of the multi-granularity feature representation layer 3 and the multi-granularity feature interaction layer 4 are connected to the input end of the feature fusion layer 5;
[0050] The output end of the feature fusion layer 5 is connected to the matching prediction layer 6 .
[0051] In the present invention, the main function of the embedding layer is to convert each word in the text into a vector with a fixed dimension, and then convert the text sequence into a matrix composed of word vectors, that is, a data form that can be understood and processed by a computer. The embedding layer uses the trained Glove word vectors to obtain the embedding search matrix of all words in the training text, and uses zero vectors to represent those that do not appear in the Glove word vector library. The word vector representation of each word is obtained through the embedding search matrix, and then the text is serialized into a matrix data form for output.
[0052] In the present invention, in the text semantic analysis, phrases are also important units of text semantic expression. A complete semantic segment usually consists of 1 to 3 words. Phrases composed of different words may have the same meaning, and phrases composed of the same words may have different meanings. If the text semantics is analyzed only from a single granularity, some similar semantic features of the text will be lost. Figure 4As shown in the figure, "all in" and "in all" are phrases composed of the same words, but the semantics they imply are different due to the different word orders. "Hard work" and "work hard" are also phrases composed of the same words, but the change of word order does not affect the similarity of semantics. "Case in point" and "for example" are two different phrases composed of different words, but the semantics they express are similar. The multi-granularity feature encoding layer uses a convolutional neural network and sets different convolution kernel parameters to simulate the word granularity information of different lengths in the text and obtain the multi-granularity features of the text.
[0053] In the present invention, a multi-granularity feature representation layer uses a convolutional neural network to perform feature compression of different granularities on a representation matrix of a text sequence, and then passes the text features compressed at different granularities through a Bi-LSTM network to extract sequence features of the text, wherein the Bi-LSTM network is a double-layer LSTM network structure, the first layer LSTM analyzes the text in a time sequence from left to right, and the second layer LSTM analyzes the text in a time sequence from right to left, respectively extracting semantic information of the text from two directions, thereby avoiding the loss of contextual information of the text.
[0054] In the present invention, while performing multi-granular text representation on the text, in order to enhance the weight of similar features between texts and enable the model to learn the semantic features of similar words between texts, the multi-granular feature information of the two text sequences obtained from the encoding layer is interactively calculated to obtain interactive attention between the two texts. By means of semantic alignment, the similar features between the two texts are weighted, making it easier for the model to focus on similar features during training, further reducing the impact of noise information. Interactive attention will highlight the similarity information between the query and the document through different weights during the text representation process, allowing the model to capture the key parts of the text. The multi-granular feature interaction layer adopts an interactive attention mechanism, which calculates the similarity between corresponding fragments in the two text sequences under the feature representation of the same granularity to obtain the similarity matrix of the two texts; the normalized weighted sum of the horizontal data in the matrix is the attention weight Att of the document to the query. D2Q , the normalized weighted sum of its longitudinal data is the attention weight Att of the query to the document Q2D , respectively using Att D2Q and Att Q2D The original feature matrix of the query and document is calculated to improve the weight information of important features in the text and obtain a new text feature matrix.
[0055] In the present invention, the input of the feature fusion layer is the output from the multi-granularity feature representation layer and the multi-granularity feature interaction layer. The input information contains the long-distance information of the text at the multi-granularity angle and the multi-granularity local feature information after the interaction of the two texts. In the feature fusion layer, the text feature representations processed by two different methods are fused. First, the representations of the two texts are fused at the same granularity, and then the text representations at all granularities are spliced to obtain the final representation of a single text sequence. The feature fusion layer uses three methods to compare the overall text representation and the local text representation, and then fuses them together. The text representation of the first text sequence at the k-granularity size is calculated by the output of the feature fusion layer by the following formula:
[0056]
[0057] Among them, E represents the element-by-element calculation of the matrix. The overall representation of the text and the local representation of the text are respectively compared by matrix concatenation, matrix subtraction and matrix multiplication. The two representations of the text are compared from multiple angles to fully integrate the multi-granularity representations between the two texts.
[0058] The multi-granularity feature representation layer obtains the global semantic information of the text and uses the BiLSTM neural network to obtain the contextual features of the text at a long distance. The multi-granularity feature interaction layer enables the model to better grasp the semantic focus of the text and generate deeper semantic information. The feature fusion layer combines the information obtained by the above two models, combines the multi-granularity feature extraction method from the overall text perspective and the local text perspective, fully integrates the representation and interaction matching models, and builds a more comprehensive text matching model.
[0059] In addition, in the present invention, the matching prediction layer predicts the logical relationship between the two texts according to the feature vectors of the two texts. The matching prediction layer adopts the same method as the MGTA model to interactively enhance the two feature vectors to capture the logical relationship between the two texts, as shown below:
[0060]
[0061] in, Represents the prediction result, L is multiple fully connected layers, “;” represents the concatenation of vectors, and “⊙” represents element multiplication.
[0062] The method of predicting logical relationship classification is used to determine whether two texts are similar or dissimilar in logical relationship. The cross entropy loss function is used, and the calculation formula is as follows:
[0063]
[0064] Working principle: A method for using a multi-granularity text matching model based on a combination of representation and interaction, including the following steps:
[0065] A. During the model training process, a set of data is input into the left and right network structures of the twin model respectively, and the text data is converted into a text feature representation vector through the word embedding layer of the model;
[0066] B. After that, the output of the embedding layer is transmitted to the multi-granularity feature representation layer and the multi-granularity feature interaction layer. In the multi-granularity feature representation layer, the text features at different granularities are globally represented. At the same time, in the multi-granularity feature interaction layer, two text features at different granularities are interactively calculated to increase the weight of the important part of the text information.
[0067] C. The obtained text representation vector and the text features after the multi-granularity interaction layer are used as input and transmitted to the feature fusion layer, where the multi-granularity text representation features and the text interaction features are fused together, so that the model can capture text information at different levels to a greater extent;
[0068] D. Finally, the generated text final vector representation is input into the matching prediction layer, and the matching result is obtained through calculation in the fully connected layer.
[0069] In summary, the present invention integrates two semantic matching models, combines global features and local matching features for analysis from a multi-granularity perspective, and obtains more comprehensive and in-depth text semantic information. As mentioned above, the above embodiments are only used to illustrate the technical solution of the present application, rather than to limit it; although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that it is still possible to modify the technical solutions described in the aforementioned embodiments, or to replace some of the technical features therein by equivalents; and these modifications or replacements do not deviate the essence of the corresponding technical solution from the spirit and scope of the technical solutions of the various embodiments of the present application.
Claims
1. A multi-granularity text matching model based on the combination of representation and interaction, characterized in that: It includes an embedding layer (1), a multi-granularity feature encoding layer (2), a multi-granularity feature representation layer (3), a multi-granularity feature interaction layer (4), a feature fusion layer (5) and a matching prediction layer (6); The embedding layer (1) is connected to the input end of the multi-granularity feature encoding layer (2); The output end of the multi-granularity feature encoding layer (2) is respectively connected to the input ends of the multi-granularity feature representation layer (3) and the multi-granularity feature interaction layer (4); The output ends of the multi-granularity feature representation layer (3) and the multi-granularity feature interaction layer (4) are connected to the input end of the feature fusion layer (5); The output end of the feature fusion layer (5) is connected to the matching prediction layer (6).
2. A multi-granularity text matching model based on a combination of representation and interaction according to claim 1, characterized in that: The embedding layer uses the trained GloV word vectors to obtain the embedding lookup matrix of all words in the training text, and uses zero vectors to represent words that do not appear in the GloV word vector library. The word vector representation of each word is obtained through the embedding lookup matrix, and then the text is serialized into a matrix data form for output.
3. The multi-granularity text matching model based on the combination of representation type and interaction type according to claim 1, characterized in that: The multi-granularity feature encoding layer adopts a convolutional neural network and sets different convolution kernel parameters to simulate word granularity information of different lengths in the text and obtain the multi-granularity features of the text.
4. The multi-granularity text matching model based on the combination of representation type and interaction type according to claim 1, characterized in that: The multi-granularity feature representation layer uses a convolutional neural network to perform feature compression of different granularities on the representation matrix of the text sequence, and then passes the text features compressed at different granularities through a Bi-LSTM network to extract the sequence features of the text, wherein the Bi-LSTM network is a two-layer LSTM network structure, the first layer LSTM analyzes the text in a time sequence from left to right, and the second layer LSTM analyzes the text in a time sequence from right to left, respectively extracting the semantic information of the text from two directions, thereby avoiding the loss of contextual information of the text.
5. The multi-granularity text matching model based on the combination of representation type and interaction type according to claim 1, characterized in that: The multi-granularity feature interaction layer adopts an interactive attention mechanism. Under the feature representation of the same granularity, the similarity between the corresponding segments in the two text sequences is calculated to obtain the similarity matrix of the two texts. The normalized weighted sum of the horizontal data in the matrix is the attention weight Att of the document to the query. D2Q , the normalized weighted sum of its longitudinal data is the attention weight Att of the query to the document Q2D , respectively using Att D2Q and Att Q2D The original feature matrix of the query and document is calculated to improve the weight information of important features in the text and obtain a new text feature matrix.
6. The multi-granularity text matching model based on the combination of representation and interaction according to claim 1, characterized in that: The feature fusion layer uses three methods to compare the overall text representation and the local text representation, and then fuses them together. The output of the text representation of the first text sequence at the k-granularity size after the feature fusion layer is calculated by the following formula: Among them, E represents the element-by-element calculation of the matrix. The overall representation of the text and the local representation of the text are respectively compared by matrix concatenation, matrix subtraction and matrix multiplication. The two representations of the text are compared from multiple angles to fully integrate the multi-granularity representations between the two texts.
7. The multi-granularity text matching model based on the combination of representation and interaction according to claim 1 is characterized in that: The matching prediction layer predicts the logical relationship between the two texts based on the feature vectors of the two texts. The matching prediction layer adopts the same method as the MGTA model to capture the logical relationship between the two texts by interactively enhancing the two feature vectors, as shown below: in, Represents the prediction result, L is a number of fully connected layers, ";" represents the concatenation of vectors, and "⊙" represents element multiplication.
8. The multi-granularity text matching model based on the combination of representation type and interaction type according to claim 7, characterized in that: The method of predicting logical relationship classification is used to determine whether two texts are similar or dissimilar in logical relationship. The cross entropy loss function is used, and the calculation formula is as follows:
9. A method for using a multi-granularity text matching model based on a combination of representation and interaction as described in claim 1, characterized in that: The method of use includes the following steps: A. During the model training process, a set of data is input into the left and right network structures of the twin model respectively, and the text data is converted into a text feature representation vector through the word embedding layer of the model; B. After that, the output of the embedding layer is transmitted to the multi-granularity feature representation layer and the multi-granularity feature interaction layer. In the multi-granularity feature representation layer, the text features at different granularities are globally represented. At the same time, in the multi-granularity feature interaction layer, two text features at different granularities are interactively calculated to increase the weight of the important part of the text information. C. The obtained text representation vector and the text features after the multi-granularity interaction layer are used as input and transmitted to the feature fusion layer, where the multi-granularity text representation features and the text interaction features are fused together, so that the model can capture text information at different levels to a greater extent; D. Finally, the generated text final vector representation is input into the matching prediction layer, and the matching result is obtained through calculation in the fully connected layer.