Long text matching method and device, electronic device and computer-readable storage medium
By dividing long text into subtext segments and calculating sequence weights and correlation weights using pre-trained models, the problem of information loss in long text matching is solved, and the accuracy of matching results is improved.
Patent Information
- Application Number
- CN202111119737.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-24
- Publication Date
- 2024-12-17
- Estimated Expiration
- 2041-09-24
AI Technical Summary
The existing long text matching algorithms have lost information when converting long text into short text, resulting in inaccurate matching results.
By dividing the target long text into multiple subtext segments, and obtaining the sentence vector and word vector of each subtext segment, combining pre-trained models (such as BERT, ERNIE, RoBERTa) to calculate the sequence weight and correlation weight, fully considering the text information, Finalize the matching result.
Improve the accuracy of long text matching results and solve the inaccuracy problem caused by information loss.
Smart Images
Figure CN113935312B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a long text matching method and device, an electronic device and a computer-readable storage medium. Background Art
[0002] Text matching is one of the basic tasks of natural language processing. In many NLP (Natural Language Processing) tasks, the idea of text matching can be applied to solve them, such as text retrieval, semantic recognition, language reasoning, and question-answer matching. The task goal of text matching is: given a question query and a text document, give the matching degree between the two and determine the matching relationship between the two. According to the length of the query and the document, the text matching task can be divided into long text matching and short text matching. Generally speaking, the length of the query in a specific task will not be too long, so long text matching in actual tasks mostly refers to the situation where the document is longer.
[0003] Most existing text matching tasks refer to short text matching, so the technology for short text matching is relatively mature, such as: representational text matching algorithm, interactive text matching algorithm, and text matching based on pre-trained models. The powerful encoding ability of pre-trained models can greatly improve the effect of text matching, so the text matching algorithm of pre-trained models is currently the most widely used.
[0004] When applying pre-trained models to long text matching, common approaches include:
[0005] 1. Directly truncate the long text, either at the head stage, or at the tail, or in the middle, and convert it into short text matching.
[0006] 2. Extract keywords or key sentences from long texts, obtain text consisting of keywords or key sentences, and convert them into short text matching.
[0007] 3. Divide the long text into multiple sub-text segments and match them with short texts respectively. Select the one with the highest matching degree as the matching score to determine the matching relationship.
[0008] From the above long text matching algorithms based on pre-trained models, it can be seen that the basic idea of long text matching is to convert long text into short text matching with relatively mature technology. The difference lies in how to convert long text into short text. Direct truncation and extraction of key words and phrases will cause information loss; while dividing the long text into multiple sub-text segments, using the sub-text segments for text matching respectively, and selecting the highest matching degree as the matching score, there is still the problem of information loss. Therefore, the existing long text matching algorithm has the problem of inaccurate matching results caused by information loss. Summary of the invention
[0009] The object of the present invention is to provide a long text matching method and device, an electronic device and a computer-readable storage medium to improve the accuracy of the matching results.
[0010] In a first aspect, an embodiment of the present invention provides a long text matching method, comprising:
[0011] Get the target long text and the question to be matched;
[0012] Dividing the target long text into a plurality of sub-text segments;
[0013] Obtaining the sentence vector and word vector corresponding to each of the sub-text segments, and the sentence vector and word vector corresponding to the question to be matched;
[0014] Determine, according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched, a sequence weight corresponding to each of the sub-text segments and a correlation weight between the sub-text segments and the question to be matched;
[0015] The matching result between the target long text and the question to be matched is determined according to the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched, as well as the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched.
[0016] Furthermore, the step of obtaining the sentence vector and the word vector corresponding to each of the sub-text segments, and the sentence vector and the word vector corresponding to the question to be matched, includes:
[0017] Input the multiple sub-text segments and the questions to be matched into a pre-trained encoding model to obtain sentence vectors and word vectors corresponding to each of the sub-text segments output by the encoding model, as well as sentence vectors and word vectors corresponding to the questions to be matched; the encoding model includes a BERT model, an ERNIE model or a RoBERTa model.
[0018] Furthermore, the step of determining the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched includes:
[0019] According to the order of the sub-text segments, the sentence vectors corresponding to the sub-text segments are concatenated to obtain a concatenated vector;
[0020] Inputting the concatenated vector into the BiLSTM network to obtain context sequence information of each sub-text segment;
[0021] Through a linear layer and a softmax function, the context sequence information of each sub-text segment is converted into a sequence weight corresponding to each sub-text segment.
[0022] Furthermore, the step of determining the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched, further includes:
[0023] Respectively calculating the inner product of the sentence vector corresponding to each of the sub-text segments and the sentence vector corresponding to the question to be matched, to obtain the inner product value corresponding to each of the sub-text segments;
[0024] The inner product values corresponding to the sub-text segments are normalized to obtain the correlation weight between each sub-text segment and the question to be matched.
[0025] Furthermore, the step of determining the matching result between the target long text and the question to be matched according to the sequence weight corresponding to each of the sub-text segments and the correlation weight with the question to be matched, and the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched, comprises:
[0026] According to the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched, a sub-enhanced vector corresponding to each of the sub-text segments and a sub-enhanced vector corresponding to each of the sub-text segments corresponding to the question to be matched are calculated;
[0027] According to the sequence weights corresponding to the sub-text segments, the sub-enhancement vectors corresponding to the sub-text segments are concatenated to obtain a first feature vector corresponding to the target long text;
[0028] According to the relevance weights corresponding to the sub-text segments, the sub-enhancement vectors corresponding to the question to be matched and corresponding to the sub-text segments are merged to obtain a second feature vector corresponding to the question to be matched;
[0029] The first feature vector and the second feature vector are matched to obtain a matching result between the target long text and the question to be matched.
[0030] Furthermore, the step of calculating the sub-enhanced vector corresponding to each sub-text segment and the sub-enhanced vector corresponding to each sub-text segment corresponding to the question to be matched according to the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched includes:
[0031] For each of the sub-text segments, a similarity matrix between the sub-text segment and the question to be matched is calculated based on the word vector corresponding to the sub-text segment and the word vector corresponding to the question to be matched;
[0032] Calculate the sub-enhancement vector corresponding to the sub-text segment according to the similarity matrix and the word vector corresponding to the question to be matched;
[0033] According to the similarity matrix and the word vector corresponding to the sub-text segment, a sub-enhanced vector corresponding to the sub-text segment and corresponding to the question to be matched is calculated.
[0034] Furthermore, the method further comprises:
[0035] Obtaining training samples, wherein the training samples include long text samples, question samples, and matching labels;
[0036] Dividing the long text sample into a plurality of sub-sample segments;
[0037] Inputting the plurality of sub-sample segments and the question sample into the encoding network to be trained to obtain sample matching data, wherein the sample matching data includes a sentence vector and a word vector corresponding to each of the sub-sample segments, and a sentence vector and a word vector corresponding to the question sample;
[0038] Determining a predicted matching result between the long text sample and the question sample according to the sample matching data;
[0039] The network parameters of the encoding network are adjusted according to the predicted matching result and the matching label to obtain a trained encoding model.
[0040] In a second aspect, an embodiment of the present invention further provides a long text matching device, comprising:
[0041] The first acquisition module is used to acquire the target long text and the question to be matched;
[0042] A field division module, used for dividing the target long text into a plurality of sub-text segments;
[0043] A second acquisition module is used to acquire the sentence vector and word vector corresponding to each of the sub-text segments, and the sentence vector and word vector corresponding to the question to be matched;
[0044] A weight determination module, used to determine the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched;
[0045] The text matching module is used to determine the matching result between the target long text and the question to be matched according to the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched, as well as the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched.
[0046] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the long text matching method described in the first aspect is implemented.
[0047] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the long text matching method described in the first aspect is executed.
[0048] The long text matching method and device, electronic device and computer-readable storage medium provided by the embodiment of the present invention, when matching the target long text with the problem to be matched, first divide the target long text into multiple sub-text segments, and obtain the sentence vector and word vector corresponding to each sub-text segment, as well as the sentence vector and word vector corresponding to the problem to be matched; then determine the sequence weight corresponding to each sub-text segment and the correlation weight between it and the problem to be matched according to the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the problem to be matched; and then determine the matching result of the target long text and the problem to be matched according to the sequence weight corresponding to each sub-text segment and the correlation weight between it and the problem to be matched, as well as the word vector corresponding to each sub-text segment and the word vector corresponding to the problem to be matched. In this way, when matching the target long text with the problem to be matched, the sequence weight and the correlation weight are introduced, and the various text information of the target long text is fully considered, thereby solving the problem of inaccurate matching results caused by information loss in the existing long text matching algorithm, and improving the accuracy of the matching results. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0050] Figure 1 A flowchart of a long text matching method provided by an embodiment of the present invention;
[0051] Figure 2 A schematic diagram of a flow chart of another long text matching method provided by an embodiment of the present invention;
[0052] Figure 3 A schematic diagram of obtaining sequence weights provided by an embodiment of the present invention;
[0053] Figure 4 A schematic diagram of obtaining a correlation weight provided by an embodiment of the present invention;
[0054] Figure 5 A schematic diagram of obtaining a sub-enhancement vector provided by an embodiment of the present invention;
[0055] Figure 6 A schematic diagram of a matching process provided by an embodiment of the present invention;
[0056] Figure 7 A schematic diagram of the structure of a long text matching device provided by an embodiment of the present invention;
[0057] Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0058] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0059] Currently, dividing a long text into multiple sub-text segments is a feasible approach. However, if the sub-text segments are used for text matching separately and the highest matching degree is selected as the matching score, there is still the problem of information loss, such as the context information of the sub-text segment itself. These inappropriate processing methods will bring adverse interference to the matching results. Based on this, an embodiment of the present invention provides a long text matching method and device, an electronic device, and a computer-readable storage medium. By segmenting long texts and using a twin network to obtain text encoding, the input restrictions of the model on long texts are broken, sequence weights and correlation weights are introduced, and various text information is fully considered through aggregation. This can solve the problem of inaccurate matching results caused by information loss in existing long text matching algorithms and improve the accuracy of matching results.
[0060] To facilitate understanding of this embodiment, a long text matching method disclosed in an embodiment of the present invention is first introduced in detail.
[0061] The embodiment of the present invention provides a long text matching method, which can be executed by an electronic device with data processing capability. Figure 1 The flowchart of a long text matching method shown in FIG. 1 includes the following steps:
[0062] Step S101, obtaining the target long text and the question to be matched.
[0063] Step S102: dividing the target long text into a plurality of sub-text segments.
[0064] The target long text can be segmented according to a given length and divided into several sub-text segments.
[0065] Step S103, obtaining the sentence vector and word vector corresponding to each sub-text segment, and the sentence vector and word vector corresponding to the question to be matched.
[0066] Multiple sub-text segments and questions to be matched can be input into a pre-trained encoding model to obtain sentence vectors (i.e., CLS vectors) and word vectors (i.e., word granularity vectors) corresponding to each sub-text segment output by the encoding model, as well as sentence vectors and word vectors corresponding to the questions to be matched. The encoding model has a variety of pre-trained language models to choose from, for example, the encoding model can be a BERT (Bidirectional Encoder Representations from Transformers) model, an ERNIE model, or a RoBERTa model.
[0067] Step S104, determining the sequence weight corresponding to each sub-text segment and the correlation weight between each sub-text segment and the question to be matched according to the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the question to be matched.
[0068] The weights in this embodiment include two types: one is the sequence weight between sub-text segments, and the other is the relevance weight between each sub-text segment and the question to be matched. The sequence weight reflects the contextual information between each split sub-text segment; the relevance weight reflects the interaction between the question to be matched and the target long text at the sentence-level, representing the importance of each sub-text segment to the question to be matched.
[0069] In a possible implementation, the sequence weight corresponding to each sub-text segment can be obtained through the following process: according to the order of each sub-text segment, the sentence vectors corresponding to each sub-text segment are concatenated to obtain a concatenated vector; the concatenated vector is input into the BiLSTM network to obtain the context sequence information of each sub-text segment; through the linear layer and the softmax function, the context sequence information of each sub-text segment is converted into the sequence weight corresponding to each sub-text segment.
[0070] The relevance weight between each sub-text segment and the question to be matched can be obtained through the following process: the inner product of the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the question to be matched is calculated respectively to obtain the inner product value corresponding to each sub-text segment; the inner product values corresponding to each sub-text segment are normalized to obtain the relevance weight between each sub-text segment and the question to be matched.
[0071] Step S105, determining the matching result between the target long text and the question to be matched according to the sequence weight corresponding to each sub-text segment and the correlation weight with the question to be matched, as well as the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched.
[0072] In some possible embodiments, step S105 can be implemented by the following process: (1) based on the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched, the sub-enhancement vector corresponding to each sub-text segment and the sub-enhancement vector corresponding to each sub-text segment corresponding to the question to be matched are calculated; (2) based on the sequence weights corresponding to each sub-text segment, the sub-enhancement vectors corresponding to each sub-text segment are concatenated to obtain the first feature vector corresponding to the target long text; (3) based on the relevance weights corresponding to each sub-text segment, the sub-enhancement vectors corresponding to each sub-text segment corresponding to the question to be matched are merged to obtain the second feature vector corresponding to the question to be matched; (4) the first feature vector and the second feature vector are matched to obtain the matching result of the target long text and the question to be matched.
[0073] In the above step (1), the enhancement matrix (i.e., the sub-enhancement vector) is defined as two vectors that use the similarity matrix between the two to represent the vector of the other party. The process of step (1) mainly performs the interaction between the sub-text segment and the question to be matched at the word-level. The specific implementation method of the above step (1) can be as follows: for each sub-text segment, according to the word vector corresponding to the sub-text segment and the word vector corresponding to the question to be matched, the similarity matrix between the sub-text segment and the question to be matched is calculated; according to the similarity matrix and the word vector corresponding to the question to be matched, the sub-enhancement vector corresponding to the sub-text segment is calculated; according to the similarity matrix and the word vector corresponding to the sub-text segment, the sub-enhancement vector corresponding to the question to be matched is calculated.
[0074] The specific implementation of the above step (4) can be as follows: perform pooling operations (such as maximum pooling or average pooling or use maximum pooling and average pooling at the same time) on the first feature vector and the second feature vector respectively, splice the pooling results of the two, and connect the splicing results to the linear layer for matching to obtain the matching result of the target long text and the problem to be matched. The matching result can be 0 or 1, 0 means that the target long text does not match the problem to be matched, and 1 means that the target long text matches the problem to be matched.
[0075] It should be noted that there is no specific order in which steps (2) and (3) above are performed.
[0076] The long text matching method provided by the embodiment of the present invention, when matching the target long text with the problem to be matched, first divides the target long text into multiple sub-text segments, and obtains the sentence vector and word vector corresponding to each sub-text segment, as well as the sentence vector and word vector corresponding to the problem to be matched; then, according to the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the problem to be matched, determines the sequence weight corresponding to each sub-text segment and the correlation weight between the problem to be matched; and then, according to the sequence weight corresponding to each sub-text segment and the correlation weight between the problem to be matched, as well as the word vector corresponding to each sub-text segment and the word vector corresponding to the problem to be matched, determines the matching result of the target long text and the problem to be matched. In this way, when matching the target long text with the problem to be matched, the sequence weight and the correlation weight are introduced, and the various text information of the target long text is fully considered, thereby solving the problem of inaccurate matching results caused by information loss in the existing long text matching algorithm, and improving the accuracy of the matching results.
[0077] For ease of understanding, the present invention provides a long text matching method based on the BERT twin network, which adopts the method of dividing the long text into sub-text segments, and uses the BERT model corresponding to the BERT twin network to obtain the vector representation of different sub-text segments for downstream matching tasks. In the downstream matching, the sub-text segments and the query are interacted at the sentence-level and word-level, and the bidirectional LSTM (Long Short-Term Memory) is used to ensure the context information of the sub-text segments, which solves the problem of inaccurate matching results caused by information loss in the existing long text matching process using pre-trained models.
[0078] Specifically, see Figure 2 The flowchart of another long text matching method is shown in FIG. 1 . The specific flowchart of the long text matching method is as follows:
[0079] 1. Data input: Get the target long text document and the query to be matched, split the document according to the given length, and divide it into several sub-text segments (3 sub-text segments will be used as an example later).
[0080] 2. Weight acquisition:
[0081] (1) Sequence weights between sub-text segments: Figure 3 As shown in the figure, each sub-text segment is encoded by the BERT twin network to obtain the CLS vector output by the BERT model: CLS1, CLS2, CLS3. The CLS vector can represent the overall semantics of the sub-text segment. The CLS vectors corresponding to each sub-text segment are concatenated, and the context sequence information of each sub-text segment is obtained through BILSTM (i.e. bidirectional LSTM), and then converted into sequence weights: v1, v2, v3 by the linear layer and softmax function.
[0082] (2) The relevance weight of the sub-text segment and the query: Figure 4 As shown in the figure, first obtain the CLS vectors of each sub-text segment encoded by the BERT model: CLS1, CLS2, CLS3, and at the same time obtain the CLS vector of the query encoded by the BERT model: CLSq; then calculate the inner products of CLS1, CLS2, CLS3 and CLSq respectively, and normalize them to obtain the correlation weights: w1, w2, w3.
[0083] 3. Enhanced vector (taking sub-text segment 1 as an example to introduce the process of obtaining the enhanced vector):
[0084] like Figure 5As shown, the word vectors s1 and sq of sub-text segment 1 and query encoded by the BERT model are obtained respectively, and the similarity matrix e1 is calculated according to the following formula:
[0085] e1=s1·sq T .
[0086] The enhanced vector s1z corresponding to sub-text segment 1 is calculated from the similarity matrix e1 and the word vector sq, and the enhanced vector sqz1 corresponding to the query is calculated from the similarity matrix e1 and the word vector s1:
[0087] s1z=softmax(e1)·sq;
[0088] sqz1=softmax(e1 T )·s1.
[0089] Similarly, the enhanced vectors s2z and s3z corresponding to sub-text segment 2 and sub-text segment 3, and the enhanced vectors sqz2 and sqz3 corresponding to the query, which correspond to sub-text segment 2 and sub-text segment 3, respectively, can be obtained.
[0090] 4. Aggregation: The aggregation process is mainly the merging and splicing of enhanced vectors, including the merging of several (3 in this case) enhanced vectors corresponding to the query, and the splicing of several (3 in this case) enhanced vectors corresponding to the sub-text segments, in order to integrate the information of each part. This step requires the obtained relevance weight and sequence weight. The specific merging and splicing process is as follows:
[0091] Merge: sqz = w1 × sqz1 + w2 × sqz2 + w3 × sqz3;
[0092] Concatenation: sz = concat([v1×s1z, v2×s2z, v3×s3z]).
[0093] 5. Matching: sqz and sz obtained by the aggregation process represent the feature vectors of the query and the target long text after interaction at two levels, respectively, and are used in the matching process. Specifically, Figure 6 As shown in FIG. 1 , the maximum pooling operation is performed on sqz and sz respectively, and then the pooling results of the two are concatenated and connected to the linear layer for matching to obtain the matching result.
[0094] The embodiment of the present invention provides a long text matching method based on the BERT twin network. The method adopts a method of segmenting the long text and uses the BERT model corresponding to the BERT twin network to obtain the encoding of the sub-text segment and the query, so that the model is no longer limited by the length of the text. In order to make full use of the information and sequence information of each sub-text segment, the full interaction with the query at the sentence-level and word-level is added, and the sub-text segment and the query are aggregated using weight information. This method can effectively improve the accuracy of long text matching.
[0095] In addition, the embodiment of the present invention also provides a training process of the above coding model, as follows:
[0096] (1) Obtain training samples, which include long text samples, question samples and matching labels, where the matching labels are 0 or 1; (2) Divide the long text samples into multiple sub-sample segments; (3) Input the multiple sub-sample segments and the question samples into the encoding network to be trained to obtain sample matching data, which includes the sentence vector and word vector corresponding to each sub-sample segment, and the sentence vector and word vector corresponding to the question sample; (4) Determine the predicted matching results of the long text sample and the question sample based on the sample matching data; (5) Adjust the network parameters of the encoding network based on the predicted matching results and the matching labels to obtain a trained encoding model.
[0097] For the parts of the above training process that are not described in detail, reference can be made to the corresponding contents of the aforementioned embodiments and will not be repeated here.
[0098] Corresponding to the above-mentioned long text matching method, the embodiment of the present invention also provides a long text matching device, see Figure 7 A schematic diagram of the structure of a long text matching device is shown, the device comprising:
[0099] A first acquisition module 71 is used to acquire a target long text and a question to be matched;
[0100] A field division module 72, used for dividing the target long text into a plurality of sub-text segments;
[0101] A second acquisition module 73, used to acquire the sentence vector and word vector corresponding to each sub-text segment, and the sentence vector and word vector corresponding to the question to be matched;
[0102] A weight determination module 74 is used to determine the sequence weight corresponding to each sub-text segment and the correlation weight between the sub-text segment and the question to be matched according to the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the question to be matched;
[0103] The text matching module 75 is used to determine the matching result between the target long text and the question to be matched based on the sequence weight corresponding to each sub-text segment and the correlation weight between the sub-text segment and the question to be matched, as well as the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched.
[0104] The long text matching device provided by the embodiment of the present invention, when matching the target long text with the problem to be matched, first divides the target long text into multiple sub-text segments, and obtains the sentence vector and word vector corresponding to each sub-text segment, as well as the sentence vector and word vector corresponding to the problem to be matched; then, according to the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the problem to be matched, determines the sequence weight corresponding to each sub-text segment and the correlation weight between it and the problem to be matched; and then, according to the sequence weight corresponding to each sub-text segment and the correlation weight between it and the problem to be matched, as well as the word vector corresponding to each sub-text segment and the word vector corresponding to the problem to be matched, determines the matching result of the target long text and the problem to be matched. In this way, when matching the target long text with the problem to be matched, the sequence weight and the correlation weight are introduced, and the various text information of the target long text is fully considered, thereby solving the problem of inaccurate matching results caused by information loss in the existing long text matching algorithm, and improving the accuracy of the matching results.
[0105] Furthermore, the above-mentioned second acquisition module 73 is specifically used to: input multiple sub-text segments and questions to be matched into a pre-trained encoding model to obtain sentence vectors and word vectors corresponding to each sub-text segment output by the encoding model, as well as sentence vectors and word vectors corresponding to the questions to be matched; the encoding model includes a BERT model, an ERNIE model or a RoBERTa model.
[0106] Furthermore, the weight determination module 74 is specifically used to: concatenate the sentence vectors corresponding to each sub-text segment according to the order of each sub-text segment to obtain a concatenated vector; input the concatenated vector into the BiLSTM network to obtain the context sequence information of each sub-text segment; and convert the context sequence information of each sub-text segment into a sequence weight corresponding to each sub-text segment through a linear layer and a softmax function.
[0107] Furthermore, the weight determination module 74 is also used to: respectively calculate the inner product of the sentence vector corresponding to each sub-text segment and the sentence vector corresponding to the question to be matched, to obtain the inner product value corresponding to each sub-text segment; and normalize the inner product values corresponding to each sub-text segment to obtain the correlation weight between each sub-text segment and the question to be matched.
[0108] Furthermore, the above-mentioned text matching module 75 is specifically used to: calculate the sub-enhancement vector corresponding to each sub-text segment and the sub-enhancement vector corresponding to the question to be matched according to the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched; splice the sub-enhancement vectors corresponding to each sub-text segment according to the sequence weights corresponding to each sub-text segment to obtain the first feature vector corresponding to the target long text; merge the sub-enhancement vectors corresponding to each sub-text segment corresponding to the question to be matched according to the relevance weights corresponding to each sub-text segment to obtain the second feature vector corresponding to the question to be matched; match the first feature vector and the second feature vector to obtain the matching result of the target long text and the question to be matched.
[0109] Furthermore, the text matching module 75 is also used to: for each sub-text segment, calculate the similarity matrix between the sub-text segment and the question to be matched based on the word vector corresponding to the sub-text segment and the word vector corresponding to the question to be matched; calculate the sub-enhancement vector corresponding to the sub-text segment based on the similarity matrix and the word vector corresponding to the question to be matched; calculate the sub-enhancement vector corresponding to the sub-text segment corresponding to the question to be matched based on the similarity matrix and the word vector corresponding to the sub-text segment.
[0110] Furthermore, the above-mentioned device also includes a model training module connected to the second acquisition module 73, and the model training module is used to: obtain training samples, which include long text samples, question samples and matching labels; divide the long text samples into multiple sub-sample segments; input the multiple sub-sample segments and the question samples into the encoding network to be trained to obtain sample matching data, and the sample matching data includes the sentence vector and word vector corresponding to each sub-sample segment, and the sentence vector and word vector corresponding to the question sample; determine the predicted matching results of the long text sample and the question sample according to the sample matching data; adjust the network parameters of the encoding network according to the predicted matching results and the matching labels to obtain the trained encoding model.
[0111] The implementation principle and technical effects of the device provided in this embodiment are the same as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0112] See also Figure 8 An embodiment of the present invention further provides an electronic device 100, comprising: a processor 80, a memory 81, a bus 82 and a communication interface 83, wherein the processor 80, the communication interface 83 and the memory 81 are connected via the bus 82; the processor 80 is used to execute an executable module stored in the memory 81, such as a computer program.
[0113] The memory 81 may include a random access memory (RAM) and may also include a non-volatile memory (NVM), such as at least one disk storage. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 83 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0114] The bus 82 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0115] Among them, the memory 81 is used to store programs, and the processor 80 executes the program after receiving the execution instruction. The method executed by the process definition device disclosed in any embodiment of the above-mentioned embodiment of the present invention can be applied to the processor 80 or implemented by the processor 80.
[0116] The processor 80 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 80. The above processor 80 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 81, and the processor 80 reads the information in the memory 81 and completes the steps of the above method in combination with its hardware.
[0117] The embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the long text matching method described in the above method embodiment is executed. The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk or an optical disk, and other media that can store program codes.
[0118] In all examples shown and described herein, any specific values should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.
[0119] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the method and computer program product according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.
[0120] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.
[0121] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0122] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0123] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. A long text matching method, It is characterized in that include: Get the target long text and the question to be matched; Dividing the target long text into a plurality of sub-text segments; Obtaining the sentence vector and word vector corresponding to each of the sub-text segments, and the sentence vector and word vector corresponding to the question to be matched; Determine, according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched, a sequence weight corresponding to each of the sub-text segments and a correlation weight between the sub-text segments and the question to be matched; Determine a matching result between the target long text and the question to be matched according to a sequence weight corresponding to each of the sub-text segments and a correlation weight with the question to be matched, as well as a word vector corresponding to each of the sub-text segments and a word vector corresponding to the question to be matched; The step of determining the matching result between the target long text and the question to be matched according to the sequence weight corresponding to each of the sub-text segments and the correlation weight with the question to be matched, as well as the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched, comprises: calculating the sub-enhancement vector corresponding to each of the sub-text segments and the sub-enhancement vector corresponding to the question to be matched according to the word vector corresponding to each of the sub-text segments and the word vector corresponding to the question to be matched; splicing the sub-enhancement vectors corresponding to each of the sub-text segments according to the sequence weights corresponding to each of the sub-text segments to obtain the first feature vector corresponding to the target long text; merging the sub-enhancement vectors corresponding to each of the sub-text segments according to the correlation weights corresponding to each of the sub-text segments to obtain the second feature vector corresponding to the question to be matched; matching the first feature vector and the second feature vector to obtain the matching result between the target long text and the question to be matched; The step of calculating the sub-enhancement vector corresponding to each sub-text segment and the sub-enhancement vector corresponding to the question to be matched according to the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched comprises: for each sub-text segment, calculating the similarity matrix between the sub-text segment and the question to be matched according to the word vector corresponding to the sub-text segment and the word vector corresponding to the question to be matched; calculating the sub-enhancement vector corresponding to the sub-text segment according to the similarity matrix and the word vector corresponding to the question to be matched; and calculating the sub-enhancement vector corresponding to the sub-text segment according to the similarity matrix and the word vector corresponding to the sub-text segment.
2. The long text matching method according to claim 1, It is characterized in that The step of obtaining the sentence vector and the word vector corresponding to each of the sub-text segments, and the sentence vector and the word vector corresponding to the question to be matched, comprises: Input the multiple sub-text segments and the questions to be matched into a pre-trained encoding model to obtain sentence vectors and word vectors corresponding to each of the sub-text segments output by the encoding model, as well as sentence vectors and word vectors corresponding to the questions to be matched; the encoding model includes a BERT model, an ERNIE model or a RoBERTa model.
3. The long text matching method according to claim 1, It is characterized in that The step of determining the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched comprises: According to the order of the sub-text segments, the sentence vectors corresponding to the sub-text segments are concatenated to obtain a concatenated vector; Inputting the concatenated vector into the BiLSTM network to obtain context sequence information of each sub-text segment; Through a linear layer and a softmax function, the context sequence information of each sub-text segment is converted into a sequence weight corresponding to each sub-text segment.
4. The long text matching method according to claim 1, It is characterized in that The step of determining the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched, further includes: Respectively calculating the inner product of the sentence vector corresponding to each of the sub-text segments and the sentence vector corresponding to the question to be matched, to obtain the inner product value corresponding to each of the sub-text segments; The inner product values corresponding to the sub-text segments are normalized to obtain the correlation weight between each sub-text segment and the question to be matched.
5. The long text matching method according to claim 2, It is characterized in that The method further comprises: Obtaining training samples, wherein the training samples include long text samples, question samples, and matching labels; Dividing the long text sample into a plurality of sub-sample segments; Inputting the plurality of sub-sample segments and the question sample into the encoding network to be trained to obtain sample matching data, wherein the sample matching data includes a sentence vector and a word vector corresponding to each of the sub-sample segments, and a sentence vector and a word vector corresponding to the question sample; Determining a predicted matching result between the long text sample and the question sample according to the sample matching data; The network parameters of the encoding network are adjusted according to the predicted matching result and the matching label to obtain a trained encoding model.
6. A long text matching device, It is characterized in that include: The first acquisition module is used to acquire the target long text and the question to be matched; A field division module, used for dividing the target long text into a plurality of sub-text segments; A second acquisition module is used to acquire the sentence vector and word vector corresponding to each of the sub-text segments, and the sentence vector and word vector corresponding to the question to be matched; A weight determination module, used to determine the sequence weight corresponding to each of the sub-text segments and the correlation weight between the sub-text segments and the question to be matched according to the sentence vectors corresponding to the sub-text segments and the sentence vectors corresponding to the question to be matched; A text matching module, used for determining a matching result between the target long text and the question to be matched according to a sequence weight corresponding to each of the sub-text segments and a correlation weight with the question to be matched, as well as a word vector corresponding to each of the sub-text segments and a word vector corresponding to the question to be matched; The text matching module is specifically used to: calculate the sub-enhancement vector corresponding to each sub-text segment and the sub-enhancement vector corresponding to each sub-text segment and the sub-enhancement vector corresponding to the question to be matched according to the word vector corresponding to each sub-text segment and the word vector corresponding to the question to be matched; concatenate the sub-enhancement vectors corresponding to each sub-text segment according to the sequence weights corresponding to each sub-text segment to obtain the first feature vector corresponding to the target long text; merge the sub-enhancement vectors corresponding to each sub-text segment corresponding to the question to be matched according to the relevance weights corresponding to each sub-text segment to obtain the second feature vector corresponding to the question to be matched; match the first feature vector and the second feature vector to obtain the matching result of the target long text and the question to be matched; The above-mentioned text matching module is also used to: for each sub-text segment, calculate the similarity matrix between the sub-text segment and the question to be matched based on the word vector corresponding to the sub-text segment and the word vector corresponding to the question to be matched; calculate the sub-enhancement vector corresponding to the sub-text segment based on the similarity matrix and the word vector corresponding to the question to be matched; calculate the sub-enhancement vector corresponding to the sub-text segment corresponding to the question to be matched based on the similarity matrix and the word vector corresponding to the sub-text segment.
7. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, It is characterized in that When the processor executes the computer program, the method according to any one of claims 1 to 5 is implemented.
8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is performed.