A long text matching method combining noise filtering and divide-and-conquer strategy

Abstract

This invention discloses a long text matching method combining noise filtering and a divide-and-conquer strategy. The method includes: constructing a long text matching model, which comprises a keyword extraction layer, an association extraction layer, and a filtering layer. The keyword extraction layer extracts keywords from the text; the filtering layer filters noise from the text based on sentence similarity to obtain a denoised text sequence; and the association extraction layer further removes keywords from the denoised text sequence to obtain the remaining associated text. The long text matching model is trained with the optimization objective of minimizing a set overall loss function, which reflects the global matching distribution and combines the keyword and association matching distributions. For the target text, real-time text matching is performed using the trained long text matching model. This invention improves the generalization ability and accuracy of text matching.

Description

Technical Field

[0001] This invention relates to the field of text matching technology, and more specifically, to a long text matching method that combines noise filtering and a divide-and-conquer strategy. Background Technology

[0002] Most current mainstream text matching models are designed for short texts and cannot effectively solve the problem of matching long texts. Compared with short texts, long texts have richer semantics and more complex logical relationships. Therefore, research on long text matching is extremely necessary.

[0003] In existing technologies, text matching schemes include the Match-Ignition method (Pang L, Lan Y, Cheng X. Match-ignition: Plugging pagerank into transformer for long-form textmatching [C] / / Proceedings of the 30th ACM International Conference on Information & Knowledge Management. 2021: 1396-1405) and the Divide and Conquer method (Zou Y, Liu H, Gui T, et al. Divide and Conquer: Text Semantic Matching with Disentangled Keywords and Intents [J]. arXiv preprint arXiv: 2203.02898, 2022.).

[0004] The main idea of ​​the Match-Ignition method is to insert the well-known PageRank algorithm into the Transformer to identify and filter sentence-level and word-level noise during the matching process. Noisy sentences are usually easy to detect because existing research shows that their similarity can be explicitly evaluated through word overlap, so PageRank can be directly used to filter this information based on the sentence similarity graph. However, this approach overemphasizes noise filtering, filtering sentence-level noise first and then word-level noise, ignoring the different levels of matching granularity contained within long sentences.

[0005] In the Divide and Conquer method, queries typically contain content requiring different levels of matching granularity. Specifically, keywords represent factual information such as behaviors, entities, and events that should be strictly matched, while intent conveys abstract concepts and ideas that can be expressed in various ways. This work proposes a simple yet effective text semantic matching training strategy that separates keywords from intent using a divide-and-conquer approach. However, this scheme is only suitable for short text matching and cannot be well applied to long text matching. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a long text matching method that combines noise filtering and a divide-and-conquer strategy. This method includes the following steps:

[0007] A long text matching model is constructed, which includes a keyword extraction layer, an association extraction layer, and a filtering layer. The keyword extraction layer is used to extract keywords from the text, the filtering layer is used to filter noise in the text based on the similarity between sentences to obtain a denoised text sequence, and the association extraction layer is used to remove keywords from the denoised text sequence to obtain the remaining associated text.

[0008] The long text matching model is trained with the optimization objective of minimizing the set overall loss function, which reflects the global matching distribution and combines the keyword and association matching distributions.

[0009] For the target text, real-time text matching is performed using the trained long text matching model.

[0010] Compared with existing technologies, the advantages of this invention lie in providing a long text matching method that combines noise filtering and a divide-and-conquer strategy. It first extracts keywords from long documents to avoid filtering out key information, and proposes a divide-and-conquer approach by performing sentence-level noise filtering on the long document before matching keywords and associations separately. Furthermore, the divide-and-conquer strategy utilizes gates, enabling the model to better understand which keywords and associations are more important during prediction, effectively improving the accuracy of long text matching.

[0011] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0013] Figure 1 This is a flowchart of a long text matching method combining noise filtering and a divide-and-conquer strategy according to an embodiment of the present invention.

[0014] Figure 2 This is an overall framework diagram of a long text matching model that combines noise filtering and a divide-and-conquer strategy according to an embodiment of the present invention.

[0015] Figure 3 This is a state transition diagram of a deterministic finite automaton according to an embodiment of the present invention. Detailed Implementation

[0016] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0017] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0018] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0019] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0020] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0021] This invention proposes a long text matching method combining noise filtering and a divide-and-conquer strategy (LMNFDC model). In short, given a pair of long text input sequences, the model first extracts keywords from the long text using keyword extraction techniques and then filters noise from the long text using a graph structure method. Next, for the noise-filtered text, standard global matching is performed, and information other than keywords (referred to as associations in this paper) is retrieved based on the extracted keywords. Keywords and associations are then matched separately. Finally, the KL divergence between the global matching distribution and the combined keyword and association matching distribution is minimized to ensure that the distributions of the combined solutions to the original problem and its subproblems are similar.

[0022] See Figure 1 As shown, the long text matching method combining noise filtering and a divide-and-conquer strategy includes the following steps:

[0023] Step S110: Construct a long text matching model, which includes a keyword extraction layer, a filtering layer, and an association extraction layer.

[0024] In one embodiment, the framework of the proposed long text matching model LMNFDC, which combines noise filtering and a divide-and-conquer strategy, is as follows: Figure 2 As shown, the LMNFDC model as a whole includes a keyword extraction layer, a filtering layer, and an association extraction layer.

[0025] 1) Keyword extraction layer

[0026] For example, the keyword extraction layer uses the TextRank algorithm, which can obtain the keywords of the text based solely on the information in the text corpus itself.

[0027] Specifically, firstly, a set of words extracted according to certain rules is used as nodes, and the relationships between words are used as edges. For example, co-occurrence is chosen as the standard for the relationship between words, that is, if two nodes appear in the same window, then there is an edge between the two words. Finally, a word graph is constructed for each sentence. The TextRank method uses the concept of network graph to construct a word graph G(V, E), where the set of words is V and the set of links between words is E. The formal description of this method is as shown in formula (1):

[0028]

[0029] Wherein, T(v) i ) represents node v i TextRank value, T(v j ) represents node v j The TextRank value, where d is the damping coefficient, for example, d is set to 0.85; In(v i ) is v i The in-degree represents the direction pointing to node v. i A set of nodes; Out(v j ) is v j The out-degree represents the node v j The set of nodes it points to; w ji Represents node v j to v i The weight of an edge is the number of times the edge appears, w. jk It is node v j to node v k The weights of the edges between nodes are determined; finally, the nodes are sorted by score, and the number of keywords can be 1 / 3 of the number of nodes, which is used as input for the subsequent pre-trained model.

[0030] 2) Filter layer

[0031] For example, the filtering layer uses the PageRank graph ranking algorithm. First, a graph is constructed to represent the relationships between sentences, where each sentence in the document is a node, and the similarity (Sim) between sentences is the weight of the edges, measured by word overlap. Then, two long text segments T to be matched are... m and T n Divide into single sentences, using T m = [m1, m2, ..., m L1 ], T n = [n1, n2, ..., n L2 ] indicates that L1 and L2 are respectively T m and T The number of sentences in n forms a set T = {m1, m2, ..., m}. L1 n1, n2, ..., n L2 By evaluating the sentence similarity in the sentence set T, a sentence similarity graph is constructed to measure the proportion of repeated words in two sentences, as shown in formula (2):

[0032]

[0033] Among them, w k |·| represents the words in a sentence, and |·| represents the length of the sentence or set of words, m. i n j These are two sentences in a joint sentence set T. Since sentence extraction uses a weighted graph, the definition of sentence similarity is the same as in TextRank. Therefore, the formula for calculating sentence similarity in PageRank is:

[0034]

[0035] Among them, WT(v j ) represents node v j TextRank value, WT(v i ) represents node v i The TextRank value, d is the damping coefficient, In(v i ) represents a pointer to node v i A set of nodes, Out(v j ) is node v j The set of nodes pointed to, w ji Represents node v j to v i The weight of the edge between the two nodes, i.e., the similarity Sim(v) between the two nodes. j v iTo ensure that the sentence similarity graph has sparse links, stop words in the sentences are removed before calculating similarity. Finally, the PageRank algorithm is applied to construct the sentence similarity graph to obtain the importance score of each sentence. Then, the sentences with the highest importance score (λ) are concatenated into a text sequence, for example, starting with [CLS] and separating with [SEP], which is used as the input to the subsequent pre-trained model.

[0036] 3) Association Extraction Layer

[0037] For example, the association extraction layer uses a deterministic finite automaton (DFA) algorithm to extract the remaining text after filtering and keyword removal, referred to as association in this paper. The DFA algorithm is driven by the text to be searched, where "deterministic" means that the state and the event causing the state transition are deterministic, with no other possibilities; and "finite" means that the number of states and events is exhaustive. DFA can be used to implement text filtering, its idea being that given a state and an input, the next deterministic output state can be reached. The mathematical model of DFA contains five elements, represented as follows:

[0038] M DFA =S,∑,δ,S0,F (4)

[0039] Where S represents a finite non-empty set of states, ∑ is a finite input table consisting of 256 characters, and δ is M. DFA The transformation function from S×∑ to S is given by S0∈S, where S0∈S represents the initial state and F∈S represents the set of final states.

[0040] The core of the DFA algorithm is the transition function δ, which defines a given series of states that, when a specific input character is triggered, guarantees only one given subsequent state. Its mathematical expression is:

[0041] event + state = next state (5)

[0042] Here, event represents an event (such as an input character event), state represents the current state, and nextstate represents the next state.

[0043] Figure 3This is a diagram illustrating the state transition principle of a Dependency-Assisted Graph (DFA), where (S, U, V, Q) are states, and a and b are actions. State S and action a transition to U, i.e., S→a→U. Each input corresponds to only one state, and each character is checked only once. The DFA algorithm improves extraction efficiency and is very suitable for keyword matching and replacement. In one embodiment, the data structure of the DFA search tree is first determined, and all starting nodes are stored in a HashMap data structure. Then, all words in the keyword dictionary are retrieved, and each word is iterated once. Finally, multiple search trees are constructed sequentially according to the sentence order. After the DFA search tree is built, character matching is performed. The first node of the sentence is matched with the search tree, and the matched keyword is replaced with the symbol "[mask]" until the child node is null.

[0044] Step S120: Train a long text matching model based on a set loss function, which reflects the global matching distribution and combines keyword and association matching distributions.

[0045] For example, set two training objectives: the classification loss of the standard matching model and the KL-divergence loss following a divide-and-conquer strategy.

[0046] 1) Text semantic matching based on Roberta

[0047] For example, using the Roberta model for text semantic matching can be viewed as a sentence pair classification task. For instance, for sentences a and b, using S... a and S b S represents the semantic vector representation of the two sentences respectively. a ={w a1 w a2 , ..., w ai}, S b ={w b1 w b2 ,…,w bi The usual approach is to use the semantic representation of the sentences to predict whether sentences a and b are semantically similar, i.e., y = ξ(S a S b ), where w ai Sentence S a The i-th token in the string, w bi Sentence S bThe i-th token in the input is given as [CLS]a[SEP]b[SEP]. In actual input, it is fed into the model in the form [CLS]a[SEP]b[SEP]. The Roberta pre-trained language model belongs to the MLM language model category and has a bidirectional attention mechanism, allowing tokens to pay attention to each other. [CLS], as the first token of the sentence, possesses the semantic information of the entire sentence, and since [CLS] is a self-created token, it does not have any real semantic information. Therefore, in S... a and S b In this context, the vector at position [CLS] is typically used to predict y. However, considering that [CLS] is a global token and relatively simple, it lacks sufficient semantic representation of the sentence. Therefore, in this embodiment of the invention, the vector at position [SEP] is added for prediction. The average of the two vectors predicts y, as shown below:

[0048]

[0049] Where H cls H is the vector at position [CLS]. sep Let L be the vector at position [SEP], and L represent the sentence length. This yields the semantic representation S of the sentence. cls+sep Then, it is mapped to the corresponding category label, and then Softmax is performed to obtain the probability of the corresponding label. The formula is expressed as:

[0050] P(y|S a S b ) = Softmax(S cls+sep ·W T (7)

[0051] Where W∈R KxH R represents the set of real numbers, K represents the number of labels in the category, and H represents the vector dimension of the hidden layer. Finally, by fine-tuning the Roberta pre-trained language model, the standard cross-entropy loss is calculated, expressed as:

[0052] L cls+sep =-logP(y|S a S b (8)

[0053] Among them, L cls+sep It is the classification loss value.

[0054] 2) Divide and conquer matching strategy

[0055] To achieve a deeper semantic representation of the sentence, in one embodiment, the original problem is divided into two easier sub-problems: keywords and associations. By extracting the keywords and associations from the sentence and then merging the two sub-problems, the semantic representation of the sentence is enhanced. Segmenting the sentence into keywords and associations means dividing the original problem into two parts: (1) classifying the entire sentence; and (2) merging the keywords and associations after each classification. As mentioned above, this paper uses CLS+SEP to represent the entire sentence, and the final calculation involves classifying and merging the keywords and associations.

[0056] Specifically, the classification and merging of keywords and associations is similar to the overall sentence classification, using vectors at the corresponding positions of keywords and associations for result prediction. Considering the varying importance of different keywords and associations, the model needs to be able to understand which keywords and associations are more important during prediction. Therefore, in one embodiment, a gating unit is designed for control. For example, a cls vector H is used. cls As a query, the associated vector H it and keyword vector H kw The combined vector serves as the key. A query is then used to retrieve the importance score of the keyword and related terms. This score represents the gate of the keyword and related terms, and the specific formula is as follows:

[0057] Gate it =H cls ·H it Gate it ∈R Lit×1 (9)

[0058] Gate kw =H cls ·H kw Gate kw ∈R Lkw×1 (10)

[0059] H IT =Gate it ·H it (11)

[0060] H KW =Gate kw ·H it (12)

[0061] P(y|S a S b =Softmax(H·W) T ), H∈(H IT H KW (13)

[0062] Among them, Gate it The key represents the importance score of the related terms, Lit represents the length of the related terms, Lkw represents the length of the keywords, and Gate represents the importance score of the related terms. kw H represents the keyword importance score. IT H represents the hidden layer vector of the last layer of the conjunctions. KW This represents the hidden layer vector of the last layer of keywords. H represents the vector symbol, which is not specific. In the formula, H represents H1. KW Or H IT W represents the learnable parameters of the model, used to transform the vector after softmax.

[0063] Finally, the joint probability of keywords and associations is:

[0064]

[0065] Among them, y kw y represents the probability of a keyword. it The probability of a conjunction is expressed as follows: This represents the semantic vector representation of the keywords in sentence a. Let y represent the semantic vector representation of the keywords in sentence b. it Indicates the probability of conjunctions. This represents the semantic vector representation of the conjunctions in sentence a. The semantic vector representation of the conjunctions in sentence b.

[0066] The above describes the combined distribution of keywords and related terms.

[0067] Since both subproblems follow the same objective, we can determine the outcome from the joint probability distribution P(y) of the two subproblems. i y j ), and the probability distribution relationship Q(y) after their combination is obtained, which is expressed as:

[0068]

[0069] Among them, c m c n The target category represents the degree of matching, if c m >c n , representing c m The matching degree is higher than c n For example, if the target label is [0, 1] and the category is 1, it means a match; if the category is 0, it means a no match.

[0070] In one embodiment, a method is employed to reduce the bidirectional KL-divergence between the two distributions to ensure that the overall matching distribution (i.e., the original problem) is similar to the combined distribution of solutions to the subproblems, thereby enhancing the learning of the global matching model, as expressed in:

[0071]

[0072] Among them, L 联合 D represents the loss value of the joint distribution of keywords and related terms. KL This represents the KL divergence.

[0073] Finally, the loss function L cls+sep and L 联合 Together, they can be used to train the model, as shown below:

[0074] L = L cls+sep +L 联合 (17)

[0075] Where L represents the total loss value.

[0076] Step S130: For the target long text, use the trained long text matching model to perform matching.

[0077] Once the long text matching model is trained, it can be used for real-time text matching. It should be understood that the training process of the long text matching model involved in this invention can be performed offline on a server or in the cloud. The trained model can be embedded into an electronic device to achieve real-time long text matching. This electronic device can be a terminal device or a server. Terminal devices include any terminal device such as mobile phones, tablets, personal digital assistants (PDAs), point-of-sale (POS) terminals, in-vehicle computers, and smart wearable devices. Servers include, but are not limited to, application servers or web servers, and can be independent servers, cluster servers, or cloud servers.

[0078] To further verify the effectiveness of this invention, experiments were conducted on two Chinese long-text datasets: the Chinese News Same Events Dataset (CNSE) and the Chinese News Same Reports Dataset (CNSS). The CNSE dataset contains 29,063 news pairs, with articles sourced from long-form reports by major domestic online media outlets and manually labeled based on whether a set of news articles belonged to the same event. Similarly, the CNSS dataset contains 33,503 news pairs, with articles also sourced from long-form reports by major domestic online media outlets and manually labeled based on whether a set of news reports belonged to popular information on a specific topic.

[0079] The average number of characters in the two sets of long texts is 734, with a maximum of 21,791. The split ratio of each data set remains at 3:1:1 (training set:validation set:test set), as shown in Table 1. This dataset has the following characteristics:

[0080] (1) Label: A label manually marked to determine whether they are similar, with a content of 0 or 1;

[0081] (2) doc_id1 and doc_id2: the length of the news text;

[0082] (3) title1 and title2: the titles of the news text;

[0083] (4) content1 and content2: The specific content of the news text;

[0084] (5) keyword1 and keyword2: keywords in news reports;

[0085] (6) Category 1 and Category 2: The category to which the news text belongs, covering multiple fields such as politics, sports, and entertainment;

[0086] (7) time1 and time2: timestamps of news releases.

[0087] Using label and content tags as input to our model, although the dataset provides keywords, the coverage is limited, and some keywords are redundant and cannot be directly used. When constructing negative samples, we select document pairs containing similar keywords, and the similarity in TF-IDF must be greater than a certain threshold, thus increasing the difficulty of evaluating pattern recognition.

[0088] Table 1 shows the partitioning of the two datasets.

[0089]

[0090] The experimental results were evaluated using two commonly used methods: accuracy (Acc) and F1 score. Accuracy (Acc) represents the proportion of correctly predicted samples in the total sample. The F1 score considers both precision (Pre) and recall (R). Precision (Pre) represents the percentage of correctly predicted positive samples, while recall (R) represents the percentage of correctly predicted samples with a true positive label. The relevant calculation formulas are as follows:

[0091]

[0092]

[0093]

[0094]

[0095] Wherein, FP represents the total number of samples with negative true labels and positive predictions, TP represents the total number of samples with positive true labels and positive predictions, TN represents the total number of samples with negative true labels and negative predictions, and FN represents the total number of samples with positive true labels and negative predictions.

[0096] The baseline of this invention includes traditional methods for similarity matching based on terms, representation-based models, interaction-based models, hybrid models, and pre-trained models. The experiment selected several of the better modes for comparison and analysis.

[0097] 1) Traditional methods based on terminology

[0098] BM25: The BM25 algorithm is a classic probabilistic detection algorithm, often used to search for relevance scores and calculate the relevance of a sentence to each document in a document set.

[0099] LDA (Learning-Derivative-Action) is a probabilistic topic model, a three-layer Bayesian probabilistic generative model that considers documents, topics, and words. Its main idea is to view long documents as a probability distribution of latent topics, and topics as a probability distribution of words.

[0100] SimNet's encoding method involves concatenating a series of feature vectors based on probabilistic statistical models, such as TF, TF-IDF, and BM25, and then using a classifier to obtain the prediction results.

[0101] PGBN is a hierarchical topic model that uses bag-of-words (BOW) features to extract the proportion of multiple topics for performing classification, thereby obtaining sentence semantics at multiple levels. Hierarchical topics are interpreted as cluster centers of sentences.

[0102] 2) Representation-based methods:

[0103] DSSM: This method maps two texts to a semantic space of the same dimension and then calculates semantic similarity using cosine distance to predict the final result.

[0104] C-DSSM: This method first introduces a graph convolution algorithm to extract local information, and then uses max pooling to extract global information. The method for calculating semantic similarity is the same as DSSM.

[0105] ARC-I: The main idea is to use graph convolutional networks to extract features from the text to be matched, and then use a multilayer perceptron to obtain sentence similarity.

[0106] SMASH: The encoding method uses different attention mechanisms to represent the document at three levels: word, sentence, and document, and uses a classifier model to predict the final result.

[0107] 3) Interactive approach:

[0108] ARC-II: This method models the data by applying convolutional networks multiple times. Each word in the two sentences to be matched is represented by a word vector. The word vectors from both sentences are then combined pairwise to form a matrix. Repeated convolution operations are then used to obtain a vector representing the overall correlation between the two sentences. These vectors are then merged into a multi-level perceptron to obtain the final result. The difference between this method and ARC-I is that it considers the interaction between the two texts, thus improving the performance.

[0109] MatchPyramid: This method uses a two-dimensional convolutional neural network to extract features from each document, and then uses dot products to construct a word-level interactive matrix. Using this matrix as input, the final prediction value is obtained.

[0110] CIG was proposed by Liu et al. in 2019. This method uses "concept" as a node to conceptualize the document, uses the construction of Siamese networks to perform similarity matching, and uses classification methods to obtain the final predicted value.

[0111] 4) Hybrid-based methods

[0112] DUET: This model enables web search by learning to use both local and distributed representations of text.

[0113] RE2: This method uses an attention mechanism to generate a feature representation for each document, aligns and fuses the feature representations, so that the final feature expression has interactive information.

[0114] 5) Methods based on pre-trained language models:

[0115] BERT: This method is based on fine-tuning a large-scale pre-trained language model for text matching tasks.

[0116] SBERT: A pre-trained language model based on BERT. This algorithm acquires the semantics of search queries and document queries, encodes and models them in the same vector space, and finally calculates their similarity for comparison.

[0117] MatchBERT: MatchBERT is a Siamese network-based text matching model built upon the BERT pre-trained model. This network consists of two BERT encoders with shared parameters, used to learn the vector representations of two documents to be matched. Cosine similarity is then used to measure the similarity between the two representation vectors to determine if the two documents match. The difference between MatchBERT and SBERT is that MatchBERT focuses only on word-level information, not sentence or paragraph-level information.

[0118] The performance comparison results of the LMNFDC model of this invention with each baseline model are shown in Table 2, in percentage (%). The best result is indicated in bold.

[0119] Table 2. Experimental results of various models on the CNSE and CNSS datasets.

[0120]

[0121] In summary, the model proposed in this invention achieves excellent performance on both datasets, which can be mainly attributed to two reasons: First, the noise filtering mechanism removes data noise, allowing the model to perform high-quality fitting and reducing redundant text; second, the introduction of separate keyword and association matching mechanisms decomposes the long text matching problem into two simpler sub-problems: keyword matching and association matching. This allows the model to capture important word information while also focusing on the overall picture, avoiding the drawback of vector fusion for each word in the CIG model and improving model performance; furthermore, this model has a simpler process, is faster, and has stronger generalization ability due to the introduction of the noise filtering channel.

[0122] To better study the effectiveness of the LMNFDC model, ablation experiments were also conducted to investigate the contribution of each module to the experimental results of this model. The ablation experiments included the following two models:

[0123] (1) Ablation Model #1: LMNF model, which does not use the divide-and-conquer strategy, but only uses the PageRank filtering method;

[0124] (2) Ablation Model #2: LMDC model, which does not use PageRank filtering, but only uses a divide-and-conquer strategy;

[0125] (3) Ablation Model #3: LMNFDC model, which uses both PageRank filtering method and divide-and-conquer strategy, namely the long text matching model LMNFDC proposed in this invention that combines noise filtering and divide-and-conquer strategy.

[0126] Table 3 shows the experimental results of the three models on the CNSE and CNSS datasets, in percentage (%). The best results are indicated in bold. The following conclusions can be drawn from these experimental results:

[0127] Table 3 Ablation Experiment Results

[0128]

[0129] In the CNSE dataset, the model using the divide-and-conquer strategy (ablation model #3) improved its Acc and F1 scores by 1.78% and 2.07% respectively compared to the model without the divide-and-conquer strategy (ablation model #1), demonstrating the significant impact of the divide-and-conquer strategy on the model's performance. However, it can also be seen that using the PageRank filtering method (ablation model #3) resulted in varying degrees of decrease in both Acc and F1 scores (4.68%–6.24%) compared to not using the PageRank filtering method (ablation model #2). Analysis revealed that this is because the CNSE dataset uses the standard of matching news articles based on whether they report the same event; extensive filtering can easily filter out keywords indicating the same event.

[0130] In the CNSS dataset, it can be seen that both the PageRank filtering method and the divide-and-conquer strategy are crucial. Without the divide-and-conquer strategy (ablation model #1), the Acc and F1 scores would drop from 90.38% and 90.35% to 89.64% and 89.25%, respectively. At the same time, without the PageRank filtering method (ablation model #2), the experimental performance of the model would decrease (1.56% to 1.62%).

[0131] Furthermore, the experiment showed the best results on the CNSS dataset, indicating that this invention is more suitable for determining whether long articles belong to the same series of topics.

[0132] In summary, this invention first employs keyword extraction technology to extract keywords from long texts, then uses the PageRank algorithm to filter out unimportant and noisy sentences, and then performs standard global matching on the noise-filtered text. Based on the obtained keywords, it then extracts data other than keywords (called associations), and further matches keywords and associations separately. Finally, during training, it minimizes the global matching distribution and combines the bidirectional KL-divergence between the keyword and association matching distributions to ensure that the combined solutions of the original problem and its subproblems are similar. Verification shows that this invention significantly improves the accuracy of long text matching and is also applicable to short text matching.

[0133] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0134] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0135] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0136] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0137] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0138] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0139] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0141] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A long text matching method combining noise filtering and a divide-and-conquer strategy, comprising the following steps: A long text matching model is constructed, which includes a keyword extraction layer, an association extraction layer, and a filtering layer. The keyword extraction layer is used to extract keywords from the text, the filtering layer is used to filter noise in the text based on the similarity between sentences to obtain a denoised text sequence, and the association extraction layer is used to remove keywords from the denoised text sequence to obtain the remaining associated text. The long text matching model is trained with the goal of minimizing the set overall loss function. The loss function reflects the global matching distribution and combines the keyword and association matching distributions. For the target text, real-time text matching is performed using the trained long text matching model; The overall loss function is set as follows: in: in, This represents the total loss value. and These represent the semantic vector representations of the two sentences respectively. Sentence The i-th token in the array, Sentence The i-th token in the array, , This represents the probability of the category label corresponding to the statement. and To represent a sentence, It is the classification loss value. The loss value representing the joint distribution of keywords and related terms; The probability of the category label for the statement is set as follows: in, , The vector at position [CLS] The vector is the position of [SEP], where [CLS] is the first token of the sentence. It is input into the language model in the form [CLS]a[SEP]b[SEP], where W∈R K x H R represents the set of real numbers, K represents the number of labels in the category, and H represents the vector dimension of the hidden layer. Indicates the length of the sentence; in, Obtained by following these steps: Using cls vector H cls As a query, the associated vector H it and keyword vector H kw The combined vector serves as the key. A query is then used to retrieve the importance scores of the keywords and related terms, expressed as follows: The joint probability of keywords and associations is expressed as: Based on joint probability distribution The probability distribution relationship after combination is obtained. , is represented as: in, c m , c n The target category represents the degree of matching, if c m > c n , representing c m The matching degree is higher than c n , Indicates the importance score of conjunctions. Indicates the length of conjunctions, Indicates keyword length. This indicates the importance score of the keywords. This represents the hidden layer vector of the last layer of the conjunctions. H represents the hidden layer vector of the last layer of keywords. or , The table shows the learnable parameters. Represents the probability of a keyword. Indicates the probability of conjunctions. This represents the semantic vector representation of the keywords in sentence a. This represents the semantic vector representation of the keywords in sentence b. Indicates the probability of conjunctions. The semantic vector representation of the conjunctions in sentence a.

2. The method according to claim 1, characterized in that, The keyword extraction layer performs the following steps: For each sentence, words are used as nodes, and the relationships between words are used as edges to construct a word graph G(V, E), where V is the set of words, E is the set of links between words, and co-occurrence relations are selected as the standard for the relationships between words. For the aforementioned word graph, the TextRank value of each node is calculated as a score, expressed as follows: in, Representative node TextRank value, d It is the set damping coefficient. It is a node in-degree, It is a node The degree of exit, Represents a node To the node The weight of the edges between them is the number of times the edge appears. It is a node To the node The weight of the edges between them; Sort each node by score and extract a predetermined number of nodes as keywords.

3. The method according to claim 1, characterized in that, The filter layer performs the following steps: Construct a graph to represent the relationships between sentences, where each sentence in the document is a node, and the similarity between sentences is the weight of the edges. The similarity is measured by the degree of word overlap. The two long text segments Tm and Tn to be matched are split into single sentences, and then... , It means that among them and They are and The number of sentences in the text forms a set. ; By evaluating the sentence similarity in a sentence set T, a sentence similarity graph is constructed, measuring the proportion of repeated words in two sentences, as shown below: in, Indicates words in a sentence. Indicates the length of a sentence or set of words. These are two sentences in the combined sentence set T; The similarity between sentences after removing stop words is represented as follows: in, d The damping coefficient is... node in-degree, It is a node The degree of exit, It is a node and nodes similarity between ; The importance score of each sentence is evaluated based on the similarity between nodes, and the top-ranked sentences are selected. The sentences are concatenated into a text sequence, which is then used as the text sequence for denoising.

4. The method according to claim 1, characterized in that, The association extraction layer uses a deterministic finite automaton algorithm to extract the remaining text after filtering and removing keywords, as the association text.

5. The method according to claim 1, characterized in that, The language model mentioned is Roberta.

6. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

7. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

Images