Semantic enhancement text matching method and system based on achievement transformation large model

Abstract

The invention belongs to the technical field of large model application, and particularly relates to a semantic enhancement text matching method and system based on an achievement transformation large model. The semantic enhancement text matching method based on the achievement transformation large model comprises five parts, namely preparation of data and knowledge, enhancement of multi-level semantic knowledge, learning of semantic difference, intelligent weight distribution of semantic information and prediction of text similarity. The preparation of data and knowledge is used for constructing basic data and knowledge resources required by a text matching task through text preprocessing and knowledge base construction. And enhancement of multi-level semantic knowledge: enriching semantic representation of the text by injecting external knowledge at three levels of word level, phrase level and sentence pattern level. And semantic difference learning: learning fine-grained semantic features by comparing a learning framework, and improving the recognition capability of the model on the text semantic difference. Intelligent weight distribution of semantic information is carried out, and dynamic balance and optimal combination of different source information are realized by evaluating feature importance and dynamically adjusting weights. And predicting the text similarity, calculating the similarity score of the text pair by integrating the feature information of each stage, and providing an interpretable prediction result. The complete text matching method and system are designed based on the achievement transformation large model, the problems of insufficient knowledge utilization, incomplete semantic representation, fixed feature weight and the like of the large model in a text matching task are effectively solved, the text matching effect of the large model in an actual application scene is improved, and the text matching efficiency is improved. And a feasible solution is provided for achievement transformation of a large model technology.

Description

Background Art

[0002] With the rapid development of big model technology, how to effectively transform big models into results has become a hot topic in current research. Big models for transformation of results need to have strong capabilities in semantic understanding and knowledge application, among which text matching is a basic and critical task. In intelligent customer service scenarios, it is necessary to quickly match user questions with similar questions in the knowledge base; in search engines, it is necessary to accurately judge the relevance of queries and documents; in question-answering systems, it is necessary to identify the semantic equivalence relationship between questions.

[0003] At present, the large-scale model for achievement transformation mainly uses word vector-based methods and deep learning-based methods for text matching. The word vector-based method obtains text representation by calculating the weighted average or pooling operation of word vectors. Although it has high computational efficiency, it ignores word order and context information. Deep learning-based methods, such as convolutional neural networks and recurrent neural networks, can capture local features and sequence information, but often rely on a large amount of labeled data. In the actual application of large-scale models for achievement transformation, text matching tasks face multiple challenges. Due to the lack of effective integration of domain knowledge, the matching effects of these models in professional fields are often not ideal. At the same time, the semantic representation of existing large-scale models for achievement transformation is not comprehensive enough, and fails to fully consider semantic information at multiple levels such as words, phrases, and sentence patterns. In addition, these models lack the ability to recognize subtle semantic differences and are prone to misjudgment. Summary of the invention

[0004] In order to solve at least one technical problem existing in the above-mentioned background technology, the present invention provides a semantically enhanced text matching method and system based on a large model of achievement transformation, which realizes the hierarchical injection of semantic knowledge by constructing a multi-level knowledge base, improves the model's recognition ability of semantic differences through comparative learning, and realizes adaptive fusion of features through a dynamic weight allocation mechanism. In order to promote the practical application of this method, a corresponding achievement transformation system is also developed to realize the implementation of this method in actual application scenarios, providing new ideas for the development of text matching technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solution:

[0006] The first aspect of the present invention provides a semantically enhanced text matching method based on a large model of achievement transformation, which comprises:

[0007] Preparation of data and knowledge. Build basic data and knowledge resources, including cleaning and standardizing text data, building training sets and test sets; at the same time, build and organize knowledge bases at three levels: word level, phrase level, and sentence level, and use pre-trained language models to obtain the basic semantic representation of text, providing a data foundation for subsequent processing.

[0008] Enhancement of multi-level semantic knowledge. The semantic representation of the text is enriched through knowledge injection. First, the knowledge subset related to the current text is retrieved from the knowledge base. Then, semantic enhancement is performed at the sentence, word, and phrase levels, including calculation of attention weights and knowledge fusion. Finally, semantic knowledge at different levels is integrated to obtain an enhanced text representation.

[0009] Semantic difference learning. Focus on improving the semantic differentiation ability of the model. By constructing positive and negative sample pairs, using contrastive learning methods to learn the semantic difference features between texts, and using strategies such as dynamic temperature parameter adjustment and online difficult sample mining to enhance the model's differentiation ability.

[0010] Intelligent weight allocation of semantic information. To achieve a dynamic balance of different semantic information, first evaluate the value of various semantic features, then dynamically adjust the weights based on the feature value scores, and finally achieve interaction and fusion between different features through the attention mechanism to ensure the optimal combination of information.

[0011] Prediction of text similarity. Generating the final similarity judgment result mainly includes integrating the feature information obtained in multiple stages, calculating the similarity score of the text pair through a multi-layer perceptron, and finally outputting the judgment result in combination with the preset threshold, and providing corresponding explainability analysis.

[0012] The second aspect of the present invention provides a semantically enhanced text matching system based on a large model of achievement transformation, which includes:

[0013] The data and knowledge preparation module is responsible for building basic data and knowledge resources, including cleaning and standardizing text data, building training sets and test sets, and building and organizing knowledge bases at the word level, phrase level, and sentence level. It also uses pre-trained language models to obtain the basic semantic representation of text, providing a data foundation for subsequent processing.

[0014] The multi-level semantic knowledge enhancement module is responsible for enriching the semantic representation of the text through knowledge injection, including retrieving the knowledge subset related to the current text from the knowledge base, and then performing semantic enhancement at the sentence, word and phrase levels. Finally, the semantic knowledge at different levels is integrated to obtain the enhanced text representation.

[0015] The semantic difference learning module improves the semantic differentiation ability of the model, including building positive and negative sample pairs, using contrastive learning methods to learn the semantic difference features between texts, and adopting strategies such as dynamic temperature parameter adjustment and online difficult sample mining to enhance the model's differentiation ability;

[0016] The semantic information intelligent weight allocation module is responsible for achieving a dynamic balance between different semantic information, including evaluating the value of various semantic features, and then dynamically adjusting the weights based on the feature value scores. Finally, the attention mechanism is used to achieve interaction and fusion between different features to ensure the optimal combination of information.

[0017] The text similarity prediction module is responsible for generating the final similarity judgment result, including integrating the feature information obtained in multiple stages, calculating the similarity score of the text pair through a multi-layer perceptron, and finally outputting the judgment result in combination with the preset threshold and providing corresponding explainability analysis.

[0018] A third aspect of the present invention provides a computer-readable storage medium.

[0019] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps in the semantically enhanced text matching method based on a large model of achievement transformation as described above.

[0020] A fourth aspect of the present invention provides a computer device.

[0021] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the semantically enhanced text matching method based on the achievement transformation model as described above are implemented.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] (1) The present invention proposes a semantically enhanced text matching method based on a large model of achievement transformation. This method uses a pre-trained language model as the basic semantic representation layer, and realizes the hierarchical injection and fusion of semantic knowledge by constructing a three-layer knowledge base at the word level, phrase level, and sentence level. The importance of knowledge at different levels is dynamically adjusted through the attention mechanism, which effectively enhances the semantic representation ability of the text and improves the accuracy of text matching.

[0024] (2) The present invention designs an innovative knowledge retrieval and fusion mechanism. By combining a multi-step retrieval strategy of exact matching, rule matching, and semantic similarity calculation, efficient knowledge retrieval is achieved. The attention-based knowledge fusion method is used to dynamically integrate the retrieved knowledge into the text representation according to its relevance and importance, which not only ensures the accuracy of knowledge injection, but also avoids the interference of irrelevant knowledge.

[0025] (3) The present invention constructs a semantic difference learning framework based on contrastive learning. By designing a difficult negative sample mining strategy and a dynamic temperature parameter adjustment mechanism, the model's ability to recognize subtle semantic differences is enhanced. At the same time, contrastive learning is performed simultaneously at multiple semantic levels, making full use of semantic information at different levels, so that the model can better distinguish semantically similar and dissimilar text pairs.

[0026] (4) The present invention proposes an adaptive feature weight allocation method. By evaluating the contribution and stability of different features to similarity judgment, the feature weights are dynamically adjusted to achieve adaptive allocation of feature importance. At the same time, a feature interaction mechanism is introduced to capture the correlation between different features, further improving the model's discriminative ability.

[0027] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0029] Figure 1 It is a flow chart of a semantically enhanced text matching method based on a large achievement transformation model according to an embodiment of the present invention;

[0030] Figure 2 is a flow chart of data and knowledge preparation of an embodiment of the present invention;

[0031] Figure 3 is a flow chart of multi-level semantic knowledge enhancement according to an embodiment of the present invention;

[0032] Figure 4 is a flowchart of semantic difference learning according to an embodiment of the present invention;

[0033] Figure 5 is a flow chart of intelligent weight allocation of semantic information according to an embodiment of the present invention;

[0034] Figure 6 4 is a flowchart of text similarity prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0036] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0037] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0038] Embodiment 1

[0039] Reference Figure 1 , this embodiment provides a text similarity matching method based on multi-level semantic knowledge enhancement, which includes:

[0040] S101: Preparation of data and knowledge: cleaning and standardizing the input text pairs, building a knowledge base including word level, phrase level and sentence level, and using the pre-trained language model to obtain the basic semantic representation of the text.

[0041] S102: Enhancement of multi-level semantic knowledge. Retrieve the knowledge subset related to the current text from the knowledge base, and then perform semantic enhancement at the sentence, word and phrase levels. Finally, fuse the knowledge at different levels through the attention mechanism to obtain the enhanced semantic representation.

[0042] S103: Learning semantic differences, constructing positive and negative sample pairs, and using contrastive learning to allow the model to learn to capture subtle differences between text pairs, thereby improving the model's semantic distinction capabilities.

[0043] S104: Intelligent weight allocation of semantic information, evaluating the importance of various types of information, dynamically adjusting the weights of different information according to the characteristics of the text, and realizing adaptive fusion of information.

[0044] S105: Prediction of text similarity, integrating the feature information obtained in the previous stages, calculating the similarity scores of the text pairs and outputting the final evaluation results.

[0045] Reference Figure 2 , This embodiment provides a flowchart of the steps of data and knowledge preparation, and explains the preparation of data and knowledge in detail.

[0046] Step 200: Preprocessing and marking of text pairs. Preprocess the input text (cleaning, word segmentation). Step 200 can be further divided into the following steps:

[0047] Step 200-1: Build a text pair dataset, which can be obtained from a public dataset such as LCQMC (Large Scale Chinese Question Matching Dataset). The text format in the dataset is (text1, text2, label), where label indicates whether text1 and text2 are similar. The similar label value is 1, otherwise the label value is 0.

[0048] Step 200-2: After obtaining the data, divide the data set into training set D and training set D according to the ratio of 8:2. train and the test set D test ;

[0049] Step 200-3: Standardize the text pairs (text1, text2) in each data set in step 200-2, including removing special characters, unifying upper and lower case, etc.;

[0050] Step 200-4: Perform word segmentation and part-of-speech tagging on the text to obtain a standardized word sequence (text1', text2'). The text segmentation and part-of-speech tagging can use the jieba open source toolkit;

[0051] Step 200-4: After processing in steps 200-3 and 200-4, the training set D is obtained. train ' and test set D test '.

[0052] Step 201: Organize and structure knowledge resources. Construct a three-level knowledge base structure, including a word-level knowledge base, a phrase-level knowledge base, and a sentence-level knowledge base. Step 201 can be further divided into the following steps:

[0053] Step 201-1: The word-level knowledge base is mainly constructed by integrating multiple authoritative Chinese word knowledge resources. The data sources include the extended version of the Synonym Cilin, the Synonym Cilin of the Information Retrieval Laboratory of Harbin Institute of Technology, and Chinese Wordnet. The construction of the word-level knowledge base can also use the synonyms toolkit to process and extract the semantic relationship between words, including synonyms, antonyms, hyponyms, and other information;

[0054] Step 201-2: The construction of phrase-level knowledge base is based on resources such as the modern Chinese phrase structure tree of the Institute of Computing Technology of the Chinese Academy of Sciences, common phrases in the CCL corpus, fixed collocations in encyclopedia entries, and professional terminology libraries. Use LTP tools to perform phrase recognition and semantic role labeling to identify and extract common phrase structure patterns. For each phrase, record its structure pattern, similar phrases, components, and usage areas;

[0055] Step 201-3: The sentence-level knowledge base is mainly constructed based on the Peking University Chinese Grammar Information Dictionary, Tsinghua University Syntax Knowledge Base, Chinese Sentence Template Library, and common sentences in professional literature. Stanford Core NLP and Harbin Institute of Technology LTP tools are used for syntactic analysis to extract the structural patterns and semantic information of sentences. The knowledge base records information such as syntactic templates, semantic types, similar sentences, and sentence constraints;

[0056] Step 201 - 4 : The data in step 201 - 1 to step 201 - 3 are stored and managed through MongoDB, and a word index is established to support fast retrieval.

[0057] Step 202: Basic semantic representation of the text. Use a pre-trained language model (such as BERT) to obtain the semantic representation of the text. Step 202 can be further refined into the following steps:

[0058] Step 202-1: Obtain the word sequence text1' obtained in step 200-4, input it into a pre-trained language model (such as BERT), and obtain a context-related vector representation text1";

[0059] Step 202-2: Obtain the word sequence text2' obtained in step 200-4, input it into a pre-trained language model (such as BERT), and obtain a context-related vector representation text2";

[0060] Step 202-3: After processing in steps 202-1 and 202-2, the training set D is obtained. train " and test set D test ”.

[0061] Reference Figure 3 This embodiment provides a flowchart of the steps of multi-level semantic knowledge enhancement, which describes the enhancement of multi-level semantic knowledge in detail, including:

[0062] Step 300: Matching of relevant knowledge subsets. Retrieve knowledge related to the text pair (text1', text2') in step 200-4 from the constructed knowledge base. Step 300 can be further divided into the following steps:

[0063] Step 300-1: Retrieve the knowledge related to the words appearing in text text1' and text2' from the word-level knowledge base. First, perform an exact match on the words in the text, directly search for exactly the same entries in the knowledge base. For the words that cannot be exactly matched, use a rule-based fuzzy match, including handling word form changes (such as "美丽" and "美丽的"), standardization processing (such as traditional and simplified Chinese conversion), and then perform the match again. Finally, for the words that still cannot be matched, calculate the semantic similarity between the vector representation of the word in the text and the entries in the knowledge base, sim(w,k) = cos(vec(w),vec(k)), where w is the word in the text, k is the entry in the knowledge base, vec(·) is the word vector representation function, and cos(·) is the cosine similarity calculation function. When the similarity is greater than the preset threshold θ (usually taken as 0.8), add the corresponding knowledge entries to the result set. The knowledge retrieved by the three steps is merged to finally obtain the complete word-level knowledge subsets related to text1' and text2' respectively;

[0064] Step 300-2: Retrieve the relevant knowledge from the phrase-level knowledge base. First, it is necessary to identify the phrase structures in text text1' and text2' respectively. Use the sliding window method to extract consecutive n words as candidate phrases (n is usually taken as 2-4). Then, adopt a three-step retrieval strategy: First, perform an exact match on the candidate phrases, directly search for exactly the same phrase entries in the knowledge base; for the phrases that cannot be found, use a rule-based fuzzy match, including handling word order changes (such as "性能优秀" and "优秀的性能"), synonym replacement (such as "性能卓越" and "性能优异"), and then perform the match again. Finally, calculate the semantic similarity between the candidate phrase and the phrase in the phrase-level knowledge base, sim(p,s) = cos(vecs(p),vecs(s)), where p is the phrase in the text, s is the phrase in the phrase-level knowledge base, vecs(·) represents the function that maps the phrase to the vector space, and cos(·) is the cosine similarity calculation function. When the similarity exceeds the threshold β (usually taken as 0.75), add the corresponding phrase knowledge to the result set. Finally, the knowledge retrieved by the three steps is merged to obtain the phrase knowledge subsets related to text text1' and text2' respectively;

[0065] Step 300-3: Retrieve relevant knowledge from the sentence-level knowledge base. First, use the dependency syntactic analysis tool to obtain the structural representation of sentences text1 and text2, extract the subject, predicate, and object and other grammatical components of the sentences to form a sentence template template(s), and then adopt a three-step retrieval strategy. First, the sentence template is accurately matched, and the completely identical sentence template is searched in the sentence-level knowledge base. For unmatched sentences, rule-based fuzzy matching is used, including processing the position changes of grammatical components (such as "subject + predicate + object" and "subject + object + predicate"), the replacement of grammatical components (such as replacing negative words with positive words) and then matching again. Finally, the structural similarity between the sentence template and the sentence in the sentence-level knowledge base is calculated sim(ss,kk)=align(template(ss),template(kk)), where ss is the input sentence, kk is the sentence in the sentence-level knowledge base, align(·) represents the similarity calculation function based on the alignment of grammatical components. When the similarity is greater than the threshold ∈ (generally 0.7), the corresponding sentence knowledge is added to the result set. Finally, the knowledge retrieved in the three steps is merged to obtain the sentence knowledge subsets related to sentences text1 and text2 respectively.

[0066] Step 301: Enhancement of sentence-level semantics. Based on the sentence knowledge retrieved in step 300-3, semantic enhancement is performed on the text. Step 301 can be further refined into the following steps:

[0067] Step 301-1: respectively obtain the sentence structure representations and corresponding sentence knowledge subsets of text1 and text2 in step 300-3;

[0068] Step 301-2: For each sentence knowledge, calculate its attention weight ɑ=softmax(f(template(ss), template(k))) with the original sentence, where f(·) is a correlation calculation function, which can be dot product attention, scaled dot product attention, additive attention, etc. ss is the input sentence, and k is the subset of sentence knowledge related to sentence ss;

[0069] Step 301-3: Integrate sentence knowledge into the original text representation according to the attention weight to obtain sentence-level enhanced semantic representations H_sent1 and H_sent2 of text1 and text2. Specifically, take text1 as an example for description. For the input sentence ss (i.e., text1), there are its original sentence representation template(ss) and m related sentence knowledge {k1, k2, ..., km} retrieved from the knowledge base and their template representations {template(k1), template(k2), ..., template(km)}, as well as the attention weights {α1, α2, ..., αm} obtained by the correlation calculation function. First, the representations of the m sentence knowledge are weighted and summed according to the attention weights to obtain the sentence knowledge enhancement vector s_enhanced. Then, the original sentence representation template(ss) is fused with the knowledge enhancement vector s_enhanced to obtain the final sentence representation s_final = σ(W·[template(ss);s_enhanced]+b), where [;] represents the vector concatenation operation, W and b are learnable parameters, and σ is the activation function (such as tanh or ReLU).

[0070] Step 302: Enhancement of word-level semantics. The text is semantically enhanced based on the word-level knowledge retrieved in step 300-1. Step 302 can be further refined into the following steps:

[0071] Step 302-1: Obtain each word in text1' and text2' in step 300-1 and its corresponding word-level knowledge subset;

[0072] Step 302-2: For each knowledge entry of a word, calculate its attention weight β=softmax(g(vec(w),vec(j))) with the original word, where g(·) is a correlation calculation function, which can use dot product attention, scaled dot product attention, additive attention, etc., w is the input word, and j is the word-level knowledge subset related to word w;

[0073] Step 302-3: Integrate word-level knowledge into the original text representation according to the attention weight to obtain word-level enhanced semantic representations H_word1, H_word2. Specifically, take text1' as an example. For each word w in text1', there is its original representation vec(w) and m related knowledge items {j1, j2, ...jm} retrieved from the knowledge base and their representations {vec(j1), vec(j2), ...vec(jm)}, as well as the attention weights {β1, β2, ..., βm} obtained by the correlation calculation function. First, the vector representations of the m knowledge items are weighted and summed according to the attention weights to obtain the knowledge enhancement vector m_enhanced. Then the original word vector vec(w) is fused with the knowledge enhancement vector m_enhanced to obtain the final word representation w_enhanced = σ(W·[vec(w);m_enhanced]+b), where [;] represents the vector concatenation operation, W and b are learnable parameters, and σ is the activation function (such as tanh or ReLU).

[0074] Step 303: Enhancement of phrase-level semantics. Based on the phrase knowledge retrieved in step 300-2, the text is semantically enhanced. Step 303 can be further refined into the following steps:

[0075] Step 303-1: Obtain the phrases identified in text1' and text2' in step 300-2 and their corresponding phrase knowledge subsets;

[0076] Step 303-2: For each phrase knowledge, calculate its attention weight γ=softmax(h(vecs(p),vecs(s))) with the original phrase, where h(·) is a correlation calculation function, which can be dot product attention, scaled dot product attention, additive attention, etc., p is the input phrase, s is the sentence p related sentence knowledge subset;

[0077] Step 303-3: Integrate phrase knowledge into the original text representation according to the attention weight to obtain phrase-level enhanced semantic representations H_phrase1 and H_phrase2. Specifically, text1' is used as an example for description. For the phrase p identified in text1', there are its original representation vecs(p) and n related phrase knowledge {s1, s2, ..., sn} retrieved from the knowledge base and their representations {vecs(s1), vecs(s2), ..., vecs(sn)}, as well as the attention weights {γ1, γ2, ..., γn} obtained by the correlation calculation function. First, the vector representations of the n phrase knowledge are weighted and summed according to the attention weights to obtain the phrase knowledge enhancement vector p_enhanced. Then the original phrase vector vecs(p) is fused with the knowledge enhancement vector p_enhanced to obtain the final phrase representation p_final = σ(W·[vecs(p);p_enhanced]+b), where [;] represents the vector concatenation operation, W and b are learnable parameters, and σ is the activation function (such as tanh or ReLU).

[0078] Step 304: Fusion of multi-level semantic knowledge. Integrate the semantic knowledge of different levels obtained in steps 301, 302, and 303. Step 304 can be further divided into the following steps:

[0079] Step 304-1: Calculate the attention weights between the levels. For text1 and text2, calculate the attention weights λ=softmax(W[H_sent;H_word;H_phrase]) of the three levels of semantic representation (sentence level H_sent, word level H_word and phrase level H_phrase), respectively, where [;] represents the vector concatenation operation, W is a learnable parameter matrix, and the obtained λ={λ1,λ2,λ3} corresponds to the weights of the three levels respectively;

[0080] Step 304-2: Fuse semantic representations at different levels. For each text, perform weighted summation of its three-level semantic representations according to the corresponding attention weights to obtain a fusion vector text_fusion = λ1·H_sent+λ2·H_word+λ3·H_phrase, and then obtain the final multi-level semantic enhanced representation text_enhanced = σ(W·text_fusion+b) through nonlinear transformation, where W and b are learnable parameters and σ is an activation function (such as tanh or ReLU);

[0081] Step 304 - 3 : Obtain multi-level semantically enhanced representations text1_enhanced and text2_enhanced of text1 and text2 respectively, as inputs for subsequent similarity calculations.

[0082] Reference Figure 4 This embodiment provides a flowchart of the steps of semantic difference learning, which describes the semantic difference learning in detail, including:

[0083] Step 400: constructing positive and negative sample pairs. Construct training samples based on the text representations text1_enhanced and text2_enhanced obtained in step 304. Step 400 can be further refined into the following steps:

[0084] Step 400-1: For the positive sample pair (text1_enhanced, text2_enhanced, 1), directly use the text pair with similar semantics;

[0085] Step 400-2: For the negative sample pair (text1_enhanced, text3_enhanced, 0), randomly select text3 from the training set that is semantically dissimilar to text1 to construct;

[0086] Step 400-3: Construct difficult negative samples, and select samples that are close to text1 in the semantic space but have dissimilar labels as text3 to improve the model's discrimination ability.

[0087] Step 401: Learning semantic difference features. Learning text semantic difference features based on contrastive learning method. Step 401 can be further refined into the following steps:

[0088] Step 401 - 1: Calculate the similarity between the positive sample pairs sim_p = f(text1_enhanced, text2_enhanced), where f(·) is a similarity calculation function;

[0089] Step 401 - 2: Calculate the similarity between the negative sample pairs sim_n = f(text1_enhanced, text3_enhanced);

[0090] Step 401 - 3 : Construct a contrast loss function L = -log(exp(sim_p / τ) / (exp(sim_p / τ)+Σexp(sim_n / τ))), where τ is a temperature parameter, and learn discriminative features by minimizing the loss function.

[0091] Step 402: Enhancement of differentiation capability. The semantic differentiation capability of the model is enhanced through various strategies. The step 402 can be further refined into the following steps:

[0092] Step 402-1: using a dynamically adjusted temperature parameter τ, gradually lowering the temperature as training progresses, so that the model gradually focuses on more fine-grained semantic differences;

[0093] Step 402-2: Introduce multi-scale contrastive learning, perform contrastive learning at different semantic levels simultaneously, and enhance the distinguishing ability of the model;

[0094] Step 402-3: Apply the online difficult sample mining strategy to dynamically select the most challenging negative samples in the current batch for learning.

[0095] Reference Figure 5 This embodiment provides a flowchart of the steps of intelligent weight allocation of semantic information, which describes the intelligent weight allocation of semantic information in detail, including:

[0096] Step 500: Evaluate the value of various types of information. Evaluate the value of the semantic difference features obtained in step 401. Step 500 can be further divided into the following steps:

[0097] Step 500-1: Calculate the contribution of each semantic feature to similarity judgment, using a gradient-based feature importance evaluation method;

[0098] Step 500-2: Evaluate the stability of each feature and examine the consistency of the features on different samples;

[0099] Step 500 - 3 : The value score of each feature is obtained by combining contribution and stability: value = α·contribution+(1-α)·stability, where α is a balance parameter.

[0100] Step 501: Dynamic adjustment of weights. Dynamically adjust weights based on feature value scores. Step 501 can be further broken down into the following steps:

[0101] Step 501-1: Initialize feature weights and set basic weight values;

[0102] Step 501-2: Dynamically update the weight w_new=softmax(η·value+w_old) according to the feature value score, where η is the learning rate and w_old is the original weight;

[0103] Step 501 - 3 : Apply a smoothing strategy to avoid drastic changes in weights, w_smooth = β·w_new+(1-β)·w_old, where β is the smoothing coefficient.

[0104] Step 502: Fusion of multi-faceted information. Perform weighted fusion of information from different aspects. Step 502 can be further refined into the following steps:

[0105] Step 502-1: Obtain various feature representations after weight adjustment;

[0106] Step 502-2: Use the attention mechanism to calculate the interaction relationship between different features, and obtain the interaction matrix I=attention(F_weighted), where F_weighted is the weighted feature;

[0107] Step 502 - 3 : Perform feature fusion based on the interaction matrix to obtain the final feature representation F_final=fusion(F_weighted,I), where fusion(·) is the feature fusion function.

[0108] Reference Figure 6 This embodiment provides a flowchart of the steps of text similarity prediction, which describes the prediction of text similarity in detail, including:

[0109] Step 600: Integration of feature information. The feature representations obtained in step 502 are integrated. Step 600 can be further divided into the following steps:

[0110] Step 600 - 1: Obtain feature representations F_final1 and F_final2 of two texts respectively;

[0111] Step 600 - 2: construct a feature interaction representation, including feature difference diff = |F_final1-F_final2| and feature product prod = F_final1⊙F_final2;

[0112] Step 600 - 3 : Concatenate the original features with the interactive features to obtain a complete feature representation F_complete = [F_final1; F_final2; diff; prod], where [;] represents a vector concatenation operation.

[0113] Step 601: Calculation of text similarity. Calculate the similarity score of the text pair based on the integrated features. Step 601 can be further refined into the following steps:

[0114] Step 601-1: transform the features through a multi-layer perceptron, score=MLP(F_complete), where MLP includes multiple nonlinear transformation layers;

[0115] Step 601-2: normalize the output score, sim_score = sigmoid(score), to ensure that the similarity score is in the range of [0, 1];

[0116] Step 601 - 3 : Setting a similarity threshold θ to convert continuous similarity scores into binary classification results.

[0117] Step 602: Output of judgment result. Generate the final text similarity judgment result. The step 602 can be further divided into the following steps:

[0118] Step 602-1: Compare the similarity score with the threshold, and determine that they are similar when sim_score>θ, otherwise, determine that they are not similar;

[0119] Step 602-2: Generate a prediction output including a similarity score and a judgment result;

[0120] Step 602-3: Perform interpretability analysis on the prediction results and identify the features that contribute most to the judgment results.

[0121] Embodiment 2

[0122] This embodiment provides a semantically enhanced text matching system based on a large model of achievement transformation, which includes:

[0123] The data and knowledge preparation module is responsible for building basic data and knowledge resources, including cleaning and standardizing text data, building training sets and test sets, and building and organizing knowledge bases at the word level, phrase level, and sentence level. It also uses pre-trained language models to obtain the basic semantic representation of text, providing a data foundation for subsequent processing.

[0124] The multi-level semantic knowledge enhancement module is responsible for enriching the semantic representation of the text through knowledge injection, including retrieving the knowledge subset related to the current text from the knowledge base, and then performing semantic enhancement at the sentence, word and phrase levels. Finally, the semantic knowledge at different levels is integrated to obtain the enhanced text representation.

[0125] The semantic difference learning module improves the semantic differentiation ability of the model, including building positive and negative sample pairs, using contrastive learning methods to learn the semantic difference features between texts, and adopting strategies such as dynamic temperature parameter adjustment and online difficult sample mining to enhance the model's differentiation ability;

[0126] The semantic information intelligent weight allocation module is responsible for achieving a dynamic balance between different semantic information, including evaluating the value of various semantic features, and then dynamically adjusting the weights based on the feature value scores. Finally, the attention mechanism is used to achieve interaction and fusion between different features to ensure the optimal combination of information.

[0127] The text similarity prediction module is responsible for generating the final similarity judgment result, including integrating the feature information obtained in multiple stages, calculating the similarity score of the text pair through a multi-layer perceptron, and finally outputting the judgment result in combination with the preset threshold and providing corresponding explainability analysis.

[0128] Embodiment 3

[0129] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps in the semantically enhanced text matching method based on the achievement transformation model as described above are implemented.

[0130] Embodiment 4

[0131] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps in the semantically enhanced text matching method based on the achievement transformation model as described above are implemented.

[0132] It should be appreciated by those skilled in the art that embodiments of the present invention may be in the form of methods, systems, or computer program products. Therefore, the invention may be implemented in the form of hardware, software, or a combination of hardware and software. In addition, the invention may be in the form of a computer program product on a computer-usable storage medium (including but not limited to a disk storage and an optical storage, etc.), which contains one or more computer-usable program codes.

[0133] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A semantically enhanced text matching technology based on a large model of achievement transformation, characterized in that: It includes the preparation of data and knowledge, the enhancement of multi-level semantic knowledge, the learning of semantic differences, the intelligent weight allocation of semantic information, and the prediction of text similarity. Among them, the preparation of data and knowledge includes text preprocessing (cleaning, word segmentation), data set division, knowledge base (word level, phrase level, sentence level) construction and the acquisition of basic semantic representation of text. The enhancement of multi-level semantic knowledge is achieved through the matching of relevant knowledge subsets, the enhancement of sentence level semantics, the enhancement of word level semantics, and the enhancement of phrase level semantics, and finally the fusion of multi-level semantic knowledge. The learning of semantic differences is achieved by constructing positive and negative sample pairs, learning semantic difference features, and enhancing the ability to distinguish to achieve the modeling of text semantic differences. The intelligent weight allocation of semantic information includes the evaluation of the value of various types of information, the dynamic adjustment of weights, and the fusion of multi-faceted information to achieve adaptive combination of features. The prediction of text similarity generates the final prediction result through the integration of feature information, the calculation of text similarity, and the output of judgment results.

2. A semantically enhanced text matching technology based on a large model of achievement transformation is used to prepare data and knowledge, characterized in that: Build and organize knowledge bases at three levels: word level, phrase level, and sentence level, and use pre-trained language models to obtain the basic semantic representation of the text.

3. A semantic enhancement text matching technology based on a large model of achievement transformation to enhance multi-level semantic knowledge, characterized in that: According to claim 2, a knowledge subset related to the current text is retrieved from the knowledge base, and then semantic enhancement is performed at three levels: sentence, word and phrase.

4. A semantically enhanced text matching technology based on a large model of achievement transformation for a semantically differentiated learning process, characterized in that: According to claim 3, the semantic difference features between texts are learned using a contrastive learning method.

5. A semantically enhanced text matching technology based on a large model of achievement transformation for a semantically differentiated learning process, characterized in that: According to claim 4, strategies such as dynamic temperature parameter adjustment and online difficult sample mining are adopted to enhance the distinguishing ability of the model.

6. A semantically enhanced text matching technology based on a large model of achievement transformation for intelligent weight allocation of semantic information, characterized in that: According to claims 3 and 5, the value of each type of semantic features is evaluated, and then the weight is dynamically adjusted based on the feature value score.

7. A semantically enhanced text matching technology based on a large model of achievement transformation for intelligent weight allocation of semantic information, characterized in that: According to claim 6, the attention mechanism is used to achieve interaction and fusion between different features to ensure the optimal combination of information.

8. A semantically enhanced text matching technology based on a large model of achievement transformation is used to predict text similarity, characterized in that: According to claim 7, the similarity scores of the text pairs are calculated by a multi-layer perceptron, the judgment results are output in combination with the preset threshold, and corresponding explainability analysis is provided.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps in a semantically enhanced text matching method based on a large model of achievement transformation as described in any one of claims 1 to 8 are implemented.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the semantically enhanced text matching method based on the achievement transformation model as described in any one of claims 1 to 8 are implemented.

Images