Text element recognition method and system of adaptive integration technology based on multilevel feature fusion

Abstract

The invention discloses a text element recognition method and system based on a multilevel feature fusion adaptive integration technology, which is applied to the field of natural language processing, and comprises the following steps: preprocessing a data text, and training a plurality of text element recognition sub-models based on the preprocessed data text; performing multi-level feature extraction on the preprocessed data text, performing feature fusion, training a weight learning device of a plurality of text element recognition sub-models, and outputting dynamic weights of the plurality of text element recognition sub-models; and based on the output results and the confidence scores of the plurality of text element recognition sub-models, combining with the dynamic weight, and weighting to obtain final text element recognition output. According to the method, the problem of low recognition precision caused by incomplete feature extraction and poor model adaptability in the prior art is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to a text element recognition method and system based on an adaptive integration technology of multi-level feature fusion. Background Art

[0002] Case element recognition is the key support and research hotspot for the research of automation technologies such as similar case recommendation, sentence prediction, and information extraction. There are several problems in the research on case element recognition: the traditional case element recognition method mainly uses the text features of the judgment document as the category prediction of the case element label, ignoring the judicial semantic information of the case element label; the distribution of case elements in the judgment document is often uneven, which makes it difficult for the model to fully learn the semantic features of small sample elements and effectively recognize them; unlike general multi-classification problems, case element recognition belongs to multi-label classification problems, and a sample may belong to 0-N categories at the same time, and there is often correlation between multiple categories.

[0003] In the field of natural language processing, text element recognition is a key task, especially in patent and judicial texts. Due to the complexity and diversity of the text, accurate identification of elements faces huge challenges. Traditional recognition methods mostly rely on a single model, which has limited adaptability and is difficult to handle multi-level features in the text, resulting in insufficient recognition accuracy and robustness.

[0004] In the existing technology, pre-trained language models such as BERT and ALBERT have been widely used in natural language processing tasks. However, these models usually focus on specific text features and it is difficult to fully capture the multi-level features in the text.

[0005] Therefore, how to provide a text element recognition method and system based on multi-level feature fusion adaptive integration technology that can effectively integrate the prediction results of multiple models to improve the overall prediction performance is an urgent problem that technicians in this field need to solve. Summary of the invention

[0006] In view of this, the present invention proposes a text element recognition method and system based on an adaptive integration technology of multi-level feature fusion.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] A text element recognition method based on multi-level feature fusion adaptive integration technology, comprising:

[0009] Step 1: Preprocess the data text, and train multiple text element recognition sub-models based on the preprocessed data text;

[0010] Step 2: Perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train weight learners of multiple text element recognition sub-models, and output dynamic weights of multiple text element recognition sub-models;

[0011] Step 3: Based on the output results and confidence scores of the multiple text element recognition sub-models, combined with dynamic weights, weighted results are obtained to obtain the final text element recognition output.

[0012] Optionally, in step 1, the data text is preprocessed, including data cleaning, denoising and standardization.

[0013] Optionally, in step 1, multiple text element recognition sub-models include but are not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, and BiLSTM-CRF model.

[0014] Optionally, in step 2, multi-level feature extraction is performed on the preprocessed data text, including: lexical feature extraction, syntactic feature extraction and deep semantic feature extraction.

[0015] Optionally, in step 2, feature fusion is performed as follows:

[0016] F 融合 =αF 词汇 +βF 句法 +γF 语义 ;

[0017] Among them, F 融 is the feature after multi-level feature fusion; α, β, and γ are the feature weighting coefficients of lexical features, syntactic features, and deep semantic features respectively; F 词汇 is a lexical feature; F 句法 is a syntactic feature; F 语义 It is a deep semantic feature.

[0018] Optionally, in step 2, weight learners of multiple text element recognition sub-models are trained to output dynamic weights of multiple text element recognition sub-models as follows:

[0019] W 模型 =softmax(f 权重学习器 (F 融合 ));

[0020] Among them, W 模型 is the dynamic weight of the text element recognition sub-model; softmax is the normalized softmax function; f 权重学习器 is the calculation function of the weight learner; F 融合 It is the feature after multi-level feature fusion.

[0021] Optionally, in step 2, in the process of training the weight learners of multiple text element recognition sub-models, it also includes: updating the parameters of the weight learners using stochastic gradient descent and back propagation algorithms.

[0022] Optionally, in step 3, based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, the final text element recognition output is weighted as follows:

[0023]

[0024] Among them, S 输出 is the final text element recognition output; n is the number of text element recognition sub-models; W 模型，i is the dynamic weight of the i-th text element recognition sub-model; P 模型，i is the output result of the i-th text element recognition sub-model; C 模型，i The confidence of the i-th text feature recognition sub-model.

[0025] Optionally, in step 3, the confidence of the text element recognition sub-model is calculated by a softmax function.

[0026] The present invention also provides a text element recognition system based on multi-level feature fusion adaptive integration technology using a text element recognition method based on multi-level feature fusion adaptive integration technology, comprising:

[0027] A sub-model training module is used to pre-process the data text and train multiple text element recognition sub-models based on the pre-processed data text;

[0028] Dynamic weight acquisition module: used to perform multi-level feature extraction and feature fusion on the preprocessed data text, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models;

[0029] Text element recognition output module: Based on the output results and confidence scores of multiple text element recognition sub-models, combined with dynamic weights, the final text element recognition output is weighted.

[0030] It can be seen from the above technical solutions that compared with the prior art, the present invention proposes a text element recognition method and system based on an adaptive integration technology of multi-level feature fusion. By fusing multi-level features, including lexical features, syntactic features and deep semantic features, these features are used to identify judicial text elements in the patent field, solving the problem that the traditional single model is difficult to fully handle complex text features, and using multiple sub-models, such as ALBERT, BERT-CRF, etc., by training the pre-processed judicial text data, fully learning the text features, using the adaptive integration algorithm based on deep neural network to learn the weights of the fused features, and improving the prediction performance of the model through dynamic weight adjustment, significantly improving the accuracy of element recognition, and combining the weights of each sub-model output by the weight learner with the prediction results and their confidence. In the training process of the weight learner, the model parameters are optimized by using stochastic gradient descent and back propagation algorithms, which effectively reduces the prediction error of the model. Experiments were conducted on the LAIC2023 dataset to verify the significant effect of this method in improving recognition accuracy and robustness, and the adaptability of the model under different types of texts is enhanced. In summary, the present invention effectively solves the recognition accuracy problem caused by incomplete feature extraction and poor model adaptability in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0032] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0034] Embodiment 1:

[0035] Embodiment 1 of the present invention discloses a text element recognition method based on an adaptive integration technology of multi-level feature fusion, such as Figure 1 As shown, including:

[0036] Step 1: Preprocess the data text and train multiple text element recognition sub-models based on the preprocessed data text.

[0037] Preprocess the data text, including: data cleaning, removing noise and redundant information in the text, denoising, using regular expressions and other text processing techniques to remove noise characters and special symbols in the text, standardization processing, and unified text format to ensure data quality.

[0038] Multiple text element recognition sub-models, including but not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, BiLSTM-CRF model. These models are good at different types of feature processing, which helps to capture different dimensions of information in the text.

[0039] Step 2: Perform multi-level feature extraction and feature fusion on the preprocessed data text, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models.

[0040] Perform multi-level feature extraction on the preprocessed data text, including: lexical feature extraction, syntactic feature extraction and deep semantic feature extraction.

[0041] Lexical feature extraction: Statistical methods such as term frequency-inverse document frequency (TF-IDF) are used to extract lexical features to reflect the importance of words in the document.

[0042] Syntactic feature extraction: Extract the syntactic structure of the text through dependency syntactic analysis tools (such as Stanford Parser), identify grammatical relationships such as subject, predicate and object, and capture sentence structure information.

[0043] Semantic feature extraction: Extract deep semantic features based on pre-trained language models (such as BERT) to capture implicit meaning in context to improve the understanding of complex legal terms.

[0044] Feature fusion is as follows:

[0045] F 融合 =αF 词汇 +βF 句法 +γF 语义 ;

[0046] Among them, F 融 is the feature after multi-level feature fusion; α, β, and γ are the feature weighting coefficients of lexical features, syntactic features, and deep semantic features respectively; F 词汇 is a lexical feature; F 句法 is a syntactic feature; F 语义 It is a deep semantic feature.

[0047] Train the weight learners of multiple text element recognition sub-models and output the dynamic weights of multiple text element recognition sub-models as follows:

[0048] W 模型 =softmax(f 权重学习器 (F 融合 ));

[0049] Among them, W 模型 is the dynamic weight of the text element recognition sub-model; softmax is the normalized softmax function; f 权重学习器 is the calculation function of the weight learner; F 融合 It is the feature after multi-level feature fusion.

[0050] The process of training the weight learners of multiple text element recognition sub-models also includes: using stochastic gradient descent and back propagation algorithms to update the parameters of the weight learners to minimize the prediction error and optimize the performance of the weight learners.

[0051] Step 3: Based on the output results and confidence scores of the multiple text element recognition sub-models, combined with dynamic weights, the final text element recognition output is weighted. After obtaining the text element recognition output results, post-processing operations are performed to further improve the accuracy and consistency of the recognition results through rule correction, cross-sentence verification, etc.

[0052] Based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, the final text element recognition output is weighted as follows:

[0053]

[0054] Among them, S 输出 is the final text element recognition output; n is the number of text element recognition sub-models; W 模型，i is the dynamic weight of the i-th text element recognition sub-model; P 模型，i is the output result of the i-th text element recognition sub-model; C 模型，i The confidence of the i-th text feature recognition sub-model.

[0055] The confidence of the text element recognition sub-model is calculated by the softmax function.

[0056] Embodiment 2:

[0057] Embodiment 2 of the present invention discloses a specific application of a text element recognition method based on a multi-level feature fusion adaptive integration technology described in Embodiment 1 on a LAIC2023 patent field judicial text dataset, including:

[0058] The text element recognition method based on the adaptive integration technology of multi-level feature fusion described in Example 1 of the present invention and four baseline models BiLSTM-CRF, BERT-CRF, BERT-Softmax, and BERT-LSTM-CRF are used to recognize the text elements of the LAIC2023 patent field judicial text dataset, and the structured case element information, including the subject, behavior, legal basis, etc., is output through the final rule correction, cross-sentence verification and other means. Precision, Recall and F1 value are selected as evaluation indicators, and the final recognition comparison results are shown in Table 1.

[0059] Table 1 Comparison results of various model experiments

[0060]

[0061] From Table 1, it can be seen that the method proposed in Example 1 of the present invention outperforms the baseline model in all indicators on the test set and shows better performance. Specifically, the Precision, Recall and F1 values ​​of Example 1 of the present invention on the test set reached 0.654, 0.649 and 0.652 respectively, which are significantly higher than the indicators of other baseline models.

[0062] By comparing the baseline models, we can find that:

[0063] Although the BiLSTM-CRF model has the highest F1 value of 0.749 on the validation set, its Precision, Recall and F1 values ​​on the test set are 0.649, 0.642 and 0.646 respectively. Although it performs well on the validation set, its comprehensive performance on the test set fails to exceed the method of Example 1 of the present invention. This model has a significant improvement over other models, which may be because softmax performs well in classification tasks, but in sequence labeling tasks, it may not be as effective as CRF, a model specifically used for sequence labeling, because CRF can better consider the dependencies between labels, while Softmax focuses on the independent prediction of a single label.

[0064] The F1 value of the BERT-CRF model on the validation set is 0.700, which is lower than that of BiLSTM-CRF. The Precision, Recall and F1 values ​​of the model on the test set are 0.538, 0.558 and 0.548 respectively. The overall performance is not as good as that of BiLSTM-CRF and the method of Example 1 of the present invention, especially in terms of F1 value.

[0065] In general, the method proposed in Example 1 of the present invention outperforms all baseline models on the test set, especially in terms of F1 value, which is improved by 0.104, 0.244, and 0.259 compared with BERT-CRF, BERT-Softmax, and BERT-LSTM-CRF models, respectively. This shows that the method in Example 1 of the present invention has good comprehensive performance in all indicators, proving its effectiveness and superiority in processing related tasks.

[0066] Embodiment 3:

[0067] Embodiment 3 of the present invention discloses a text element recognition system based on multi-level feature fusion adaptive integration technology using the text element recognition method based on multi-level feature fusion adaptive integration technology described in Embodiment 1, comprising:

[0068] A sub-model training module is used to pre-process the data text and train multiple text element recognition sub-models based on the pre-processed data text;

[0069] Dynamic weight acquisition module: used to perform multi-level feature extraction and feature fusion on the preprocessed data text, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models;

[0070] Text element recognition output module: Based on the output results and confidence scores of multiple text element recognition sub-models, combined with dynamic weights, the final text element recognition output is weighted.

[0071] The embodiment of the present invention discloses a method and system for identifying text elements based on an adaptive integration technology of multi-level feature fusion. By fusing multi-level features, including lexical features, syntactic features and deep semantic features, these features are used to identify judicial text elements in the patent field, solving the problem that the traditional single model is difficult to fully process complex text features, and using multiple sub-models, such as ALBERT, BERT-CRF, etc., by training the pre-processed judicial text data, fully learning the text features, using the adaptive integration algorithm based on the deep neural network to learn the weights of the fused features, and improving the prediction performance of the model through dynamic weight adjustment, significantly improving the accuracy of element recognition, and combining the weights of each sub-model output by the weight learner with the prediction results and their confidence. In the training process of the weight learner, the model parameters are optimized by using the stochastic gradient descent and back propagation algorithms, effectively reducing the prediction error of the model, and conducting experiments on the LAIC2023 data set to verify the significant effect of the method in improving recognition accuracy and robustness, and the adaptability of the model under different types of texts is enhanced. In summary, the embodiment of the present invention effectively solves the recognition accuracy problem caused by incomplete feature extraction and poor model adaptability in the prior art.

[0072] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0073] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text element recognition method based on multi-level feature fusion and adaptive integration technology, characterized in that: include: Step 1: Preprocess the data text, and train multiple text element recognition sub-models based on the preprocessed data text; Step 2: Perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train weight learners of multiple text element recognition sub-models, and output dynamic weights of multiple text element recognition sub-models; Step 3: Based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, weighted results are obtained to obtain the final text element recognition output.

2. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 1, the data text is preprocessed, including data cleaning, denoising and standardization.

3. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 1, the multiple text element recognition sub-models include but are not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, and BiLSTM-CRF model.

4. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 2, multi-level feature extraction is performed on the preprocessed data text, including: lexical feature extraction, syntactic feature extraction and deep semantic feature extraction.

5. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 4 is characterized in that: In step 2, feature fusion is as follows: F 融合 =αF 词汇 +βF 句法 +γF 语义 ; Among them, F 融 is the feature after multi-level feature fusion; α, β, γ are the feature weighting coefficients of the lexical feature, syntactic feature and deep semantic feature respectively; F 词汇 is the vocabulary feature; F 句法 is the syntactic feature; F 语义 is the deep semantic feature.

6. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 2, a weight learner of a plurality of the text element recognition sub-models is trained, and the dynamic weights of the plurality of the text element recognition sub-models are outputted as follows: W 模型 =softmax(f 权重学习器 (F 融合 )); Among them, W 模型 is the dynamic weight of the text element recognition sub-model; softmax is the normalized softmax function; f 权重学习器 is the calculation function of the weight learner; F 融合 It is the feature after multi-level feature fusion.

7. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 2, in the process of training the weight learners of the multiple text element recognition sub-models, it also includes: updating the parameters of the weight learners using stochastic gradient descent and back propagation algorithms.

8. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 3, based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, the final text element recognition output is weighted as follows: Among them, S 输出 is the final text element recognition output; n is the number of text element recognition sub-models; W 模型，i is the dynamic weight of the i-th text element recognition sub-model; P 模型，i is the output result of the i-th text element recognition sub-model; C 模型，i The confidence of the i-th text feature recognition sub-model.

9. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 3, the confidence of the text element recognition sub-model is calculated by the softmax function.

10. A text element recognition system based on multi-level feature fusion adaptive integration technology using the text element recognition method based on multi-level feature fusion adaptive integration technology described in any one of claims 1 to 9, characterized in that: include: A sub-model training module is used to pre-process the data text and train multiple text element recognition sub-models based on the pre-processed data text; Dynamic weight acquisition module: used to perform multi-level feature extraction and feature fusion on the preprocessed data text, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models; Text element recognition output module: based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, weighted to obtain the final text element recognition output.

Images