A text element recognition method and system based on an adaptive integration technology of multi-level feature fusion

Abstract

The present invention discloses a text element recognition method and system based on an adaptive integration technology of multi-level feature fusion, which is applied to the field of natural language processing, and includes: preprocessing data texts, and training multiple text element recognition sub-models based on the preprocessed data texts; extracting multi-level features from the preprocessed data texts, and performing feature fusion to train a weight learner of multiple text element recognition sub-models, and outputting dynamic weights of multiple text element recognition sub-models; based on the output results and confidence scores of multiple text element recognition sub-models, combining with the dynamic weights, and obtaining the final text element recognition output through weighted summation. The present invention effectively solves the recognition accuracy problem caused by incomplete feature extraction and poor model adaptability in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and particularly to a method and system for text element recognition based on an adaptive integration technology of multi-level feature fusion. Background Art

[0002] Case element recognition is a key support and research hotspot for automated technology research such as similar case recommendation, sentence prediction, information extraction, etc. There are the following problems in the research on case element recognition: Traditional case element recognition methods mainly use the text features of judgment documents as the category prediction of case element labels, ignoring the judicial semantic information of case element labels; In judgment documents, the distribution of case elements is often unbalanced, resulting in the model being unable to fully learn the semantic features of small-sample elements and making it difficult to effectively identify them; Different from general multi-classification problems, case element recognition belongs to a multi-label classification problem, where a sample may belong to 0 - N categories at the same time, and there is often a correlation between multiple categories.

[0003] In the field of natural language processing, text element recognition is a key task. Especially in patent and judicial texts, due to the complexity and diversity of the text, accurately identifying elements faces huge challenges. Most traditional recognition methods rely on a single model, and this method has limited adaptability and is difficult to handle multi-level features in the text, resulting in insufficient recognition accuracy and robustness.

[0004] In the prior art, pre-trained language models such as BERT and ALBERT have been widely applied to natural language processing tasks. However, these models usually focus on specific text features and are difficult to comprehensively capture multi-level features in the text.

[0005] Therefore, how to provide a method and system for text element recognition based on an adaptive integration technology of multi-level feature fusion that can effectively integrate the prediction results of multiple models to improve the overall prediction performance is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0006] In view of this, the present invention proposes a method and system for text element recognition based on an adaptive integration technology of multi-level feature fusion.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A method for text element recognition based on an adaptive integration technology of multi-level feature fusion, comprising:

[0009] Step 1: Preprocess the data text, and train multiple text element recognition sub-models based on the preprocessed data text;

[0010] Step 2: Perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models;

[0011] Step 3: Based on the output results and confidence scores of multiple said text element recognition sub-models, combined with the dynamic weights, obtain the final text element recognition output through weighting.

[0012] Optionally, in Step 1, preprocessing the data text includes: data cleaning, denoising, and normalization processing.

[0013] Optionally, in Step 1, multiple text element recognition sub-models include, but are not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, BiLSTM-CRF model.

[0014] Optionally, in Step 2, performing multi-level feature extraction on the preprocessed data text includes: lexical feature extraction, syntactic feature extraction, and deep semantic feature extraction.

[0015] Optionally, in Step 2, the feature fusion is as follows:

[0016] F 融合 = αF 词汇 + βF 句法 + γF 语义 ;

[0017] where F 融 is the feature after multi-level feature fusion; α, β, and γ are the feature weighting coefficients of lexical features, syntactic features, and deep semantic features respectively; F 词汇 is the lexical feature; F 句法 is the syntactic feature; F 语义 is the deep semantic feature.

[0018] Optionally, in Step 2, training the weight learners of multiple text element recognition sub-models and outputting the dynamic weights of multiple text element recognition sub-models is as follows:

[0019] W 模型 = softmax(f 权重学习器 (F 融合 ));

[0020] where W 模型 is the dynamic weight of the text element recognition sub-model; softmax is the normalized softmax function; f 权重学习器 is the calculation function of the weight learner; F 融合 is the feature after multi-level feature fusion.

[0021] Optionally, in step 2, during the process of training the weight learners of multiple text element recognition sub-models, it further includes: updating the parameters of the weight learners by using the stochastic gradient descent and backpropagation algorithms.

[0022] Optionally, in step 3, based on the output results and confidence scores of multiple text element recognition sub-models, combined with the dynamic weights, the weighted final text element recognition output is obtained as follows:

[0023]

[0024] where S 输出 is the final text element recognition output; n is the number of text element recognition sub-models; W 模型，i is the dynamic weight of the i-th text element recognition sub-model; P 模型，i is the output result of the i-th text element recognition sub-model; C 模型，i is the confidence of the i-th text element recognition sub-model.

[0025] Optionally, in step 3, the confidence of the text element recognition sub-model is calculated by the softmax function.

[0026] The present invention also provides a text element recognition system based on a multi-level feature fusion-based adaptive integration technology for a text element recognition method using a multi-level feature fusion-based adaptive integration technology, including:

[0027] A sub-model training module, configured to preprocess the data text and train multiple text element recognition sub-models based on the preprocessed data text;

[0028] A dynamic weight acquisition module: configured to perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models;

[0029] A text element recognition output module: based on the output results and confidence scores of multiple text element recognition sub-models, combined with the dynamic weights, the weighted final text element recognition output is obtained.

[0030] As can be seen from the above technical solutions, compared with the prior art, the present invention proposes a method and system for text element recognition based on an adaptive integration technology with multi-level feature fusion. By fusing multi-level features, including lexical features, syntactic features, and deep semantic features, these features are used to recognize judicial text elements in the patent field, solving the problem that traditional single models are difficult to comprehensively process complex text features. Moreover, by using multiple sub-models, such as ALBERT, BERT-CRF, etc., through training on preprocessed judicial text data, the text features are fully learned. An adaptive integration algorithm based on a deep neural network is used to learn the weights of the fused features, and the prediction performance of the model is improved by dynamically adjusting the weights, significantly improving the accuracy of element recognition. The weights of each sub-model output by the weight learner are combined with the prediction results and their confidence levels. During the training process of the weight learner, the model parameters are optimized by adopting the stochastic gradient descent and backpropagation algorithms, effectively reducing the prediction error of the model. Experiments are carried out on the LAIC2023 dataset to verify the significant effect of this method in improving the recognition accuracy and robustness, and the adaptability of the model under different types of texts is enhanced. In summary, the present invention effectively solves the recognition accuracy problem in the prior art caused by incomplete feature extraction and poor model adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained according to the provided drawings.

[0032] Figure 1 It is a schematic flow chart of the method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0033] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0034] Embodiment 1:

[0035] Embodiment 1 of the present invention discloses a method for text element recognition based on an adaptive integration technology with multi-level feature fusion, as Figure 1 shown, including:

[0036] Step 1: Preprocess the data text and train multiple text element recognition sub-models based on the preprocessed data text.

[0037] Preprocess the data text, including: data cleaning to remove noise and redundant information in the text, denoising to remove noise characters and special symbols in the text using regular expressions and other text processing techniques, and normalization to unify the text format to ensure data quality.

[0038] Multiple text element recognition sub-models, including but not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, BiLSTM-CRF model. These models are each good at different types of feature processing and help capture different-dimensional information in the text.

[0039] Step 2: Perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train the weight learners of multiple text element recognition sub-models, and output the dynamic weights of multiple text element recognition sub-models.

[0040] Perform multi-level feature extraction on the preprocessed data text, including: lexical feature extraction, syntactic feature extraction, and deep semantic feature extraction.

[0041] Lexical feature extraction: Use statistical methods such as term frequency-inverse document frequency (TF-IDF) to extract features at the lexical level, reflecting the importance of words in the document.

[0042] Syntactic feature extraction: Extract the syntactic structure of the text through a dependency parsing tool (such as Stanford Parser), identify grammatical relationships such as subject, predicate, and object to capture sentence structure information.

[0043] Semantic feature extraction: Based on a pre-trained language model (such as BERT), extract deep semantic features to capture implicit meanings in the context to improve the understanding of complex legal terms.

[0044] Feature fusion is as follows:

[0045] F 融合 = αF 词汇 + βF 句法 + γF 语义 ;

[0046] Where F 融 is the feature after multi-level feature fusion; α, β, and γ are the feature weighting coefficients of lexical features, syntactic features, and deep semantic features respectively; F 词汇 is the lexical feature; F 句法 is the syntactic feature; F 语义 is the deep semantic feature.

[0047] Train the weight learner of multiple text element recognition sub - models, and output the dynamic weights of multiple text element recognition sub - models as follows:

[0048] W 模型 = softmax(f 权重学习器 (F 融合 ));

[0049] Among them, W 模型 is the dynamic weight of the text element recognition sub - model; softmax is the normalized softmax function; f 权重学习器 is the calculation function of the weight learner; F 融合 is the feature after multi - level feature fusion.

[0050] In the process of training the weight learner of multiple text element recognition sub - models, it also includes: using the stochastic gradient descent and backpropagation algorithms to update the parameters of the weight learner to minimize the prediction error and optimize the performance of the weight learner.

[0051] Step 3: Based on the output results and confidence scores of multiple text element recognition sub - models, combined with the dynamic weights, perform weighted summation to obtain the final text element recognition output. After obtaining the text element recognition output result, perform post - processing operations, such as rule correction and cross - sentence verification, to further improve the accuracy and consistency of the recognition result.

[0052] Based on the output results and confidence scores of multiple text element recognition sub - models, combined with the dynamic weights, perform weighted summation to obtain the final text element recognition output as follows:

[0053]

[0054] Among them, S 输出 is the final text element recognition output; n is the number of text element recognition sub - models; W 模型，i is the dynamic weight of the i - th text element recognition sub - model; P 模型，i is the output result of the i - th text element recognition sub - model; C 模型，i is the confidence of the i - th text element recognition sub - model.

[0055] The confidence of the text element recognition sub - model is calculated by the softmax function.

[0056] Embodiment 2:

[0057] Embodiment 2 of the present invention discloses the specific application of the text element recognition method using the adaptive integration technology based on multi - level feature fusion described in Embodiment 1 on the judicial text dataset in the LAIC2023 patent field, including:

[0058] Using the text element recognition method of an adaptive integration technology based on multi-level feature fusion described in Embodiment 1 of the present invention and four baseline models BiLSTM-CRF, BERT-CRF, BERT-Softmax, and BERT-LSTM-CRF to perform text element recognition on the judicial text dataset in the LAIC2023 patent field, and output structured case element information including involved parties, behaviors, legal bases, etc. through means such as final rule correction and cross-sentence verification. Precision, Recall, and F1 values are selected as evaluation indicators, and the final recognition comparison results are shown in Table 1.

[0059] Table 1 Experimental comparison results of each model

[0060]

[0061] From Table 1, it can be seen that: The method proposed in Embodiment 1 of the present invention has better performance than the baseline models in terms of various indicators on the test set. Specifically, the Precision, Recall, and F1 values of Embodiment 1 of the present invention on the test set reached 0.654, 0.649, and 0.652 respectively, which are significantly higher than the indicators of other baseline models.

[0062] Comparing each baseline model, it can be found that:

[0063] Although the BiLSTM-CRF model has the highest F1 value of 0.749 on the validation set, its Precision, Recall, and F1 values on the test set are 0.649, 0.642, and 0.646 respectively. Although it performs well on the validation set, its comprehensive performance on the test set does not exceed the method of Embodiment 1 of the present invention. This model has a relatively large improvement compared to other models. This may be because softmax performs well in classification tasks, but in sequence labeling tasks, it may not be as effective as a model like CRF that is specifically designed for sequence labeling, because CRF can better consider the dependencies between labels, while Softmax mainly focuses on the independent prediction of individual labels.

[0064] The BERT-CRF model has an F1 value of 0.700 on the validation set, which is lower than that of BiLSTM-CRF. The Precision, Recall, and F1 values of this model on the test set are 0.538, 0.558, and 0.548 respectively. Its comprehensive performance is inferior to that of BiLSTM-CRF and the method of Embodiment 1 of the present invention, especially with a large gap in the F1 value.

[0065] Overall, the method proposed in Embodiment 1 of the present invention performs better than all baseline models on the test set. In particular, compared with the BERT-CRF, BERT-Softmax, and BERT-LSTM-CRF models, the F1 value is improved by 0.104, 0.244, and 0.259 respectively. This shows that the method of Embodiment 1 of the present invention has good comprehensive performance in various indicators, proving its effectiveness and superiority in dealing with related tasks.

[0066] Embodiment 3:

[0067] Embodiment 3 of the present invention discloses a text element recognition system based on a multi-level feature fusion adaptive integration technology using the text element recognition method of the multi-level feature fusion adaptive integration technology described in Embodiment 1, including:

[0068] A sub-model training module, configured to preprocess data texts and train multiple text element recognition sub-models based on the preprocessed data texts;

[0069] A dynamic weight acquisition module: configured to perform multi-level feature extraction on the preprocessed data texts, perform feature fusion, train weight learners of multiple text element recognition sub-models, and output dynamic weights of multiple text element recognition sub-models;

[0070] A text element recognition output module: based on the output results and confidence scores of multiple text element recognition sub-models, combined with dynamic weights, weighted to obtain the final text element recognition output.

[0071] An embodiment of the present invention discloses a method and system for text element recognition based on an adaptive integration technology of multi-level feature fusion. By fusing multi-level features, including lexical features, syntactic features, and deep semantic features, these features are used to recognize judicial text elements in the patent field, solving the problem that traditional single models are difficult to comprehensively process complex text features. Moreover, multiple sub-models, such as ALBERT, BERT-CRF, etc., are used to train the preprocessed judicial text data to fully learn the text features. An adaptive integration algorithm based on a deep neural network is used to learn the weights of the fused features, and the prediction performance of the model is improved by dynamic weight adjustment, significantly improving the accuracy of element recognition. The weights of each sub-model output by the weight learner are combined with the prediction results and their confidence levels. During the training process of the weight learner, the model parameters are optimized by using the stochastic gradient descent and backpropagation algorithms, effectively reducing the prediction error of the model. Experiments are carried out on the LAIC2023 dataset to verify the significant effect of this method in improving the recognition accuracy and robustness, and the adaptability of the model under different types of texts is enhanced. In summary, the embodiment of the present invention effectively solves the recognition accuracy problem in the prior art caused by incomplete feature extraction and poor model adaptability.

[0072] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments, and the same or similar parts among the various embodiments can be referred to each other. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0073] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be obvious to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown herein, but will be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A text element recognition method based on multi-level feature fusion and adaptive integration technology, characterized in that: include: Step 1: Preprocess the data text, and train multiple text element recognition sub-models based on the preprocessed data text; Step 2: Perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train weight learners of multiple text element recognition sub-models, and output dynamic weights of multiple text element recognition sub-models; Step 3: Based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, weighted to obtain the final text element recognition output; In step 2, a weight learner of a plurality of the text element recognition sub-models is trained, and the dynamic weights of the plurality of the text element recognition sub-models are outputted as follows: W 模型 =softmax(f 权重学习器 (F 融合 )); Among them, W 模型 is the dynamic weight of the text element recognition sub-model; softmax is the normalization function; f 权重学习器 is the calculation function of the weight learner; F 融合 It is the feature after multi-level feature fusion; In step 3, based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, the final text element recognition output is weighted as follows: Among them, S 输出 is the final text element recognition output; n is the number of text element recognition sub-models; W 模型，i is the dynamic weight of the i-th text element recognition sub-model; P 模型，i is the output result of the i-th text element recognition sub-model; C 模型，i The confidence of the i-th text feature recognition sub-model.

2. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 1, the data text is preprocessed, including data cleaning, denoising and standardization.

3. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 1, the multiple text element recognition sub-models include but are not limited to: ALBERT model, BERT-CRF model, BERT-LSTM-CRF model, BERT-Softmax model, and BiLSTM-CRF model.

4. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 2, multi-level feature extraction is performed on the preprocessed data text, including: lexical feature extraction, syntactic feature extraction and deep semantic feature extraction.

5. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 4 is characterized in that: In step 2, feature fusion is as follows: F 融合 =αF 词汇 +βF 句法 +γF 语义 ; Among them, F 融合 is the feature after multi-level feature fusion; α, β, γ are the feature weighting coefficients of the lexical feature, syntactic feature and deep semantic feature respectively; F 词汇 is the vocabulary feature; F 句法 is the syntactic feature; F 语义 is the deep semantic feature.

6. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 2, in the process of training the weight learners of the multiple text element recognition sub-models, it also includes: updating the parameters of the weight learners using stochastic gradient descent and back propagation algorithms.

7. The text element recognition method based on the adaptive integration technology of multi-level feature fusion according to claim 1 is characterized in that: In step 3, the confidence of the text element recognition sub-model is calculated by the softmax function.

8. A text element recognition system based on multi-level feature fusion adaptive integration technology using the text element recognition method based on multi-level feature fusion adaptive integration technology described in any one of claims 1 to 7, characterized in that: include: A sub-model training module is used to pre-process the data text and train multiple text element recognition sub-models based on the pre-processed data text; Dynamic weight acquisition module: used to perform multi-level feature extraction on the preprocessed data text, perform feature fusion, train weight learners of multiple text element recognition sub-models, and output dynamic weights of multiple text element recognition sub-models; Text element recognition output module: based on the output results and confidence scores of the multiple text element recognition sub-models, combined with the dynamic weights, weighted to obtain the final text element recognition output.

Images