Element extraction method, device and equipment based on multi-task learning and medium

Abstract

The present disclosure provides an element extraction method and device based on multi-task learning, equipment and medium, the method comprises: obtaining the text data of the target field for preprocessing to construct the field corpus, based on the field corpus, the pre-training language model is adaptively trained to obtain the field language representation model; construct a multi-task learning model; based on the sample data set carrying the labeled information and the preset field knowledge graph, the multi-task learning model is jointly trained; the target text to be processed is input into the trained multi-task learning model, and the initial prediction results of multiple extraction tasks are output in parallel; the initial prediction results are processed by the arbitration layer located at the output end of the multi-task learning model to solve the conflict, and the processing results are converted into the element extraction results in the preset structured data format. Therefore, the extraction accuracy can be improved by multi-task fusion and knowledge constraint, and the logical consistency of the element extraction results can be ensured by using the arbitration layer to eliminate conflicts.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to a method, apparatus, device, and medium for element extraction based on multi-task learning. Background Technology

[0002] With the popularization of artificial intelligence technology, natural language processing (NLP) technology is widely used in the automated information processing of professional documents (such as professional agreements and contracts) to achieve rapid extraction of key text elements. While existing element extraction schemes can perform basic text analysis, they still face the following significant technical bottlenecks in practical applications:

[0003] First, most conventional pre-trained language models are trained on general corpora. When directly applied to highly specialized domain texts, they lack sufficient awareness of technical terms and specific syntactic patterns. Such "general language models" struggle to accurately capture deep semantic features when dealing with highly rigorous professional texts, thus limiting the accuracy of low-level feature representations.

[0004] Secondly, feature extraction in the target domain typically encompasses multiple sub-tasks such as entity recognition, relation extraction, and text clause classification. Traditional methods often treat these tasks in isolation or employ simple stacked architectures, failing to establish effective underlying encoding sharing mechanisms and feature fusion networks. The training process of existing models largely relies solely on labeled data, lacking external domain knowledge graphs or business rules as constraints. This results in low collaborative extraction efficiency when processing complex texts with overlapping information, making it difficult to guarantee the accuracy and reliability of prediction results.

[0005] Furthermore, when a model outputs predictions for multiple tasks in parallel, logical conflicts between results are easily generated due to prediction biases or limitations in understanding local contexts between tasks. For example, the model may identify a subject entity, but fail to classify it into any reasonable logical category during classification or relation extraction. Because existing technologies typically lack post-verification and arbitration mechanisms for the results of parallel multi-task outputs, the final output structured element data suffers from numerous logical inconsistencies, making it difficult for downstream risk warning or knowledge graph construction systems to directly call and integrate it.

[0006] In summary, there is an urgent need in this field for an element extraction method that can deeply adapt to the professional domain context, efficiently integrate multi-task features, and automatically eliminate logical conflicts in parallel prediction, so as to effectively ensure the accuracy and logical consistency of the output results. Summary of the Invention

[0007] This disclosure aims to at least partially address one of the technical problems in the related art.

[0008] Therefore, the purpose of this disclosure is to propose a feature extraction method, device, electronic device, and storage medium based on multi-task learning. By adaptively training the target domain corpus and combining it with a multi-task shared structure, the pre-trained language model's ability to represent and extract features from professional texts is effectively improved. At the same time, by introducing knowledge constraints during training and adding an arbitration layer at the output end, not only is the accuracy of multi-task concurrent processing enhanced, but logical conflicts in the prediction process are also automatically eliminated, thereby ensuring the logical consistency and reliability of the final structured feature extraction results.

[0009] To achieve the above objectives, the first aspect of this disclosure proposes a feature extraction method based on multi-task learning, comprising: Text data from the target domain is acquired and preprocessed to construct a domain corpus. Based on the domain corpus, a pre-trained language model is adaptively trained to obtain a domain language representation model. A multi-task learning model is constructed, wherein the multi-task learning model includes a shared underlying encoder based on the domain language representation model and multiple task decoding networks connected in parallel, wherein the multiple task decoding networks perform feature fusion through a shared attention network; The multi-task learning model is jointly trained based on a sample dataset with labeled information and a preset domain knowledge graph. The target text to be processed is input into the trained multi-task learning model, and the initial prediction results of multiple extraction tasks are output in parallel. The initial prediction results are processed by an arbitration layer located at the output of the multi-task learning model to resolve conflicts, and the processing results are converted into feature extraction results in a preset structured data format.

[0010] To achieve the above objectives, the feature extraction apparatus based on multi-task learning proposed in the second aspect of this disclosure includes: The acquisition module is used to acquire text data of the target domain, preprocess it to build a domain corpus, and adaptively train the pre-trained language model based on the domain corpus to obtain a domain language representation model. The model building module is used to build a multi-task learning model, wherein the multi-task learning model includes a shared underlying encoder based on the domain language representation model and multiple task decoding networks connected in parallel, wherein the multiple task decoding networks perform feature fusion through a shared attention network; The training module is used to jointly train the multi-task learning model based on a sample dataset carrying labeled information and a preset domain knowledge graph. The output module is used to input the target text to be processed into the trained multi-task learning model and output the initial prediction results of multiple extraction tasks in parallel. The processing module is used to perform conflict resolution processing on the initial prediction results through the arbitration layer located at the output end of the multi-task learning model, and convert the processing results into feature extraction results in a preset structured data format.

[0011] The electronic device proposed in the third aspect of this disclosure includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the feature extraction method based on multi-task learning as proposed in the first aspect of this disclosure.

[0012] The fourth aspect of this disclosure provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the feature extraction method based on multi-task learning as proposed in the first aspect of this disclosure.

[0013] A fifth aspect of this disclosure provides a computer program product in which, when instructions are executed by a processor, the element extraction method based on multi-task learning as proposed in a first aspect of this disclosure is performed.

[0014] The feature extraction method, apparatus, electronic device, and storage medium based on multi-task learning disclosed herein preprocess text data from the target domain to construct a domain corpus, and adaptively train a pre-trained language model based on the domain corpus to obtain a domain language representation model; construct a multi-task learning model, wherein the multi-task learning model includes a shared underlying encoder based on the domain language representation model and multiple parallel task decoding networks, which fuse features through a shared attention network; jointly train the multi-task learning model based on a sample dataset with labeled information and a pre-defined domain knowledge graph; input the target text to be processed into the trained multi-task learning model, and output initial prediction results for multiple extraction tasks in parallel; resolve conflicts in the initial prediction results through an arbitration layer located at the output of the multi-task learning model, and convert the processed results into feature extraction results in a pre-defined structured data format. Therefore, by adaptively training the target domain corpus and combining it with a multi-task sharing structure, the pre-trained language model's ability to represent and extract features from professional texts is effectively improved. At the same time, introducing knowledge constraints during training and adding an arbitration layer at the output not only enhances the accuracy of multi-task concurrent processing but also automatically eliminates logical conflicts in the prediction process, thereby ensuring the logical consistency and reliability of the final structured element extraction results.

[0015] Additional aspects and advantages of this disclosure will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this disclosure. Attached Figure Description

[0016] The above and / or additional aspects and advantages of this disclosure will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 This is a flowchart illustrating a feature extraction method based on multi-task learning proposed in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a feature extraction method based on multi-task learning proposed in another embodiment of this disclosure; Figure 3 This is a schematic diagram of the structure of a feature extraction device based on multi-task learning according to an embodiment of this disclosure; Figure 4 This is a block diagram of an electronic device according to an embodiment of the present application. Detailed Implementation

[0017] Embodiments of this disclosure are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are used only to explain this disclosure, and should not be construed as limiting this disclosure. Rather, embodiments of this disclosure include all variations, modifications, and equivalents falling within the spirit and scope of the appended claims.

[0018] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0019] Figure 1 This is a flowchart illustrating a feature extraction method based on multi-task learning proposed in one embodiment of this disclosure.

[0020] It should be noted that the execution subject of the feature extraction method based on multi-task learning in this embodiment is a feature extraction device based on multi-task learning. This device can be implemented by software and / or hardware. The device can be configured in an electronic device, which may include, but is not limited to, a terminal, a server, etc., such as a mobile phone, a PDA, etc.

[0021] like Figure 1 As shown, this feature extraction method based on multi-task learning includes: S101: Obtain text data from the target domain, preprocess it to construct a domain corpus, and adaptively train the pre-trained language model based on the domain corpus to obtain a domain language representation model.

[0022] The target domain refers to a pre-defined application scenario or business scope with a specific industry background and professional terminology.

[0023] Domain corpora refer to standardized text collections that are specifically collected, organized, and preprocessed for a target domain, reflecting the specific professional vocabulary, syntactic patterns, and textual structures of that domain.

[0024] Among them, pre-trained language models refer to deep neural network models that have been initially trained using massive amounts of unlabeled text data and have acquired basic general language comprehension capabilities and the ability to represent underlying lexical and syntactic features.

[0025] Domain-specific language representation models refer to feature representation models that, after being trained on top of basic pre-trained language models using target domain corpora, possess the ability to understand semantics in that domain.

[0026] In this embodiment, when text data of the target domain is acquired and preprocessed to construct a domain corpus, and a domain language representation model is obtained by adaptively training a pre-trained language model based on the domain corpus, redundant noise in the original text can be effectively removed by performing standardized preprocessing and continued adaptive training on the text data of a specific domain. This transforms the general language model into a domain language representation model that deeply fits the specific context, thereby improving the model's ability to perceive and capture professional terms and unique syntactic patterns in a specific domain, and providing an accurate semantic foundation for the high-quality extraction of key features in the future.

[0027] S102: Construct a multi-task learning model, which includes a shared underlying encoder based on a domain language representation model and multiple parallel task decoding networks. The multiple task decoding networks fuse features through a shared attention network.

[0028] Among them, the multi-task learning model refers to a deep learning architecture that can learn and execute multiple related extraction tasks simultaneously, and improve the generalization ability of each sub-task and the overall collaborative prediction performance by sharing underlying feature representations.

[0029] In this context, the shared underlying encoder can refer to a network module in a multi-task model structure that is used to uniformly receive input text and extract common underlying semantic features, and is visible and shared by all downstream task networks.

[0030] The task decoding network refers to an independent network branch connected after the shared underlying encoder, specifically designed for feature mapping and result output for a particular extraction task.

[0031] Attention networks are mechanisms that dynamically assign different attention weights to input features based on their importance, thereby optimizing the fusion of feature information across different tasks.

[0032] In this embodiment, by constructing a multi-task learning model that includes a shared underlying encoder and multiple parallel decoding networks, and introducing a shared attention network, the semantic features of different feature extraction tasks can be shared and deeply integrated at the underlying level. This not only avoids the computational redundancy caused by the fragmented processing of multiple extraction tasks and improves the efficiency of concurrent processing, but also effectively captures the potential correlation between each sub-task, thereby enhancing the collaborative representation effect of multi-dimensional features in the fusion process.

[0033] S103: Jointly train a multi-task learning model based on a sample dataset with labeled information and a pre-defined domain knowledge graph.

[0034] Domain knowledge graphs refer to structured knowledge bases that organize and store objective entities, business concepts, and their interrelationships within a specific professional domain in the form of graph structures.

[0035] In this embodiment, by combining a sample dataset with labeled information with a pre-defined domain knowledge graph for joint training of the model, a deep fusion of data features and prior industry knowledge is achieved. Incorporating structured domain knowledge relationships as constraints into the joint training process effectively standardizes and limits the model's prediction boundaries, corrects learning biases, and thus significantly enhances the model's output accuracy and result reliability when facing complex and ambiguous text environments.

[0036] S104: Input the target text to be processed into the trained multi-task learning model and output the initial prediction results of multiple extraction tasks in parallel.

[0037] In this embodiment, by inputting the target text into the trained model and outputting the initial prediction results of multiple tasks in parallel, efficient concurrent extraction of multi-dimensional text elements is achieved, which greatly shortens the overall response time of the element extraction system and ensures that each specialized decoding network can efficiently output its own preliminary extraction conclusions within the same computation cycle based on a unified underlying representation, providing a complete data foundation for subsequent conflict resolution and fusion synchronization.

[0038] S105: Through the arbitration layer located at the output of the multi-task learning model, conflict resolution is performed on the initial prediction results, and the processing results are converted into feature extraction results in a preset structured data format.

[0039] The arbitration layer refers to the post-verification and processing module deployed at the output end of the multi-task learning model, which is specifically used to receive the parallel output results of multiple tasks and resolve logical conflicts between the results according to preset logic.

[0040] Element extraction refers to the process of automatically identifying and classifying key information (such as subject, time, amount, business terms, etc.) with specific semantic attributes from unstructured target domain text.

[0041] Structured data format refers to data organized according to fixed rules or templates (such as key-value pairs or tree-like label structures), which facilitates direct parsing and storage by computer systems or downstream business modules.

[0042] In this embodiment, text data from the target domain is preprocessed to construct a domain corpus. A pre-trained language model is then adaptively trained based on this corpus to obtain a domain language representation model. A multi-task learning model is constructed, comprising a shared underlying encoder based on the domain language representation model and multiple parallel task decoding networks. These networks fuse features through a shared attention network. The multi-task learning model is jointly trained based on a sample dataset with labeled information and a pre-defined domain knowledge graph. The target text to be processed is input into the trained multi-task learning model, which outputs initial prediction results for multiple extraction tasks in parallel. An arbitration layer at the output of the multi-task learning model resolves conflicts in the initial prediction results and converts the processed results into feature extraction results in a pre-defined structured data format. Therefore, by adaptively training the target domain corpus and combining it with a multi-task sharing structure, the pre-trained language model's ability to represent and extract features from professional texts is effectively improved. At the same time, introducing knowledge constraints during training and adding an arbitration layer at the output not only enhances the accuracy of multi-task concurrent processing but also automatically eliminates logical conflicts in the prediction process, thereby ensuring the logical consistency and reliability of the final structured element extraction results.

[0043] Optionally, in some embodiments, when preprocessing text data in the target domain, the following steps may be taken: performing text cleaning, word segmentation, and length standardization on the text data to obtain intermediate text; and then labeling entities in the intermediate text using a preset sequence labeling format. Thus, by performing standardized preprocessing operations such as text cleaning, word segmentation, and length standardization, and in conjunction with a standard sequence labeling format, redundant and interfering information in the original input text can be effectively removed, providing standardized, high-quality basic sample data for the subsequent training of multi-task learning models, thereby improving the stability of the model's extraction performance.

[0044] Text cleaning refers to the data preprocessing steps that remove invalid characters, noise interference, or non-standard formatting symbols from the original acquired text.

[0045] Among them, length unification processing refers to the operation of adjusting the input text sequence of varying lengths to a fixed input dimension required by the model through methods such as truncation or edge padding.

[0046] Among them, the preset sequence labeling format refers to the standardized labeling rules that assign specific labels to each basic character or word in a text sequence to accurately indicate whether it belongs to the boundary and category of a certain entity.

[0047] An entity refers to an independent information fragment or word combination that objectively exists in the text and has a specific meaning, attribute, or category.

[0048] Optionally, in some embodiments, the multiple task decoding networks include an entity recognition network, a relation extraction network, and a text classification network. When the multiple task decoding networks fuse features through a shared attention network, it can be done by: constructing the task output layer of a multi-task learning model; based on preset learning rates, using the shared attention network to perform a fully connected operation on the outputs of at least two task decoding networks, and inputting the result of the fully connected operation into a shared hidden layer to achieve information sharing. Thus, by constructing a multi-task decoding network covering entity recognition, relation extraction, and text classification, and utilizing a shared attention network combined with different learning rates for fully connected and hidden layer information sharing, this network structure can effectively capture and fuse deep feature associations between different extraction tasks, significantly improving feature interaction efficiency and overall collaborative prediction performance during multi-task concurrent processing.

[0049] Among them, entity recognition network refers to the task decoding branch specifically used to locate and classify noun phrases with specific business attributes from text.

[0050] Among them, relation extraction network refers to the task decoding branch specifically used to determine and extract the inherent logical relationship between two or more entities in text.

[0051] Among them, the text classification network refers to the task decoding branch specifically used to identify and classify the business intent and clause category of a text paragraph or sentence.

[0052] Fully connected operation refers to a type of operation in neural networks, in which each node in one layer is connected to all nodes in the previous layer, in order to integrate input features from various dimensions.

[0053] The shared hidden layer refers to the internal processing layer in the network architecture that receives features fused from multiple task branches and provides a unified high-order feature representation for each task.

[0054] Optionally, in some embodiments, the joint training of the multi-task learning model based on a sample dataset carrying labeled information and a preset domain knowledge graph can be achieved by: constructing a joint loss function and assigning corresponding loss weights to different extraction tasks within the joint loss function; constructing a training set and a validation set based on the sample dataset; determining the changes in loss values ​​in the training set and validation set based on the joint loss function; iterating hyperparameters according to the changes in loss values, and using the hyperparameters when the loss values ​​remain stable as the target parameters of the multi-task learning model. Here, logical relationships extracted based on the preset domain knowledge graph or preset rule base are incorporated as constraints into the joint training process of the multi-task learning model. Thus, by assigning customized weights to different tasks in the joint loss function, the training bias problem caused by sample imbalance in some extraction tasks can be effectively alleviated. Furthermore, by dynamically iterating the loss values ​​to determine the optimal hyperparameters and incorporating domain knowledge relationships as hard constraints into the training process, the prediction accuracy and boundary control capability of the model in complex contexts can be further improved.

[0055] The joint loss function is a mathematical function that combines the independent loss functions of each subtask in a multi-task model according to specific rules. It is used to measure the overall prediction error of the model and guide parameter iteration.

[0056] The loss weight refers to the importance coefficient assigned to each subtask in the joint loss function to balance the learning progress and sample ratio of different extraction tasks.

[0057] Hyperparameters refer to network configuration parameters (such as learning rate and batch size) that need to be manually set before model training begins, rather than being automatically learned and updated during model training.

[0058] The rule base refers to a set of judgment conditions that are pre-compiled and stored, reflecting the business logic, condition constraints, or industry standards of the target domain.

[0059] Figure 2 This is a flowchart illustrating a feature extraction method based on multi-task learning proposed in another embodiment of this disclosure.

[0060] like Figure 2 As shown, this feature extraction method based on multi-task learning includes: S201: Obtain text data from the target domain, preprocess it to construct a domain corpus, and adaptively train the pre-trained language model based on the domain corpus to obtain a domain language representation model.

[0061] S202: Construct a multi-task learning model, which includes a shared underlying encoder based on a domain language representation model and multiple parallel task decoding networks. The multiple task decoding networks fuse features through a shared attention network.

[0062] S203: Jointly train a multi-task learning model based on a sample dataset with labeled information and a pre-defined domain knowledge graph.

[0063] S204: Input the target text to be processed into the trained multi-task learning model and output the initial prediction results of multiple extraction tasks in parallel.

[0064] The descriptions of S201-S204 can be found in the above embodiments, and will not be repeated here.

[0065] S205: Compare the initial prediction results of the decoding network outputs of different tasks to identify logically inconsistent entities or relationships as conflict targets.

[0066] Among them, logical inconsistency refers to the state in which different results output by a multi-task model in parallel contradict each other in terms of common sense logic or business rules (such as an entity being identified as a subject, but not being classified into any reasonable category).

[0067] S206: Aggregate the initial prediction confidence, contextual features, and inter-task dependencies of the conflicting target across all task decoding networks to generate an arbitration evidence set.

[0068] Among them, the arbitration evidence set refers to a comprehensive collection of multi-dimensional information, including the original predicted probability values ​​of the model, textual context features, and logical dependencies between different extraction tasks, which is aggregated to resolve conflicts in prediction results.

[0069] S207: Input the arbitration evidence set into the preset rule engine or lightweight discriminant model to generate an adjudication instruction, modify or eliminate the initial prediction results corresponding to the conflict target based on the adjudication instruction, and convert the processing results into element extraction results in a preset structured data format.

[0070] The rule engine is a computational component that, based on preset logical judgment conditions, reasons and judges the input set of arbitration evidence and outputs the final decision result.

[0071] Lightweight discriminant models are machine learning models with fewer parameters and lower computational resource consumption, specifically designed for quickly distinguishing or classifying a small number of specific features (such as conflicting evidence).

[0072] In other words, in this embodiment, after outputting the initial prediction results of multiple extraction tasks in parallel, the initial prediction results output by different task decoding networks can be compared to identify logically inconsistent entities or relationships as conflict targets. The initial prediction confidence, contextual features, and inter-task dependencies of the conflict targets in all task decoding networks are aggregated to generate an arbitration evidence set. This arbitration evidence set is then input into a preset rule engine or lightweight discriminant model to generate an adjudication instruction, and the initial prediction results corresponding to the conflict targets are corrected or eliminated based on the adjudication instruction. Thus, a post-arbitration mechanism including conflict comparison, evidence aggregation, and instruction adjudication is constructed to address potential logical deviations during multi-task parallel prediction. This allows for accurate location of logically inconsistent conflict targets and automatic correction or elimination based on multi-dimensional contextual features and dependency evidence, effectively overcoming the error risks caused by the limitations of a single model's contextual understanding.

[0073] S208: Use knowledge distillation and model quantization techniques to compress the parameters of the trained multi-task learning model, and deploy the compressed model as a callable interface service.

[0074] Knowledge distillation is a model compression technique that guides lightweight models to learn and approximate the output distribution of complex, deep, large models, thereby reducing model size while retaining core extraction capabilities.

[0075] Model quantization technology refers to the technique of converting parameters in a neural network model that were originally represented by high-precision numerical values ​​into low-precision numerical values ​​in order to reduce model storage space and inference computing resource consumption.

[0076] Interface services refer to software services that encapsulate trained models into standardized calling modules, allowing external systems to directly obtain feature extraction results by sending network requests.

[0077] S209: Obtain corrected data for the feature extraction results as incremental annotation samples.

[0078] Incremental labeled samples refer to new business text data that has been manually checked or corrected after the system has actually run and generated preliminary extraction results, which are used for subsequent model supplementary training.

[0079] S210: Incremental training of multi-task learning models based on incrementally labeled samples.

[0080] Incremental training refers to the process of fine-tuning and updating the model's parameters locally using newly collected incremental labeled samples, based on the model's existing parameters, to adapt it to new data features.

[0081] In this embodiment, after converting the processing results into feature extraction results in a preset structured data format, the trained multi-task learning model is compressed using knowledge distillation and model quantization techniques. The compressed model is then deployed as a callable interface service. Correction data for the feature extraction results is obtained as incremental annotation samples. Based on these incremental annotation samples, the multi-task learning model is incrementally trained. Therefore, by using knowledge distillation and model quantization techniques to lightweight compress the trained model, the computational resource consumption for post-deployment interface calls can be significantly reduced, and the response rate can be improved. Simultaneously, establishing a closed-loop feedback and incremental training mechanism based on correction data ensures that the extraction system can continuously adapt to the constantly updated text specifications and iterative needs of business scenarios.

[0082] In summary, based on the above embodiments, this disclosure combines a pre-trained language model with a multi-task learning mechanism for the following specific technical path for extracting text elements in a target domain: Step 1: Data Preparation and Preprocessing For text data in the target domain, it is necessary to acquire and organize standard text datasets of multiple categories to construct a domain corpus. First, a corpus is created for the text of the target category and preprocessed. The preprocessing process includes text cleaning, word segmentation, length standardization, and standardized entity annotation using a preset sequence labeling format. This constructs a high-quality labeled dataset that forms the cornerstone of model performance.

[0083] Step 2: Domain Adaptive Pre-training The general pre-trained language model is further pre-trained using the constructed domain corpus. This step significantly improves the model's ability to perceive specific domain-specific terminology and syntactic patterns, enabling the model to complete its professional transformation into a domain-specific language representation model.

[0084] Step 3: Constructing a multi-task learning model architecture The design incorporates a model architecture that includes a shared underlying encoder and multiple task-specific decoding networks. Each task decoding network is responsible for parallel extraction tasks such as entity recognition, relation extraction, and text clause classification. This multi-task learning model fuses information from different extraction tasks through a shared attention network. For example, it performs fully connected operations on the results of at least two tasks with different preset learning rates and inputs them into the same shared hidden layer to achieve information sharing of underlying features.

[0085] Step 4: Model Training and Optimization The multi-task learning model is jointly trained using a sample dataset with labeled information. The training process mainly involves: Joint loss function design: Different loss weights are assigned to different extraction tasks in the joint loss function to balance the learning progress of each task; for specific extraction tasks with imbalanced samples, a preset loss optimization strategy is used for adjustment.

[0086] Hyperparameter iteration and tuning: Monitor the changes in loss values ​​on the training and validation sets, and use the model hyperparameters that keep the loss values ​​stable as the target parameters.

[0087] Domain knowledge injection: Logical relationships extracted from a pre-defined domain knowledge graph or rule base are incorporated as constraints into the joint training process to improve the accuracy and reliability of the model output.

[0088] Step 5: Reasoning and Post-processing After training, the target text to be processed is input into the multi-task learning model, which outputs initial prediction results for multiple extraction tasks in parallel. Then, the crucial post-processing steps are performed: Results Fusion and Conflict Resolution: To resolve logical conflicts caused by prediction biases between tasks or limitations in contextual understanding, an arbitration layer is established at the output of the multi-task learning model to integrate, verify, and correct the initial prediction results. Conflict identification: Compare the outputs of decoding networks for different tasks, and mark and identify entities or relationships with logical inconsistencies as conflict targets.

[0089] Evidence collection: Aggregate the initial prediction confidence, contextual features, and inter-task dependencies of the conflicting target across all task decoding networks to generate an arbitration evidence set.

[0090] Arbitration decision: Input the set of arbitration evidence into a preset rule engine or lightweight discriminant model to generate an award instruction, and correct or eliminate contradictory prediction results based on the instruction.

[0091] Structured output: Convert the raw processing results output by the model into feature extraction results in a preset structured data format so that downstream systems can directly call and execute subsequent business processes.

[0092] Step 6: Deployment and Continuous Iteration The trained model is optimized for parameter compression and lightweighting using techniques such as knowledge distillation and model quantization to meet the requirements of practical applications for response speed and resource consumption. Ultimately, it is deployed as a callable interface service. After system launch, a feedback loop mechanism is established: manually corrected data of the feature extraction results is acquired as incremental annotation samples, and the model is periodically incrementally trained based on this data to continuously adapt to new text types and domain-specific changes.

[0093] Figure 3This is a schematic diagram of the structure of a feature extraction device based on multi-task learning proposed in an embodiment of this disclosure.

[0094] like Figure 3 As shown, the feature extraction device 30 based on multi-task learning includes: The acquisition module 301 is used to acquire text data of the target domain for preprocessing in order to build a domain corpus, and to adaptively train the pre-trained language model based on the domain corpus to obtain a domain language representation model. The model building module 302 is used to build a multi-task learning model, wherein the multi-task learning model includes a shared underlying encoder based on a domain language representation model and multiple task decoding networks connected in parallel, and the multiple task decoding networks fuse features through a shared attention network. Training module 303 is used to jointly train a multi-task learning model based on a sample dataset with labeled information and a preset domain knowledge graph. The output module 304 is used to input the target text to be processed into the trained multi-task learning model and output the initial prediction results of multiple extraction tasks in parallel. The processing module 305 is used to perform conflict resolution processing on the initial prediction results through the arbitration layer located at the output of the multi-task learning model, and convert the processing results into feature extraction results in a preset structured data format.

[0095] It should be noted that the foregoing explanation of the feature extraction method based on multi-task learning also applies to the feature extraction device based on multi-task learning in this embodiment, and will not be repeated here.

[0096] In this embodiment, text data from the target domain is preprocessed to construct a domain corpus. A pre-trained language model is then adaptively trained based on this corpus to obtain a domain language representation model. A multi-task learning model is constructed, comprising a shared underlying encoder based on the domain language representation model and multiple parallel task decoding networks. These networks fuse features through a shared attention network. The multi-task learning model is jointly trained based on a sample dataset with labeled information and a pre-defined domain knowledge graph. The target text to be processed is input into the trained multi-task learning model, which outputs initial prediction results for multiple extraction tasks in parallel. An arbitration layer at the output of the multi-task learning model resolves conflicts in the initial prediction results and converts the processed results into feature extraction results in a pre-defined structured data format. Therefore, by adaptively training the target domain corpus and combining it with a multi-task sharing structure, the pre-trained language model's ability to represent and extract features from professional texts is effectively improved. At the same time, introducing knowledge constraints during training and adding an arbitration layer at the output not only enhances the accuracy of multi-task concurrent processing but also automatically eliminates logical conflicts in the prediction process, thereby ensuring the logical consistency and reliability of the final structured element extraction results.

[0097] According to embodiments of this application, this application also provides an electronic device and a readable storage medium.

[0098] Figure 4 This is a block diagram of an electronic device according to an embodiment of the present application.

[0099] like Figure 4 As shown, the electronic device includes: The memory 401, the processor 402, and the computer instructions stored in the memory 401 and executable on the processor 402.

[0100] When processor 402 executes instructions, it implements the feature extraction method based on multi-task learning provided in the above embodiments.

[0101] Furthermore, electronic devices also include: Communication interface 403 is used for communication between memory 401 and processor 402.

[0102] The memory 401 is used to store computer instructions that can be executed on the processor 402.

[0103] Memory 401 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0104] The processor 402 is used to implement the feature extraction method based on multi-task learning in the above embodiments when executing the program.

[0105] If the memory 401, processor 402, and communication interface 403 are implemented independently, then the communication interface 403, memory 401, and processor 402 can be interconnected via a bus to complete communication between them. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0106] Optionally, in a specific implementation, if the memory 401, processor 402, and communication interface 403 are integrated on a single chip, then the memory 401, processor 402, and communication interface 403 can communicate with each other through an internal interface.

[0107] Processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application.

[0108] This application also proposes a computer program product that, when executed by an instruction processor, implements the feature extraction method based on multi-task learning according to the embodiments of this application.

[0109] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0110] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0111] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0112] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0113] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0114] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0115] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0116] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A feature extraction method based on multi-task learning, characterized in that, include: Text data from the target domain is acquired and preprocessed to construct a domain corpus. Based on the domain corpus, a pre-trained language model is adaptively trained to obtain a domain language representation model. A multi-task learning model is constructed, wherein the multi-task learning model includes a shared underlying encoder based on the domain language representation model and multiple task decoding networks connected in parallel, wherein the multiple task decoding networks perform feature fusion through a shared attention network; The multi-task learning model is jointly trained based on a sample dataset with labeled information and a preset domain knowledge graph. The target text to be processed is input into the trained multi-task learning model, and the initial prediction results of multiple extraction tasks are output in parallel. The initial prediction results are processed by an arbitration layer located at the output of the multi-task learning model to resolve conflicts, and the processing results are converted into feature extraction results in a preset structured data format.

2. The method as described in claim 1, characterized in that, The preprocessing of the acquired text data in the target domain includes: The text data is cleaned, segmented, and its length is standardized to obtain intermediate text. The entities in the intermediate text are labeled using a preset sequence labeling format.

3. The method as described in claim 1, characterized in that, The multiple task decoding networks include an entity recognition network, a relation extraction network, and a text classification network; The multiple task decoding networks perform feature fusion through a shared attention network, including: Construct the task output layer of the multi-task learning model; Based on different preset learning rates, the shared attention network is used to perform a fully connected operation on the outputs of at least two task decoding networks, and the result of the fully connected operation is input into a shared hidden layer to achieve information sharing.

4. The method as described in claim 1, characterized in that, The joint training of the multi-task learning model based on the sample dataset carrying labeled information and the preset domain knowledge graph includes: Construct a joint loss function, and assign corresponding loss weights to different extraction tasks in the joint loss function; Based on the aforementioned sample dataset, a training set and a validation set are constructed; Based on the joint loss function, determine the changes in the loss values ​​of the training set and the validation set; Hyperparameters are iterated based on the changes in the loss value, and the hyperparameters when the loss value remains stable are used as the target parameters of the multi-task learning model. The logical relationships extracted based on the preset domain knowledge graph or preset rule base are incorporated as constraints into the joint training process of the multi-task learning model.

5. The method as described in claim 1, characterized in that, The conflict resolution process for the initial prediction results via the arbitration layer located at the output of the multi-task learning model includes: By comparing the initial prediction results of the decoding network outputs of different tasks, entities or relationships with logical inconsistencies are identified as conflict targets. The initial prediction confidence, contextual features, and inter-task dependencies of the conflicting target in all task decoding networks are aggregated to generate an arbitration evidence set. The arbitration evidence set is input into a preset rule engine or lightweight discriminant model to generate an adjudication instruction, and the initial prediction results corresponding to the conflict target are corrected or eliminated based on the adjudication instruction.

6. The method as described in claim 1, characterized in that, After converting the processing results into feature extraction results in a preset structured data format, the method further includes: The multi-task learning model that has been trained is compressed using knowledge distillation and model quantization techniques, and the compressed model is deployed as a callable interface service. Obtain the corrected data of the extracted elements as incremental labeled samples; Based on the incrementally labeled samples, the multi-task learning model is incrementally trained.

7. A feature extraction device based on multi-task learning, characterized in that, include: The acquisition module is used to acquire text data of the target domain, preprocess it to build a domain corpus, and adaptively train the pre-trained language model based on the domain corpus to obtain a domain language representation model. The model building module is used to build a multi-task learning model, wherein the multi-task learning model includes a shared underlying encoder based on the domain language representation model and multiple task decoding networks connected in parallel, wherein the multiple task decoding networks perform feature fusion through a shared attention network; The training module is used to jointly train the multi-task learning model based on a sample dataset carrying labeled information and a preset domain knowledge graph. The output module is used to input the target text to be processed into the trained multi-task learning model and output the initial prediction results of multiple extraction tasks in parallel. The processing module is used to perform conflict resolution processing on the initial prediction results through the arbitration layer located at the output end of the multi-task learning model, and convert the processing results into feature extraction results in a preset structured data format.

8. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the steps of the method according to any one of claims 1-6.

Images