Entity extraction and annotation method, device and storage medium

Abstract

This application discloses an entity extraction and annotation method, device, and storage medium, including: obtaining text data to be processed; extracting dynamic indicators reflecting text characteristics from the text data; determining the combined weights of a rule engine, a statistical model, and a deep learning model based on the dynamic indicators; configuring a hybrid recognition model based on the combined weights, and identifying entities in the text data using the hybrid recognition model; performing boundary verification and type annotation on the recognition results, and outputting structured entity information. This application optimizes multi-model collaborative decision-making through a dynamic weight allocation mechanism, significantly improving the extraction accuracy and annotation efficiency of key entities such as time and place in task texts, and realizing the integration of intelligent recognition of task elements and structured annotation.

Description

Technical Field

[0001] The present application relates to the technical field of text recognition, and in particular to an entity extraction and annotation method, device, and storage medium. Background Art

[0002] In some document analysis processes, accurately extracting key entity information such as time, location, and related individuals is a fundamental step in document analysis and processing. To achieve entity information extraction, the general approach is to perform entity recognition using a dictionary of predefined regular expressions and keywords. While this approach has good extraction results for highly structured text (such as tabular data in a fixed format), it is difficult to adapt to non-standardized expressions and cannot effectively process entities associated with contextual semantics. For example, if the text to be analyzed contains a non-standardized description of "around 8 p.m.," or if there is a description that requires determining entity relationships based on contextual semantics, this approach requires inferring the relationships between people in the technical problem. For specific cases, a large number of rules need to be manually maintained, resulting in high maintenance costs.

[0003] A more prominent issue is that existing solutions generally use a single model architecture that cannot accommodate the recognition needs of different entity types. This technical limitation directly leads to the need for manual secondary verification and integration of the output results of different models in actual task processing, seriously affecting task processing efficiency.

[0004] The above content is only used to assist in understanding the technical solution of this application and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide an entity extraction and annotation method, device and storage medium, aiming to solve the technical problem that the existing technology cannot realize the integrated processing of entity recognition and annotation for the specified standard processing text.

[0006] To achieve the above objectives, this application proposes a method for entity extraction and annotation, which includes:

[0007] Get the text data to be processed;

[0008] Extracting dynamic indicators reflecting text characteristics from the text data;

[0009] Determine the combined weight of the rule engine, statistical model and deep learning model based on the dynamic indicators;

[0010] configuring a hybrid recognition model according to the combined weights, and identifying entities in the text data by using the hybrid recognition model;

[0011] Perform boundary checking and type labeling on the recognition results, and output structured entity information.

[0012] In one embodiment, the step of extracting a dynamic indicator reflecting text characteristics from the text data includes:

[0013] Analyze the distribution pattern of relevant clause standard paragraphs in the text data and determine the target area for standard professional terminology concentration;

[0014] The relevance between the text representation and the field to which the text information belongs is evaluated according to the distribution law and the target area, and the dynamic indicator is extracted according to the evaluation result.

[0015] In one embodiment, the step of determining the combined weight of the rule engine, the statistical model, and the deep learning model based on the dynamic indicator includes:

[0016] Select a corresponding weight distribution strategy according to the application scenario type of the text data;

[0017] Allocate decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicators;

[0018] The step of allocating decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicator includes:

[0019] Determining the model characteristics corresponding to the dynamic indicators in a pre-established data relationship table;

[0020] Increase and / or decrease the decision weight of the data model corresponding to the model feature according to the weight allocation strategy.

[0021] In one embodiment, the entity extraction and annotation method further includes:

[0022] Real-time monitoring of the recognition performance of the rule engine, statistical model, and deep learning model on the text data;

[0023] The initial weight distribution of the rule engine, statistical model and deep learning model is dynamically modified based on the monitoring results and the determination results of the preset weight adjustment conditions.

[0024] In one embodiment, the step of configuring a hybrid recognition model according to the combined weights and identifying entities in the text data using the hybrid recognition model includes:

[0025] Analyze the output results of the hybrid recognition model, where the output results are entity candidate sets output in parallel by each data model;

[0026] Prioritizing the entities in the entity candidate set and filtering out entities below a confidence threshold;

[0027] The filtering result is used as the recognition result of the entity.

[0028] In one embodiment, the step of prioritizing the entities in the entity candidate set and filtering out entities below a confidence threshold comprises:

[0029] Sort the entities with high priority by confidence;

[0030] The ranking result is adjusted using the combined weights, and entities below a confidence threshold are filtered out from the adjusted ranking result using a weighted voting mechanism.

[0031] In one embodiment, the step of performing boundary checking and type labeling on the recognition results and outputting structured entity information includes:

[0032] Correcting the entity boundary position in the recognition result according to the punctuation mark distribution characteristics in the recognition result;

[0033] The target entity is obtained by filtering out the recognition results that do not meet the requirements using the preset entity length threshold;

[0034] Perform type annotation on the target entity and output structured entity information.

[0035] In one embodiment, the step of performing type annotation on the target entity and outputting structured entity information includes:

[0036] Recognize the text data through a bidirectional LSTM-CRF model, and predict the entity boundary area of ​​the target entity in the text data;

[0037] Correcting the position of the entity boundary area based on the syntax analysis tree to determine the annotation range of the target entity, and performing entity annotation on the annotation range;

[0038] Execute the step of outputting structured entity information.

[0039] In addition, to achieve the above-mentioned purpose, the present application also proposes an entity extraction and labeling device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the computer program is configured to implement the steps of the entity extraction and labeling method described above.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by the processor, the steps of the entity extraction and labeling method described above are implemented.

[0041] One or more technical solutions proposed in this application have at least the following technical effects:

[0042] The technical solution of this application obtains text data to be processed; extracts dynamic indicators reflecting text characteristics from the text data; determines the combined weights of the rule engine, statistical model, and deep learning model based on the dynamic indicators; configures a hybrid recognition model based on the combined weights, and identifies entities in the text data through the hybrid recognition model; performs boundary verification and type annotation on the recognition results, and outputs structured entity information. By optimizing multi-model collaborative decision-making through a dynamic weight allocation mechanism, the extraction accuracy and annotation efficiency of key entities such as time and place in task texts are significantly improved, achieving the technical effect of integrating intelligent recognition of task elements with structured annotation. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0045] Figure 1 A flowchart of the first embodiment of the entity extraction and annotation method of this application is provided;

[0046] Figure 2 Schematic diagram of the device structure of the hardware operating environment involved in the entity extraction and annotation method in the embodiment of the present application.

[0047] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0048] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0049] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0050] The main solution of the embodiment of the present application is: obtaining text data to be processed; extracting dynamic indicators reflecting text characteristics from the text data; determining the combination weights of the rule engine, statistical model and deep learning model based on the dynamic indicators; configuring a hybrid recognition model according to the combination weights, and identifying entities in the text data through the hybrid recognition model; performing boundary checking and type labeling on the recognition results, and outputting structured entity information.

[0051] Existing solutions generally use a single model architecture, which is unable to address the recognition needs of different entity types. This technical limitation directly leads to the need for manual secondary verification and integration of the output results of different models in actual task text processing, which seriously affects the processing efficiency of task text.

[0052] This application provides a solution that optimizes multi-model collaborative decision-making through a dynamic weight allocation mechanism, significantly improves the extraction accuracy and annotation efficiency of key entities such as time and place in task texts, and achieves the technical effect of integrating intelligent recognition and structured annotation of task elements.

[0053] Based on this, the embodiment of the present application provides a method for entity extraction and annotation, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the entity extraction and annotation method of this application. In this embodiment, the entity extraction and annotation method includes steps S10 to S50:

[0054] Step S10, obtaining text data to be processed;

[0055] Step S20, extracting dynamic indicators reflecting text characteristics from the text data;

[0056] Step S30, determining the combined weight of the rule engine, the statistical model, and the deep learning model based on the dynamic indicators;

[0057] Step S40, configuring a hybrid recognition model according to the combined weights, and identifying entities in the text data using the hybrid recognition model;

[0058] Step S50: perform boundary verification and type labeling on the recognition results, and output structured entity information.

[0059] This embodiment provides an integrated processing method for entity extraction and annotation in text data to be processed, which is essentially achieved through a multi-model collaborative mechanism with dynamic weight allocation. When the text data to be processed is obtained, dynamic indicators that can reflect the text characteristics are extracted from the text data to be processed. The dynamic indicators of text characteristics are used to quantify a set of key parameters that reflect the semantic characteristics, structural complexity and context relevance of the text, specifically including language complexity, domain term density and context relevance. Among them, the language complexity can be evaluated by calculating indicators such as the proportion of long and difficult sentences and the depth of nested clauses to assess the difficulty of text structuring. For example, when the Flesch-Kincaid readability index of the colloquial segment in the inquiry text is less than 30, it is marked as a text with high language complexity; the domain term density can be used to calculate the ratio of the standard entity word frequency to the general vocabulary. When the density is greater than 15%, it is determined to be highly professional content; the context relevance is based on the strength of the logical relationship between the entity items analyzed on the co-occurrence line, such as the co-occurrence frequency of "abnormal behavior individual-specific equipment".

[0060] Furthermore, a sliding window algorithm is used to detect fluctuations in local text features and identify mixed paragraphs, such as cross-modal texts containing both account flow data and inquiry ratios.

[0061] Specifically, the step of extracting dynamic indicators reflecting text characteristics from the text data includes:

[0062] Analyze the distribution pattern of relevant clause standard paragraphs in the text data and determine the target area for standard professional terminology concentration;

[0063] The relevance between the text representation and the field to which the text information belongs is evaluated according to the distribution law and the target area, and the dynamic indicator is extracted according to the evaluation result.

[0064] In this embodiment, quantitative analysis of standard text features is performed to provide a decision basis for subsequent model weight allocation. Specifically, a hierarchical analysis method is used to locate the relevant clause standard paragraphs of the text data to be processed, and the located results are subjected to structural feature analysis and distribution rule modeling.

[0065] In the structural feature analysis, a paragraph structure fingerprint is established by identifying the standard paragraph start mark, including references to standard clauses. For example, the expression type of the conclusion is ascertained after inspection, such as the conclusion type considered by this organization. The paragraph structure fingerprint uses regular expression matching tag words + context window verification (the first 5 words and the last 10 words) to form a three-dimensional fingerprint vector containing position, type, and hierarchy.

[0066] In the distribution law modeling, spatial distribution analysis is used to calculate the standard paragraph spacing and detect paragraph clustering. For example, on average, one key rule clause appears every two pages; the relevant rule clause paragraphs for specific abnormal events appear concentrated at the front of the text, etc.

[0067] Then, a time series curve of "normative expression intensity" was constructed using time series analysis to identify its periodic pattern, such as the confirmation of normative clauses appearing once every three rounds of questions and answers in the inquiry record text.

[0068] Based on the structural feature analysis and distribution rule modeling of the above positioning results, the target area where the professional terms are located is located in the processing results. During the positioning process, the professional terms can be matched based on the basic terminology library. Specifically, by loading the standard terminology knowledge base to build an enhanced Trie tree to achieve efficient matching, so as to locate the professional target area according to the matching results.

[0069] Among them, a multi-level Trie tree structure dedicated to the standardization field is constructed, and its core processing includes knowledge base preprocessing, hybrid index construction, and dynamic loading mechanism. Specifically, the knowledge base preprocessing can stratify the existing standard terms into multiple categories, establish parent-child node associations for designated nested terms, and add weight factors to each node; the hybrid index construction combines character-level Trie with word-level jump tables, and uses first character triggering + hash-assisted positioning for long terms to achieve O(1) average query efficiency; the dynamic loading mechanism needs to preload core term branches according to the application scenario type, and the memory usage is controlled within the predetermined memory amount.

[0070] Furthermore, through context expansion detection, the term influence radius of the located professional term is determined for the target area. Specifically, the specific value of the influence radius can be derived through empirical research, for example, the influence radius R = 5 sentences, and can be set accordingly based on the specific application scenario type represented by the text data to be processed.

[0071] As shown above, target regions are merged based on the determined term influence radius to generate continuous target regions. This merging is achieved using the DBSCAN clustering algorithm. Specifically, the positioning processing of the target regions where the professional terms are located can be characterized by the following example.

[0072] In addition, based on the target area positioning of the professional terminology obtained from the above analysis, it is also necessary to evaluate its relevance to the field to which the text information belongs to improve accuracy. This evaluation can be achieved through a preset regulatory clause relevance evaluation mechanism, which has a multi-dimensional evaluation system, including expression professionalism scoring and field consistency analysis. The expression professionalism scoring is achieved by using Legal-BERT to extract paragraph embeddings and calculate the cosine similarity between the target area positioning of the professional terminology and the regulatory clause text benchmark set. Based on the calculation results of the cosine similarity, the text perplexity is calculated in the pre-built language model of the field to which the text information belongs, and the consistency index is obtained based on the calculation results.

[0073] Then, dynamic indicators are generated from the consistency indicators. These dynamic indicators are output as an indicator matrix. In the indicator matrix, the indicator names include standard density, expression standardization, domain consistency, and structural complexity. The parameters represented by each indicator name have corresponding calculation methods and normalization ranges. Specific calculation methods include TermDensity / MAX_Density, Sigmoid(Ps*2 - 1), Consistency, and 1 - (standard paragraph spacing / MAX_spacing). The normalization range for the calculation method is related to the type of application scenario represented by the text data to be processed.

[0074] In this embodiment, a hybrid detection method combining structural analysis and semantic understanding is used to perform correlation analysis on the text data being processed. Dynamic parameters that reflect the characteristics of the corpus are then used to accurately select the regions where these dynamic indicators reside. This improves the accuracy of standard paragraph recognition and the precision of terminology region location. Furthermore, the multi-granularity evaluation system used to assess processing targets also improves the consistency of relevance assessment with manual judgment.

[0075] Based on the dynamic indicators extracted from the text data to be processed, the dynamic allocation of multi-model weights is determined. Specifically, the multi-module includes a rule engine, a statistical model, and a deep learning model. That is, based on the dynamic indicators, the rule engine, the statistical model, and the deep learning model are weighted to form a combined weight.

[0076] Among them, a weight distribution strategy library is pre-built, and a data table based on the mapping relationship between application scenario type and weight strategy is provided in the weight distribution strategy library, which includes multiple weight distribution strategies, so as to allocate the decision weight of each data model based on the weight distribution strategy. That is, the step of determining the combined weight of the rule engine, statistical model and deep learning model based on the dynamic indicator includes:

[0077] Select a corresponding weight distribution strategy according to the application scenario type of the text data;

[0078] Allocate decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicators;

[0079] The step of allocating decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicator includes:

[0080] Determining the model characteristics corresponding to the dynamic indicators in a pre-established data relationship table;

[0081] Increase and / or decrease the decision weight of the data model corresponding to the model feature according to the weight allocation strategy.

[0082] In the mapping relationship data table between application scenario types and weight strategies set in the weight allocation strategy library, the multiple weight allocation strategies stored can be divided into three types: term-dominant strategy, sentence-sensitive strategy and association-intensive strategy, which are respectively suitable for different application scenario types.

[0083] Specifically, each of the weight allocation strategies is provided with an initial weight allocation, and based on the initial weight allocation including the rule engine benchmark weight, the statistical model benchmark weight, and the deep learning model benchmark weight, the basic weight values ​​of each of the benchmark weights are different, and the basic weight values ​​are dynamically adjusted according to the complexity of the features used and the term density.

[0084] The dynamic adjustment process can be defined as a dynamic weight calculation process. In the dynamic weight calculation process, a basic strategy can be determined according to an application scenario type classifier, wherein the application scenario type classifier can obtain the initial weights of each data model by inputting the dynamic indicators of the text data to be processed, that is, the initial weights of the rule engine, statistical model, and deep learning model. The dynamic indicators to be input into the application scenario type classifier are the event causes, high-frequency entity type distribution, and data flow pattern characteristics marked in the document header of the text data to be processed, and the initial weights of the data models are set to 0.3-0.5-0.2 respectively.

[0085] Furthermore, the initial weights are adaptively adjusted using the dynamic indicators. Since each data model has different focuses on data processing, the adaptive adjustment is essentially a dynamic adjustment of the initial weights based on the data calculation tendencies of each data model (rule engine, statistical model, and deep learning model). The details are as follows:

[0086] The dynamic weight adjustment based on the rule engine is to increase the initial weight of the rule engine based on the baseline weight value for term density, standard clause references and complexity. For example, for every fixed percentage increase in term density, such as 5%, the weight is increased by a preset value, such as 0.1 (upper limit + 0.3); when a standard clause reference is detected, the weight is instantly increased by a preset value, such as 0.15; when the complexity is greater than the preset complexity threshold, the weight is reduced by a preset unit value, such as when the complexity is >0.8, the weight is reduced by 0.05 / 0.1 units. Among them, the baseline weight value based on the rule engine can be set according to the information represented by the initial weight, such as the above-mentioned 5% increase by 0.1, etc. An example of weight adjustment based on the above-mentioned rule engine is: the initial weight of a task is 0.6. When the term density is detected to reach a preset upper limit value, such as 25%, the final weight is increased to the upper limit weight value, such as 0.75.

[0087] The dynamic weight adjustment based on the statistical model is to increase or decrease the initial weight value based on complexity, uncommon word combinations and context stability, wherein the numerical range of the complexity is determined, and the initial weight of the statistical model is adjusted according to the data relationship between the baseline weight value and the complexity, and the baseline weight value is linearly correlated with the complexity, for example: when the complexity is in a preset numerical range, such as the range of 0.5-0.7, the weight is linearly positively correlated with the complexity; when an uncommon word combination is found, the initial weight is reduced by the baseline weight value, for example, the weight is reduced by 0.1; and, based on the comparison result of the context stability with the preset stability threshold, the initial weight is increased, for example, the weight compensation is increased by 0.05; in actual applications, the application scenario of the dynamic weight adjustment based on the statistical model can be: when a certain type of task text has a specific word, the statistical model weight is reduced from 0.5 to 0.4.

[0088] The dynamic weight adjustment based on the deep learning model is to adjust the initial weight of the deep learning model based on the text relevance, cross-paragraph reference and rule coverage of the text data to be processed. For example, the weight increases by 0.08 (upper limit + 0.25) for every 0.1 increase in relevance; when cross-paragraph reference is detected, the weight increases instantly by 0.1; when the rule coverage is >70%, the weight decreases by 0.05.

[0089] Finally, constraints are applied to the adjusted weights of each data model (rule engine, statistical model, and deep learning model). These constraints include, but are not limited to, setting upper and lower limits for each data model's weights and the sum of their weights. For example, ensure that the sum of rule engine weights ∈ [0.15, 0.8]; statistical model weights ∈ [0.1, 0.6]; and deep learning model weights ∈ [0.2, 0.75] is strictly equal to 1. Specifically, this can be determined based on the application scenario of the text data to be processed or through detailed analysis.

[0090] In this embodiment, in the weight allocation of each data model, prior knowledge of the application scenario type is provided through the basic allocation strategy, and the initial weight of each data model is adjusted with dynamic indicators to more accurately reflect the real-time characteristics of the text. Then, the two are fused through weighted decision-making, which not only maintains the commonality of the type but also captures the characteristics of individual cases, further improving the accuracy of entity recognition of the text data to be processed.

[0091] Furthermore, the entity extraction and labeling method further includes:

[0092] Real-time monitoring of the recognition performance of the rule engine, statistical model, and deep learning model on the text data;

[0093] The initial weight distribution of the rule engine, statistical model and deep learning model is dynamically modified based on the monitoring results and the determination results of the preset weight adjustment conditions.

[0094] In this embodiment, in the process of text data recognition, the fixed weight distribution scheme cannot adapt to the dynamic changes of entities in text features. Therefore, when the dynamic weight distribution shown above is applied, the initial weight of each data model is further adjusted by real-time monitoring of the performance of each data model, so that the entity recognition effect of each data model is consistent with the diversity of the text data to be processed.

[0095] In the specific implementation process, an effectiveness detection index system and weight adjustment trigger conditions are set up. The test results of the effectiveness detection index system are used to determine whether the weight adjustment trigger conditions are met. The initial weights of each data model (rule engine, statistical model, and deep learning model) are dynamically adjusted based on the results of the determination. The effectiveness detection index system is equipped with multi-dimensional detection indicators for performing performance tests on the rule engine, statistical model, and deep learning model respectively, as shown below:

[0096] Rule engine: Real-time statistics of rule matching rate (number of successful matches / number of attempted matches) and number of rule conflicts;

[0097] Statistical model: monitors entity boundary accuracy (based on sliding window evaluation) and type confusion matrix;

[0098] Deep learning model: Tracking attention focus (key entity attention weights) and associated reasoning accuracy.

[0099] Specifically, the weight adjustment trigger conditions set for the multi-dimensional detection indicators are determined by the set response mechanism. The trigger conditions for each level of the response mechanism are different. For example, when a model's core indicator is 10% lower than the benchmark value (such as a rule matching rate <65%), a first-level response is triggered; when the indicator is 20% lower than the benchmark value for three consecutive paragraphs (such as a statistical model boundary accuracy rate <58%), a second-level response is triggered; and when a systemic failure occurs (such as a complete error in the deep learning model's associative reasoning), a third-level response is triggered. Specifically, the trigger conditions can be set according to the entity recognition results of the text data for each data model.

[0100] In the process of dynamically adjusting the initial weight of each data model according to the determination results shown above, the adjustment can be specifically made based on the response mechanism set in the weight adjustment trigger condition, that is, the initial weight of each data model is adjusted differentially according to the warning level, which can be shown as follows:

[0101] The initial weight adjustment mechanism of the rule engine can be defined as follows: a first-level warning reduces the weight and activates the similar rule expansion search; a second-level warning adjusts the weight downward and activates the backup rule set; a third-level warning pauses the engine and transfers the weight to other models;

[0102] The initial weight adjustment mechanism of the statistical model adjustment can be defined as follows: the first-level warning can compensate by increasing the weight and reducing the context window; the second-level warning triggers feature engineering reconstruction to dynamically float the weight value; the third-level warning switches the feature template and resets the weight to the baseline value;

[0103] The initial weight adjustment mechanism of the deep learning model can be limited to increasing the weight and enhancing the attention bias during the first-level warning; activating the adversarial sample filtering and locking the weight upper limit during the second-level warning; and loading the domain adaptation copy during the third-level warning, which periodically doubles the initial weight.

[0104] Specifically, this embodiment constructs a closed-loop feedback system to enable the initial weights of each data model to evolve dynamically with the implementation of regulatory clauses. This not only preserves the regulatory rigor of the rule engine, but also fully leverages the adaptability of the data-driven model, further improving the technical effect of entity recognition accuracy. The dynamic weight adjustment values ​​shown above can be derived based on the real-time detection results of each data model. Furthermore, based on the initial weight adjustment results of each data model, the final weight values ​​are integrated to obtain the combined weights of the rule engine, statistical model, and deep learning model to perform entity recognition on the text data to be processed.

[0105] Furthermore, the rule engine, statistical model, and deep learning model perform entity recognition in parallel, i.e., the three data models simultaneously perform entity recognition on the text data and output the recognition results. Taking into account the differences in data processing methods of the various data models, a hybrid recognition model can be configured based on the combined weights of the rule engine, statistical model, and deep learning model, and entity recognition can be performed in the text data using the hybrid recognition model.

[0106] Specifically, in the hybrid recognition model, the data structures of the rule engine, statistical model, and deep learning model are respectively configured according to the combined weights, or the combined weights of the rule engine, statistical model, and deep learning model are paralleled to form the data processing layer of the hybrid recognition model, and then corresponding processing channels are set for the configured rule engine, statistical model, and deep learning model, and the settings of the processing channels are adapted to the characteristics of the recognition entities of each data model. The specific details can be as follows:

[0107] The working mode of the rule engine processing channel is hierarchical matching based on the standard knowledge base. For example, the first layer is mandatory rule matching (such as the number of standard clauses and standard terminology); the second layer is inference rule execution; the third layer is semantic rule verification; the output characteristics of this data model focus on high precision and low recall rate, and come with standard logical constraints.

[0108] The feature processing setting parameters of the statistical model processing channel include character-level features, context window features, and application scenario type features; its output characteristics focus on balance judgment, general entity stability, and whether it is unable to process unregistered words.

[0109] The network structure of the deep learning model channel includes the Legal-BERT pre-training layer, the BiLSTM sequence encoding layer, and the multi-head attention mechanism; the output characteristics focus on high recall rate (91%-93%) and the ability to discover potential associations. This data structure requires a lot of computing power support.

[0110] As shown above, entities in the text data are identified based on the hybrid recognition model, and entity recognition results are output. The entity recognition results are output in parallel and can be output in the form of an entity data set. Subsequently, the output results of the entity recognition results are analyzed, and the specific entity content is derived based on the analysis results. That is, the steps of configuring the hybrid recognition model based on the combined weights and identifying entities in the text data using the hybrid recognition model include:

[0111] Analyze the output results of the hybrid recognition model, where the output results are entity candidate sets output in parallel by each data model;

[0112] Prioritizing the entities in the entity candidate set and filtering out entities below a confidence threshold;

[0113] The filtering result is used as the recognition result of the entity.

[0114] As shown above, entity recognition is performed on the text data based on the configured hybrid recognition model and the entity recognition results are output. In a specific implementation, each data model independently outputs an entity set, and the output entity set forms an original candidate pool for specific entity analysis. Due to the recognition characteristics of each data model, the output entity set has different text features. For example, the output result of the rule engine is an entity with a mandatory mark; the output result of the statistical model output is an entity fragment with a probability value; and the output result of the deep learning output is an entity and its relationship with attention weights.

[0115] In the original candidate pool, the output results of each data model are used as the entity candidate set for priority sorting. In the priority sorting, a three-level priority system is set up, which is sorted by normative key entities, task feature entities and basic description entities respectively. Among them, the normative key entities are entities involving non-normative behavior constituent elements; the task feature entities are entities that reflect the particularity of the task; and the basic description entities are general task element entities, such as time and place.

[0116] According to the sorting results of the priority sorting shown, the confidence of each entity in the priority sorting is calculated respectively. In this embodiment, each entity is calculated using the preset entity final confidence calculation formula, and the entity final confidence Score calculation formula is: Score = αS_rule + βS_stat + γ*S_dl, where α / β / γ are the current model combination weights (such as 0.5 / 0.3 / 0.2), S_rule is the rule engine output score (1 for successful match, otherwise 0), S_stat is the statistical model probability value (0-1), and S_dl is the deep learning normalized attention value (0-1).

[0117] The above-mentioned operation of calculating the confidence of the entities in the priority sorting queue can be limited to performing the confidence calculation only for high-priority entities, that is, directly excluding low-priority entities. The specific method can be determined according to the specific settings of the priority sorting. After the confidence is calculated for the high-priority entities in the priority sorting queue, further sorting is performed based on the calculation results of the confidence calculation. That is, the step of prioritizing the entities in the entity candidate set and filtering out entities below the confidence threshold includes:

[0118] Sort the entities with high priority by confidence;

[0119] The ranking result is adjusted using the combined weights, and entities below a confidence threshold are filtered out from the adjusted ranking result using a weighted voting mechanism.

[0120] When ranking high-priority entities based on confidence calculation results, the combined weights used to adjust the ranking results are essentially setting a differentiated filtering threshold based on the application scenario type. Since the application scenario type can also be expressed through combined weights, it can also be limited to setting a differentiated filtering threshold based on the combined weights, and adjusting the confidence-ranked entities based on the differentiated filtering threshold. Specifically, the differentiated filtering threshold is set to the threshold corresponding to the application scenario type, and the threshold corresponding to each application scenario type can be set based on the application scenario. To this end, the confidence-ranked entities are adjusted based on the numerical limit of the differentiated filtering threshold.

[0121] Furthermore, overlapping entity adjudication and type contradiction resolution processing are performed based on the ranking results of the confidence ranking to exclude duplicate entities and contradictory entities. Wherein, the overlapping entity adjudication can adjudicate the same entity by comparing the contribution weights of each data model, logical rationality verification, contextual consistency and the retained manual review marks, so as to exclude duplicate entities; and exclude contradictory entities through a pre-set type priority matrix. In the rules of the type priority matrix, it can be set that the rule engine type annotation has the highest authority, when the statistical model conflicts with deep learning, the annotation that matches the application scenario type is given priority, and an expert knowledge base query is initiated for the type that cannot be adjudicated. After excluding the entities that do not meet the requirements in the ranking results of the confidence ranking shown, the final entity recognition result is obtained.

[0122] Based on the entity recognition result, boundary checking and type labeling are performed in the text data to be processed to output structured entity information. Specifically, based on the entity represented in the recognition result, text information that conforms to the entity is marked in the text data of the text data to be processed. For example, if the entity is the time when an event occurs, the date is marked in the shown text data, and the entity parameter is indicated as night. Specifically, it can be defined as the name of the task item represented by the text data to be processed, and the mark of the entity is the filled-in parameter that reflects the name of the task item.

[0123] The step of performing boundary checking and type labeling on the recognition results and outputting structured entity information includes:

[0124] Correcting the entity boundary position in the recognition result according to the punctuation mark distribution characteristics in the recognition result;

[0125] The target entity is obtained by filtering out the recognition results that do not meet the requirements using the preset entity length threshold;

[0126] Perform type annotation on the target entity and output structured entity information.

[0127] Since the text data to be processed is the task content expressed in words, a boundary verification process is provided based on the text data. In the boundary verification process, punctuation-guided boundary correction and entity length threshold filtering are provided, wherein the punctuation-guided boundary correction establishes three types of correction rules by analyzing the punctuation features of the entity context, including standard document-specific punctuation: forcing the complete retention of the content within "《》", enumerated punctuation: identifying parallel entities separated by "", " and ";" and statement termination punctuation: using "." and "?" as entity boundary hard constraints. In addition, a punctuation position weight matrix is ​​provided to perform probability adjustments on the first and last characters of the entity. When the entity boundary falls between quotation marks or brackets, it is automatically expanded to a complete symbol pair.

[0128] The entity length threshold filtering system incorporates a dynamic length threshold system, including basic threshold settings, such as for names (2-4 characters), and application scenario type adjustments. This system can relax numerical description lengths and text position compensation to address financial anomalies, and can also extend the length of the main text relative to the title by a certain percentage. Furthermore, a sliding window is used to assess entity rationality, and overly long entities are segmented, semantically compressed, and fully annotated. Specifically, based on the entity length threshold filtering process, entities in the recognition results are filtered to obtain target entities. Furthermore, these target entities are annotated with their types to output structured entity information.

[0129] Furthermore, based on the results of boundary checking and type annotation of the above-mentioned entity, when outputting the structured entity information of the target entity, a preset entity recognition mechanism is used, that is, the steps of performing type annotation on the target entity and outputting the structured entity information include:

[0130] Recognize the text data through a bidirectional LSTM-CRF model, and predict the entity boundary area of ​​the target entity in the text data;

[0131] Correcting the position of the entity boundary area based on the syntax analysis tree to determine the annotation range of the target entity, and performing entity annotation on the annotation range;

[0132] Execute the step of outputting structured entity information.

[0133] In this embodiment, a domain-optimized annotation system is pre-built, which can perform corresponding annotation processing for designated entity labels and special tags. Specifically, the standard entity labels include multiple main types + multiple subcategories (such as "related items-item names"), and the special tags are used to mark controversial entities and cross-sentence entities with disputes, wherein the tag objects of the controversial entities need to be manually compounded, and the cross-sentence entities can be specially associated to clarify the tagging scope of the entities. Furthermore, in order to improve the annotation effect of the domain-optimized annotation system, the domain-optimized annotation system can be trained based on attention-guided training, and its training scheme includes applying multiple loss weights to standard terms, establishing attention biases for risk behavior constituent elements, and using adversarial samples to enhance model robustness. Specifically, it can be implemented based on a specific training scheme.

[0134] In this manner, the entity boundary region of the target entity is marked in the text data according to the constructed domain-optimized annotation system. The entity boundary region is the marking range selected for the target entity by the domain-optimized annotation system. Furthermore, the specific location of the entity boundary region is corrected using the syntax analysis tree to further determine the marking range of the target entity.

[0135] During the syntactic parse tree-assisted correction process, an analysis mechanism based on dependency syntax is implemented. This analysis mechanism includes, but is not limited to, core word alignment, modifier absorption, and ambiguous structure processing. Core word alignment ensures that entities contain complete dependency subtrees, modifier absorption automatically expands boundaries to include necessary modifiers, and ambiguous structure processing prioritizes parsing paths that conform to standard semantics. In practical applications, this analysis mechanism can be configured by setting a syntactic analyzer enhanced for the domain to which the text belongs, thereby further correcting the position of the entity boundary region of the target entity. When configuring the syntactic analyzer enhanced for the domain to which the text belongs, the analysis mechanism must incorporate sentence structure rules specific to the domain to which the text belongs, optimize long-distance dependency parsing capabilities, and support cross-paragraph reference analysis to correct the position of the entity boundary region of the target entity. For example, the original boundary of "electronic account" (missing key modifiers) can be corrected to "external network account used by a certain user" (a complete semantic unit).

[0136] As shown above, after the position of the entity boundary area of ​​the target entity is corrected according to the syntactic analysis tree, the final annotation range of the target entity is obtained, and the entity is annotated based on the annotation range. In actual application, after the annotation range is annotated, the target entity needs to be output in a structured form in combination with the annotated annotation range to clearly indicate the entity object.

[0137] Specifically, the target entity and its annotated range are output through a structured output specification. The structured output specification must clearly define the entity's core attributes, including text content (original surface form), standardized expression (standardized expression), and location information (document ID + start and end offsets). Furthermore, the entity's core attributes must indicate its type system, characterized by primary types (person / object / behavior, etc.), subtypes (related participants / related objects, etc.), and extended attributes (time range, etc.). Finally, the target entity and its annotated range must highlight the relationship between other target entities and their annotated ranges, primarily including inter-entity relationships and evidence chain directions.

[0138] Furthermore, before the target entity and its annotated scope are output as structured entity information, they need to be screened for output quality control. The output quality control specifically includes a verification mechanism and fault-tolerant processing, and the target entity and its annotated scope are respectively subjected to regulatory compliance checks, logical consistency verifications, and evidence chain integrity assessments. The regulatory compliance checks must comply with the regulatory risk behavior composition, the logical consistency verifications are used to verify the timeline and causal relationship of the target entity and its annotated scope, and the evidence chain integrity assessments are used to determine the coverage of key elements of the target entity and its annotated scope.

[0139] Based on the verification mechanism's results for the target entity and its tag range, the verification results are then subjected to appropriate fault-tolerance processing. For example, a dispute flag is added when the confidence level is determined to be below a threshold, multiple tag ranges are retained as candidate entity tag ranges for ambiguous entities, and a manual review channel is initiated for mandatory verification of the target entity if the target entity cannot be determined. Finally, the structured entity information for the successfully verified and fault-tolerant target entity and its tag range is output.

[0140] In this embodiment, multi-model collaborative decision-making is optimized through a dynamic weight allocation mechanism, which significantly improves the extraction accuracy and annotation efficiency of key entities such as time and place in task text data, and realizes the integration of intelligent recognition and structured annotation of task elements.

[0141] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the entity extraction and annotation method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0142] The present application provides an entity extraction and labeling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the entity extraction and labeling method in the above-mentioned embodiment 1.

[0143] Reference below Figure 2 , which shows a schematic diagram of the structure of an entity extraction and annotation device suitable for implementing the embodiments of the present application. The entity extraction and annotation device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), etc., as well as fixed terminals such as digital TVs and desktop computers. Figure 2 The entity extraction and annotation device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0144] like Figure 2 As shown, the entity extraction and annotation device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the entity extraction and annotation device. The processing device 1001, the ROM 1002, and the RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, a magnetic tape, hard disk, etc.; and communication devices 1009. The communication devices 1009 can allow the entity extraction and annotation device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows an entity extraction and annotation device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have instead.

[0145] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are performed.

[0146] The entity extraction and annotation device provided in this application, which utilizes the entity extraction and annotation method of the aforementioned embodiment, can resolve the technical problem of the prior art being unable to achieve integrated entity recognition and annotation processing for specified, standardized text data to be processed. Compared with the prior art, the beneficial effects of the entity extraction and annotation device provided in this application are the same as those of the entity extraction and annotation method provided in the aforementioned embodiment, and the other technical features of the entity extraction and annotation device are the same as those disclosed in the method of the previous embodiment, and are not further described here.

[0147] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0148] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0149] The present application provides a storage medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, and the computer-readable program instructions are used to execute the entity extraction and labeling method in the above embodiment.

[0150] The computer-readable storage medium provided herein may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems, or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including, but not limited to, wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0151] The computer-readable storage medium may be included in the entity extraction and annotation device; or it may exist independently without being assembled into the entity extraction and annotation device.

[0152] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the entity extraction and annotation device, the entity extraction and annotation device implements the technical content of the entity extraction and annotation method embodiment shown above.

[0153] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0154] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0155] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0156] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the entity extraction and annotation method described above. This computer-readable storage medium addresses the technical issue of prior art inability to achieve integrated entity recognition and annotation for specified, standardized text data to be processed. Compared to prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the entity extraction and annotation method provided in the aforementioned embodiments, and are not further elaborated here.

Claims

1. A method for entity extraction and annotation, characterized in that: The entity extraction and annotation method comprises the following steps: Get the text data to be processed; Extracting dynamic indicators reflecting text characteristics from the text data; Determine the combined weight of the rule engine, statistical model and deep learning model based on the dynamic indicators; configuring a hybrid recognition model according to the combined weights, and identifying entities in the text data by using the hybrid recognition model; Perform boundary checking and type annotation on the recognition results, and output structured entity information; The step of determining the combined weight of the rule engine, the statistical model, and the deep learning model based on the dynamic indicator includes: Select a corresponding weight distribution strategy according to the application scenario type of the text data; Allocate decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicators; The step of allocating decision weights to the rule engine, statistical model, and deep learning model using the weight allocation strategy and the dynamic indicator includes: Dynamic weight adjustment based on the rule engine is to increase the initial weight of the rule engine based on the baseline weight value for term density, regulatory clause references, and complexity; Dynamic weight adjustment based on the statistical model is to increase or decrease the initial weight of the statistical model based on complexity, uncommon word combinations, and context stability; The dynamic weight adjustment based on the deep learning model is to adjust the initial weight of the deep learning model based on the text relevance, cross-paragraph reference and rule coverage of the text data to be processed; Furthermore, the entity extraction and annotation method further includes: Real-time monitoring of the recognition performance of the rule engine, statistical model, and deep learning model on the text data; The initial weight distribution of the rule engine, statistical model and deep learning model is dynamically modified based on the monitoring results and the determination results of the preset weight adjustment conditions.

2. The entity extraction and annotation method according to claim 1, characterized in that: The step of extracting dynamic indicators reflecting text characteristics from the text data comprises: Analyzing the distribution pattern of relevant clause standard paragraphs in the text data and determining the target area where the professional terminology is located, wherein a hierarchical analysis method is used to locate the relevant clause standard paragraphs of the text data to be processed, and structural feature analysis and distribution rule modeling are performed on the located results, and based on the structural feature analysis and distribution rule modeling of the located results, the target area where the professional terminology is located is located in the processed results; The relevance between the text representation and the field to which the text information belongs is evaluated according to the distribution law and the target area, and the dynamic indicator is extracted according to the evaluation result.

3. The entity extraction and annotation method according to claim 1, characterized in that: The step of configuring a hybrid recognition model according to the combined weights and identifying entities in the text data by using the hybrid recognition model comprises: Analyze the output results of the hybrid recognition model, where the output results are entity candidate sets output in parallel by each data model; Prioritizing the entities in the entity candidate set and filtering out entities below a confidence threshold; The filtering result is used as the recognition result of the entity.

4. The entity extraction and annotation method according to claim 3, characterized in that: The step of prioritizing the entities in the entity candidate set and filtering the entities below the confidence threshold comprises: Sort the entities with high priority by confidence; The ranking result is adjusted using the combined weights, and entities below a confidence threshold are filtered out from the adjusted ranking result using a weighted voting mechanism.

5. The entity extraction and labeling method according to claim 1, wherein: The step of performing boundary checking and type labeling on the recognition results and outputting structured entity information includes: Correcting the entity boundary position in the recognition result according to the punctuation mark distribution characteristics in the recognition result; The target entity is obtained by filtering out the recognition results that do not meet the requirements using the preset entity length threshold; Perform type annotation on the target entity and output structured entity information.

6. The entity extraction and labeling method according to claim 5, characterized in that: The step of performing type annotation on the target entity and outputting structured entity information includes: Recognize the text data through a bidirectional LSTM-CRF model, and predict the entity boundary area of ​​the target entity in the text data; Correcting the position of the entity boundary area based on the syntax analysis tree to determine the annotation range of the target entity, and performing entity annotation on the annotation range; Execute the step of outputting structured entity information.

7. An entity extraction and annotation device, characterized in that: The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the entity extraction and annotation method according to any one of claims 1 to 6.

8. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the entity extraction and labeling method according to any one of claims 1 to 6 are implemented.

Images