Text entity recognition model construction method and device based on large model data augmentation

Abstract

The application provides a text entity recognition model construction method and device based on large model data enhancement, which comprises the following steps: an initial model containing preprocessing, syntax dependency analysis, data correction and enhancement, context coding, syntax enhancement, representation fusion and sequence decoding processing units is constructed; training samples are preprocessed to obtain preprocessed texts, and syntax analysis is performed to build a preliminary dependency graph; a large language model is used to correct and enhance the texts and the dependency graph, so that a text sequence and a dependency graph used in the subsequent process are obtained; an initial representation vector is coded from the text sequence, and an enhanced vector is combined with the dependency graph; a prediction label containing word element entity annotation is output after fusion and decoding; a function algorithm loss containing a conditional random field structure loss is used to calculate the loss, convergence is judged, parameters are updated and training is continued until a target model is obtained. The method uses a large language model to optimize and expand data, combines with syntax dependency to strengthen features, and improves the ability of complex text named entity recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, in particular to a text entity recognition model construction method and device based on large model data enhancement. BACKGROUND

[0002] With the continuous development of information technology, text named entity recognition, as a key task in natural language processing, is widely used in public opinion monitoring, search engines, knowledge graph construction, medical text analysis and other scenarios. Its goal is to automatically identify entities with specific semantic categories from raw text, such as names, place names, organizations, times, etc. In recent years, pre-trained language models based on deep learning, especially the sequence labeling method combining BERT and conditional random field (CRF), have become the mainstream solution for Chinese NER. This kind of method captures the context semantics through BERT, and then models the transition constraints between labels through CRF, achieving good performance. However, the complexity of Chinese language, the diversity of grammatical structure, and the ambiguity of named entity boundaries still pose challenges to the recognition accuracy of existing models, especially in corpora with complex syntactic relationships or uneven distribution of entity categories.

[0003] Although the current BERT+CRF-based named entity recognition method has achieved high accuracy on multiple public datasets, this kind of method still faces many technical bottlenecks in practical applications. First, the model's ability to perceive Chinese syntactic structure is insufficient. BERT, as a context representation model, does not explicitly use syntactic dependency relationships in language in its attention mechanism. Especially when dealing with complex sentence patterns, modifier structures and entity nesting, it is often difficult to accurately capture the structural relationship between entities and context, resulting in unclear entity boundary judgment or label prediction errors. Second, in the model training process, the learning of entity labels is treated as an equivalent goal, ignoring the frequency distribution difference of entity categories in data and the ambiguity level of sample context. This strategy cannot fully focus on rare labels, and is prone to bias towards high-frequency categories, resulting in decreased recognition performance for low-frequency, professional or domain-specific entities. In addition, existing methods have weak perception ability for uncertain samples, and are prone to make wrong predictions in low-confidence areas, affecting the stability and reliability of overall performance. SUMMARY

[0004] To solve the above technical problems, the technical solution adopted by the present application is as follows:

[0005] According to the first aspect of the present application, a text entity recognition model construction method based on large model data enhancement is provided, which comprises the following steps:

[0006] S100, constructing an initial text entity recognition model, the model comprising a text preprocessing processing unit, a syntax dependency analysis processing unit, a data correction and enhancement processing unit, a context representation coding processing unit, a syntax dependency enhancement processing unit, a representation fusion processing unit, and a sequence decoding processing unit;

[0007] S200, inputting training sample data into the model for text preprocessing to generate preprocessed text;

[0008] S300, performing syntax dependency analysis on the preprocessed text to construct a preliminary dependency graph;

[0009] S400, correcting the preprocessed text and the preliminary dependency graph using a large language model to obtain corrected text and corrected dependency graph, and generating enhanced text and enhanced dependency graph based on the correction result, obtaining a final text sequence and a final dependency graph for subsequent processing, the final text sequence comprising the corrected text and the enhanced text, and the final dependency graph comprising the corrected dependency graph and the enhanced dependency graph;

[0010] S500, encoding the final text sequence to obtain an initial representation vector sequence;

[0011] S600, based on the final dependency graph and the initial representation vector sequence, performing structure enhancement on the initial representation vector sequence to obtain an enhanced representation vector sequence;

[0012] S700, fusing the initial representation vector sequence and the enhanced representation vector sequence to obtain a fused representation vector sequence;

[0013] S800, performing sequence decoding on the fused representation vector sequence to output a predicted label sequence containing word element entity category labels;

[0014] S900, based on the predicted label sequence and the real label sequence, calculating the model loss using a preset loss function; if the loss satisfies the convergence condition, the current model is taken as the target model; otherwise, updating the model parameters and returning to S200 for continuous training; wherein the preset loss function comprises a conditional random field structure loss.

[0015] According to the second aspect of the present application, an electronic device is provided, comprising a processor and a memory; the processor is used to execute the steps of the method of the first aspect of the present application by calling the programs or instructions stored in the memory.

[0016] The text entity recognition model construction method based on large model data enhancement provided by the embodiment of the application aims to improve the accuracy and robustness of entity recognition in Chinese text. The method first constructs an initial text entity recognition model comprising text preprocessing, syntax dependency analysis, data correction and enhancement, context representation coding, syntax dependency enhancement, representation fusion and sequence decoding function processing units; then the training samples are sequentially preprocessed to generate preprocessed text, and a preliminary dependency graph is constructed through syntax dependency analysis; the preprocessed text and the preliminary dependency graph are corrected and enhanced using a large language model to obtain a final text sequence and a final dependency graph for subsequent processing; the text sequence is coded to obtain an initial representation vector sequence, and the initial representation vector sequence is structurally enhanced in combination with the dependency graph to obtain an enhanced representation vector sequence; after fusing the two types of vector sequences, sequence decoding is performed to output a predicted label sequence containing word element entity category labels; finally, the model loss is calculated based on a preset loss function comprising a conditional random field structure loss, whether the model converges is judged by the loss, and if not, the parameters are updated for further training until the target text entity recognition model is obtained. The method realizes data quality optimization and expansion through a large language model, and effectively improves the entity recognition capability of the model for complex text by combining syntax dependency structure to strengthen feature representation.

[0017] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the application, nor is it intended to limit the scope of the application. Other features of the application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0019] Figure 1 The flowchart of the text entity recognition model construction method based on large model data enhancement provided by the embodiment of the application. DETAILED DESCRIPTION

[0020] The technical solutions in the embodiments of the application will be described clearly and completely in the following with reference to the drawings in the embodiments of the application. Obviously, the described embodiments are only some of the embodiments of the application, not all. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0022] It is to be understood that some of the example embodiments are described in terms of a process or method depicted as a flowchart. Although a flowchart can describe operations as a sequential process, many of the operations can be performed in parallel, concurrently or simultaneously. In addition, the order of the operations can be re-arranged. A process can be terminated when its operations are completed, but can also have additional steps not included in a figure. A process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0023] The present application aims at the problems of weak structural understanding ability and serious label class bias in Chinese named entity recognition task, and proposes a text entity recognition model construction method based on large model data enhancement by fusing syntax dependency enhancement and dynamic label weight mechanism. Figure 1 As shown in the figure, the method comprises the following steps:

[0024] S100, constructing an initial text named entity recognition model.

[0025] In the embodiment of the present application, the text named entity recognition model is an end-to-end deep learning model fusing syntax structure and context semantics, which mainly comprises processing units for realizing text preprocessing, syntax dependency analysis, data correction and enhancement, context representation coding, syntax dependency enhancement, representation fusion and sequence decoding, i.e. comprising a text preprocessing processing unit, a syntax dependency analysis processing unit, a data correction and enhancement processing unit, a context representation coding processing unit, a syntax dependency enhancement processing unit, a representation fusion processing unit and a sequence decoding processing unit.

[0026] S200, inputting training sample data into the model for text preprocessing to generate preprocessed text.

[0027] In the embodiments of the present application, the batch setting of the training sample data needs to be determined in combination with the model training efficiency and the hardware resource adaptability. Specifically, the number of samples contained in a single batch can be set to 16, 32 or 64 Chinese text sequences. The actual value can be dynamically adjusted according to the GPU memory of the training device: when the average length of the samples is relatively long (such as more than 512 word units), a smaller batch setting of 16 per batch is adopted to avoid GPU memory overflow caused by excessive data; for short text samples, a larger batch setting of 64 per batch can be adopted to improve the parallel computing efficiency. The training sample data contains Chinese text sequences to be processed and corresponding entity label sequences.

[0028] The preprocessing unit performs standardization preprocessing operations on the Chinese text sequence to generate structured preprocessed text. The specific operations include:

[0029] Text cleaning: removing noise data (such as redundant spaces, special symbols such as # and @, and garbled characters) in the Chinese text sequence, and unifying the text format (such as standardizing “2025” to “2025” and “10000” to “10000”);

[0030] Word segmentation processing: using a Chinese word segmentation tool (such as Jieba word segmentation tool or LTP word segmentation tool) to cut continuous Chinese text into discrete word unit sequences, for example, cutting “Beijing Haidian District Zhongguancun Street” into “Beijing Haidian District Zhongguancun Street”, to ensure that the word unit boundary is adapted to the input requirements of subsequent syntactic analysis and semantic encoding;

[0031] Encoding conversion: performing character-level or subword-level encoding (such as subword division based on BPE word segmentation algorithm) on the word unit sequence after word segmentation, converting each word unit into an index or embedding vector format (such as converting the word unit “mobile phone” into the index “1256” or a 768-dimensional embedding vector) recognizable by the model, and providing standardized input for the context representation encoding unit;

[0032] Length adaptation: according to the preset text length threshold (such as 512 word units or 1024 word units) of the model, truncating the super-long text from the end of the sequence, and padding the short text with a preset “PAD” symbol at the end of the sequence to ensure the consistency of the length of the text sequence input to the subsequent module. S300, syntactic dependency analysis is performed on the preprocessed text to construct a preliminary dependency graph.

[0033] In the embodiment of the present application, the syntax dependency analysis processing unit calls a Chinese syntax analysis tool (such as an LTP dependency analysis module, a HanLP syntax analysis interface, an open-source StanfordParser Chinese version, or a HanLP dependency syntax library) to perform syntax structure analysis on the preprocessed text (a token sequence that has been segmented and cleaned) output by S200, extract the dependency relationship between tokens, and further construct a preliminary dependency graph.

[0034] In the embodiment of the present application, the preliminary dependency graph can be represented as G=(V, E), where V is a node set: each node uniquely corresponds to a token in the preprocessed text and contains the token text content and the position index (such as the first token or the second token) in the text sequence; E is an edge set: each edge connects a pair of tokens that have a syntax dependency relationship, and the edge can carry a specific dependency relationship type label (such as a “subject-predicate relationship”, a “determiner-noun relationship”, a “verb-object relationship”, or a “adverb-noun relationship”), or can be simplified to a binary connected relationship that only indicates the existence of a dependency relationship according to subsequent calculation requirements; the dependency graph as a whole adopts an undirected graph structure to completely retain bidirectional dependency information (such as the mutual association of “subject-predicate”). Based on the preprocessed text (a standardized token sequence that has been segmented and encoded) output by S200, the preliminary dependency graph is constructed, and the process is specifically as follows:

[0035] (1) Syntax dependency relationship analysis

[0036] The syntax dependency analysis processing unit calls the above tool to analyze the token sequence and outputs a structured result: including the “parent node token index” (that is, the position of the other token in the sequence that has a dependency relationship with the current token) and the “dependency relationship type label” (such as “subject-predicate”, “verb-object”, or “determiner-noun”) of each token.

[0037] Example: for the preprocessed text token sequence “Xiaoming / bought / a / mobile phone”, the analysis result is “bought” (parent node: Xiaoming, relationship: subject-predicate), “mobile phone” (parent node: bought, relationship: verb-object), and “a” (parent node: bought, relationship: adverb).

[0038] (2) Graph structure mapping

[0039] (3) Optional graph simplification processing

[0040] If the subsequent syntax dependency enhancement unit is sensitive to the complexity of the graph structure, the preliminary dependency graph can be simplified: only the "connectivity" of the edge is retained (i.e., the relationship label of the edge is deleted), forming a binary undirected graph that only represents "whether there is a dependency relationship", to reduce the computational complexity of subsequent graph neural networks (such as GCN, GAT).

[0041] To enhance the modeling ability of the dependency graph for text structure information, the syntax dependency analysis processing unit can additionally perform the following optimization operations (not necessary steps, selected according to actual task requirements):

[0042] Add adjacent edges: add undirected edges for nodes corresponding to "sequentially adjacent word units" in the preprocessed text (such as "buy-ed" and "ed-phone"), supplementing the local sequential information of the text;

[0043] Fuse semantic labels: combine semantic role labeling results (such as calling the PropBank Chinese semantic role library) to add "agent-action" and "action-patient" semantic labels to some edges, realizing joint encoding of syntax structure and semantic information.

[0044] Those skilled in the art should understand that constructing a syntax dependency graph based on a preprocessed Chinese text is a mature technology in the field of natural language processing, and its core value lies in providing "explicit word structure association information" for the subsequent syntax dependency enhancement unit, helping the model capture long-distance word dependency relationships that traditional semantic encoding cannot cover, and thus improving the accuracy of text entity recognition.

[0045] S400, correcting the preprocessed text and the preliminary dependency graph to obtain a corrected text and a corrected dependency graph; based on the correction result, generating an enhanced text and an enhanced dependency graph using a large language model; obtaining a text sequence and a dependency graph for subsequent processing, the final text sequence including the corrected text and the enhanced text, and the final dependency graph including the corrected dependency graph and the enhanced dependency graph.

[0046] This step is performed by a data correction and enhancement processing unit in the model, and the input is the output result of the previous step: core input: preprocessed text generated by S200 (denoted as X, which has been cleaned, segmented, and encoded adapted), preliminary dependency graph constructed by S300 (denoted as G, containing word nodes and dependency edges); auxiliary input: initial entity label sequence provided by the training sample (denoted as Y, corresponding to the word units of the preprocessed text X).

[0047] The data correction and enhancement processing unit calls a large language model (such as Qwen3-14b, Llama3, or a customized Chinese large language model), conducts consistency and accuracy review on the input X, G, and Y, identifies and corrects three types of core problems:

[0048] (1) Text level errors: such as pre-processing stage residual word / sub-word segmentation errors (e.g., missegmenting “Zhongguancun Avenue” into “Zhongguancun Avenue” / “Street”), missing special characters (e.g., miswriting the date “2025-09” as “202509”);

[0049] (2) Dependency graph level errors: such as missing dependency edges in the initial dependency graph (e.g., the “verb-object relationship” edge of “buy a phone” is not labeled), incorrect dependency relationship types (e.g., mislabeling “definite center relationship” as “subject-predicate relationship”);

[0050] (3) Label-text / Figure 1 consistency errors: such as entity labels that contradict the context (e.g., mislabeling “Beijing City” as “person name”), labels that do not match the dependency structure (e.g., the subject word in the “subject-predicate relationship” is not labeled with an “entity” label). Based on Chinese grammar rules, entity labeling specifications, and contextual semantics, the large language model outputs the corrected data: corrected text sequence (denoted as XC): standardized text after correcting word segmentation and character errors; corrected dependency graph (denoted as GC): undirected graph after completing dependency edges and correcting relationship types; revised entity label sequence (denoted as YC): label sequence consistent with the semantics / structure of XC and GC.

[0051] Based on the corrected data (XC, GC, YC), the data correction and enhancement processing unit continues to generate three types of enhanced samples using the large language model to ensure that the samples cover more scenarios:

[0052] 1) Entity replacement enhancement: maintain the syntactic structure of GC (such as subject-predicate, definite center, and verb-object relationships) unchanged, replace the entities in XC with entities of the same category (e.g., replace “Beijing City” with “Shanghai City” and “A phone” with “B phone”), generate enhanced text XE1, enhanced dependency graph GE1 (structure unchanged, only node word updated), and enhanced label YE1 (label type unchanged, only entity text updated);

[0053] (2) Context rewriting enhancement: adjust the modifying words / syntactic details associated with the dependency edges in XC (e.g., rewrite “quickly buy a phone” as “quickly purchase a new phone”), update the associated node words in GC (maintain the dependency relationship type unchanged), generate XE2, GE2, and YE2 (label sequence consistent with the original YC);

[0054] (3) Adversarial sample enhancement: generate ambiguous / complex modification structures near the entity boundaries (e.g., rewrite “a company located in Beijing” as “a technology company located near Chaoyang District in Beijing”), increase syntactic complexity to improve model robustness, generate XE3, GE3, and YE3 (ensure that the entity core boundary labels are accurate).

[0055] To avoid low-quality augmented samples affecting training results, the data correction and augmentation unit uses three metrics to screen the augmented samples:

[0056] Semantic fluency: The naturalness of the enhanced text is evaluated using the perplexity of a large language model, and samples with perplexity below a preset threshold (e.g., perplexity < 50) are filtered out.

[0057] dependence Figure 1 Consistency: Verify whether the edge associations of the enhanced dependency graph conform to Chinese syntax rules (e.g., "verb-object relationship" requires connecting the verb and the object word), and remove samples with structural contradictions;

[0058] Label consistency: Compare the logical consistency between the enhanced sample label and the original corrected sample label (e.g., entities originally labeled "place name" should still be labeled "place name" after enhancement), and remove samples with incorrect labels.

[0059] Finally, the enhanced sample set after filtering is merged with the original sample set after correction to form the final text sequence and final dependency graph for subsequent S500 (context representation encoding), and the corresponding label sequence is the merged label sequence.

[0060] S500, the final text sequence is encoded to obtain an initial representation vector sequence.

[0061] In this embodiment of the invention, the text sequence is the final text sequence output by S400 (including the word sequence of the corrected sample and the enhanced sample after filtering). The context representation encoding processing unit takes a pre-trained Chinese language model as its core component. Its core function is to transform the word semantics of the text sequence into a vector representation that integrates global context information, so as to provide basic semantic features for subsequent syntactic dependency enhancement.

[0062] The context representation encoding processing unit uses a pre-trained model adapted to Chinese semantics, including but not limited to: Chinese BERT (such as BERT-Base,Chinese), BERT-wwm (a full-word mask version of Chinese BERT), RoBERTa-Chinese (a Chinese pre-trained model with optimized training strategies), etc.; the model structure adopts a multi-layer Transformer encoder architecture, and the number of layers can be selected according to task requirements (such as 12 layers, 24 layers, corresponding to vector dimensions of 768 and 1024), balancing semantic capture capability and computational efficiency.

[0063] The text sequence input to the encoding unit needs to undergo model adaptation processing first:

[0064] Symbol supplement: Add the special symbol "[CLS]" (used to aggregate global text semantics) to the beginning of the word sequence of the final text sequence, and add "[SEP]" (used to identify sequence boundaries) to the end;

[0065] Position Encoding: Assign a position index to each token, generate a position encoding vector and superimpose it with the token's own embedding vector (such as WordPiece embedding), ensuring that the model captures the sequential information of the text;

[0066] Length Adaptation: If the token sequence length of the final text sequence exceeds the maximum input limit of the model (such as BERT-Base's maximum of 512 tokens), it is truncated according to the rules determined in S200; if the length is insufficient, it is filled with the "[PAD]" symbol to ensure uniformity of the input sequence length.

[0067] The context representation encoding processing unit generates the initial representation vector sequence through the following steps:

[0068] Embedding layer operation: Convert the adapted token sequence into low-dimensional word embedding vectors (such as dimension 768), superimpose the position encoding and segment encoding (if processing single text sequence, segment encoding is uniformly 0), and form an initial embedding matrix;

[0069] Transformer encoder operation: The embedding matrix is processed by multiple layers of Transformer encoder, each layer calculates the semantic correlation weight between tokens through multi-head self-attention mechanism (such as assigning high attention weight to "Beijing City - Zhongguancun" type of semantically related tokens), and then through the feedforward neural network (FFN) to nonlinearly transform the attention output, strengthening the semantic feature expression;

[0070] Vector sequence output: After encoding, extract the output vector corresponding to each token (excluding the "[PAD]" filler symbol) to form the initial representation vector sequence H0 = {h01, h02, …, h0 j , …, h0 m}, where: h01 is the initial representation vector of the jth valid token in the final text sequence, j takes values from 1 to m, m is the number of valid tokens in the final text sequence; h0 j ∈ R d , d is the vector dimension, which matches the model structure, such as 768, 1024, which can be dynamically adjusted according to the complexity of the entity recognition task); This vector sequence can capture the polysemy of tokens (such as the different semantics of "A" in "A mobile phone" and "eat A") and long-distance semantic correlation (such as cross-sentence entity reference relationships).

[0071] The generated initial representation vector sequence H0 will be sent synchronously to the syntactic dependency enhancement processing unit of S600 (for structural feature fusion) and the representation fusion processing unit of S700 (for subsequent feature integration), realizing the collaborative modeling basis of "semantic features-structural features".

[0072] S600, performing structure enhancement on the initial representation vector sequence based on the final dependency graph and the initial representation vector sequence, to obtain an enhanced representation vector sequence.

[0073] This step is performed by a syntax dependency enhancement processing unit. In the embodiments of the present application, the syntax dependency enhancement processing unit can adopt a graph convolution network (GCN), which can be a one-layer or multi-layer graph convolution network, for injecting syntax structure information into the word vector representation. The core difference of the one-layer or multi-layer graph convolution network lies in the information propagation range, feature abstraction capability and adaptability to complex syntax structure, and the specific differences are as follows:

[0074] 1. Characteristics of one-layer GCN

[0075] Information propagation range: only the first-order neighbors of the word (directly dependent words such as the directly associated words in the subject-predicate relationship, the subject-object relationship, etc.) are aggregated for feature aggregation. For example, for the sentence "B sets up headquarters in Shenzhen", the first-order neighbors of "B" are "set up" (subject-predicate relationship), and the first-order neighbors of "Shenzhen" are "in" (adverbial relationship) and "set up" (object-verb relationship), and the one-layer GCN only fuses the features of these directly associated words.

[0076] Feature abstraction capability: the output enhanced vector mainly retains the shallow structure information of the original syntax relationship, and focuses on capturing the local semantic association between the word and the directly dependent word, and the feature dimension is consistent with the initial representation vector (such as 768 dimensions).

[0077] Computational cost: small parameter size (only one set of convolution weight matrix and bias), fast inference speed, suitable for resource-constrained scenarios or simple syntax structure texts (such as short sentences, colloquial texts).

[0078] Applicable scenarios: when the entities in the text are mostly short-distance dependencies (such as entities composed of adjacent words) and the syntax ambiguity is small, the one-layer GCN can inject basic structure information while ensuring efficiency, avoiding overfitting.

[0079] 2. Characteristics of multi-layer GCN

[0080] Information propagation range: multi-order neighbor information is accumulated and propagated through multi-layer stacking. For example, a two-layer GCN can aggregate "neighbors of first-order neighbors" (two-order dependency), and for the above sentence, the two-order neighbors of "B" can include "headquarters" (indirectly associated through "set up"), thereby capturing the long-distance dependency structure of "B-set up-headquarters". The larger the number of layers, the longer the syntax dependency path that can be covered (but limited by the length of the text, usually 2-3 layers).

[0081] Feature abstraction: Each layer of GCN performs nonlinear transformation and aggregation on the features output by the previous layer, gradually realizing deep abstraction of syntactic features. For example, the first layer focuses on direct dependency relations, and the second layer can integrate "dependency relations" (such as the combination pattern of "subject-predicate relations + verb-object relations") to enhance the structural modeling ability of nested entities (such as "B Technology Co., Ltd.") and cross-sentence entities.

[0082] Computational cost: The parameter size increases linearly with the number of layers (each layer corresponds to independent weights and biases), and the computational complexity increases (multiple graph convolution operations are required), but the risk of overfitting can be alleviated through residual connections and dropout regularization.

[0083] Applicable scenarios: When the text has complex syntactic structures (such as long sentences and nested clauses), and the entity boundaries are ambiguous (such as "Beijing Haidian Zhongguancun Avenue"), multi-layer GCN can capture global structural associations through multi-order information propagation, improving the robustness of entity recognition.

[0084] In the syntactic dependency enhancement processing unit of the text named entity recognition model, the number of layers of the graph convolution network (GCN) needs to be determined comprehensively considering the task characteristics, data size, syntactic dependency distance, and model complexity. The specific selection strategy is as follows:

[0085] When the model needs to focus on local syntactic relationship modeling or is limited by computing resources, one layer of GCN is preferred, and the specific applicable conditions include:

[0086] (1) Short-distance dependency dominant task:

[0087] Most entities in Chinese text (such as basic personal names and place names) depend on first-order syntactic relationships (such as subject-predicate and attributive relationships), for example, in "Professor Li chaired the meeting", the attributive relationship between "Li" and "professor" can be fully captured through first-order neighbor aggregation, without the need for deep propagation.

[0088] (2) Small sample or low resource scenario:

[0089] If the training data is small (such as less than 10,000 sentences in the domain corpus), multi-layer GCN is prone to overfitting due to excessive parameters, and one layer of GCN can reduce the risk of overfitting by simplifying the structure, while also reducing the training time (single round training efficiency improved by 30%-50%).

[0090] (3) Lightweight deployment requirements:

[0091] For edge devices or real-time inference scenarios (such as mobile NER tools), one layer of GCN can reduce the memory usage (more than 50% reduction compared to multi-layer models) and meet the low-latency requirements (single sentence inference time ≤ 50ms).

[0092] When the task needs to capture long-distance syntactic dependencies or complex entity structures, multi-layer GCN (usually 2-3 layers, rarely more than 4 layers) needs to be used, and the specific applicable conditions include:

[0093] (1) Long entity or nested entity recognition:

[0094] The recognition of Chinese compound entities (such as "Some Institute of Computing Technology") or nested entities (such as "Liu Someone" in "The author Liu Someone of XXX") depends on multi-hop syntactic relationships. Multi-layer GCN can aggregate more distant structural information (such as second-order neighbor features) through layer-by-layer propagation, improving boundary positioning accuracy.

[0095] (2) Complex syntactic structure text:

[0096] For long sentence-intensive texts (single sentence word number > 50) such as news reports and academic papers, multi-layer GCN can model cross-sentence dependencies (such as the reference relationship between "Zhang San" and "Zhang Si" in "The chairman of the company Zhang San announced that his son Zhang Si will take over as CEO").

[0097] (3) High resource support scenarios:

[0098] When the training data is sufficient (domain corpus ≥ 100,000 sentences) and the computing resources are allowed (such as equipped with GPU memory ≥ 16GB), multi-layer GCN can improve feature expression ability by increasing depth, but needs to be combined with residual connection (to alleviate gradient disappearance) and dropout regularization (dropout rate of each layer is set to 0.1-0.2) to suppress overfitting.

[0099] In an embodiment, for general Chinese NER tasks (such as news, social media), 2-layer GCN is recommended to balance local and medium-distance syntactic modeling, and residual connection is used to improve training stability. In another embodiment, an adaptive layer number strategy is introduced in model training, which dynamically switches the number of layers according to the length of the input text (short sentences ≤ 30 words use 1 layer, long sentences > 30 words use 2 layers), balancing accuracy and efficiency. Through the above strategies, the optimal configuration of GCN layer number can be achieved in different scenarios, ensuring that syntactic structure information is effectively injected into word vector representation, and finally improving the overall performance of Chinese named entity recognition.

[0100] The input of the syntactic dependency enhancement processing unit includes the final dependency graph and the initial representation vector sequence output by the context representation encoding module. Through graph convolution operation, each layer of GCN aggregates features based on the adjacency matrix of the dependency graph: for the jth word, its enhanced vector is generated by aggregating its initial vector and the first-order neighbor word vector associated with the dependency relationship. The convolution operation formula of the kth layer of the graph convolution network is: where h j kis the enhanced representation vector of the jth wordpiece after the kth layer graph convolution operation, k is valued from 1 to L, L is the number of layers of the graph convolution network, N(j) is the number of first-order neighbor nodes of the jth wordpiece in the dependency graph, W g k is the trainable convolution weight matrix of the kth layer, b g k is the trainable convolution bias term of the kth layer, σ() is a nonlinear activation function, which can be a ReLU activation function, · represents dot multiplication, h0 p is the initial representation vector of the pth wordpiece, p is valued from 1 to N(j)+1.

[0101] S700, the initial representation vector sequence and the enhanced representation vector sequence are fused to obtain a fused representation vector sequence.

[0102] This step is performed by a representation fusion processing unit. In the embodiments of the present application, the representation fusion processing unit dynamically balances the contribution weights of the two features by using a gated fusion mechanism to realize adaptive fusion of the initial semantic representation and the syntactic structure enhanced representation, that is, the initial representation vector sequence and the enhanced representation vector sequence received are fused. Specifically, the fused representation vector hc j of the jth wordpiece in the fused representation vector sequence satisfies the following condition: hc j =z j ⊙h0 j +(1-z j )⊙h1 j ; where j is valued from 1 to m, m is the number of wordpieces in the fused representation vector sequence, z j is the gating coefficient of the jth wordpiece, z j =σ(Wz•[h0 j ,h1 j ]+bz),[h0 j ,h1 j ] represents the concatenation operation of the initial representation vector h0 j and the enhanced representation vector h1 j of the jth wordpiece, Wz is a trainable weight matrix, bz is a trainable bias term, σ() is a nonlinear activation function, which can be a Sigmoid function, the output range is [0, 1], and is a pointwise multiplication operation.

[0103] S800, the fused representation vector sequence is sequentially decoded to output a predicted label sequence containing wordpiece entity category labels.

[0104] This step is performed by a sequence decoding processing unit. The sequence decoding processing unit inputs the fused representation vector sequence Hc={hc1,hc2,……,hc j , …, hcm The core function is to derive a globally optimal entity annotation sequence from the fused features by modeling the dependency between tags, and to provide a prediction result for the model loss calculation of S900.

[0105] S800 can specifically include:

[0106] (1) Emission score calculation

[0107] The fused representation vector sequence Hc is input into the linear transformation layer of the sequence decoding processing unit, and each word element fused vector is mapped to a "word element-tag" matching score (i.e., an emission score) through a trainable weight matrix, and finally an emission score matrix E is generated. Wherein, E ∈ R m×C C is the preset number of entity label categories, for example, C = 7, including "O" (non-entity), "B-PER" (person name start), "I-PER" (person name middle), "B-LOC" (location name start), "I-LOC" (location name middle), "B-ORG" (organization start), "I-ORG" (organization middle). E[j][c] represents the emission score of the jth word element matching the cth label, and the higher the score, the stronger the local possibility that the word element belongs to the label.

[0108] (2) Transition score modeling

[0109] In order to avoid local annotation bias (such as "B-ORG" mislabeled "O"), the sequence decoding processing unit introduces a trainable label transition matrix T to model the reasonable transition relationship of adjacent word elements. Wherein, T ∈ R C×C The elements T cp，cc in T represent the transition probability score of the current word element annotation cc when the previous word element label is cp, and the higher the score, the more consistent the label transition with the language logic (for example, T[B-ORG][I-ORG] score is much higher than T[B-ORG][O], which conforms to the rule that "organization name start is often followed by middle words").

[0110] Based on the emission score matrix E and the transition score matrix T, the Viterbi algorithm is used to calculate the label sequence with the maximum sum of emission score + transition score, that is, the globally optimal prediction result.

[0111] The Viterbi algorithm records the "cumulative maximum score" and "optimal predecessor label" of each word element corresponding to each type of label through dynamic programming, and finally backtracks from the mth word element to obtain the globally optimal label sequence, avoiding the overall annotation error caused by local high emission score, such as "Beijing" being annotated as "B-LOC", and the subsequent "city" needing to be annotated as "I-LOC" rather than "O" with slightly higher local score.

[0112] (4) Annotation system and output format

[0113] The sequence decoding processing unit supports a main flow entity labeling system, and can be selected according to task requirements: a BIO system: suitable for a non-embedded entity scene, and labels include “B-X” (entity X start word), “I-X” (entity X intermediate word), and “O” (non-entity), for example, “Beijing (I-LOC)”; a BIOES system: suitable for a single-word entity and nested entity scene, and “E-X” (entity X end word) and “S-X” (entity X single-word) are added on the basis of the BIO system, for example, “Li (B-PER) Bai (E-PER)” and “Beijing (S-LOC)”.

[0114] Finally, a structured prediction label sequence Y is output pred ={Y pred1 , Y pred2 , …, Y predj , …, Y predm}, where Y predj is an entity category labeling result of the jth word element, and is completely aligned with the format of the final label sequence (real label) in S400, and is used for loss calculation of S900.

[0115] The sequence decoding processing unit effectively solves the core pain points in Chinese entity recognition through joint modeling of “local features (emission scores) + global logic (transition scores)”: avoiding “local optimal deviation”: for example, after “A (B-ORG, indicating A company)”, even if the local emission score of “hand” is slightly higher, “I-ORG” will still be preferentially labeled based on the high transition score of T[B-ORG][I-ORG] (A mobile phone is a product name, and context needs to be combined here, and the decoding unit can adjust the global score); improving complex entity recognition accuracy: for nested entities (such as “Beijing Haidian Zhongguancun (B-LOC) Street (I-LOC)”) and long entities (such as “a certain automation research institute (B-ORG)”), the boundary and category labeling consistency can be ensured through label transition logic.

[0116] S900, based on the prediction label sequence and the real label sequence, a preset loss function is used to calculate the model loss; if the loss meets the convergence condition, the current model is taken as a target model; otherwise, the model parameters are updated and S200 is returned to continue training; wherein the preset loss function includes a conditional random field structure loss.

[0117] In the embodiment of the application, the convergence condition can be dynamically configured according to training requirements, including but not limited to:

[0118] Loss stability judgment: the decline amplitude of the training set loss is less than a preset threshold (such as 10 −5 ) in K consecutive iteration periods (such as K=5);

[0119] Performance target: the entity recognition F1 value on the validation set reaches the preset threshold (such as 0.92) and there is no improvement for 3 consecutive periods;

[0120] Maximum iteration limit: the total number of training rounds reaches the preset upper limit (such as 100 rounds) and is forcibly terminated.

[0121] In the embodiment of the application, the model full-link parameters are updated by the back propagation algorithm, including:

[0122] Context representation encoding processing unit (fine-tuning parameters of the BERT pre-training model);

[0123] Syntax dependency enhancement processing unit (convolution weights and biases of GCN);

[0124] Gating fusion processing unit (weight matrix, bias term);

[0125] Sequence decoding processing unit (label transition matrix T, linear transformation layer weight).

[0126] In the embodiment of the application, a multi-dimensional optimization strategy is adopted in the training process:

[0127] Early stopping mechanism (EarlyStopping): when the performance of the validation set does not improve for N rounds (such as N=5) in succession, the training is terminated to avoid overfitting;

[0128] Learning rate decay: cosine annealing or step decay strategy is adopted, the initial learning rate is set to 5e-5, and every M rounds (such as M=10

[0129] ) is decayed to 1 / 2 of the current value;

[0130] Dynamic label weight fine-tuning: every P rounds (such as P=3) the class static weight is recalculated based on the latest training data to enhance the adaptability to rare classes;

[0131] Optimizer selection: AdamW optimizer (weight decay coefficient is set to 1e-2) is adopted, adaptive learning rate adjustment and L2 regularization are used to suppress overfitting, and the model convergence speed and generalization performance are improved.

[0132] Through the above mechanism, the model can effectively balance the training stability and entity recognition accuracy while ensuring the convergence efficiency, and finally output the target model with excellent generalization ability.

[0133] Further, the preset loss function adopted in the embodiment of the application is a conditional random field structure loss with dynamic label weight fusion, which balances the loss contribution of different word elements through class static weight and prediction confidence weight, and is used to quantify the global difference between the predicted label sequence and the real label sequence.

[0134] Further, the model loss L satisfies the following conditions:

[0135] ;

[0136] where L is the model loss, and a and b are weighting coefficients, and a+b=1, which can be set by task configuration or optimized by cross-validation.

[0137] where, is the class static weight of the true label yi of the ith token, i is valued from 1 to n, and n is the total number of tokens in the training sample data. , f yi is the frequency of yi in the training data, that is, the proportion of the number of times that the label yi appears in the token annotation results of all training samples to the total number of annotations, yi∈Y, Y is a set of pre-set entity category labels, containing all possible entity categories and non-entity labels defined by the task, such as “B-PER” “I-PER” “B-ORG” “O”, and the elements in the set are de-duplicated category labels. ε is a smoothing coefficient, which can be, for example, 10 -6 , to prevent numerical explosion caused by extremely small frequency. The physical meaning of the calculation formula of the class static weight is that the smaller the frequency of a rare category is, the greater the corresponding class static weight is, and the error penalty for the rare category is heavier in loss calculation, which improves the recognition ability of the model for long-tail categories.

[0138] is the confidence weight of the ith token, , Entropy(P i ) represents the entropy value of the label prediction probability distribution P i of the ith token, and the greater the entropy is, the more uncertain the prediction is, such as “[0.5, 0.5]” has a greater entropy than “[0.9, 0.1]”. The physical meaning of the calculation formula of the confidence weight is that the greater the uncertainty of the prediction (the greater the entropy is), the greater the confidence weight is, which forces the model to preferentially optimize the annotation error of difficult-to-predict tokens.

[0139] L CRF，i is the prediction error of the ith token, that is, the CRF structure loss of the ith token.

[0140] Further, . Where A y（i-1），yi is the transition score from the true label y(i-1) of the i-1th token to the true label yi of the ith token in the label transition probability matrix, A y（i-1），yi is a trainable parameter for modeling the syntax logic between labels, such as “B-ORG” being more likely to be followed by “I-ORG”. i，yi is the score of the fusion feature vector of the ith token being predicted as the true label yi, which is calculated by a linear layer and reflects the matching degree of the token feature and the label. yi pY is a preset entity category label set, containing all possible entity categories defined by the task and non-entity labels such as "B-PER", "I-PER", "B-ORG", "O", etc., and the elements in the set are de-duplicated category labels.

[0141] The physical meaning of the calculation formula of the prediction error of the i-th word element is to minimize the negative log-likelihood, which is equivalent to maximizing the joint probability of the true label sequence (satisfying the word feature matching degree and the label dependency logic at the same time), avoiding local optimal labeling, such as predicting a single word element correctly, but the label sequence violates the grammar rules.

[0142] In summary, the present application is based on "text entity recognition model construction and application based on large language model", through the whole process technical design of "data quality optimization-feature deep fusion-training dynamic regulation-scene flexible adaptation", multi-dimensional technical breakthrough is realized in Chinese entity recognition task, the overall technical effect is as follows:

[0143] 1. Data level: solve the pain points of "low data quality, insufficient sample diversity", consolidate the training foundation The present application passes through data correction and enhancement processing unit, relies on large language model (such as Qwen3-14b) to carry out double optimization to preprocessed text and preliminary dependency graph: on the one hand, accurately identify and correct word segmentation errors (such as "Zhongguancun Avenue" is corrected to "Zhongguancun Avenue"), missing dependency edges (such as completing "subject-predicate relationship" edge), label and context contradiction (such as "Beijing" mislabeled "B-PER" is corrected to "B-LOC") and other problems, improve the annotation accuracy of training data by 15%-25%; on the other hand, through "entity replacement, context rewriting, adversarial sample generation" to generate diversified enhanced samples, combined with perplexity and dependency Figure 1 consistency screening, the effective training sample amount is expanded by 2-3 times, and complex scenes such as "long entity (such as some automation research institute), nested entity (such as 'Beijing Haidian Zhongguancun Avenue') are covered, completely solving the problems of "data noise interference training, sample coverage not complete leading to poor generalization" in traditional methods.

[0144] 2. Feature level: Achieve "semantic + syntax" dual feature cooperation, break through the core difficulty of Chinese entity recognition The invention constructs a "dual feature fusion architecture": the context representation coding processing unit (based on Chinese BERT / RoBERTa) captures deep semantics (such as the "brand" semantics of "A" in "A mobile phone"), solves the problem of polysemous word ambiguity; the syntax dependency enhancement processing unit (based on GCN / GAT) injects dependency graph structure information into vector representation (such as the verb-object relationship of "buy-mobile" to strengthen entity association), solves the problem of long-distance word dependence modeling; then the gating mechanism of the representation fusion processing unit dynamically balances the weight of the two types of features, focusing on semantic features to improve efficiency in short sentences, and focusing on syntactic features to ensure boundary accuracy in long sentences, finally the F1 value of Chinese entity recognition is improved by 8%-12%, especially the recall rate of recognition of obscure place names and special organization names is improved by more than 20%.

[0145] 3. Training level: Dynamically regulate loss weight to alleviate the problem of "class imbalance and insufficient training of difficult samples" The invention uses a "conditional random field loss that fuses static weight and confidence weight" through the model training optimization unit: the static weight based on the class frequency gives higher loss weight to low-frequency entities (such as "historical relic names"), solving the imbalance problem of "overtraining of high-frequency entities (such as names and place names) and insufficient training of low-frequency entities"; the confidence weight based on the prediction entropy value strengthens the training of difficult samples with fuzzy prediction (such as the "boundary fuzzy entity 'Beijing Xizhan South Road'"), avoiding overfitting of the model to "easy samples". In actual training, this mechanism improves the recognition accuracy of low-frequency entities by 18%-23% and speeds up model convergence by 30%.

[0146] 4. Application level: Architecture flexible adaptation, covering multi-domain Chinese entity recognition scenarios The "modular processing unit architecture" (preprocessing, syntax analysis, coding, decoding, etc. units independently cooperate) of the invention has strong expansibility:

[0147] Model construction end: Support flexible replacement of core units (such as replacing the coding unit with the medical field pre-training model "BioBERT" and replacing the syntax analysis unit with HanLP), without the need to restructure the overall architecture to adapt to news, medical, legal, and other vertical fields. For example, in medical text recognition, after replacing BioBERT, the recognition accuracy of disease names and drug names is improved by 10%-15%;

[0148] Entity recognition application end: optimize preprocessing (such as Jieba word segmentation adaptation Chinese semantic) for Chinese text characteristics, decoding system (BIOES adaptation single word entity), can be directly applied to "Chinese news word segmentation entity recognition" "medical record entity extraction" "legal document agency name recognition" and other scenes, without additional adaptation development, significantly reduce the landing cost.

[0149] The target model constructed by the text entity recognition model construction method based on large model data enhancement provided by the embodiment of the application can realize end-to-end entity recognition and labeling in various Chinese text scenes with the core characteristics of "strong data adaptability, high recognition accuracy and excellent scene expansion", and the general application process is: "obtaining text to be processed→text format adaptation→inputting the target model→outputting structured labeling results→supporting downstream tasks", and the operation details and application value of different scenes are as follows:

[0150] 1. News reporting scene: helping news content structuring and hot spot analysis

[0151] Application operation steps:

[0152] Text acquisition: through news information platform API, web crawler (within the scope of compliance) or media database, batch acquisition of news text to be processed (including politics, finance, entertainment and other fields), and the text format can be HTML, TXT or JSON;

[0153] Format adaptation: light preprocessing is performed on the obtained news text (if the text contains web tags, the tags need to be removed Etc. HTML tags, keep pure text content), without additional complex processing (target model built-in preprocessing unit can automatically complete word segmentation, format uniformity);

[0154] Model input: input the adapted news text into the target model in batches (single or batch, adjusted according to hardware resources), and the model automatically calls the built-in "preprocessing-syntactic analysis-coding-fusion-decoding" whole process processing unit;

[0155] Result output: the model outputs a structured prediction label sequence in 1-3 seconds (single text), and the annotated content includes "person name (B-PER / I-PER), place name (B-LOC / I-LOC), organization name (B-ORG / I-ORG), event name (B-EVT / I-EVT), file name (B-FIL / I-FIL)" and other entity types;

[0156] Downstream application: import the annotated results into data analysis tools (such as Python Pandas, Tableau), count the frequency of different news entities (such as the number of mentions of "a new energy enterprise" in financial news), or associate the relationship between entities (such as the association between "a leader" and "a meeting"), support news content archiving, hot event tracking, and media public opinion analysis.

[0157] Scenario adaptation advantages:

[0158] In view of the characteristics of news text "high information density, many emerging entities (such as new industry enterprise name 'certain live e-commerce' )", the model has "low-frequency entity recognition ability" through large model data enhancement training, which can accurately capture small organizations, and new event-related entities; At the same time, the semantic and syntactic fusion mechanism can solve the problem of "Beijing roast duck" being a dish name or the boundary ambiguity of "Beijing + roast duck", ensuring that the annotation accuracy is above 92%.

[0159] 2. Medical record scenario: support electronic medical record structuring and clinical analysis

[0160] Application operation steps:

[0161] Text acquisition: export the medical record text (such as hospitalization records, medical records, examination reports) to be processed from the hospital electronic medical record system (EMR), the format is mostly PDF or XML, and the data privacy needs to be ensured through compliance authorization;

[0162] Format adaptation: use PDF parsing tools (such as PyPDF2) to extract pure text of medical records, remove "doctor's signature" "page number" and other non-core content, and split "doctor's advice" "examination results" and other key paragraphs (for accurate annotation);

[0163] Model input: Input the split medical record text into the target model one by one (ensure single processing in medical scenarios to avoid privacy disclosure risks), and the model automatically loads the pre-trained weight adapted to the medical field (replace the built-in context encoding unit with BioBERT in advance);

[0164] Result output: The model outputs specialized entity labeling for the medical scenario, including "disease name (B-DIS / I-DIS), drug name (B-DRG / I-DRG), examination item (B-EXM / I-EXM), human organ (B-ORG / I-ORG, distinguished from institution name), and symptom (B-SYM / I-SYM)", with a labeling accuracy of over 95% (core entities);

[0165] Downstream application: Enter the labeling results into the hospital clinical data center (CDR) to build a standardized electronic medical record library (solve the problem of structuring handwritten medical records), or assist in clinical research (such as statistical analysis of "type 2 diabetes" patient commonly used drug types, and analysis of the correlation probability of "hypertension" and "cardiovascular disease").

[0166] Scenario adaptation advantages:

[0167] The pain points of medical text "professional terminology-intensive, same disease different name ('heart attack' = 'acute myocardial infarction')" can be solved by the model "medical exclusive sample library constructed by large model data enhancement", realizing unified labeling of synonymous entities; at the same time, the syntax dependency enhancement module can capture the association structure of "disease-examination-drug" (such as "perform XX examination → diagnose XX disease → give XX drug"), avoiding information gaps caused by isolated labeling.

[0168] 3. Legal document scenario: enabling legal text electrification and case search

[0169] Application operation steps:

[0170] Text acquisition: Obtain the legal text to be processed (such as civil judgment, ruling, and lawyer letter) from China Judgments and Documents Network and internal court database, in formats such as PDF or DOC, ensuring that the text does not contain classified information;

[0171] Format adaptation: Extract core paragraphs such as "party information", "litigation request", "court determination", and "judgment main text" from the text, remove "header and footer" and "seal watermark", and unify the text encoding to UTF-8;

[0172] Model input: Batch input the core paragraphs into the target model, and the model automatically switches the syntax analysis unit to a legal field adaptation tool (such as HanLP legal syntax analysis module) to optimize the recognition logic of "legal reference" and "party type" entities;

[0173] Output: The output of the annotation results includes "natural persons (B-PER / I-PER), legal persons (B-COM / I-COM), law firms (B-LAW / I-LAW), courts (B-COU / I-COU), legal provisions (B-REG / I-REG), and case numbers (B-CAS / I-CAS)", and can distinguish between "plaintiff" and "defendant" role attributes.

[0174] Downstream applications: Import the annotated results into a legal big data platform for case search (input "copyright infringement + XX court" to quickly match similar judgments), legal document archiving (store by "case type + entity type"), or assist lawyers in analyzing "the frequency of a certain legal provision in similar cases".

[0175] Advantages of scenario adaptation:

[0176] In view of the characteristics of legal documents, such as "long organization names ('XX Provincial High People's Court Civil Trial First Division') and diverse legal provision formats ('Article 1001 of the Civil Code' = 'Article 1001 of the Civil Code')", the model can achieve standardized annotation through legal samples optimized by the data correction module. The global consistency modeling (CRF) of sequence decoding can avoid unreasonable jumps such as "B-REG (legal provision start) followed by O (non-entity)", ensuring the standardization of annotation.

[0177] 4. Other extended scenarios: flexible adaptation to multi-domain text processing

[0178] In addition to the above scenarios, the target model can be adapted to more fields through simple "core unit replacement", and the application steps remain the same:

[0179] Financial report scenario: After obtaining the annual report and research report text of listed companies, replace the context encoding unit of the model with a financial pre-training model (such as FinBERT), which can accurately identify entities such as "company name (B-COM), financial product (B-PRO), and financial indicator (B-IND)", supporting financial data structuring and investment risk analysis;

[0180] Social media scenario: After obtaining text from Weibo and Douyin comment sections, the model automatically enhances the recognition of "online nicknames (B-USR), topic tags (B-HAS), and associated entities of memes (such as '[doge]' corresponding to 'humorous content')", assisting in social public opinion monitoring.

[0181] In summary, the application steps of the target model have both universality and scenario flexibility. The universal process ensures that users can quickly get started, and the scenario-based unit replacement (such as encoding models and syntax tools) meets the individual needs of different fields, enabling once construction and multiple scenario reuse, significantly reducing the cost of implementing Chinese text entity recognition in various industries.

[0182] The embodiment of the present application further provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are configured to execute the method according to the embodiment of the present application.

[0183] The embodiment of the present application further provides a computer readable storage medium, which stores computer executable instructions, and the computer executable instructions are used for executing the method according to the embodiment of the present application.

[0184] It should be understood that the steps shown above can be reordered, added to, or deleted from, using various forms of flow. For example, the steps described in the present application can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present application can be achieved, which are not limited herein.

[0185] The above detailed description does not constitute a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A text entity recognition model construction method based on large model data enhancement, characterized in that, The method comprises the following steps: S100, constructing an initial text named entity recognition model, the model comprising a text preprocessing unit, a syntax dependency analysis processing unit, a data correction and enhancement processing unit, a context representation coding processing unit, a syntax dependency enhancement processing unit, a representation fusion processing unit, and a sequence decoding processing unit; S200, inputting training sample data into the text named entity recognition model, and generating preprocessed text by preprocessing the text through the preprocessing unit; S300, performing syntax dependency analysis on the preprocessed text through the syntax dependency analysis processing unit to construct a preliminary dependency graph; S400, correcting the preprocessed text and the preliminary dependency graph by using a large language model through the data correction and enhancement processing unit to obtain a corrected text and a corrected dependency graph, and generating an enhanced text and an enhanced dependency graph based on the correction result; obtaining a final text sequence and a final dependency graph for subsequent processing, the final text sequence comprising the corrected text and the enhanced text, and the final dependency graph comprising the corrected dependency graph and the enhanced dependency graph; S500, encoding the final text sequence through the context representation coding processing unit to obtain an initial representation vector sequence; S600, performing structural enhancement on the initial representation vector sequence based on the final dependency graph and the initial representation vector sequence through the syntax dependency enhancement processing unit to obtain an enhanced representation vector sequence; wherein for each word element, the enhanced representation vector of the word element is generated by aggregating the initial representation vector of the word element and the first-order neighbor word element vector associated with the dependency relationship; S700, fusing the initial representation vector sequence and the enhanced representation vector sequence through the representation fusion processing unit to obtain a fused representation vector sequence; S800, performing sequence decoding on the fused representation vector sequence through the sequence decoding processing unit to output a predicted label sequence containing word element entity category labels; S900, calculating the model loss by using a preset loss function based on the predicted label sequence and a real label sequence; if the loss meets the convergence condition, taking the current model as a target model; otherwise, updating the model parameters and returning to S200 for continuous training; wherein the preset loss function comprises a conditional random field structure loss.

2. The method of claim 1, wherein, The model loss meets the following conditions: ； wherein L is a model loss, and a and β are weighting coefficients, is a class static weight of a true label yi of an i-th token, i is valued from 1 to n, and n is a total number of tokens in training sample data, , f yi is a frequency of occurrence of yi in training data, and ε is a smoothing coefficient, is a confidence weight of an i-th token, , Entropy(P i ) represents an entropy value of a label prediction probability distribution P i of an i-th token, and L CRF，i is a prediction error of an i-th token.

3. The method of claim 2, wherein, wherein A y（i-1），yi is the transition score from the true label y(i-1) of the i-1th wordpiece to the true label yi of the ith wordpiece in the label transition probability matrix s i，yi is the score of the fusion feature vector of the ith wordpiece predicting the true label yi yi p is the predicted label of the ith wordpiece, and Y is a preset label set.

4. The method of claim 1, wherein, The context representation coding processing unit adopts a pre-trained BERT model.

5. The method of claim 1, wherein, The syntax dependency enhancement processing unit adopts a graph convolution network.

6. The method of claim 1, wherein, The representation fusion processing unit fuses the initial representation vector sequence and the enhanced representation vector sequence by using a gated fusion mechanism.

7. The method of claim 6, wherein, a fusion representation vector hc representing the jth token in the sequence of fusion representation vectors j satisfies the following condition: hc j = z j ⊙ h0 j + (1 - z j ) ⊙ h1 j ; where j takes values from 1 to m, m is the number of tokens in the sequence of fusion representation vectors, z j is a gating coefficient for the jth token, z j = σ(Wz•[h0 j , h1 j ] + bz), [h0 j , h1 j ] represents a concatenation operation of the initial representation vector h0 j and the enhanced representation vector h1 j for the jth token, Wz is a trainable weight matrix, bz is a trainable bias term, σ() is a non-linear activation function, and ⊙ is an element-wise multiplication operation.

8. An electronic device, comprising: The device comprises a processor and a memory; the processor is configured to execute the steps of the method according to any one of claims 1 to 7 by invoking programs or instructions stored in the memory.

Images