Named entity recognition system based on speech domain migration and model fusion

Abstract

The invention relates to a named entity recognition system based on language domain migration and model fusion, a training process of the system is composed of a source domain and a target domain, the source domain comprises training of multiple groups of corpora and transmits parameters to the target domain, and the target domain receives parameter transmission from the source domain and outputs corresponding labels of texts; the system comprises three modules, namely an Albert pre-training module, a TNER encoder and a CRF decoder module, the system inputs a text sequence and outputs a corresponding label sequence; in the source domain training stage, after the model is subjected to CRF decoding, a loss function is calculated, model parameters are adjusted, and then the model parameters are continuously applied to training of the next field in the source domain; in a target domain prediction stage, the model directly returns a corresponding label after CRF decoding; the problem that the actual application performance is reduced due to the fact that an existing named entity recognition system faces cross-lingual domain notation data scarcity is solved, cross-lingual domain named entity recognition is achieved, and original data support is provided for a knowledge graph construction task.

Description

Technical Field

[0001] The present invention relates to a natural language processing technology, and in particular to a named entity recognition system based on register transfer and model fusion. Background Art

[0002] Named entity recognition technology plays a vital role in the field of natural language processing. Its core function is to automatically extract entities with specific meanings from unstructured text data, such as names of people, places, institutions, time expressions, numerical expressions, etc. It plays a vital role in information extraction and structuring, semantic understanding and intelligent question answering, machine translation and cross-language processing, sentiment analysis and public opinion monitoring, text summarization and automatic generation, and decision-making assistance and intelligent analysis.

[0003] However, the existing named entity recognition systems have the following shortcomings: First, for most practical application scenarios, especially those professional fields (such as specific industry terms, emerging network languages, etc.), the available annotated data is extremely scarce. Due to the differences in semantic features between domains, these domains with insufficient label data cannot use domains with sufficient labels for named entity recognition training (that is, for example, the geological domain with insufficient labels cannot use the medical domain text with sufficient labels for cross-domain named entity recognition training). Secondly, most traditional named entity recognition systems are trained based on large-scale annotated data sets. These models can achieve good recognition results with sufficient data support. However, in practical applications, especially in emerging fields or specific scenarios, only a small amount or even very little annotated data can be obtained. In this case, traditional named entity recognition systems often suffer from overfitting, resulting in a significant drop in performance in test sets or actual applications.

[0004] Related patent application (patent) number: 202310160997.6 Name: Named entity recognition method and recognition device, which can realize named entity recognition for text, but the system is based on a large sample and cannot realize cross-domain named entity recognition. In addition, the system is based on manual annotation and cannot realize automatic recognition without manual work. Summary of the invention

[0005] In view of the problem that the existing named entity recognition system faces the problem of reduced performance in actual applications due to the scarcity of cross-domain annotation data, a named entity recognition system based on domain transfer and model fusion is proposed. Based on the idea of ​​transfer learning and model fusion, the system realizes cross-domain named entity recognition through transfer learning based on model parameters, and provides original data support for the knowledge graph construction task.

[0006] The technical problems to be solved by the present invention are embodied in the following aspects:

[0007] (1) Improve cross-domain recognition capabilities. This allows the system to adapt to different fields and contexts, and to identify and classify named entities in specific fields across different domains, thus reducing reliance on data in specific fields.

[0008] (2) Enhance the effect of model fusion. Different named entity recognition models may perform better on different types of text or entities. Model fusion can fully utilize these advantages. When a single model faces complex and changing text data, it may cause recognition errors due to certain defects or deviations. Model fusion can reduce the occurrence of such errors and improve the overall robustness of the system by complementing multiple models.

[0009] (3) Improve recognition efficiency and accuracy. Domain transfer and model fusion technology can accelerate the system's recognition process by optimizing algorithms and reducing computational redundancy. By combining domain transfer and model fusion technology, the named entity recognition system can more accurately identify named entities in text, including those entities that may have different meanings or forms in different domains.

[0010] The technical solution of the present invention is:

[0011] A named entity recognition system based on register transfer and model fusion. The training process of this system consists of a source domain and a target domain. The source domain contains training on multiple groups of corpora and transfers parameters to the target domain. The target domain accepts the parameter transfer from the source domain and outputs the corresponding label of the text.

[0012] This system treats the training of different registers as different tasks. By learning many different registers, the underlying model parameters are shared cyclically. Transfer learning based on parameter sharing realizes knowledge transfer between different registers by sharing the training parameters of different registers.

[0013] This system consists of three modules, namely, Albert pre-training module, T_NER encoder and CRF decoder module; its overall architecture includes input layer, pre-training layer, encoding layer, decoding layer and output layer connected in sequence, among which, Albert pre-training module, T_NER encoder and CRF decoder module correspond to pre-training layer, encoding layer and decoding layer respectively; the system inputs text sequence and outputs corresponding label sequence;

[0014] In the source domain training phase, after the model is decoded by CRF, the loss function is calculated and the model parameters are adjusted, and then the model parameters are further applied to the training of the next domain in the source domain; in the target domain prediction phase, the model directly returns the corresponding label after CRF decoding;

[0015] The Albert pre-training module converts the text in the source domain from characters to word vectors;

[0016] The T_NER encoder承接pre-training module, with its input being the word vectors obtained from Albert training and its output being the probability combination of the label sequence corresponding to the text, connecting to the CRF decoder module; The T_NER encoder is composed of a linear combination of BiLSTM and BiGRU, combining the two as the learning structure for sequence features;

[0017] The CRF decoder module has a structure centered around CRF, aiming to provide more constraints for the probability combination splicing obtained by the BiLSTM+BiGRU encoder; This part has two functions. In the source domain training structure, it conducts cyclic transmission of model parameters, passing the model parameters back to the output position of the pre-training module to participate in the model training of the next set of corpora; In the target domain extraction structure, it conducts probability combination conversion, converting the probability combination into the corresponding label and outputting it.

[0018] Furthermore, the source domain and the target domain are as follows:

[0019] From a structural perspective, macroscopically, the data processing of the source domain can be represented by the following formula:

[0020]

[0021] Among them, p n represents the output result of the source domain training structure, which is the model parameters of the cell; p n will be passed to the target domain as one of the inputs to the target domain; x i represents the i-th group of texts, and y i represents the label corresponding to the i-th group of texts, and n represents the n-th source domain;

[0022] Microscopically, the source domain is composed of a linear combination of multiple cells, and the cell is a special design structure of T_NER; Texts and corresponding labels in different language domains are input into the cell in sequence. For each cell, its mathematical expression is as follows:

[0023]

[0024] Among them, pi represents the output of the i-th layer of the cell, 0 < i < n + 1, representing the neural parameters of the i-th layer of the cell passed to the i + 1-th layer. The i-th layer of the cell corresponds to the i-th layer of texts; (x i , y i , p i-1 ) represents the input of the i-th layer of the cell, x i represents the i-th group of texts, y i represents the label corresponding to the i-th group of texts, and P i-1 represents the neural parameters passed from the i - 1-th layer of the cell to the i-th layer of the cell; Represents the neural network function of the cell; where i∈[0,n], p0=0;

[0025] For the target domain extraction structure, its structure contains only one cell, usually only one set of training data, and its logic can be expressed by the following mathematical formula:

[0026] y t =θ(p n ,x t ) (3)

[0027] x t The text representing the target domain, p n represents the model parameters obtained by source domain training, n represents the number of groups of corpora in the source domain; y t Represents x t The corresponding label.

[0028] Furthermore, the operating principle of the T_NER encoder can be described by the following formula:

[0029] i tL =σ(W i x t +U i h (t-1)L +b i ) (4)

[0030] f tL =σ(W f x t +U f h (t-1)L +b f ) (5)

[0031] o tL =σ(W o x t +U o h (t-1)L +b o ) (6)

[0032] g tL =tanh(W g x t +U g h (t-1)L +b g ) (7)

[0033] c tL =f tL *c (t-1)L +i tL *g tL (8)

[0034] htL =o t *tanh(c t ) (9)

[0035] r tG =σ(W r x t +U r h (t-1)G +b r ) (10)

[0036] u tG =σ(W u x t +U u h (t-1)G +b u ) (11)

[0037] g tG =tanh(W g x t +U g (r t *h (t-1)G )) (12)

[0038] h tG =(1-u tG )*h (t-1)G +u tG *g tG (13)

[0039]

[0040] In the above formula, the subscript *L represents the process parameters and output results of BiLSTM neurons, and the subscript *G represents the process parameters and output results of BiGRU neurons; where x t represents the trained word vector, h tL represents the probability of the text corresponding to the label output by the BiLSTM neuron, h tG represents the probability of the text corresponding to the label output by the BiGRU neuron, h t Represents the probabilistic sequence concatenation obtained by mixed training.

[0041] Furthermore, the CRF loss function is used as the loss function of the model. The CRF loss function takes the sequence label as the path. After the same text sequence passes through the BiLSTM+BiGRU encoder, different label sequences will be obtained. The probability of occurrence of different sequences is different. The CRF loss function aims to maximize the probability of the correct sequence. The CRF loss functions of BiLSTM and BiGRU are calculated separately, and are not calculated uniformly. The formula is as follows:

[0042]

[0043] In the above formula, the subscript L represents the loss function of the BiLSTM part; the subscript G represents the loss function of the BiGRU part; m in the formula represents the total number of objects in the text sequence, j represents each object; P RealPath refers to the correct tag sequence, Right now Refers to all paths that appear in BiLSTM and BiGRU. In this model, the loss functions of the BiLSTM part and the BiGRU part are directly added together, linearly summed, and then back-propagated separately to calculate parameter corrections.

[0044] The beneficial effects of the present invention are:

[0045] (1) Improve cross-domain recognition capabilities. The system can apply the named entity recognition rules and patterns learned from one domain (such as the news domain) to another domain (such as the medical domain), thereby improving the cross-domain named entity recognition capabilities. Through domain migration, the system can adapt to data in new domains more quickly, reduce dependence on annotated data in new domains, and reduce data annotation costs.

[0046] (2) Improve recognition accuracy and robustness. Model fusion technology can effectively improve recognition accuracy by integrating the prediction results of multiple different named entity recognition models (such as rule-based models, statistics-based models, and deep learning-based models). Different models may capture features in the text from different perspectives, so the fused model can understand the text more comprehensively and reduce misrecognition.

[0047] (3) Broaden application scenarios. Due to the application of domain transfer and model fusion technology, the system can better adapt to different application scenarios, such as news, medical, legal, financial and other fields. This multi-domain adaptability gives the system a wider range of application prospects. In response to the needs of different fields, the system can customize and optimize the named entity recognition model to provide recognition results that are more in line with the characteristics of the field.

[0048] (4) Compared with patent 202310160997.6, the present invention improves cross-domain recognition capabilities. The system can apply the named entity recognition rules and patterns learned from one domain (such as the news field) to another domain (such as the medical field), thereby improving the cross-domain named entity recognition capabilities. Through domain migration, the system can be independent of manual work, can adapt to data in new fields more quickly, reduce dependence on annotated data in new fields, and reduce data annotation costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1It is a flowchart of the operation of the named entity recognition system based on register transfer and model fusion of the present invention;

[0050] Figure 2 This is a diagram of the architecture of the named entity recognition system based on register transfer and model fusion of the present invention;

[0051] Figure 3 It is the structural diagram of T_NER encoder of the present invention;

[0052] Figure 4 It is a logic diagram of a specific embodiment of the named entity recognition system of the present invention. DETAILED DESCRIPTION

[0053] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0054] The training process of this system is as follows Figure 1 As shown in the figure, the training process consists of a source domain and a target domain. The source domain contains training for multiple groups of corpora and passes parameters to the target domain. The target domain accepts parameter transfer from the source domain and outputs the corresponding labels of the text. Since the target domain labels are insufficient, as many corpora with sufficient annotations as possible need to be used in the source domain. However, these corpora belong to different domains. When calculating the loss function of named entity recognition, their loss paths are different from the perspective of language structure. How to extract the common features of language through many different domains? In response to the above problems, this model is designed with multi-tasks based on hard parameter sharing. This system treats the training of different domains as different tasks. By learning many different domains (including a large amount of text and corresponding labels), the underlying model parameters are shared cyclically. This learning method can alleviate the overfitting of the model at the domain level caused by a single domain, and can also perform data enhancement on the other hand. The noise of different domains tends to different directions, and multi-domain mixed training can better highlight the common features of language.

[0055] At the same time, since the register of the training text is different from the register of the text to be tested, this model is designed based on transfer learning for this situation. This system is based on parameter sharing transfer learning, which realizes knowledge transfer between different registers by sharing the training parameters of different registers.

[0056] From a structural perspective, in general, data processing in the source domain can be represented by the following formula:

[0057]

[0058] Among them, p nRepresents the output result of the source domain training structure, which is the model parameter of the cell. p n Will be passed to the target domain as one of the inputs to the target domain. x i Represents the i-th group of texts, y i Represents the label corresponding to the i-th group of texts, and n represents the n-th source domain.

[0059] Microscopically, the source domain is composed of a linear combination of multiple cells (neurons). A cell is a special design structure of T_NER (a Python library for Transformer named entity recognition). Different domain texts and their corresponding labels are sequentially input into the cell in order. For each cell, its mathematical expression is as follows:

[0060]

[0061] Among them, pi represents the output of the i-th layer of the cell (0 < i < n + 1), representing the neural parameter of the i-th layer of the cell passed to the (i + 1)-th layer (the i-th layer of the cell corresponds to the i-th layer of text). (x i , y i , p i-1 ) represents the input of the i-th layer of the cell, x i represents the i-th group of texts, y i represents the label corresponding to the i-th group of texts, p i-1 represents the neural parameter passed from the (i - 1)-th layer of the cell to the i-th layer of the cell. Represents the neural network function of the cell. Among them, i ∈ [0, n], and p0 = 0.

[0062] For the target domain extraction structure, its structure only contains one cell and usually only contains a set of data to be trained. Its logic can be represented by the following mathematical formula:

[0063] y t = θ(p n , x t ) (3)

[0064] x t represents the text of the target domain, p n represents the model parameters obtained from the source domain training, and n represents that there are n groups of corpora in the source domain. y t represents x t The corresponding label.

[0065] This system includes three major modules, namely the Albert pre-training module, the T_NER encoder, and the CRF decoder module.

[0066] Its overall architecture is as Figure 2As shown; the system inputs the text sequence "I love Shanghai" and outputs the corresponding label sequence "O", "O", "B-LOC", "I-LOC".

[0067] In the source domain training phase, after the model is decoded by CRF, the loss function is calculated and the model parameters are adjusted, and then the model parameters are further applied to the training of the next field in the source domain; in the target domain prediction phase, the model directly returns the corresponding label after CRF decoding.

[0068] The Albert pre-training module converts the text in the source domain from characters to word vectors. Compared with the traditional Bert, Albert is lighter and more suitable for task processing in small sample areas. It can also speed up subsequent training.

[0069] The T_NER encoder takes over the pre-training module. Its input is the word vector obtained by Albert training. Its output is the probability combination of the label sequence corresponding to the text, which is connected to the CRF decoder module. The T_NER encoder is a linear combination of BiLSTM (bidirectional long short-term memory network) and BiGRU (bidirectional gated recurrent unit). Through research, it is found that BiLSTM is more suitable for feature learning of long texts, while BiGRU is more suitable for feature learning of short texts. Therefore, this model combines the two as a learning structure for sequence features. The structure of the T_NER encoder is as follows: Figure 3 :

[0070] Its operating principle can be described by the following formula:

[0071] i tL =σ(W i x t +U i h (t-1)L +b i ) (4)

[0072] f tL =σ(W f x t +U f h (t-1)L +b f ) (5)

[0073] o tL =σ(W o x t +U o h (t-1)L +b o ) (6)

[0074] g tL =tanh(W g x t +U g h(t-1)L +b g ) (7)

[0075] c tL =f tL *c (t-1)L +i tL *g tL (8)

[0076] h tL =o t *tanh(c t ) (9)

[0077] r tG =σ(W r x t +U r h (t-1)G +b r ) (10)

[0078] u tG =σ(W u x t +U u h (t-1)G +b u ) (11)

[0079] g tG =tanh(W g x t +U g (r t *h (t-1)G )) (12)

[0080] h tG =(1-u tG )*h (t-1)G +u tG *g tG (13)

[0081]

[0082] In the above formula, the subscript *L represents the process parameters and output results of BiLSTM neurons, and the subscript *G represents the process parameters and output results of BiGRU neurons. t represents the trained word vector, h tL Represents the probability of the text corresponding to the label output by the BiLSTM neuron, that is, Figure 4 L L i(i∈(1,n)),h tG Represents the probability of the text corresponding to the label output by the BiGRU neuron, that is Figure 4 L Li(i∈(1,n)),h t represents the probability sequence concatenation obtained by mixed training, that is, Figure 4 Li(i∈(1,n)) in the formula. For other general contents in the formula, please refer to the relevant papers on LSTM and GRU (Long Short-Term Memory, Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling)

[0083] The CRF decoder module is a structure with CRF (conditional random field) as the core. Its purpose is to provide more constraints for the probability combination splicing obtained by the BiLSTM+BiGRU encoder. This part has two functions: in the source domain training structure, the model parameters are cyclically transferred and the model parameters are transferred back to the output position of the pre-training module to participate in the model training of the next set of corpus; in the target domain extraction structure, the probability combination conversion is performed, the probability combination is converted into the corresponding label, and then output.

[0084] This experiment uses the CRF loss function as the loss function of the model. The CRF loss function uses the sequence label as a path. After the same text sequence passes through the BiLSTM+BiGRU encoder, different label sequences will be obtained. The probability of occurrence of different sequences is different. The CRF loss function aims to maximize the probability of the correct sequence. This study calculates the CRF loss function of BiLSTM and BiGRU separately, and does not perform a unified calculation. The formula is as follows:

[0085]

[0086] In the above formula, the subscript L represents the loss function of the BiLSTM part; the subscript G represents the loss function of the BiGRU part; m in the formula represents the total number of objects in the text sequence, and j represents each object. RealPath refers to the correct tag sequence, (Right now ) refers to all paths that appear in BiLSTM and BiGRU, that is, Figure 4 All L L 1-L L 4 and L G 1-L G 4. In this model, the loss functions of the BiLSTM part and the BiGRU part are directly added together, linearly summed, and then back-propagated separately to calculate the parameter correction.

[0087] Implementation example 1: Use this system to extract labels that exist in the news and social media fields as source domains and the geography field as the target domain.

[0088] (1) Label-oriented data analysis

[0089] Assume that the news and social media fields are previously accumulated data, and the geography field is the field to which the text to be identified belongs. Before using the system for named entity recognition tasks, label-oriented data analysis must be performed first. The data features of the news, social media, and geography fields are analyzed. After analysis, it is noted that the data in these three fields all contain "person names", "place names", and "organization names".

[0090] (2) Named Entity Recognition Training for the Source Domain

[0091] The news and social media domains with sufficient labels are used as source domains and put into the named entity recognition system based on domain transfer and model fusion for training.

[0092] First, the news domain text is put into the system. At this time, the system is in the initialization state, and the parameters of the T_Named Entity Recognition encoder module are original values. After the news domain text passes through the Albert pre-training module, the T_NER encoder module and the CRF decoder module, the model parameters that can reflect the language characteristics of the domain are obtained, and the parameters are fed back to the T_NER encoder module to participate in the next round of training.

[0093] Secondly, the social media domain text is put into the system. At this time, the system is in a round of source domain training, and the parameters of the T_NER encoder module are the results of training in the news domain. After the social media domain text passes through the Albert pre-training module, the T_NER encoder module and the CRF decoder module, the model parameters that integrate the news domain and the social media domain are obtained, and the parameters are fed back to the T_NER encoder module to participate in the training of the target domain.

[0094] (3) Named Entity Recognition for the Target Domain

[0095] The model trained based on the source domain is used to perform named entity recognition on the target domain of geography.

[0096] Put the geographic domain text into the system. At this time, the system is in the source domain training completed state, and the parameters of the T_NER encoder module are the results after the source domain training. After the geographic domain text passes through the Albert pre-training module, the T_NER encoder module and the CRF decoder module, the label corresponding to the text is obtained. The training is completed.

[0097] The above-mentioned embodiment only expresses one implementation mode of the present invention, and its description is relatively specific and detailed, but it cannot be understood as limiting the scope of the invention patent. It should be pointed out that for ordinary technicians in this field, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be based on the attached claims.

Claims

1. A named entity recognition system based on register transfer and model fusion, characterized in that: The training process of this system consists of a source domain and a target domain. The source domain contains training for multiple groups of corpora and passes parameters to the target domain. The target domain accepts the parameter transfer from the source domain and outputs the corresponding label of the text. This system treats the training of different registers as different tasks. By learning many different registers, the underlying model parameters are shared cyclically. Transfer learning based on parameter sharing realizes knowledge transfer between different registers by sharing the training parameters of different registers. This system consists of three modules, namely, Albert pre-training module, T_NER encoder and CRF decoder module; its overall architecture includes input layer, pre-training layer, encoding layer, decoding layer and output layer connected in sequence, among which, Albert pre-training module, T_NER encoder and CRF decoder module correspond to pre-training layer, encoding layer and decoding layer respectively; the system inputs text sequence and outputs corresponding label sequence; In the source domain training phase, after the model is decoded by CRF, the loss function is calculated and the model parameters are adjusted, and then the model parameters are further applied to the training of the next domain in the source domain; in the target domain prediction phase, the model directly returns the corresponding label after CRF decoding; The Albert pre-training module converts the text in the source domain from characters to word vectors; The T_NER encoder inherits the pre-training module. Its input is the word vector obtained by Albert training. Its output is the probability combination of the label sequence corresponding to the text, which is connected to the CRF decoder module. The T_NER encoder is a linear combination of BiLSTM and BiGRU, which are combined as a learning structure for sequence features. The CRF decoder module is a structure with CRF as the core. Its purpose is to provide more constraints for the probability combination concatenation obtained by the BiLSTM+BiGRU encoder. This part has two functions. In the source domain training structure, it performs a cyclic transfer of model parameters and passes the model parameters back to the output position of the pre-training module to participate in the model training of the next set of corpus. In the target domain extraction structure, it performs probability combination conversion, converts the probability combination into corresponding labels, and outputs them.

2. The named entity recognition system based on register transfer and model fusion according to claim 1, characterized in that: The source domain and target domain are as follows: From a structural perspective, in general, data processing in the source domain can be represented by the following formula: Among them, p n Represents the output result of the source domain training structure, which is the model parameter of the cell; p n Will be passed to the target domain as one of the inputs of the target domain; x i represents the i-th group of text, y i represents the label corresponding to the i-th group of text, and n represents the n-th source domain; Microscopically, the source domain is a linear combination of multiple cells. Cell is a special design structure of T_NER. Different register texts and corresponding labels are input into the cell in sequence. For each cell, its mathematical expression is as follows: Where pi represents the output of the i-th layer cell, 0<i<n+1, and represents the neural parameter of the i-th layer cell that is passed to the i+1th layer. The i-th layer cell corresponds to the i-th layer text; (x i ,y i , p i-1 ) represents the input of the i-th layer cell, x i represents the i-th group of text, y i Indicates the label corresponding to the i-th group of text, p i-1 Represents the neural parameters transmitted from the i-1th layer cell to the i-th layer cell; Represents the neural network function of the cell; where i∈[0,n], p0=0; For the target domain extraction structure, its structure contains only one cell, usually only one set of training data, and its logic can be expressed by the following mathematical formula: y t =θ(p n x t )(3) x t The text representing the target domain, p n represents the model parameters obtained by source domain training, n represents the number of groups of corpora in the source domain; y t Represents x t The corresponding label.

3. The named entity recognition system based on register transfer and model fusion according to claim 1, characterized in that: The operating principle of the T_NER encoder can be described by the following formula: i tL =σ(W i x t +U i h (t-1)L +b i ) (4) f tL =σ(W f x t +U f h (t-1)L +b f ) (5) the tL =σ(W o x t +U o h (t-1)L +b o ) (6) g tL =tanh(W g x t +U g h (t-1)L +b g ) (7) c tL =f tL *c (t-1)L +i tL *g tL (8) h tL =o t *tanh(c t ) (9) r tG =σ(W rxt +U r h (t-1)G +b r ) (10) u tG =σ(W u x t +U u h (t-1)G +b u ) (11) g tG =tanh(W g x t +U g (r t *h (t-1)G )) (12) h tG =(1-u tG )*h (t-1)G +u tG *g tG (13) In the above formula, the subscript *L represents the process parameters and output results of BiLSTM neurons, and the subscript *G represents the process parameters and output results of BiGRU neurons; where x t represents the trained word vector, h tL represents the probability of the text corresponding to the label output by the BiLSTM neuron, h tG represents the probability of the text corresponding to the label output by the BiGRU neuron, h t Represents the probabilistic sequence concatenation obtained by mixed training.

4. The named entity recognition system based on register transfer and model fusion according to claim 1, characterized in that: The CRF loss function is used as the loss function of the model. The CRF loss function takes the sequence label as the path. After the same text sequence passes through the BiLSTM+BiGRU encoder, different label sequences will be obtained. The probability of occurrence of different sequences is different. The CRF loss function aims to maximize the probability of the correct sequence. The CRF loss functions of BiLSTM and BiGRU are calculated separately, and are not calculated uniformly. The formula is as follows: In the above formula, the subscript L represents the loss function of the BiLSTM part; the subscript G represents the loss function of the BiGRU part; m in the formula represents the total number of objects in the text sequence, j represents each object; P Real Path refers to the correct tag sequence, Right now Refers to all paths that appear in BiLSTM and BiGRU. In this model, the loss functions of the BiLSTM part and the BiGRU part are directly added together, linearly summed, and then back-propagated separately to calculate parameter corrections.

Images