Entity recognition and extraction method, device and equipment and storage medium

Abstract

The invention discloses an entity recognition and extraction method and device, equipment and a storage medium, and relates to the technical field of natural language processing, and the method comprises the steps: determining a target character in a target domain text, and carrying out the coding of the target domain text through a to-be-trained entity recognition and extraction model, so as to obtain a corresponding word segmentation result; wherein the target character is a single character in the target domain text; determining a word ordered sequence corresponding to the target character based on the word segmentation result, and training the word ordered sequence by using a to-be-trained entity recognition extraction model to determine text semantic features; determining a target text feature based on the text semantic feature and the text structure feature, training a to-be-trained entity recognition extraction model based on the target text feature, and performing convergence training based on a preset final loss function to obtain a target entity recognition extraction model, and performing entity identification and extraction operation based on the target entity identification and extraction model. In this way, the accuracy of entity extraction can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to an entity recognition extraction method, device, equipment and storage medium. Background Art

[0002] With the rapid development of science and technology, the amount of data generated by people is growing explosively. The financial industry also needs to face increasingly large text data streams and information resources. However, most of the data generated in financial scenarios are unstructured and scattered, such as credit scenario data, industry research reports, financial reports, etc. How to extract key information from unstructured and complex text data has become an urgent problem to be solved. Traditional data processing and analysis methods have been unable to meet the rapidly changing market needs and complex business challenges. Named entity recognition, as a key downstream task in the field of natural language processing, can help financial institutions extract key entity information from massive data. Through intelligent analysis of text data such as news, financial reports, transaction records and market comments, named entity recognition technology can identify important information in the text in real time, disease classification and insurance time in insurance scenarios, and information such as people, institutions, and products in research report scenarios, thereby providing strong support for decision-making. However, the current named entity recognition technology has problems such as unclear entity boundaries, rigid semantic prior knowledge fusion methods, and inability to obtain high-level token (single text in the Chinese field) level expressions. Therefore, how to extract entities with clear boundaries needs to be solved. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide an entity recognition extraction method, device, equipment and storage medium, which can extract important entities from unstructured data and improve the accuracy of entity extraction. The specific scheme is as follows:

[0004] In a first aspect, the present application discloses an entity recognition extraction method, comprising:

[0005] Data tagging is performed on the target domain text to obtain a target text corresponding to the target domain text, and the target domain text is encoded using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding word segmentation result; wherein the target text is a single character in the target domain text;

[0006] Determine the ordered sequence of words corresponding to the target text based on the word segmentation result, and train the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the semantic features of the text;

[0007] Determine a co-occurrence matrix, a syntactic dependency matrix, and a relative distance matrix between the target characters based on the target domain text, and perform feature fusion on the co-occurrence matrix, the syntactic dependency matrix, and the relative distance matrix to obtain text structure features;

[0008] The target text features are determined based on the text semantic features and the text structural features, and the entity recognition extraction model to be trained is trained based on the target text features. At the same time, the entity recognition extraction model to be trained is converged based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

[0009] Optionally, the target domain text is data-labeled to obtain a target text corresponding to the target domain text, and the target domain text is encoded using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding word segmentation result, including:

[0010] Acquire target domain text by using a preset crawler technology, and perform data tagging on the target domain text to obtain target text corresponding to the target domain text;

[0011] The target domain text is encoded using the NEZHA pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding word segmentation results.

[0012] Optionally, determining the ordered sequence of words corresponding to the target text based on the word segmentation result, and using a preset neural network model in the entity recognition extraction model to be trained to train the ordered sequence of words to determine text semantic features, includes:

[0013] Determining all words containing the target text from a preset vocabulary dictionary based on the word segmentation result, and sorting all the words based on a preset sequence formation rule to obtain a corresponding ordered sequence of words;

[0014] Training the ordered sequence of words based on a bidirectional semantic gated recurrent unit in the entity recognition extraction model to be trained to determine an aggregate feature;

[0015] The aggregated features and the original features of the target text are concatenated to obtain text semantic features.

[0016] Optionally, determining a co-occurrence matrix, a syntactic dependency matrix and a relative distance matrix between the target characters based on the target domain text, and performing feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structure features, includes:

[0017] Determine a co-occurrence matrix, a syntactic dependency matrix and a relative distance matrix between the target characters based on the co-occurrence relationship, the syntactic dependency relationship and the relative position relationship between each character in the target domain text;

[0018] Based on the structural feature fusion attention mechanism in the entity recognition extraction model to be trained, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are fused with structural features to obtain text structural features.

[0019] Optionally, the structural feature fusion attention mechanism based on the entity recognition extraction model to be trained performs structural feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structural features, including:

[0020] Concatenating the co-occurrence matrix, the syntactic dependency matrix, and the relative distance matrix to obtain structural concatenation features, and inputting the structural concatenation features into a multi-head attention mechanism in the entity recognition extraction model to be trained to obtain initial structural features;

[0021] Inputting the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix into the multi-head attention mechanism to obtain an updated co-occurrence matrix, an updated syntactic dependency matrix and an updated relative distance matrix respectively;

[0022] The initial structural features, the updated co-occurrence matrix, the updated syntactic dependency matrix and the updated relative distance matrix are sequentially concatenated and normalized to obtain text structural features.

[0023] Optionally, the preset final loss function is a loss function obtained by weighted summing a preset contrast loss function and a cross entropy loss function.

[0024] Optionally, determining target text features based on the text semantic features and the text structural features, training the to-be-trained entity recognition extraction model based on the target text features, and performing convergence training on the to-be-trained entity recognition extraction model based on a preset final loss function to obtain a target entity recognition extraction model, includes:

[0025] splicing the text semantic features and the text structural features to obtain target text features;

[0026] The target text features are applied to the dual affine classifier of the entity recognition extraction model to be trained to obtain a prediction result, and based on the prediction result and a preset final loss function, the entity recognition extraction model to be trained is converged to obtain a target entity recognition extraction model.

[0027] In a second aspect, the present application discloses an entity recognition extraction device, comprising:

[0028] A text segmentation module is used to perform data tagging on the target domain text to obtain the target text corresponding to the target domain text, and encode the target domain text using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding segmentation result; wherein the target text is a single word in the target domain text;

[0029] A semantic feature determination module, used to determine the ordered sequence of words corresponding to the target text based on the word segmentation result, and train the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the semantic features of the text;

[0030] A structural feature determination module, used for determining a co-occurrence matrix, a syntactic dependency matrix and a relative distance matrix between the target characters based on the target domain text, and performing feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structural features;

[0031] A model training module is used to determine the target text features based on the text semantic features and the text structural features, and to train the entity recognition extraction model to be trained based on the target text features, and at the same time to perform convergence training on the entity recognition extraction model to be trained based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

[0032] In a third aspect, the present application discloses an electronic device, comprising:

[0033] Memory, used to store computer programs;

[0034] A processor is used to execute the computer program to implement the aforementioned entity recognition and extraction method.

[0035] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program, which implements the aforementioned entity recognition and extraction method when executed by a processor.

[0036] It can be seen that the target domain text is data-tagged to obtain the target text corresponding to the target domain text, and the target domain text is encoded using the preset pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding word segmentation result; wherein the target text is a single word in the target domain text; based on the word segmentation result, the ordered sequence of words corresponding to the target text is determined, and the preset neural network model in the entity recognition extraction model to be trained is used to train the ordered sequence of words to determine the text semantic features; based on the target domain text, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix between the target texts are determined, and the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are feature fused to obtain text structural features; based on the text semantic features and the text structural features, the target text features are determined, and the entity recognition extraction model to be trained is trained based on the target text features, and at the same time, the entity recognition extraction model to be trained is converged based on the preset final loss function to obtain the target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model. In this way, by learning and identifying the structural features and text features of the target text, a target entity recognition and extraction model is generated. The target entity recognition and extraction model produced by this solution can extract important entities from unstructured data, improve the accuracy of entity extraction, and facilitate subsequent data analysis and decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 A flow chart of an entity recognition extraction method disclosed in this application;

[0039] Figure 2 A flow chart of a specific entity recognition extraction method disclosed in this application;

[0040] Figure 3 This is a schematic diagram of the structure of an entity recognition and extraction device disclosed in this application;

[0041] Figure 4 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0043] The current named entity recognition technology has problems such as unclear entity boundaries, rigid integration of semantic prior knowledge, and inability to obtain advanced token-level expressions. Therefore, this application will specifically introduce an entity recognition extraction method that can solve the above problems.

[0044] See also Figure 1 As shown, the embodiment of the present application discloses an entity recognition extraction method, including:

[0045] Step S11: data tagging is performed on the target domain text to obtain target characters corresponding to the target domain text, and the target domain text is encoded using a preset pre-trained model in the entity recognition extraction model to be trained to obtain corresponding word segmentation results; wherein the target characters are single characters in the target domain text.

[0046] In this embodiment, a text dataset in the financial field is obtained, and a pre-trained model is used to segment and encode the text to obtain the segmentation result and text vector of the text, wherein the target field text is data-tagged to obtain the target text corresponding to the target field text, and the target field text is encoded using the preset pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding segmentation result, including: obtaining the target field text through a preset crawler technology, and data-tagging the target field text to obtain the target text corresponding to the target field text; using the NEZHA pre-trained model in the entity recognition extraction model to be trained to encode the target field text to obtain the corresponding segmentation result. This application takes the financial field as an example, firstly obtains a text dataset in the financial field through a preset crawler technology, and then during the operation, the text dataset in the financial field can be divided into a training set and a test set according to a ratio of 9:1, and then the dataset is labeled to obtain the target text corresponding to the target field text, which is then referred to as a token. Then, the label of each document is modeled as a word-word relationship matrix, and the size of the matrix is ​​n×n, where n represents the length of the current document. Specifically, the element a in the matrix i，j As shown below:

[0047]

[0048] Specifically, the pre-trained model NEZHA (a neural contextual representation model designed specifically for Chinese language understanding) can be used to segment and encode the text to obtain the text segmentation results.

[0049] Step S12: determining an ordered sequence of words corresponding to the target text based on the word segmentation result, and training the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine text semantic features.

[0050] In this embodiment, the word ordered sequence corresponding to the target text is determined based on the word segmentation result, and the word ordered sequence is trained using the preset neural network model in the entity recognition extraction model to be trained to determine the text semantic features, including: determining all the words containing the target text from the preset vocabulary dictionary based on the word segmentation result, and sorting all the words based on the preset sequence composition rules to obtain the corresponding word ordered sequence; training the word ordered sequence based on the bidirectional semantic gated recurrent unit in the entity recognition extraction model to be trained to determine the aggregated features; splicing the aggregated features and the original features of the target text to obtain the text semantic features. That is, for each token, an ordered sequence of words containing the token is obtained, the sequence is sent to the bidirectional semantic gated recurrent unit for aggregation to obtain the token semantic features, the token vector and the token semantic features are spliced ​​to obtain the token final semantic features, and the token final semantic features are sent to the transformer for training to obtain the text semantic features. Specifically, find all words containing the token in the preset dictionary, form an ordered sequence according to the relative position of the words in the dictionary, and put the token into the ordered sequence as the last element of the ordered sequence. In actual operation, the NEZHA word segmentation result and the open source Chinese word library THUOCL can be combined into a vocabulary dictionary. For each token, find all words containing the token in the dictionary; then, send the ordered sequence to the bidirectional semantic gated recurrent unit to obtain forward and reverse aggregate features. The calculation formula of the bidirectional semantic gated recurrent unit is:

[0051] r t =sigmoid(W r ·[h t-1 , x t ]+b r );

[0052] z t =sigmoid(W z ·[h t-1 , x t ]+b z );

[0053]

[0054] Among them, r t represents the reset gate, z t represents the update gate, represents the candidate hidden state, g t represents the semantic gate, h t Indicates the output at the current moment, h (t-1) represents the hidden layer output of the bidirectional semantic gated recurrent unit at the previous moment, x t represents the input at the current moment, sigmoid and tanh represent activation functions, and W r , W z , W g represents the weight parameter, b r , b z , b g represents the bias parameter, Indicates the word frequency-inverse document frequency value of the current input in the document. Among them, the word frequency-inverse document frequency value The calculation formula is:

[0055]

[0056] Among them, num i Indicates the number of times word i appears in the current document, num all-words Indicates the total number of words in the current document, sum all-docs Indicates the total number of documents in the entire corpus, sum docshavingi It represents the total number of documents in the corpus that contain word i. Finally, the forward and reverse aggregate features are concatenated with the original features of the token, and the final feature matrix of the text is sent to the tansformer for training to obtain the text semantic features.

[0057] Step S13: determining the co-occurrence matrix, syntactic dependency matrix and relative distance matrix between the target characters based on the target domain text, and performing feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structure features.

[0058] In this embodiment, the co-occurrence matrix, syntactic dependency matrix and relative distance matrix between the target characters are determined based on the target domain text, and the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are subjected to feature fusion to obtain text structural features, including: determining the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix between the target characters based on the co-occurrence relationship, the syntactic dependency relationship and the relative position relationship between each character in the target domain text; and performing structural feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix based on the structural feature fusion attention mechanism in the entity recognition extraction model to be trained to obtain text structural features. That is, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix of the words of the text are obtained according to the input text; then, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix of the words are sent to the structural feature fusion attention mechanism module for feature fusion to obtain text structural features.

[0059] Step S14: Determine the target text features based on the text semantic features and the text structural features, and train the entity recognition extraction model to be trained based on the target text features. At the same time, perform convergence training on the entity recognition extraction model to be trained based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

[0060] In this embodiment, the text semantic features and text structural features are spliced ​​as the final features of the text. Wherein, the target text features are determined based on the text semantic features and the text structural features, and the entity recognition extraction model to be trained is trained based on the target text features, and the entity recognition extraction model to be trained is converged based on the preset final loss function to obtain the target entity recognition extraction model, including: splicing the text semantic features and the text structural features to obtain the target text features; applying the target text features to the dual affine classifier of the entity recognition extraction model to be trained to obtain the prediction results, and based on the prediction results and the preset final loss function, the entity recognition extraction model to be trained is converged to obtain the target entity recognition extraction model. The preset final loss function is a loss function obtained by weighted summing of a preset contrast loss function and a cross entropy loss function. Specifically, for a certain entity segment θ starting with i and ending with j, the calculation process of the dual affine classifier is:

[0061] h start (θ) = FFNN start (x i );

[0062] h end (θ) = FFNN end (xj );

[0063]

[0064] Among them, x i Represents a token i The vector representation of x j represents the vector representation of token j, h start (θ) represents the vector x i Feed into FFNN start Hidden layer results after training, h end (θ) represents the vector x j Feed into FFNN end The hidden layer results after training, score(θ) represents the score of entity fragments starting with i and ending with j, T represents the transpose of the matrix, U m , W m represents the trainable parameters, b m Represents the bias parameter. In actual operation, the word-word relationship matrix of the predicted text can be obtained based on the biaffine result score. Then, the token-level contrast loss function, the text-level contrast loss function, and the cross-entropy loss function are weighted and summed to obtain the final loss function; the model is trained and converged according to the final loss function, and entity prediction is performed on the financial text test set, and the boundaries and categories of entities in the text are obtained by decoding. Among them, the calculation formula of the final loss function is:

[0065]

[0066] Among them, Loss represents the loss function of the current sample, LoSS cross-entropy Represents the cross entropy loss function, Loss token-Contrastive Represents the token-level contrastive learning loss function, Loss text-Contrastive represents the document-level contrastive learning loss function, i represents the i-th token in the current sample, i represents the j-th token in the current sample, T represents the length of the current sample, x represents the token category x, C represents the total number of entity categories, and Y i，j，x Indicates whether the segment starting from i and ending at j is a label of entity type x, P i，j，x represents the probability of whether the segment starting from i and ending at j is an entity of type x, γ and δ represent weighting coefficients, and h i The vector representation of i, h i + Indicates h i The positive samples are specifically the vector representations of the tokens of the same entity as i, M represents the number of negative samples for contrastive learning at the token level, and h m- Indicates h i The negative samples are specifically vector representations of tokens of entities of different classes from i, h text Indicates the feature h of the current input text text + Indicates h text The positive sample is the feature of the current text after the dropout layer, h text - Indicates h text The negative samples are specifically the features of other texts in the current batch. represents the norm of the vector, d() represents the calculation of the Euclidean distance, N represents the number of negative samples for contrastive learning at the document level, and τ and τ' represent temperature coefficients. Finally, the entity is decoded according to the word-word relationship matrix to obtain the entities and entity categories in the document.

[0067] It can be seen that the target domain text is data-tagged to obtain the target text corresponding to the target domain text, and the target domain text is encoded using the preset pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding word segmentation result; wherein the target text is a single word in the target domain text; based on the word segmentation result, the ordered sequence of words corresponding to the target text is determined, and the preset neural network model in the entity recognition extraction model to be trained is used to train the ordered sequence of words to determine the text semantic features; based on the target domain text, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix between the target texts are determined, and the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are feature fused to obtain text structural features; based on the text semantic features and the text structural features, the target text features are determined, and the entity recognition extraction model to be trained is trained based on the target text features, and at the same time, the entity recognition extraction model to be trained is converged based on the preset final loss function to obtain the target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model. That is, when training the target domain data, it is divided into the semantic side and the structural side. The semantic side can pay more attention to the category of the entity, and the structural side can pay more attention to the boundary of the entity. Specifically, on the semantic side, a bidirectional semantic gated recurrent unit is designed to scientifically integrate vocabulary information into each token, so that the model can learn the boundary of the vocabulary and the semantic knowledge carried by the vocabulary; on the structural side, a relative distance calculation score is designed to reasonably calculate the relative distance between two tokens, and a structural feature fusion attention mechanism is designed to fuse three different structural features, so that the model can pay attention to different structural features from multiple angles, which is conducive to identifying entity boundaries; during training, multi-level comparative learning at the token level and text level is added to the basic cross entropy loss function, which can better obtain the vector representation of each token without breaking the constraint of the document vector, so as to obtain better recognition results. In this way, through the entity extraction of financial documents, better entity extraction results are achieved, which can effectively improve the efficiency of information processing in the financial industry and convert unstructured data into focused structured data.

[0068] The above embodiment introduces the entire entity extraction training process, and this application will specifically introduce the method of obtaining structural features.

[0069] See also Figure 2 As shown, the embodiment of the present application discloses a specific entity recognition extraction method, including:

[0070] Step S21: Determine the co-occurrence matrix, syntactic dependency matrix and relative distance matrix between target characters based on the co-occurrence relationship, syntactic dependency relationship and relative position relationship between each character in the target domain text.

[0071] In this embodiment, the co-occurrence matrix, syntactic dependency matrix, and relative distance matrix of the words in the text are obtained according to the input text. The co-occurrence matrix represents the co-occurrence relationship between tokens, and the size of the matrix is ​​n×n, where n represents the length of the current document, and the element b in the matrix is i,j Represents a token i and token j The number of co-occurrences in the entire prediction library when the sliding window is 20. The syntactic dependency matrix represents the syntactic dependency relationship between tokens. The syntactic dependency relationship in the current text is extracted using the open source tool Hanlp. The size of the matrix is ​​n×n, where n represents the length of the current document and the element c in the matrix is i,j Represents a token i and token j The relative distance matrix represents the relative position relationship between tokens. The matrix size is n×n, where n represents the length of the current document. The elements in the matrix are i,j Represents a token i and token j The relative distance score is calculated as follows:

[0072]

[0073] Among them, dis i,j Represents a token i and token j The relative distance between them, i and j represent the position of the token, h x represents the vectorized expression of x, α is a hyperparameter for adjusting the decay rate, which can adjust the role of relative position offset in the distance metric.

[0074] Step S22: concatenate the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain structural concatenation features, and input the structural concatenation features into the multi-head attention mechanism in the entity recognition extraction model to be trained to obtain initial structural features.

[0075] In this embodiment, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are concatenated to obtain a structural concatenation feature, and the structural concatenation feature is input into the multi-head attention mechanism in the entity recognition extraction model to be trained to obtain an initial structural feature. That is, the co-occurrence matrix Co, the syntactic dependency matrix Sd and the relative distance matrix Rd are concatenated to obtain a structural concatenation feature R, and the structural concatenation feature is input into the multi-head attention mechanism to calculate the first structural feature R'.

[0076] Step S23: input the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix into the multi-head attention mechanism respectively to obtain an updated co-occurrence matrix, an updated syntactic dependency matrix and an updated relative distance matrix.

[0077] In this embodiment, the co-occurrence matrix Co, the syntactic dependency matrix Sd, and the relative distance matrix Rd are sent to the self-attention mechanism module to calculate the self-attention mechanism results of each feature matrix; the respective self-attention mechanism results are sent to different feedforward neural networks to obtain updated features Co', Sd', Rd'.

[0078] Step S24: sequentially concatenate and normalize the initial structural features, the updated co-occurrence matrix, the updated syntactic dependency matrix, and the updated relative distance matrix to obtain text structural features.

[0079] In this embodiment, after obtaining the updated co-occurrence matrix, the updated syntactic dependency matrix and the updated relative distance matrix, R', Co', Sd', Rd' are residually connected in sequence and normalized by LayerNorm (LayerNormalization, a normalization technology) to obtain text structure features.

[0080] It can be seen that in this embodiment, a structural feature fusion attention mechanism is designed to fuse three different structural features, so that the model can pay attention to different structural features from multiple angles, which is conducive to identifying entity boundaries. By extracting entities from financial documents, better entity extraction results are achieved, which can effectively improve the efficiency of information processing in the financial industry, convert unstructured data into focused structured data, and make subsequent data analysis and retrieval more convenient, which is beneficial to many application scenarios such as risk management, customer service, and market monitoring in the financial industry.

[0081] refer to Figure 3 The embodiment of the present application also discloses an entity recognition extraction device, including:

[0082] The text segmentation module 11 is used to perform data tagging on the target domain text to obtain the target text corresponding to the target domain text, and encode the target domain text using the preset pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding segmentation result; wherein the target text is a single word in the target domain text;

[0083] A semantic feature determination module 12 is used to determine the ordered sequence of words corresponding to the target text based on the word segmentation result, and train the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the semantic features of the text;

[0084] A structural feature determination module 13 is used to determine the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix between the target characters based on the target domain text, and perform feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structural features;

[0085] The model training module 14 is used to determine the target text features based on the text semantic features and the text structural features, and to train the entity recognition extraction model to be trained based on the target text features, and at the same time to perform convergence training on the entity recognition extraction model to be trained based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

[0086] It can be seen that in this embodiment, the target entity recognition and extraction model is generated by learning and identifying the structural features and text features of the target text. The target entity recognition and extraction model produced by this scheme can be used to extract important entities from unstructured data, thereby improving the accuracy of entity extraction and facilitating subsequent data analysis and decision-making.

[0087] In some specific embodiments, the text segmentation module 11 may specifically include:

[0088] A data marking unit, used to obtain target domain text by using a preset crawler technology, and to perform data marking on the target domain text to obtain a target text corresponding to the target domain text;

[0089] The text encoding unit is used to encode the target domain text using the NEZHA pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding word segmentation result.

[0090] In some specific embodiments, the semantic feature determination module 12 may specifically include:

[0091] A word sequence generating unit, configured to determine all words containing the target text from a preset vocabulary dictionary based on the word segmentation result, and to sort all the words based on a preset sequence formation rule to obtain a corresponding ordered sequence of words;

[0092] A sequence training unit, used for training the ordered sequence of words based on a bidirectional semantic gated recurrent unit in the entity recognition extraction model to be trained to determine an aggregate feature;

[0093] The feature concatenation unit is used to concatenate the aggregated features with the original features of the target text to obtain text semantic features.

[0094] In some specific embodiments, the structural feature determination module 13 may specifically include:

[0095] A relationship matrix determination unit, used to determine the co-occurrence matrix, syntactic dependency matrix and relative distance matrix between the target characters based on the co-occurrence relationship, syntactic dependency relationship and relative position relationship between each character in the target domain text;

[0096] The structural feature determination submodule is used to perform structural feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix based on the structural feature fusion attention mechanism in the entity recognition extraction model to be trained to obtain text structural features.

[0097] In some specific embodiments, the structural feature determination submodule may specifically include:

[0098] A matrix concatenation unit, used for concatenating the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain structural concatenation features, and inputting the structural concatenation features into a multi-head attention mechanism in the entity recognition extraction model to be trained to obtain initial structural features;

[0099] A matrix updating unit, used for inputting the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix into the multi-head attention mechanism to obtain an updated co-occurrence matrix, an updated syntactic dependency matrix and an updated relative distance matrix;

[0100] The matrix normalization unit is used to sequentially concatenate and normalize the initial structural features, the updated co-occurrence matrix, the updated syntactic dependency matrix and the updated relative distance matrix to obtain text structural features.

[0101] In some specific embodiments, the model training module 14 may specifically include:

[0102] A target text feature acquisition unit, used for combining the text semantic feature and the text structural feature to obtain a target text feature;

[0103] The entity extraction unit is used to apply the target text features to the dual affine classifier of the entity recognition extraction model to be trained to obtain a prediction result, and to perform convergence training on the entity recognition extraction model to be trained based on the prediction result and a preset final loss function to obtain a target entity recognition extraction model.

[0104] Furthermore, the present application also discloses an electronic device. Figure 4 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be regarded as any limitation on the scope of use of the present application.

[0105] Figure 4 A schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the entity recognition extraction method disclosed in any of the aforementioned embodiments. In addition, the electronic device 20 in this embodiment may specifically be an electronic computer.

[0106] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0107] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0108] The operating system 221 is used to manage and control the hardware devices and computer program 222 on the electronic device 20, which can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the entity recognition extraction method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0109] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the entity recognition extraction method disclosed above. The specific steps of the method can refer to the corresponding contents disclosed in the above embodiments, and will not be repeated here.

[0110] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0111] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0112] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0113] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0114] The technical solution provided by the present application is introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technicians in this field, according to the idea of ​​the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for entity recognition extraction, characterized in that: include: Data tagging is performed on the target domain text to obtain a target text corresponding to the target domain text, and the target domain text is encoded using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding word segmentation result; wherein the target text is a single character in the target domain text; Determine the ordered sequence of words corresponding to the target text based on the word segmentation result, and train the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the semantic features of the text; Determine a co-occurrence matrix, a syntactic dependency matrix, and a relative distance matrix between the target characters based on the target domain text, and perform feature fusion on the co-occurrence matrix, the syntactic dependency matrix, and the relative distance matrix to obtain text structure features; The target text features are determined based on the text semantic features and the text structural features, and the entity recognition extraction model to be trained is trained based on the target text features. At the same time, the entity recognition extraction model to be trained is converged based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

2. The entity recognition extraction method according to claim 1, characterized in that: The target domain text is data-labeled to obtain the target text corresponding to the target domain text, and the target domain text is encoded using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding word segmentation result, including: Acquire target domain text by using a preset crawler technology, and perform data tagging on the target domain text to obtain target text corresponding to the target domain text; The target domain text is encoded using the NEZHA pre-trained model in the entity recognition extraction model to be trained to obtain the corresponding word segmentation results.

3. The entity recognition extraction method according to claim 1, characterized in that: The step of determining the ordered sequence of words corresponding to the target text based on the word segmentation result, and training the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the text semantic features, includes: Determining all words containing the target text from a preset vocabulary dictionary based on the word segmentation result, and sorting all the words based on a preset sequence formation rule to obtain a corresponding ordered sequence of words; Training the ordered sequence of words based on a bidirectional semantic gated recurrent unit in the entity recognition extraction model to be trained to determine an aggregate feature; The aggregated features and the original features of the target text are concatenated to obtain text semantic features.

4. The entity recognition extraction method according to claim 1, characterized in that: The step of determining the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix between the target characters based on the target domain text, and performing feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structure features includes: Determine a co-occurrence matrix, a syntactic dependency matrix and a relative distance matrix between the target characters based on the co-occurrence relationship, the syntactic dependency relationship and the relative position relationship between each character in the target domain text; Based on the structural feature fusion attention mechanism in the entity recognition extraction model to be trained, the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix are fused with structural features to obtain text structural features.

5. The entity recognition extraction method according to claim 4 is characterized in that: The structural feature fusion attention mechanism based on the entity recognition extraction model to be trained performs structural feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structural features, including: Concatenating the co-occurrence matrix, the syntactic dependency matrix, and the relative distance matrix to obtain structural concatenation features, and inputting the structural concatenation features into a multi-head attention mechanism in the entity recognition extraction model to be trained to obtain initial structural features; Inputting the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix into the multi-head attention mechanism to obtain an updated co-occurrence matrix, an updated syntactic dependency matrix and an updated relative distance matrix respectively; The initial structural features, the updated co-occurrence matrix, the updated syntactic dependency matrix and the updated relative distance matrix are sequentially concatenated and normalized to obtain text structural features.

6. The entity recognition extraction method according to claim 1, characterized in that: The preset final loss function is a loss function obtained by weighted summing of a preset contrast loss function and a cross entropy loss function.

7. The entity recognition extraction method according to any one of claims 1 to 6, characterized in that: The step of determining target text features based on the text semantic features and the text structural features, training the entity recognition extraction model to be trained based on the target text features, and performing convergence training on the entity recognition extraction model to be trained based on a preset final loss function to obtain a target entity recognition extraction model includes: splicing the text semantic features and the text structural features to obtain target text features; The target text features are applied to the dual affine classifier of the entity recognition extraction model to be trained to obtain a prediction result, and based on the prediction result and a preset final loss function, the entity recognition extraction model to be trained is converged to obtain a target entity recognition extraction model.

8. An entity recognition extraction device, characterized in that: include: A text segmentation module is used to perform data tagging on the target domain text to obtain the target text corresponding to the target domain text, and encode the target domain text using a preset pre-trained model in the entity recognition extraction model to be trained to obtain a corresponding segmentation result; wherein the target text is a single word in the target domain text; A semantic feature determination module, used to determine the ordered sequence of words corresponding to the target text based on the word segmentation result, and train the ordered sequence of words using a preset neural network model in the entity recognition extraction model to be trained to determine the semantic features of the text; A structural feature determination module, used for determining a co-occurrence matrix, a syntactic dependency matrix and a relative distance matrix between the target characters based on the target domain text, and performing feature fusion on the co-occurrence matrix, the syntactic dependency matrix and the relative distance matrix to obtain text structural features; A model training module is used to determine the target text features based on the text semantic features and the text structural features, and to train the entity recognition extraction model to be trained based on the target text features, and at the same time to perform convergence training on the entity recognition extraction model to be trained based on a preset final loss function to obtain a target entity recognition extraction model, so as to perform entity recognition extraction operations based on the target entity recognition extraction model.

9. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, configured to execute the computer program to implement the entity recognition extraction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, which, when executed by a processor, implements the entity recognition extraction method as described in any one of claims 1 to 7.

Images