Social media text named entity recognition method and system fusing multi-scale features and syntactic information

Abstract

The invention relates to the technical field of natural language processing, in particular to a social media text named entity recognition method and system fusing multi-scale features and syntactic information.The method comprises the steps that firstly, global features are extracted through a sequence encoder XLNET based on Transformer, a dependency syntactic relation is embedded into the global features, and semantic information of an entity is enhanced; then focusing on different local ranges of the text through sliding windows with different lengths, and inputting into BiLSTM to capture multi-level local features; and finally, predicting an optimal entity label by effectively combining global context information and a plurality of local features by utilizing an attention fusion mechanism. According to the method, the named entity recognition task of the social media text is performed by combining the dependency syntactic relationship and fusing the global features extracted by the sequence encoder XLNET based on Transformer and the local features captured by focusing on the texts with different lengths, the multi-level features rich in the text are extracted, and the entity recognition accuracy of the social media text is effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing (NLP), and in particular to a method and system for identifying named entities in social media texts by integrating multi-scale features and syntactic information. Background Art

[0002] With the development of online social platforms, massive amounts of social media data have been generated. How to effectively extract valuable information from it has always been a hot topic and difficulty in the field of natural language processing. Named Entity Recognition (NER) is one of the most important and basic tasks in knowledge extraction. It recognizes and classifies entities with specific meanings in text, such as names of people, places, and organizations, and is widely used in downstream tasks such as information retrieval, knowledge graphs, question-answering systems, and public opinion analysis.

[0003] With the rise of deep learning, neural network-based NER models have shown significant advantages, including convolutional neural networks (CNN), recurrent neural networks (RNN) and pre-trained language models, which can automatically learn implicit features from data and effectively capture the contextual information of entities, thereby significantly improving the accuracy and efficiency of entity recognition.

[0004] Collobert et al. used sentences as input for predicting words, added the relative position features of words in sentences, used CNN to obtain context, and used the extracted local feature vectors to construct global feature vectors. Although this method can capture local features from a large amount of unlabeled data, it cannot solve the problem of long-range dependency. RNN is a type of recurrent neural network that takes sequence data as input. All nodes (recurrent units) are connected in a chain, allowing information to circulate within the network. Its variants LSTM and GRU are widely used in NER tasks. Huang et al. first applied the BiLSTM-CRF architecture to sequence labeling tasks, effectively utilizing the feature information of the context. Yang et al. proposed a deep recurrent neural network for sequence labeling, using GRU to encode morphological and contextual information at the character and word levels, and applying CRF field layers to predict labels. However, RNN models are prone to gradient vanishing or exploding problems when processing long sequences, and their computational efficiency is limited by the sequence length. Therefore, pre-trained language models based on the Transformer architecture (BERT, XLNET, GPT, etc.) came into being. They broke the sequence dependency through the self-attention mechanism, solved the long-range dependency problem, and supported parallel computing, which greatly improved the model's training speed and ability to process long texts. Xu et al. used BERT as an encoder and integrated the entire future and past sentence representations into the sentence representation of each cell under the BiLSTM framework through the global context mechanism to improve entity recognition performance.

[0005] However, when existing methods perform named entity recognition on social media texts, it is difficult for the NER model to achieve a high accuracy rate due to data sparsity and non-standardization. Liu et al. combined KNN with CRF models in a semi-supervised learning framework to solve the problem of insufficient information in tweets and unavailability of training data, and added a place name dictionary to improve the accuracy of NER recognition; Peng et al. proposed a named entity recognition model with joint training and learning representation for Chinese social media Sina Weibo text; Aguilar et al. proposed a multi-task network that uses fine-grained named entity classification as the main task and entity segmentation as the secondary task, using characters, words and place name dictionaries as embedded representations to capture features at different levels. Many researchers have also improved the performance of social media text NER by semantic enhancement and adding external resources. Xu et al. proposed a model for entity recognition of Chinese social media text that uses a joint learning model with the help of annotated datasets from external domains and unlabeled datasets from within the domain. The joint model uses cross-domain learning based on domain similarity and self-training semi-supervised learning, which can effectively alleviate the problem of limited model annotated datasets. Nie et al. extracted the most similarly labeled words based on the pre-trained embedding of the labels, used the attention mechanism to weight the semantic information carried by the extracted words, and applied a gate module to aggregate this information into the labeling process, thereby enhancing social media NER. Summary of the invention

[0006] The present invention aims to solve the specific technical problems faced by existing named entity recognition technologies when processing social media texts. Social media data usually contains a large amount of noise and non-standardized texts, such as slang, abbreviations and spelling errors. These characteristics make named entity recognition more difficult. When processing such data, existing named entity recognition methods often have difficulty accurately identifying and classifying entities due to sparse entities and insufficient semantic information. Therefore, the present invention proposes a social media text named entity recognition method and system that integrates multi-scale features and syntactic information. By combining dependency syntactic relationships, integrating global features extracted by Transformer-based sequence encoder XLNET and local features captured by focusing on texts of different lengths, the named entity recognition task of social media texts is performed, and rich multi-level features of the text are extracted to solve the problem of inaccurate entity recognition caused by sparse data and insufficient semantics.

[0007] To achieve the above purpose, the technical solution adopted is:

[0008] A method for named entity recognition in social media texts that integrates multi-scale features and syntactic information, comprising:

[0009] First, the global features are extracted through the Transformer-based sequence encoder XLNET, and the dependency syntactic relations are embedded into it to enhance the semantic information of the entity;

[0010] Then, sliding windows of different lengths are used to focus on different local ranges of the text and input into BiLSTM to capture multi-level local features;

[0011] Finally, the attention fusion mechanism is used to effectively combine global context information and multiple local features to predict the optimal entity label.

[0012] According to the social media text named entity recognition method integrating multi-scale features and syntactic information of the present invention, further, the global features extracted by the Transformer-based sequence encoder XLNET include:

[0013] For a given input sequence X = [x1, x2, ..., x n ], and use XLNET to embed the corresponding vector E = [e1, e2, ..., e n ], the expression is:

[0014] E = XLNet(x) = [e1,e2,...,e n ]

[0015] Among them, e i ∈R u , that is, E∈R n×u .

[0016] According to the social media text named entity recognition method integrating multi-scale features and syntactic information of the present invention, further, the dependency syntactic relationship is embedded in the global feature to include:

[0017] By collecting all the dependencies from the dependency syntactic analysis of the input sequence X, for each x i Embed the corresponding syntactic feature d i , composed of D = [d1, d2, ..., d n ]; Finally, the global feature vector E is integrated with the syntactic feature vector D to obtain the final feature representation Z = [z1, z2, ..., z n ]; the expression is as follows:

[0018] D=[d1,d2,...,d n ]

[0019] Z=σ([(E,D)]W+b)

[0020] Among them, d i ∈R v , that is, D∈R n×v , Z∈R n×t , σ represents the sigmoid function, W and b are learnable parameters.

[0021] According to the method for named entity recognition of social media texts that integrates multi-scale features and syntactic information of the present invention, further focusing on different local ranges of the text through sliding windows of different lengths includes:

[0022] Use E = [e1,e2,...,e n ] as input, through the window sliding mechanism, for each position i in the sequence, a window is constructed including position i and its left and right The local context feature subsequence of the position Here w is the window size; multiple windows are used to extract multi-level local features [S1, S2, ..., S K ], K is the number of windows.

[0023] According to the method for named entity recognition of social media texts integrating multi-scale features and syntactic information of the present invention, the multi-level local features captured by input into BiLSTM further include:

[0024] Feed S into the BiLSTM layer. The BiLSTM layer transforms the local feature sequence into a hidden state sequence through the characteristics of the recurrent neural network. The expression is as follows:

[0025] g t =σ(W g S t +U g h t-1 +b g )

[0026] f t =σ(W f S t +U f h t-1 +b f )

[0027]

[0028] o t =σ(W o S t +U o h t-1 +b o )

[0029]

[0030] Among them, g t represents the input gate, f t represents the forget gate, o t represents the output gate, c t represents the state of the memory cell at the current time step, Indicates the candidate memory cell state; St represents the input feature representation, and They represent the forward and reverse LSTM implicit feature representations at time t, respectively, and f i represents the implicit feature representation of BiLSTM; W, b and U represent learnable parameters, σ represents the sigmoid activation function, represents the Hadamard product;

[0031] By repeating the above process until all positions in the sequence are processed, a local feature tensor F = [f1, f2, ..., f n ].

[0032] According to the method for named entity recognition of social media texts that integrates multi-scale features and syntactic information of the present invention, further, the attention fusion mechanism is used to effectively combine global context information and multiple local features to predict the optimal entity label, including:

[0033] First, we obtain the global feature representation Z = [z1, z2, ..., z n ] and multi-level local feature representation F = [f1,f2,...,f n ], then calculate the attention score matrix S, perform weighted fusion on F, obtain the weighted local features, and finally fuse them with the global features to obtain the final vector A, which is expressed as follows:

[0034] A=Z+S·F

[0035] After obtaining the final embedding, the softmax function is used to decode the classification prediction sequence label and output y = [y1, y2, ..., y n ].

[0036] According to the social media text named entity recognition method integrating multi-scale features and syntactic information of the present invention, further, the calculation process of the attention score matrix S is as follows:

[0037] First, the global features and multi-level local feature vectors are concatenated through a fully connected layer as the query vector Q, Z as the value matrix, and F as the key matrix; then the dot product between the query vector and the local feature matrix is ​​calculated to obtain the attention score matrix S, which is expressed as follows:

[0038] Q=W q [Z; F]

[0039]

[0040] Among them, W qis the weight matrix in the fully connected layer, d is the dimension of the key matrix, K is the key matrix, and V is the value matrix.

[0041] Furthermore, the present invention also provides a social media text named entity recognition system integrating multi-scale features and syntactic information, which is used to implement the social media text named entity recognition method integrating multi-scale features and syntactic information as described above, and the system comprises:

[0042] The global semantic encoding module is used to extract global features through the Transformer-based sequence encoder XLNET and embed dependency syntactic relations into it to enhance the semantic information of entities;

[0043] The multi-level local feature extraction module is used to focus on different local ranges of the text through sliding windows of different lengths and input them into BiLSTM to capture multi-level local features;

[0044] The attention fusion mechanism module is used to use the attention fusion mechanism to effectively combine global context information and multiple local features to predict the optimal entity label.

[0045] The beneficial effects achieved by adopting the above technical solution are:

[0046] 1. The present invention uses the Transformer-based sequence encoder XLNET to extract rich global context information and embeds dependency syntactic relations into XLNET. Dependency syntactic analysis provides clues about the relationships between words, which helps to understand the entity context, enhances the model proposed by the method's ability to understand the deep semantics of sentences and entity boundaries, and improves recognition accuracy.

[0047] 2. The present invention uses a multi-scale local feature extraction (Multi-scale Local Feature Extractor) module to focus on different local ranges of the text through sliding windows of different lengths, effectively extract multi-level contextual local features, and obtain multi-level local contextual features.

[0048] 3. The present invention proposes a Fusion-Attention Mechanism module that dynamically balances and fuses the global features extracted by XLNET and the multi-level local features captured by Multi-scale Local Feature Extractor through a sophisticated attention allocation strategy, thereby improving the accuracy of the model in identifying entities.

[0049] 4. The experimental results compared with other methods on three real datasets show that the proposed named entity recognition method has good performance, and ablation experiments are carried out to verify the contribution of each module to the overall model, which helps the effectiveness and robustness of named entity recognition.

[0050] The above method extracts and fuses multi-level features of the text, effectively alleviating the problem of sparse entities and insufficient semantics faced by existing methods when identifying named entities in noisy and non-normalized social media data. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings of the embodiments of the present invention, wherein the drawings are only used to illustrate some embodiments of the present invention, but not to limit all embodiments of the present invention thereto.

[0052] Figure 1 is a flowchart of a method for named entity recognition in social media text that integrates multi-scale features and syntactic information according to an embodiment of the present invention;

[0053] Figure 2 It is an example diagram of case analysis of an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings of specific embodiments of the present invention to clearly and completely describe the exemplary scheme of the embodiment of the present invention. Unless otherwise defined, the technical terms or scientific terms used in the present invention should be the common meanings understood by people with ordinary skills in the field.

[0055] Social media data is noisy and non-standardized. Existing methods face the challenge of sparse entities and insufficient semantics when identifying named entities. In order to solve the above problems, this embodiment discloses a social media text named entity recognition method SEMFF-NER that integrates multi-scale features and syntactic information. Figure 1 As shown, the following steps are included:

[0056] Step S101: extract global features through the Transformer-based sequence encoder XLNET, and embed dependency syntactic relations into it to enhance the semantic information of the entity.

[0057] High-quality text representation is the key to achieving good performance in many NLP tasks. Therefore, for a given input sequence X = [x1, x2, ..., x n ], where x i Represents a word in the text; first use XLNET to embed and get the corresponding vector E = [e1, e2, ..., e n ], the expression is:

[0058] E = XLNet(x) = [e1,e2,...,e n ]

[0059] Among them, e i ∈R u , that is, E∈R n×u XLNET is an autoregressive language model based on Transformer-XL. It also introduces the bidirectional context in the autoencoder language model, so that when processing sequences, it can not only consider the context of the current word, but also effectively capture the global context dependencies, which can provide richer and more effective global context information.

[0060] Then, by collecting all the dependencies from the dependency syntactic analysis of the input sequence X, for each x i Embed the corresponding syntactic feature d i , composed of D = [d1, d2, ..., d n ], capturing the deep and rich semantic features in social media texts, and finally integrating the above two vectors to obtain the final feature representation Z = [z1, z2, ..., z n ], the expression is as follows:

[0061] D=[d1,d2,...,d n ]

[0062] Z=σ([(E,D)]W+b)

[0063] Among them, d i ∈R v , that is, D∈R n×v , Z∈R n×t , σ represents the sigmoid function, W and b are learnable parameters.

[0064] Step S102: Focus on different local ranges of the text through sliding windows of different lengths, and input them into BiLSTM to capture multi-level local features.

[0065] Although step S101 can extract effective global context information, there are common characteristics of non-standardization, ambiguous expressions and high-frequency abbreviations, which can easily lead to blurred boundaries and confusion of entity categories during entity recognition. Therefore, this case solution focuses on texts of different lengths and inputs them into BiLSTM to extract multi-scale local features. The bidirectional processing mechanism of BiLSTM allows the model to consider the context before and after the word at the same time, which is crucial for identifying entity boundaries and understanding local context.

[0066] Under the guidance of global information, use E = [e1, e2, ..., e n] as input, through the window sliding mechanism, for each position i in the sequence, a window is constructed including position i and its left and right The local context feature subsequence of the position Here w is the window size, which determines the width of the local context. Multiple windows are used to extract multi-level local features [S1, S2, ..., S K ], K is the number of windows.

[0067] Subsequently, S is fed into the BiLSTM layer. The BiLSTM layer transforms the local feature sequence into a hidden state sequence through the characteristics of the recurrent neural network. The expression is as follows:

[0068] g t =σ(W g S t +U g h t-1 +b g )

[0069] f t =σ(W f S t +U f h t-1 +b f )

[0070]

[0071] o t =σ(W o S t +U o h t-1 +b o )

[0072]

[0073] Among them, g t represents the input gate, f t represents the forget gate, o t represents the output gate, c t represents the state of the memory cell at the current time step, Indicates the candidate memory cell state; S t represents the input feature representation, and They represent the forward and reverse LSTM implicit feature representations at time t, respectively, and f i represents the implicit feature representation of BiLSTM; W, b and U represent learnable parameters, σ represents the sigmoid activation function, It stands for Hadamard product, which is bitwise multiplication.

[0074] By repeating the above process until all positions in the sequence are processed, a local feature tensor F = [f1, f2, ..., f n ].

[0075] For the NER task of social media text, this step is to, on the one hand, more accurately capture the relationship between words near the entity boundary by enhancing entity boundary detection, especially when dealing with compound entities such as "New_York_City" composed of multiple words, which can effectively identify the start and end positions of the entity. On the other hand, by using local context information, the recognition of entity types is improved. For example, when distinguishing whether "Apple" refers to fruit or a technology company, local features can assist the model in making correct judgments based on the context, thereby significantly improving the overall performance and accuracy of the model in named entity recognition in social media text.

[0076] Step S103: Use the attention fusion mechanism to effectively combine global context information and multiple local features to predict the optimal entity label.

[0077] From the above steps S101 and S102, the global feature representation Z=[z1,z2,...,z n ] and multi-level local feature representation F = [f1,f2,...,f n ], then the attention fusion mechanism is used to effectively integrate the two features by dynamically assigning weights to improve the accuracy of the model in identifying entities.

[0078] First, the global features and multi-level local feature vectors are concatenated through a fully connected layer as the query vector Q, Z as the value matrix, and F as the key matrix; then the dot product between the query vector and the local feature matrix is ​​calculated to obtain the attention score matrix S, and F is weighted and fused to obtain the weighted local features, which are finally fused with the global features to obtain the final vector A, which is expressed as follows:

[0079] Q=W q [Z; F]

[0080]

[0081] A=Z+S·F

[0082] Among them, W q is the weight matrix in the fully connected layer, d is the dimension of the key matrix, K is the key matrix, and V is the value matrix.

[0083] After obtaining the final embedding, the softmax function is used to decode the classification prediction sequence label and output y = [y1, y2, ..., y n ],y i is a label for the entity category.

[0084] Corresponding to the above method, this embodiment also proposes a social media text named entity recognition system that integrates multi-scale features and syntactic information, including:

[0085] The Global Semantic Encoder module is used to extract global features through the Transformer-based sequence encoder XLNET and embed dependency syntactic relations into it to enhance the semantic information of entities.

[0086] The Multi-scale Local Feature Extractor module is used to focus on different local ranges of the text through sliding windows of different lengths and input them into BiLSTM to capture multi-level local features.

[0087] The Fusion-Attention Mechanism module is used to use the attention fusion mechanism to effectively combine global context information and multiple local features to predict the optimal entity label.

[0088] The effectiveness and superiority of the present invention are verified by experiments. First, the benchmark dataset and the baseline method used for comparison are introduced, then the experimental settings and metrics are described in detail, and finally the results and analysis are given.

[0089] (I) Dataset

[0090] To verify the effectiveness of the proposed model, experiments were conducted on the WNUT2016, WNUT2017, and OntoNotes5.0_english (ON5e) datasets, which include 10, 6, and 18 entity types, respectively. All datasets are divided into training sets, validation sets, and test sets. These datasets cover common abbreviations, spelling errors, and non-standard spellings, making the NER task more challenging. The detailed information of the three datasets is given in Table 1.

[0091] Table 1 Statistics of the number of sentences (#Sent) and entities (#Ent) in the training set, validation set, and test set of the three benchmark datasets

[0092]

[0093] ①WNUT2016

[0094] WNUT2016 is a dataset constructed from tweets in Twitter, including ten entity types: company, facility, geo-loc, movie, music artist, other, person, product, sports team, and tv show.

[0095] ②WNUT2017

[0096] WNUT2017 collected 2,295 texts from social media data from different sources (Twitter, Reddit, YouTube, and StackExchange comments). The entity types include six types: Person, Location, corporation, product, creative-work, and group.

[0097] ③Ontonotes5.0(english)

[0098] Ontonotes5.0 is a large corpus consisting of news reports, movie scripts, phone conversations, lecture recordings, blog posts, etc. The data used in this paper is the English text data, including 18 entity types.

[0099] 2. Baseline Method

[0100] The proposed model SEMFF-NER is compared with the following benchmark methods.

[0101] 1) BERT-BiLSTM-context: A global context mechanism is proposed, which aims to integrate global sentence information into the sentence representation of each cell and is combined with BERT and BiLSTM for NER tasks.

[0102] 2) XLNET: It is an autoregressive language model based on Transformer-XL. It inherits the advantages of the autoencoder language model and can utilize richer contextual information.

[0103] 3) SANER: This method enhances semantics by finding entity-similar words from a large-scale corpus and encodes semantic information through an attentive semantic augmentation module. Then, a gate module is applied to weigh the weight of this information in different contexts for named entity recognition.

[0104] 4) AESINER: Proposes attention-enhanced key-value memory networks (KVMN) to integrate different types of syntactic information to improve NER, such as part-of-speech tags, syntactic components, etc.

[0105] 5) InferNER: This method does not rely on external sources, but utilizes information at the word, character, and sentence levels, and combines it with visual information. It introduces an attention component to calculate the attention weight probabilities of the text and the text-related visual context for the NER task of short texts.

[0106] 6) HGN: A new Hero-Gang neural network structure is proposed, including Hero and Gang modules, which simultaneously utilizes global and local information for NER tasks.

[0107] 3. Experimental settings and metrics

[0108] For the social media dataset used in the experiment, the XLNET-large-cased model was used as the encoder, and its default settings were used: 24 layers of self-attention and 1024-dimensional embedding; for the hyperparameters of the Multi-scale Local Feature Extractor module, the hidden layer size of the BiLSTM of each window was set to half the embedding dimension output by the Global Semantic Encoder module, and during the training process, Adam was used to optimize the negative log-likelihood loss function, and the learning rate was set to 0.001. In all experiments, a maximum of 50 epochs were run, the batch size was 64, and the hyperparameters were adjusted on the validation set.

[0109] For evaluation, precision (P), recall (R) and F1 value (F1) were used as metrics and the BIO tagging scheme was adopted for all datasets. F1 value was chosen as the main metric because it is the harmonic mean of precision and recall.

[0110]

[0111] Among them, TP, FP, and FN represent the number of correct recognitions, the number of errors, and the number of unrecognized entity labels by the model, respectively.

[0112] (IV) Experimental results

[0113] In order to verify the effectiveness of the model proposed in this invention, the experimental results are given in Table 2 and compared with the benchmark model.

[0114] Table 2 Comparison of the proposed model with baseline methods in terms of Precision (P), Recall (R) and F1-score (F1)

[0115]

[0116] The results show that the proposed model outperforms the baseline model in the three datasets of WNUT2016, WNUT2017 and ON5e. The F1 score of the WNUT2016 dataset has the largest improvement, while in the WNUT2017 dataset, the accuracy of entity recognition by the proposed model is greatly improved. Due to the large data size and more entity types of the ON5e dataset, only a small improvement is achieved in this dataset.

[0117] Since the model of the present invention can extract rich and deep multi-scale features in social media texts by integrating the global features extracted by XLNET and focusing on the local features captured by texts of different lengths, compared with only using XLNET to extract global features, the entity recognition performance is poor due to insufficient feature extraction. Although the BERT-BiLSTM-context model adds a global context mechanism, XLNET introduces an autoencoder language model that can utilize richer global context information. In addition, there are other benchmark methods that improve NER performance through additional external resources, such as the SANER method that enhances semantics by finding entity-similar words in large-scale corpora and the AESINER method that improves NER by integrating different types of syntactic information through an attention-integrated key-value memory network. Given the non-standardization and sparse entity nature of social media texts, dependency syntactic analysis can provide more useful information for NER tasks than other additional resources. This is because social media texts usually contain a large number of abbreviations, spelling errors, slang, and other forms of non-standard expressions, which pose challenges to entity recognition. Dependency syntactic analysis provides clues to the relationships between words, helps understand the entity context, and improves recognition accuracy. Similar to some structures of SEMFF-NER, the HGN method also achieves superior performance by combining global and local features. However, SEMFF-NER's multi-scale feature extraction and syntactic information fusion make it more accurate in identifying complex entities, especially more flexible when dealing with texts of different lengths. In addition, SEMFF-NER's Fusion-Attention Mechanism can better balance global and local features, making it more effective in dealing with complex entities.

[0118] In general, the proposed model SEMFF-NER uses richer feature representations and can effectively improve the entity recognition performance of social media texts.

[0119] (V) Ablation experiment

[0120] In order to evaluate the contribution of different modules in the SEMFF-NER model of the present invention to the model, four groups of ablation experiments were designed, as shown in Table 3.

[0121] 1) Baseline: The baseline model is set to the XLNET model.

[0122] 2) Baseline+MSLFE: Based on the baseline, a Multi-scale LocalFeatureExtractor module is added to focus on texts of different lengths and input them into BiLSTM to extract multi-scale local features. The global features extracted by XLNET are then fused with the local features through the concat method.

[0123] 3) DEP+Baseline+MSLFE: Dependencies are embedded and features extracted by the XLNET and MSLFE modules through concat operations are combined to convert text into vector representations, extract global sequence information, and enhance the model’s ability to extract deep semantic features.

[0124] 4) Baseline+MSLFE+attention: Fusion-attention Mechanism is used to fuse the global features extracted based on XLNET and the local features captured by focusing on texts of different lengths.

[0125] 5) SEMFF-NER: the model proposed by the present invention.

[0126] Table 3 Ablation experiment results on three datasets

[0127]

[0128] Experimental results show that the different modules in the model proposed in this invention have an improvement effect on the NER task of social media text. Specifically, after adding the Multi-scale Local Feature Extractor module, the performance of the model is significantly improved on all data sets, which shows that it can effectively extract useful information from texts of different lengths; secondly, after adding dependency embedding, this may be attributed to the module's ability to capture complex syntactic structures, thereby helping the model to better understand the contextual information of the text and improve the accuracy of the model's entity recognition; similarly, by integrating multi-scale features through the Fusion-attention Mechanism, the model effect is also improved. On the above three data sets, the improvement effect through embedding dependencies is better than attention fusion. This may be because dependency embedding focuses more on the understanding of syntactic structure, and social media texts often contain complex syntactic structures, which makes dependency embedding more advantageous in this case.

[0129] (VI) Case Analysis

[0130] like Figure 2As shown in the figure, in order to further verify the effectiveness of the proposed method, the experimental results of the SEMFF-NER model and the baseline method are compared and analyzed through case studies. The input sentence "British man dies workingon Qatar 2022World Cup stadium" and its corresponding real entity label "Qatar 2022World Cupstadium(Location)" are shown respectively. In this example, compared with the baseline method, the SEMFF-NER model can predict more complete entities than the baseline model. Specifically, in the baseline, only "Qatar 2022World Cup" is recognized as the Location label, and "stadium" is ignored, while our method recognizes all the words in the entity "Qatar 2022World Cupstadium". The key to SEMFF-NER's ability to provide more complete entity recognition is that its design incorporates multi-scale features and syntactic information. The model captures the features of texts of different lengths through the Multi-scale Local Feature Extractor, which enables it to recognize and understand the importance of each word in the entity. At the same time, in the syntactic analysis, "Qatar", "2022", "World", "Cup" and "stadium" form a compound noun phrase, and "stadium pobj on" reveals the role of "stadium" in the sentence, that is, as the object of the preposition "on", pointing to the workplace. SEMFF-NER uses this dependency information to enhance the accuracy of entity recognition and ensure that even the last part of the entity is not ignored.

[0131] In addition, SEMFF-NER's fusion-attention mechanism allows the model to focus on multiple local features while considering the global context, which is lacking in the baseline model. This global and local combination strategy enables SEMFF-NER to surpass the performance of the baseline model when identifying complex entities, demonstrating its superiority in the field of social media text entity recognition.

[0132] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present invention.

[0133] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0134] The units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person of ordinary skill in the art may use different methods to implement the described functions for each specific application, but such implementation is not considered to be beyond the scope of the present invention.

[0135] Those skilled in the art will appreciate that all or part of the steps in the above method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, such as a read-only memory, a disk or an optical disk. Optionally, all or part of the steps in the above embodiment can also be implemented using one or more integrated circuits, and accordingly, each module / unit in the above embodiment can be implemented in the form of hardware or in the form of software function modules. The present invention is not limited to any specific form of combination of hardware and software.

[0136] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for named entity recognition in social media texts that integrates multi-scale features and syntactic information, characterized in that: Include: First, the global features are extracted through the Transformer-based sequence encoder XLNET, and the dependency syntactic relations are embedded into it to enhance the semantic information of the entity; Then, sliding windows of different lengths are used to focus on different local ranges of the text and input into BiLSTM to capture multi-level local features; Finally, the attention fusion mechanism is used to effectively combine global context information and multiple local features to predict the optimal entity label.

2. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 1 is characterized in that: The Transformer-based sequence encoder XLNET extracts global features including: For a given input sequence X = [x1, x2, ..., x n ], and use XLNET to embed the corresponding vector E = [e1, e2, ..., e n ], the expression is: E=XLNet(x)=[e1,e2,...,e n ] Among them, e j ∈R u , that is, E∈R n×u .

3. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 2 is characterized in that: Embedding dependency syntactic relations into global features includes: By collecting all the dependencies from the dependency syntactic analysis of the input sequence X, for each x i Embed the corresponding syntactic feature d i , composed of D = [d1, d2, ..., d n ]; Finally, the global feature vector E and the syntactic feature vector D are integrated to obtain the final feature representation Z = [z1, z2, ..., z n ]; the expression is as follows: D=[d1,d2,...,d n ] Z = σ([(E, D)]W+b) Among them, d i ∈R v , that is, D∈R n×v , Z∈R n×t , σ represents the sigmoid function, W and b are learnable parameters.

4. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 3 is characterized in that: Focusing on different local ranges of text through sliding windows of different lengths includes: Use E = [e1, e2, ..., e b ] as input, through the window sliding mechanism, for each position i in the sequence, a window including position i And its left and right The local context feature subsequence of the position Here w is the window size; multiple windows are used to extract multi-level local features [S1, S2, ..., S K ], K is the number of windows.

5. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 4 is characterized in that: Input to BiLSTM captures multi-level local features including: Feed S into the BiLSTM layer. The BiLSTM layer transforms the local feature sequence into a hidden state sequence through the characteristics of the recurrent neural network. The expression is as follows: g t =σ(W g S t +U g h t-1 +b g ) f t =σ(W f S t +U f h t-1 +b f ) the t =σ(W o S t +U o h t-1 +b o ) Among them, g t represents the input gate, f t represents the forget gate, o t represents the output gate, c t represents the state of the memory cell at the current time step, Indicates the candidate memory cell state; S t represents the input feature representation, and They represent the forward and reverse LSTM implicit feature representations at time t, respectively, and f i represents the implicit feature representation of BiLSTM; W, b and U represent learnable parameters, σ represents the sigmoid activation function, represents the Hadamard product; By repeating the above process until all positions in the sequence are processed, a local feature tensor F = [f1, f2, ..., f n ].

6. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 5 is characterized in that: The attention fusion mechanism is used to effectively combine global context information and multiple local features to predict the optimal entity label including: First, we obtain the global feature representation Z = [z1, z2, ..., z n ] and multi-level local feature representation F = [f1, f2, ..., f n ], then calculate the attention score matrix S, perform weighted fusion on F, obtain the weighted local features, and finally fuse them with the global features to obtain the final vector A, which is expressed as follows: A=Z+S·F After obtaining the final embedding, the softmax function is used to decode the classification prediction sequence label and output y = [y1, y2, ..., y n ].

7. The method for named entity recognition in social media texts integrating multi-scale features and syntactic information according to claim 6 is characterized in that: The calculation process of the attention score matrix s is as follows: First, the global features and multi-level local feature vectors are concatenated through a fully connected layer as the query vector Q, Z as the value matrix, and F as the key matrix; then the dot product between the query vector and the local feature matrix is ​​calculated to obtain the attention score matrix s, which is expressed as follows: Q=W q [Z;F] Among them, W q is the weight matrix in the fully connected layer, d is the dimension of the key matrix, K is the key matrix, and V is the value matrix.

8. A social media text named entity recognition system integrating multi-scale features and syntactic information, characterized in that: The method for identifying named entities in social media texts by integrating multi-scale features and syntactic information as claimed in any one of claims 1 to 7 comprises: The global semantic encoding module is used to extract global features through the Transformer-based sequence encoder XLNET and embed dependency syntactic relations into it to enhance the semantic information of entities; The multi-level local feature extraction module is used to focus on different local ranges of the text through sliding windows of different lengths and input them into BiLSTM to capture multi-level local features; The attention fusion mechanism module is used to use the attention fusion mechanism to effectively combine global context information and multiple local features to predict the optimal entity label.

9. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images