Non-abandoned text corpus construction and named entity recognition method and system

Abstract

The invention discloses a non-disciplinary text corpus construction and named entity recognition method and system. The method comprises the steps that related text data of a large number of intangible cultural heritage is collected; based on the related text data, constructing a non-perpetual corpus; constructing an entity recognition model based on a non-perpetual corpus; and completing named entity recognition of the non-abandoned text corpus by using the entity recognition model. According to the method, a high-quality non-abandoned text corpus is constructed, the entity recognition model is provided, through a dynamic full-word mask strategy and a special feature extraction and learning method, the recognition accuracy of the named entities in the non-abandoned text is remarkably improved, and powerful support is provided for non-abandoned knowledge graph construction and cultural transmission.

Description

Technical Field

[0001] The present invention relates to the field of intangible cultural heritage text recognition, and in particular to an intangible cultural heritage text corpus construction and named entity recognition method and system. Background Art

[0002] Text is an important carrier for the dissemination of intangible cultural heritage (hereinafter referred to as ICH), carrying the traditional essence of ICH knowledge, such as the place of origin, inheritors, categories, etc. In addition, there are often a large number of words with local cultural characteristics in ICH texts, reflecting the distinct regional characteristics, uniqueness and vitality. In recent years, the protection and inheritance of ICH has become the focus of public attention and discussion. How to effectively protect, inherit and utilize ICH knowledge has become an urgent problem to be solved. By accurately identifying and mining ICH entities in the text, we can better define and understand the content of ICH texts, provide a basis for the construction of ICH knowledge graphs, and achieve the purpose of spreading ICH.

[0003] At present, the construction of named entity corpora is mostly concentrated in the fields of medicine, agriculture, geography, etc., and there are relatively few annotated corpora related to the field of intangible cultural heritage, which limits the in-depth research and application of intangible cultural heritage named entity recognition technology. Considering the particularity of intangible cultural heritage text materials, its named entities often have unique expressions and cultural connotations, so special annotation work is required for this field. Because the named entities in the field of intangible cultural heritage are complex and diverse, and often contain a large amount of dialectal and regional text content, it is difficult to fully understand their semantic features using traditional named entity recognition methods, which affects the accuracy of recognition. Therefore, it is necessary to develop a more effective entity recognition model based on the characteristics of the field of intangible cultural heritage. Summary of the invention

[0004] In view of the scarcity of text corpus in the field of intangible cultural heritage mentioned in the background technology above, the complex regional and dialect characteristics of intangible cultural heritage texts, and the inability of traditional named entity recognition methods to comprehensively and accurately identify them, the present invention designs a set of detailed named entity annotation standard methods and systems based on the relevant text materials of intangible cultural heritage.

[0005] To achieve the above objectives, the present invention provides a method for constructing an intangible cultural heritage text corpus and identifying named entities, the steps comprising:

[0006] Collect a large amount of textual materials related to intangible cultural heritage;

[0007] Based on the relevant text materials, construct an intangible cultural heritage corpus;

[0008] Based on the intangible cultural heritage corpus, construct an entity recognition model;

[0009] The entity recognition model is used to complete named entity recognition of intangible cultural heritage text corpus.

[0010] Preferably, the method for constructing the intangible cultural heritage corpus includes: adopting the BIO annotation criteria and using LabelStudio to annotate entities; at the same time, introducing the F value as an indicator for consistency evaluation during annotation, and selecting the annotation results of annotator A as a reference standard; then, calculating the annotation accuracy and recall rate of annotator B relative to A; finally, combining the P value and the R value to calculate the consistent annotation F value; the specific calculation formula is as follows:

[0011]

[0012] Preferably, the constructed entity recognition model includes: a character vector embedding module, an intangible cultural heritage feature extraction module and an entity label prediction module;

[0013] The character vector embedding module is used to embed character vectors;

[0014] The intangible cultural heritage feature extraction module is used to extract semantic features of embedded character vectors;

[0015] The entity tag prediction module is used to determine the optimal sequence output based on the extracted semantic features.

[0016] Preferably, the character vector embedding module uses RoBERTa-wwm-ext as a pre-training model to complete the embedding of the character vector; the RoBERTa-wwm-ext model adopts a dynamic full-word masking strategy in the pre-training task, and dynamically masks the sentences during the training process.

[0017] Preferably, the intangible cultural heritage feature extraction module adopts a bidirectional long short-term memory network as a core network structure, and the bidirectional long short-term memory network includes a forward LSTM and a backward LSTM;

[0018] The forward LSTM is used to learn features from the beginning to the end of the sequence;

[0019] The backward LSTM is used to learn the features from the end to the beginning of the sequence.

[0020] Preferably, in the entity tag prediction module, a conditional random field is used to determine the optimal sequence output based on the semantic features combined with the predicted tags in the context.

[0021] The present invention also provides a system for constructing an intangible cultural heritage text corpus and identifying named entities, the system being used to implement the above method, comprising: a collection module, a corpus construction module, a model construction module and a recognition module;

[0022] The acquisition module is used to collect a large amount of textual materials related to intangible cultural heritage;

[0023] The corpus construction module is used to construct an intangible cultural heritage corpus based on the relevant text data;

[0024] The model building module is used to build an entity recognition model based on the intangible cultural heritage corpus;

[0025] The recognition module is used to complete named entity recognition of the intangible cultural heritage text corpus using the entity recognition model.

[0026] Preferably, the workflow of the corpus construction module includes:

[0027] The BIO annotation criteria are adopted, and Label Studio is used to annotate entities. At the same time, the F value is introduced as an indicator for consistency evaluation during annotation, and the annotation results of annotator A are selected as the reference standard. Then, the annotation accuracy and recall rate of annotator B relative to A are calculated. Finally, the consistent annotation F value is calculated by combining the P value and the R value. The specific calculation formula is as follows:

[0028]

[0029] Compared with the prior art, the present invention has the following beneficial effects:

[0030] The present invention constructs a high-quality intangible cultural heritage text corpus and proposes an entity recognition model. Through a dynamic full-word masking strategy and special feature extraction and learning methods, it significantly improves the accuracy of named entity recognition in intangible cultural heritage texts, providing strong support for the construction of intangible cultural heritage knowledge graphs and cultural communication. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0032] Figure 1 A schematic diagram of a method flow of an embodiment of the present invention;

[0033] Figure 2 A schematic diagram of the construction process of the intangible cultural heritage corpus according to an embodiment of the present invention;

[0034] Figure 3 This is a BIO annotation example diagram of an embodiment of the present invention;

[0035] Figure 4This is a structural diagram of RoBERTa-wwm-ext and Transformer in an embodiment of the present invention;

[0036] Figure 5 This is a BiLSTM network structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] Embodiment 1

[0040] like Figure 1 The figure is a schematic diagram of the method flow of this embodiment. The steps include:

[0041] S1. Collect a large amount of relevant textual materials on intangible cultural heritage.

[0042] This embodiment takes the intangible cultural heritage text data of Sanqin region as an example, and based on the text data of relevant websites such as Shaanxi Province Intangible Cultural Heritage, a large amount of intangible cultural heritage text data is collected. In the process of text data collection, the structured and semi-structured data on major authoritative websites are collected, such as Shaanxi Province Intangible Cultural Heritage Protection Center, China Intangible Cultural Heritage Network, China Intangible Cultural Heritage Digital Museum, Shaanxi Province Intangible Cultural Heritage Database and Baidu Encyclopedia. At the same time, in order to obtain more abundant and in-depth information, offline research was also conducted to the Shaanxi Intangible Cultural Heritage Exhibition Hall and the local cultural halls in the five regions of Baoji, Weinan, Yan'an and Hanzhong, and physical books and archival records related to intangible cultural heritage were collected. After preliminary sorting and analysis, a total of 5 batches of 91 national intangible cultural heritage projects and 766 provincial intangible cultural heritage projects were successfully collected.

[0043] After data collection is completed, the noise data in these data is cleared. Given that the original data contains a large amount of unstructured content, preprocessing is performed before data annotation. This step includes deleting non-text data, links and special characters, removing stop words, and converting text formats. After these unified preprocessing, 7243 intangible cultural heritage texts were obtained, which were stored in txt format to form a raw corpus.

[0044] S2. Construct an intangible cultural heritage corpus based on relevant text materials.

[0045] This embodiment combines the unique characteristics of the intangible cultural heritage corpus and develops named entity annotation rules suitable for intangible cultural heritage entities. Specifically, the BIO annotation criteria are adopted, and Label Studio is used to annotate entities. To ensure the scientificity and accuracy of the annotation work, each text is annotated by 2 annotators and a strict consistency test is implemented. For multiple inconsistent annotations, they are submitted to experts for careful review. Through this series of rigorous steps, this embodiment ensures the accuracy and reliability of the intangible cultural heritage entity corpus, laying a solid foundation for subsequent research. Figure 2 The construction process of the intangible cultural heritage corpus is shown. Specifically, this embodiment designs 21 tags, as shown in Table 1, where the "BX" tag indicates that the entity to which an element belongs is of type X and the element is the starting part of the entity; the "IX" tag indicates that the element is located in the middle of the X-type entity; and the "O" tag indicates that the element does not belong to any specific entity type.

[0046] Table 1

[0047]

[0048] In order to more intuitively demonstrate the application of BIO criteria in intangible cultural heritage text annotation, Figure 3 An annotation example based on text corpus is provided. In the example, the input intangible cultural heritage text is carefully disassembled, entity recognition is performed on each character, and the corresponding entity category is labeled.

[0049] When evaluating intangible cultural heritage entities, in order to ensure that the gap between the annotation results of different annotators is as small as possible, the annotation consistency (Inter-RaterAgreement) indicator is introduced. Due to the differences in the professional knowledge background of the annotators, there may be deviations in their understanding of the same intangible cultural heritage entity, which will affect the consistency of the corpus. Therefore, this embodiment expects that the highest possible consistency can be achieved between the annotators, and for this reason, it is particularly important to effectively evaluate the consistency.

[0050] Specifically, set the F value as the indicator of consistency evaluation, select the annotation results of annotator A as the reference standard; then calculate the annotation accuracy (Precision) and recall (Recall) of annotator B relative to A; finally, combine the P value and R value to calculate the consistency annotation F value. The specific calculation formula is as follows, which will be used to calculate the P value, R value and the final consistency annotation F value. The formula is as follows:

[0051]

[0052] It should be emphasized that a certain order is followed when calculating entity annotation consistency. First, the consistency of the string is calculated; secondly, based on the string consistency, the type consistency is further calculated. From the entity annotation consistency evaluation data shown in Table 2 below, it can be clearly seen that after 3 rounds of iterations, the entity string consistency has increased from 0.883 to 0.953, while the type consistency has increased from 0.969 to 0.976. This shows that the impact of entity string consistency on the annotation consistency of the entire corpus is greater than that of the entity type. Therefore, when annotating, it is necessary to combine professional knowledge and unify the determination standards of entity boundaries to ensure the accuracy of annotation.

[0053] Table 2

[0054] frequency Entity string consistency Type consistency 1 0.8834 0.9694 2 0.9471 0.9734 3 0.9530 0.9758

[0055] Finally, the text corpus with multiple inconsistencies in entity consistency detection is systematically sorted. This embodiment finally determines the annotation results of these texts by consulting relevant materials and asking intangible cultural heritage research experts to review them, as shown in Table 3.

[0056] Table 3

[0057] Entity Type Number of entities Traditional skills 836 Traditional sports and acrobatics 743 Traditional folk art 710 Traditional folklore 792 Traditional drama 810 Traditional dance 702 Traditional Music 768 Traditional Medicine 745 Traditional Art 687 Folk literature 779

[0058] Through this process, a complete and accurate intangible cultural heritage corpus was constructed.

[0059] S3. Build an entity recognition model based on the intangible cultural heritage corpus.

[0060] In this embodiment, the constructed entity recognition model includes three parts: a character vector embedding module, an intangible cultural heritage feature extraction module, and an entity label prediction module. The RoBERTa-wwm-ext pre-training model is used to extract the representation of each position in the text and align it with the original label sequence. In the process of character encoding, in view of the fact that Chinese characters exist in the form of phrases, the dynamic full-word masking strategy randomly masks continuous words and dynamically generates masks each time input is provided to the model. In the process of continuous input of a large amount of data, the model will gradually adapt to different masking strategies and learn Chinese language representation. Subsequently, these representations are connected to BiLSTM for further feature extraction and integration. Finally, through the processing of CRF, the predicted sequence representation of the model is obtained.

[0061] Specifically, the character vector embedding module uses RoBERTa-wwm-ext as a pre-training model to complete the embedding of character vectors. The RoBERTa-wwm-ext model is mainly based on the Transformer architecture and adopts a dynamic full-word masking strategy in the pre-training task. This masking strategy does not perform masking during preprocessing, but randomly generates masks each time input is provided to the model, so it changes dynamically. By dynamically masking sentences during the training process, the positions that need to be masked in each round of training are different, which greatly enhances the generalization ability of the model. In particular, in view of the fact that Chinese characters exist in the form of phrases, the dynamic full-word masking strategy can more effectively learn Chinese vocabulary features by masking consecutive words instead of single characters. Table 4 shows the difference between the dynamic full-word masking strategy and the traditional masking strategy in practical applications.

[0062] Table 4

[0063]

[0064] The structure of the RoBERTa-wwm-ext model is as follows Figure 4 As shown in the figure, E1 and E2 play a key role as input character vectors. The right sub-figure details the Transformer architecture, whose core component Trm is the Transformer encoder. It is worth noting that the RoBERTa-wwm-ext model is not a single structure, but is based on a stack of multiple Transformer encoders. After being processed by the encoder, E1 and E2 are converted into more representative character vectors T1 and T2, providing a basis for subsequent feature extraction model processing.

[0065] In order to effectively learn the semantic features of the sequence (embedded character vectors) and obtain contextual dependency information, the intangible cultural heritage feature extraction module uses a bidirectional long short-term memory network (BiLSTM) as its core network structure. BiLSTM consists of two directional LSTM networks, namely forward LSTM and backward LSTM. It learns the features from the beginning to the end of the sequence through the forward LSTM, while the backward LSTM is responsible for learning the features from the end to the beginning of the sequence, thereby fully capturing the bidirectional semantic dependency of the sequence. This structure makes the feature extraction module more accurate and comprehensive when extracting semantic features. The BiLSTM model structure is as follows: Figure 5 shown.

[0066] By the cell state C t Filter the information in the network, forget useless information and remember new information to ensure that information that is useful for subsequent calculations can be effectively transmitted. At each time step, LSTM outputs a hidden state h t, the process is affected by the hidden state h at the previous moment t-1 and the current input X t The calculated forget gate f t 、Memory Gate t and output gate o t The following is the relevant calculation formula:

[0067] f t =σ(W f ·[h t-1 , X t ]+b f )

[0068] i t =σ(W i ·[h t-1 , X t ]+b i )

[0069]

[0070] o t =σ(W o ·[h t-1 , X t ]+b o )

[0071] h t =o t *tanh(C t )

[0072] In the formula, σ represents the sigmoid activation function; W represents candidate memory cell (Candidate Cell State); f Represents the weight matrix of the forget gate; W i Represents the weight matrix of the input gate; W c represents the weight matrix of candidate memory cells; W o represents the weight matrix of the output gate; b f represents the bias term of the forget gate; b i represents the bias term of the input gate; b c represents the bias term of the candidate memory cell; b o Represents the bias term of the output gate; tanh represents the hyperbolic tangent function, which is used to generate new candidate values; * represents element multiplication.

[0073] The intangible cultural heritage feature extraction module can automatically extract key features (semantic features) from the output of the upper-level encoding vector (embedded character vector). In the entity label prediction module, the conditional random field (CRF) uses these features and combines the contextual information to predict the label and determine the optimal sequence output. Through this process, the model can make full use of the contextual information of the sequence and improve the accuracy and consistency of the intangible cultural heritage entity label prediction.

[0074] CRF calculates the input sequence X = {x 1 ,x 2 ,x 3 ,...,x n} to predict the output label sequence Y = {y 1 ,y 2 ,y 3 ,...,y n}. In particular, the CRF layer can add some constraints to the final predicted labels to ensure that they are valid, for example: the label of the first character in the sentence should start with "B-" or "O", not "I-". These constraints can be automatically learned by the CRF layer from the training dataset during training. With these useful constraints, the number of invalid predicted label sequences will be significantly reduced.

[0075] The calculation of the predicted sequence score involves the elements in the CRF transfer score matrix A It represents the label y i Move to label y i+1 The score of , and the probability that the corresponding character in the BiLSTM output sequence P is predicted as a specific label The calculation formula is as follows:

[0076]

[0077] Then, by applying Softmax normalization, this embodiment can obtain the probability distribution P(Y|X) of each label sequence, as follows:

[0078]

[0079] Among them, Y X Represents the set of all possible label sequences for a given input sequence X; Represents the set Y X Any possible tag sequence in; S(X,Y) is the score of a specific tag sequence Y; is the sum of the scores of all possible label sequences.

[0080] In order to maximize the fraction of correct transition state sequences in matrix A, it is necessary to calculate the loss value. This embodiment combines the calculation method of the loss function and uses the relevant formula to find the label set that can maximize the likelihood estimate:

[0081]

[0082] Where log P(Y|X) represents the logarithmic conditional probability of the correct label sequence Y given the input sequence X.

[0083] Finally, by applying the Viterbi algorithm, the optimal prediction label sequence Y can be determined * :

[0084]

[0085] in, represents the score of any possible label sequence.

[0086] S4. Use the entity recognition model to complete named entity recognition of intangible cultural heritage text corpus.

[0087] Finally, the constructed entity recognition model is used to complete the named entity recognition of the intangible cultural heritage text corpus.

[0088] Embodiment 2

[0089] This embodiment also provides a non-material cultural heritage text corpus construction and named entity recognition system, including: a collection module, a corpus construction module, a model construction module and a recognition module; the collection module is used to collect a large amount of relevant text materials of intangible cultural heritage; the corpus construction module is used to construct an non-material cultural heritage corpus based on the relevant text materials; the model construction module is used to construct an entity recognition model based on the non-material cultural heritage corpus; the recognition module is used to use the entity recognition model to complete the named entity recognition of the non-material cultural heritage text corpus.

[0090] The following will explain in detail how the present invention solves technical problems in real life in conjunction with this embodiment.

[0091] This embodiment takes the intangible cultural heritage text data of Sanqin region as an example, based on the text data of relevant websites such as Shaanxi Province Intangible Cultural Heritage, the collection module collects a large amount of intangible cultural heritage text data. In the process of text data collection, the structured and semi-structured data on major authoritative websites are collected, such as Shaanxi Province Intangible Cultural Heritage Protection Center, China Intangible Cultural Heritage Network, China Intangible Cultural Heritage Digital Museum, Shaanxi Province Intangible Cultural Heritage Database and Baidu Encyclopedia. At the same time, in order to obtain more abundant and in-depth information, offline research was also conducted to the Shaanxi Intangible Cultural Heritage Exhibition Hall and the local cultural halls in the five regions of Baoji, Weinan, Yan'an and Hanzhong, and physical books and archival records related to intangible cultural heritage were collected. After preliminary sorting and analysis, a total of 5 batches of 91 national intangible cultural heritage projects and 766 provincial intangible cultural heritage projects were successfully collected.

[0092] After data collection is completed, the noise data in these data is cleared. Given that the original data contains a large amount of unstructured content, preprocessing is performed before data annotation. This step includes deleting non-text data, links and special characters, removing stop words, and converting text formats. After these unified preprocessing, 7243 intangible cultural heritage texts were obtained, which were stored in txt format to form a raw corpus.

[0093] The corpus construction module then constructs the intangible cultural heritage corpus based on relevant text materials.

[0094] This embodiment combines the unique characteristics of the intangible cultural heritage corpus and develops named entity annotation rules suitable for intangible cultural heritage entities. Specifically, the BIO annotation criteria are adopted, and Label Studio is used to annotate entities. To ensure the scientificity and accuracy of the annotation work, each text is annotated by 2 annotators and a strict consistency test is implemented. For multiple inconsistent annotations, they are submitted to experts for careful review. Through this series of rigorous steps, this embodiment ensures the accuracy and reliability of the intangible cultural heritage entity corpus, laying a solid foundation for subsequent research. Figure 2 The construction process of the intangible cultural heritage corpus is shown. Specifically, this embodiment designs 21 tags, as shown in Table 1, where the "BX" tag indicates that the entity to which an element belongs is of type X and the element is the starting part of the entity; the "IX" tag indicates that the element is located in the middle of the X-type entity; and the "O" tag indicates that the element does not belong to any specific entity type.

[0095] In order to more intuitively demonstrate the application of BIO criteria in intangible cultural heritage text annotation, Figure 3 An annotation example based on text corpus is provided. In the example, the input intangible cultural heritage text is carefully disassembled, entity recognition is performed on each character, and the corresponding entity category is labeled.

[0096] When evaluating intangible cultural heritage entities, in order to ensure that the gap between the annotation results of different annotators is as small as possible, the annotation consistency (Inter-RaterAgreement) indicator is introduced. Due to the differences in the professional knowledge background of the annotators, there may be deviations in their understanding of the same intangible cultural heritage entity, which will affect the consistency of the corpus. Therefore, this embodiment expects that the highest possible consistency can be achieved between the annotators, and for this reason, it is particularly important to effectively evaluate the consistency.

[0097] Specifically, set the F value as the indicator of consistency evaluation, select the annotation results of annotator A as the reference standard; then calculate the annotation accuracy (Precision) and recall (Recall) of annotator B relative to A; finally, combine the P value and R value to calculate the consistency annotation F value. The specific calculation formula is as follows, which will be used to calculate the P value, R value and the final consistency annotation F value. The formula is as follows:

[0098]

[0099] It should be emphasized that a certain order is followed when calculating entity annotation consistency. First, the consistency of the string is calculated; secondly, based on the string consistency, the type consistency is further calculated. From the entity annotation consistency evaluation data shown in Table 2 below, it can be clearly seen that after 3 rounds of iterations, the entity string consistency has increased from 0.883 to 0.953, while the type consistency has increased from 0.969 to 0.976. This shows that the impact of entity string consistency on the annotation consistency of the entire corpus is greater than that of the entity type. Therefore, when annotating, it is necessary to combine professional knowledge and unify the determination standards of entity boundaries to ensure the accuracy of annotation.

[0100] Finally, the text corpus with multiple inconsistencies in entity consistency detection is systematically sorted. This embodiment finally determines the annotation results of these texts by consulting relevant materials and asking intangible cultural heritage research experts to review them, as shown in Table 3.

[0101] The model building module builds an entity recognition model based on the intangible cultural heritage corpus.

[0102] In this embodiment, the constructed entity recognition model includes three parts: a character vector embedding module, an intangible cultural heritage feature extraction module, and an entity label prediction module. The RoBERTa-wwm-ext pre-training model is used to extract the representation of each position in the text and align it with the original label sequence. In the process of character encoding, in view of the fact that Chinese characters exist in the form of phrases, the dynamic full-word masking strategy randomly masks continuous words and dynamically generates masks each time input is provided to the model. In the process of continuous input of a large amount of data, the model will gradually adapt to different masking strategies and learn Chinese language representation. Subsequently, these representations are connected to BiLSTM for further feature extraction and integration. Finally, through the processing of CRF, the predicted sequence representation of the model is obtained.

[0103] Specifically, the character vector embedding module uses RoBERTa-wwm-ext as a pre-training model to complete the embedding of character vectors. The RoBERTa-wwm-ext model is mainly based on the Transformer architecture and adopts a dynamic full-word masking strategy in the pre-training task. This masking strategy does not perform masking during preprocessing, but randomly generates masks each time input is provided to the model, so it changes dynamically. By dynamically masking sentences during the training process, the positions that need to be masked in each round of training are different, which greatly enhances the generalization ability of the model. In particular, in view of the fact that Chinese characters exist in the form of phrases, the dynamic full-word masking strategy can more effectively learn Chinese vocabulary features by masking consecutive words instead of single characters. Table 4 shows the difference between the dynamic full-word masking strategy and the traditional masking strategy in practical applications.

[0104] The structure of the RoBERTa-wwm-ext model is as follows Figure 4 As shown in the figure, E1 and E2 play a key role as input character vectors. The right sub-figure details the Transformer architecture, whose core component Trm is the Transformer encoder. It is worth noting that the RoBERTa-wwm-ext model is not a single structure, but is based on a stack of multiple Transformer encoders. After being processed by the encoder, E1 and E2 are converted into more representative character vectors T1 and T2, providing a basis for subsequent feature extraction model processing.

[0105] In order to effectively learn the semantic features of the sequence (embedded character vectors) and obtain contextual dependency information, the intangible cultural heritage feature extraction module uses a bidirectional long short-term memory network (BiLSTM) as its core network structure. BiLSTM consists of two directional LSTM networks, namely forward LSTM and backward LSTM. It learns the features from the beginning to the end of the sequence through the forward LSTM, while the backward LSTM is responsible for learning the features from the end to the beginning of the sequence, thereby fully capturing the bidirectional semantic dependency of the sequence. This structure makes the feature extraction module more accurate and comprehensive when extracting semantic features. The BiLSTM model structure is as follows: Figure 5 shown.

[0106] By the cell state C t Filter the information in the network, forget useless information and remember new information to ensure that information that is useful for subsequent calculations can be effectively transmitted. At each time step, LSTM outputs a hidden state h t , the process is affected by the hidden state h at the previous moment t-1 and the current input X t The calculated forget gate f t 、Memory Gate t and output gate o t The following is the relevant calculation formula:

[0107] f t =σ(W f ·[h t-1 , X t ]+b f )

[0108] i t =σ(W i ·[h t-1 , X t ]+b i )

[0109]

[0110] o t =σ(W o ·[h t-1 , X t ]+b o )

[0111] h t =o t *tanh(C t )

[0112] In the formula, σ represents the sigmoid activation function; W represents candidate memory cell (Candidate Cell State);f Represents the weight matrix of the forget gate; W i Represents the weight matrix of the input gate; W c represents the weight matrix of candidate memory cells; W o represents the weight matrix of the output gate; b f represents the bias term of the forget gate; b i represents the bias term of the input gate; b c represents the bias term of the candidate memory cell; b o Represents the bias term of the output gate; tanh represents the hyperbolic tangent function, which is used to generate new candidate values; * represents element multiplication.

[0113] The intangible cultural heritage feature extraction module can automatically extract key features (semantic features) from the output of the upper-level encoding vector (embedded character vector). In the entity label prediction module, the conditional random field (CRF) uses these features and combines the contextual information to predict the label and determine the optimal sequence output. Through this process, the model can make full use of the contextual information of the sequence and improve the accuracy and consistency of the intangible cultural heritage entity label prediction.

[0114] CRF calculates the input sequence X = {x 1 ,x 2 ,x 3 ,...,x n} to predict the output label sequence Y = {y 1 ,y 2 ,y 3 ,...,y n}. In particular, the CRF layer can add some constraints to the final predicted labels to ensure that they are valid, for example: the label of the first character in the sentence should start with "B-" or "O", not "I-". These constraints can be automatically learned by the CRF layer from the training dataset during training. With these useful constraints, the number of invalid predicted label sequences will be significantly reduced.

[0115] The calculation of the predicted sequence score involves the elements in the CRF transfer score matrix A It represents the label y i Move to label y i+1 The score of , and the probability that the corresponding character in the BiLSTM output sequence P is predicted as a specific label The calculation formula is as follows:

[0116]

[0117] Then, by applying Softmax normalization, this embodiment can obtain the probability distribution P(Y|X) of each label sequence, as follows:

[0118]

[0119] Among them, Y X Represents the set of all possible label sequences for a given input sequence X; Represents the set Y X Any possible tag sequence in; S(X,Y) is the score of a specific tag sequence Y; is the sum of the scores of all possible label sequences.

[0120] In order to maximize the fraction of correct transition state sequences in matrix A, it is necessary to calculate the loss value. This embodiment combines the calculation method of the loss function and uses the relevant formula to find the label set that can maximize the likelihood estimate:

[0121]

[0122] Where logP(Y|X) represents the logarithmic conditional probability of the correct label sequence Y given the input sequence X.

[0123] Finally, by applying the Viterbi algorithm, the optimal prediction label sequence Y can be determined * :

[0124]

[0125] in, represents the score of any possible label sequence.

[0126] Finally, the recognition module uses the constructed entity recognition model to complete the named entity recognition of the intangible cultural heritage text corpus.

[0127] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for constructing an intangible cultural heritage text corpus and identifying named entities, characterized in that the steps include: Collect a large amount of textual materials related to intangible cultural heritage; Based on the relevant text materials, construct an intangible cultural heritage corpus; Based on the intangible cultural heritage corpus, construct an entity recognition model; The entity recognition model is used to complete named entity recognition of intangible cultural heritage text corpus.

2. The method for constructing an intangible cultural heritage text corpus and identifying named entities according to claim 1, characterized in that: The method for constructing the intangible cultural heritage corpus includes: adopting the BIO annotation criteria and using Label Studio to annotate entities; at the same time, introducing the F value as an indicator for consistency evaluation during annotation, and selecting the annotation results of annotator A as a reference standard; then, calculating the annotation accuracy and recall rate of annotator B relative to A; finally, combining the P value and the R value to calculate the consistent annotation F value; the specific calculation formula is as follows:

3. The method for constructing an intangible cultural heritage text corpus and identifying named entities according to claim 1, characterized in that: The constructed entity recognition model includes: character vector embedding module, intangible cultural heritage feature extraction module and entity label prediction module; The character vector embedding module is used to embed character vectors; The intangible cultural heritage feature extraction module is used to extract semantic features of embedded character vectors; The entity tag prediction module is used to determine the optimal sequence output based on the extracted semantic features.

4. The method for constructing an intangible cultural heritage text corpus and identifying named entities according to claim 3, characterized in that: The character vector embedding module uses RoBERTa-wwm-ext as a pre-training model to complete the embedding of character vectors; the RoBERTa-wwm-ext model adopts a dynamic full-word masking strategy in the pre-training task and dynamically masks the sentences during the training process.

5. The method for constructing an intangible cultural heritage text corpus and identifying named entities according to claim 3, characterized in that: The intangible cultural heritage feature extraction module adopts a bidirectional long short-term memory network as the core network structure, and the bidirectional long short-term memory network includes a forward LSTM and a backward LSTM; The forward LSTM is used to learn features from the beginning to the end of the sequence; The backward LSTM is used to learn the features from the end to the beginning of the sequence.

6. The method for constructing an intangible cultural heritage text corpus and identifying named entities according to claim 3, characterized in that: In the entity tag prediction module, a conditional random field is used to determine the optimal sequence output based on the semantic features combined with the predicted tags in the context.

7. A system for constructing an intangible cultural heritage text corpus and identifying named entities, the system being used to implement the method described in any one of claims 1 to 6, characterized in that: include: Acquisition module, corpus building module, model building module and recognition module; The acquisition module is used to collect a large amount of textual materials related to intangible cultural heritage; The corpus construction module is used to construct an intangible cultural heritage corpus based on the relevant text data; The model building module is used to build an entity recognition model based on the intangible cultural heritage corpus; The recognition module is used to complete named entity recognition of the intangible cultural heritage text corpus using the entity recognition model.

8. The intangible cultural heritage text corpus construction and named entity recognition system according to claim 7, characterized in that: The workflow of the corpus construction module includes: The BIO annotation criteria were adopted and Label Studio was used to annotate entities. The F value was introduced as an indicator for consistency evaluation during annotation, and the annotation results of annotator A were selected as the reference standard. Next, calculate the annotation accuracy and recall rate of annotator B relative to A; finally, calculate the consistent annotation F value by combining the P value and R value; the specific calculation formula is as follows:

Images