A method for naming entity recognition of ancient books of warm diseases

Abstract

The application discloses a kind of for warm disease ancient books named entity recognition method.BIOES method is marked to relevant warm disease ancient books, and is preprocessed into the text format needed;Text is processed by word segmentation tool, and word level granularity text is obtained;The vector representation of word and character is obtained by BERT pre-training model;The result of word segmentation is added after text, and each word and character is assigned a position coding using fusion position coding algorithm;The vector representation after position coding is input and trained based on the network of Transform, and relative position information is added;The label corresponding to each word is output by CRF layer;Finally, the named entity recognition model constructed can be used for entity recognition to warm disease ancient books text.This method makes full use of the lexical information of Chinese, while effectively avoiding the final recognition error caused by word segmentation error.The application designs an efficient position coding method for classical Chinese, and uses Transform as the main part of the model, which improves the accuracy and recall rate of the method.

Description

Technical Field

[0001] This invention relates to the field of natural language processing, and in particular to a method for annotating named entities in ancient books on febrile diseases based on a fusion positional coding Transformer. Background Technology

[0002] Named entity recognition (NER) is a task in natural language processing that automatically extracts predefined types of entities from unstructured text, specifically extracting entity types and boundaries. Chinese NER differs from English NER; in English, a single character represents a word, while in Chinese, a word is often composed of multiple characters. This makes identifying entity boundaries more challenging. The text in the ancient Chinese medical text *Wenbing Guji* is written in Classical Chinese, which has two characteristics compared to modern Chinese: both character-level and word-level tokens contain important information, and it contains numerous special sentence structures, such as inverted sentences and judgment sentences.

[0003] Currently, Chinese named entity recognition mainly relies on gridded processing, with two main methods: the Lattice Lstm algorithm and the Lattice Transformer algorithm. The Lattice Lstm algorithm builds a language model using a bidirectional long short-term model, which can alleviate the long-range dependency problem of RNNs to some extent. However, it still requires a limit on sentence length and cannot handle complex and varied classical Chinese texts. The Lattice Transformer algorithm effectively solves the long-range dependency problem by establishing the correlation between every two words in a sentence using a self-attention mechanism. Since the self-attention mechanism cannot reflect word position information, it generally requires the word position information to be added to the input. Existing algorithms only use conventional relative position encoding and have not designed a special encoding method for the vertical domain of classical Chinese texts related to the Wenbing disease, thus performing poorly when handling special sentence structures. Summary of the Invention

[0004] In view of this, in order to solve the problems existing in the background technology, the purpose of this invention is to propose a named entity recognition method for ancient books on febrile diseases based on fusion positional coding Transformer. By designing a positional coding method for classical Chinese texts of ancient books on febrile diseases, special sentence structures and other problems can be handled more effectively without reducing the amount of character tokens and word tokens.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0006] A named entity recognition method for ancient texts on febrile diseases, comprising:

[0007] S1: Obtain ancient texts on febrile diseases, perform entity annotation on the text of sentences in the ancient texts on febrile diseases, and obtain entity annotation information;

[0008] S2: The text of sentences in the ancient book on febrile diseases is segmented using a word segmentation tool to obtain word-level granular text.

[0009] S3: Based on the word segmentation results of the ancient texts on febrile diseases, construct dictionaries and lexicons for characters and words respectively;

[0010] S4: Add the words obtained from the word segmentation of the text to the end of each sentence in the ancient book on febrile diseases, thereby reconstructing a new text sequence. Then, add two absolute position codes, one for the beginning and one for the end, to each character in the original text and each newly added word in the new text sequence.

[0011] S5: Construct an entity recognition model consisting of a BERT pre-trained model, a fusion positional encoding module, a Transformer layer, and a CRF layer. In this model, the new text sequence is first processed using the dictionary to generate the index input required by the BERT preprocessing model. After shallow extraction of syntactic and semantic information by the BERT preprocessing model, a distributed representation of each character and word in the new text sequence is output. Then, within the fusion positional encoding model, each character and word in the new text sequence is assigned a positional encoding with learnable parameters, integrating the head absolute positional encoding and tail absolute positional encoding of each character and word into their respective distributed representations, thus forming a distributed representation with positional encoding. Next, in the Transformer layer, the distributed representation of each character and word in the new text sequence with positional encoding is input into the Transformer network, where relative positional information is incorporated into the network structure to obtain the label probability space for each character and word. Finally, the sentence encoding result is input into the Conditional Random Field model to obtain the entity recognition result.

[0012] S6: Train the entity recognition model using the data obtained from the preprocessing in S1 to S4, and then use the trained entity recognition model to perform entity recognition on the ancient texts on febrile diseases to be identified.

[0013] Preferably, in step S1, entity annotation is performed on the ancient text of febrile diseases using annotation tools or manual annotation to obtain annotation information. The annotation information is then decomposed into sentences and annotations, and the sentences and annotations are stored in a sentence list and an annotation list, respectively.

[0014] Preferably, in step S2, the word segmentation tool jieba is used to decompose the text of sentences in ancient books on febrile diseases into word-level representations.

[0015] Preferably, in step S3, the information of the statement, entity annotation, and dictionary is saved as a josn file.

[0016] Preferably, step S4 specifically includes the following sub-steps:

[0017] Step S41: For each sentence in the ancient book on febrile diseases, add the words from the word segmentation results to the end of the original sentence text in their respective order in the original sentence text sequence to form a new text sequence;

[0018] Step S42: Assign two absolute position codes, the head and the tail, to each character of the original sentence text in the new text sequence. The head absolute position code and the tail absolute position code of each character are both the absolute position of the character in the original sentence text.

[0019] Step S43: Give two absolute position codes, the beginning and the end, to each word newly added to the end of the original text in the new text sequence. The beginning absolute position code of each word is the absolute position of the first character of the word in the original text, and the end absolute position code of each word is the absolute position of the last character of the word in the original text.

[0020] Preferably, in the new text sequence output by the BERT preprocessing model, the distributed representation of each character and word is 128-dimensional.

[0021] Preferably, the fusion position encoding algorithm executed in the fusion position encoding model is as follows:

[0022] In the new text sequence, each character and word is represented as a token. The distributed representation of each token is divided into pairs of adjacent dimensions. For each token's distributed representation, the absolute position codes of the token's head and tail are used as position information. Each pair of two-dimensional elements in the distributed representation undergoes an independent linear transformation with a learnable parameter θ. The formula for the linear transformation of any two dimensions [x, y] is as follows:

[0023]

[0024] In the formula: n represents the position information used for the current linear transformation, which is either the head absolute position code or the tail absolute position code of the token; the learnable parameter θ for different two-dimensional elements is not shared and is learned independently.

[0025] For each token's distributed representation, after linear transformation of all dimensions, two distributed representations X with positional encodings are obtained. The distributed representation X with header absolute positional encoding is obtained by linear transformation using the token's header absolute positional encoding as the positional information. head After linear transformation using the absolute position code of the token's tail as position information, a distributed representation X with a tail absolute position code is obtained. tail .

[0026] Preferably, in the Transformer layer, the distributed representation of each character and word in the new text sequence with position encoding is input into the Transformer network, and the relative position information is incorporated into the network structure to obtain the label probability space of each character and word. This specifically includes the following sub-steps:

[0027] S51: Input the distributed representation X with position encoding into the Transformer network, and perform a linear transformation on the distributed representation X of each token:

[0028] q = XW q k = XW k v=XW v

[0029] In the formula: q, k, and v represent the query, key, and value in the attention mechanism, respectively, and when X = X head When q, k, and v are respectively q head k head v head When X = X tail When q, k, and v are respectively q tail k tail v tail ;

[0030] S52: For any two tokens in the new text sequence, first calculate four sub-attentions:

[0031]

[0032]

[0033]

[0034]

[0035] In the formula: q represents the m-th token respectively head q tail ,and k represents the nth token respectively head v head k tail v tail ;

[0036] The four sub-attentions are then concatenated or superimposed to obtain the total attention Z;

[0037] S53: After obtaining the total attention Z between each pair of tokens in the new text sequence, it is fed into a fully connected layer; the fully connected layer has two layers, the first layer uses ReLU activation function, and the second layer uses a linear activation function. The output of the fully connected layer is expressed as:

[0038] FFN(Z) = max(0, ZW1+b1)W2+b2

[0039] In the formula: max represents the maximum value operation, W1 and W2 are two learnable weights, and b2 represents the bias;

[0040] S54: The output of the fully connected layer is passed through the decoding module of Transformer to obtain the label probability space of each token in the new text sequence.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] (1) The method of the present invention makes full use of Chinese vocabulary information and effectively avoids the final recognition error caused by word segmentation error;

[0043] (2) The method of the present invention provides an efficient positional coding method for the diverse special sentence structures in classical Chinese;

[0044] (3) The method of the present invention uses Transformer as the model backbone, which overcomes the problems of non-parallelism and long-range dependence of traditional RNN, LSTM and other models, thereby improving the accuracy and recall of entity recognition methods. Attached Figure Description

[0045] Figure 1 This is a flowchart of a named entity recognition method for ancient books on febrile diseases in an embodiment of the present invention;

[0046] Figure 2 This is an overall architecture diagram of a named entity recognition method for ancient books on febrile diseases according to the present invention;

[0047] Figure 3 This is a schematic diagram of the self-attention mechanism that integrates relative position information in this invention. Detailed Implementation

[0048] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited thereto.

[0049] like Figure 1 As shown, in a preferred embodiment of the present invention, a named entity recognition method for ancient books on febrile diseases is provided, the steps of which are as follows:

[0050] S1: Obtain ancient texts on febrile diseases, perform entity annotation on the text of sentences in the ancient texts on febrile diseases, and obtain entity annotation information.

[0051] In a preferred embodiment of the present invention, in step S1, the ancient text on febrile diseases can be annotated with entities using annotation tools or manual annotation to obtain annotation information. The annotation information is then decomposed into statements and annotations, and the statements and annotations are stored in a statement list and an annotation list, respectively.

[0052] S2: The text of sentences in the ancient book on febrile diseases is segmented using a word segmentation tool to obtain word-level granular text.

[0053] In a preferred embodiment of the present invention, the word segmentation tool jieba can be used in step S2 to decompose the text of sentences in ancient books on febrile diseases into word-level representations.

[0054] S3: Based on the word segmentation results of the ancient books on febrile diseases, dictionaries and thesauri can be constructed for characters and words respectively.

[0055] In a preferred embodiment of the present invention, in step S3, the information of the statements, entity annotations, and dictionaries can be saved as a josn file.

[0056] S4: Add the words obtained from the word segmentation of the text to the end of each sentence in the ancient book on febrile diseases, thereby reconstructing a new text sequence. Then, add two absolute position codes, one for the beginning and one for the end, to each character in the original text and each newly added word in the new text sequence.

[0057] In a preferred embodiment of the present invention, step S4 specifically includes the following sub-steps:

[0058] Step S41: For each sentence in the ancient book on febrile diseases, add the words from the word segmentation results to the end of the original sentence text in their respective order in the original sentence text sequence to form a new text sequence;

[0059] Step S42: Assign two absolute position codes, the head and the tail, to each character of the original sentence text in the new text sequence. The head absolute position code and the tail absolute position code of each character are both the absolute position of the character in the original sentence text.

[0060] Step S43: Give two absolute position codes, the beginning and the end, to each word newly added to the end of the original text in the new text sequence. The beginning absolute position code of each word is the absolute position of the first character of the word in the original text, and the end absolute position code of each word is the absolute position of the last character of the word in the original text.

[0061] S5: Construct an entity recognition model consisting of a BERT pre-trained model, a fusion positional encoding module, a Transformer layer, and a CRF layer. In this model, the new text sequence is first used to generate the index input required by the BERT preprocessing model using the dictionary and thesaurus. After the BERT preprocessing model extracts syntactic and semantic information at a shallow level, it outputs a distributed representation of each character and word in the new text sequence. Then, within the fusion positional encoding model, each character and word in the new text sequence is assigned a positional encoding with learnable parameters, so that the head absolute positional encoding and tail absolute positional encoding of each character and word are integrated into their respective distributed representations, thus forming a distributed representation with positional encoding. Next, in the Transformer layer, the distributed representation of each character and word in the new text sequence with positional encoding is input into the Transformer network, and relative positional information is incorporated into the network structure to obtain the label probability space of each character and word. Finally, the sentence encoding result is input into the Conditional Random Field model to obtain the entity recognition result.

[0062] In a preferred embodiment of the present invention, in the new text sequence output by the above-mentioned BERT preprocessing model, the distributed representation of each character and word is 128-dimensional.

[0063] In a preferred embodiment of the present invention, the fusion position encoding algorithm executed in the above-described fusion position encoding model is as follows:

[0064] In the new text sequence, each character and word is represented as a token. The distributed representation of each token is divided into pairs of adjacent dimensions. For each token's distributed representation, the absolute positional codes of the token's head and tail are used as positional information n. A linear transformation with a learnable parameter θ is performed independently on each pair of two-dimensional elements in the distributed representation. This linear transformation can be achieved by explicitly adding absolute positional information to the encoding. For a two-dimensional real vector [x, y] with positional information n, it can be treated as a complex number and multiplied by e. inθ We obtain the identity

[0065] (x+yi)e inθ =(x cos nθ-y sin nθ)+i(x sin nθ+y cos nθ)

[0066] Therefore, the linear transformation formula for any two dimensions [x,y] can be expressed as:

[0067]

[0068] In the formula: the symbol → represents linear transformation; n represents the position information used for the current linear transformation, which is either the head absolute position code or the tail absolute position code of the token.

[0069] The linear transformation described above can impart absolute positional information to [x, y]. Since the distributed representation of each token yields a 128-dimensional row vector, the coordinate transformation between each pair of dimensions can be performed differently to enhance flexibility. In other words, the learnable parameter θ for linear transformations between different pairs of elements is not shared; each is learned independently.

[0070] It's important to note that the aforementioned positions require separate absolute positional codes for the token's header and tail. Therefore, any linear transformation along any two dimensions [x, y] needs to be performed once using two different positional codes. For each token's distributed representation, after linear transformations across all dimensions, two distributed representations X with positional codes are obtained. The distributed representation X with a header absolute positional code is obtained by performing linear transformations using the token's header absolute positional code as the positional information. head After linear transformation using the absolute position code of the token's tail as position information, a distributed representation X with a tail absolute position code is obtained. tail .

[0071] In a preferred embodiment of the present invention, the distributed representation of each character and word in the new text sequence with position encoding is input into the Transformer network in the Transformer layer, and the relative position information is incorporated into the network structure to obtain the label probability space of each character and word. This specifically includes the following sub-steps:

[0072] S51: Input the obtained distributed representation X with position encoding into the Transformer network, and perform a linear transformation on the distributed representation X of each token:

[0073] q = XW q k = XW k v=XW v

[0074] In the formula: q, k, and v represent the query, key, and value in the attention mechanism, respectively, and when X = X head When q, k, and v are respectively q head k head v head When X = X tail When q, k, and v are respectively q tail k tail v tail ;

[0075] S52: Due to the self-attention mechanism, the above-mentioned e inθ This can help introduce relative position information. Assume q m and k nLet be two-dimensional row vectors at positions m and n. Since they are two-dimensional, they can be treated as complex numbers for calculation. The key to Attention lies in the inner product of vectors, which can be represented as a complex number:

[0076]

[0077] Where * represents the conjugate complex number, the multiplication on the right is ordinary complex multiplication, and Re[] represents taking the real part of the result.

[0078] After explicitly adding the absolute position information, then putting it into the inner product, we get...

[0079]

[0080] This means the inner product only depends on the relative position mn, thus completing the fusion of absolute and relative position information. Therefore, in this step, each token has two positional encodings: head and tail absolute position encodings, which, when combined, yield four sets of q, k, and v. For any two tokens in the new text sequence, four sub-attentions are first calculated:

[0081]

[0082]

[0083]

[0084]

[0085] In the formula: q represents the m-th token respectively head q tail ,and k represents the nth token respectively head v head k tail v tail ;

[0086] The four sub-attentions are then concatenated or superimposed to obtain the total attention Z.

[0087] S53: After obtaining the total attention Z between each pair of tokens in the new text sequence, it is fed into a fully connected layer; the fully connected layer has two layers, the first layer uses ReLU activation function, and the second layer uses a linear activation function. The output of the fully connected layer is expressed as:

[0088] FFN(Z) = max(0, ZW1+b1)W2+b2

[0089] In the formula: max represents the maximum value operation, W1 and W2 are two learnable weights, and b2 represents the bias;

[0090] S54: The output of the fully connected layer is passed through the decoding module of Transformer to obtain the label probability space of each token in the new text sequence.

[0091] S6: Train the entity recognition model using the data obtained from the preprocessing in S1 to S4, and then use the trained entity recognition model to perform entity recognition on the ancient texts on febrile diseases to be identified.

[0092] It should be noted that the training process of the above entity recognition model is existing technology and can be implemented by referring to existing model training methods; no limitations are imposed on it. In the actual application reasoning process, the ancient texts on febrile diseases to be recognized also need to be preprocessed in the same way as the training samples before being input into the model.

[0093] The named entity recognition method for ancient books on febrile diseases shown in S1 to S6 above will be applied to a specific example to demonstrate its technical effectiveness.

[0094] Example

[0095] In this example, the specific process of the named entity recognition method used for ancient books on febrile diseases is shown in S1 to S6 above, and will not be repeated in full. The following mainly focuses on... Figure 1 The flowchart shown illustrates the specific implementation of each step, along with case studies.

[0096] 1) Obtain ancient texts on febrile diseases, and use annotation tools combined with manual annotation to perform entity annotation on the ancient texts on febrile diseases to obtain annotation information;

[0097] 2) The sentence text in the ancient book on febrile diseases was segmented using the word segmentation tool jieba to obtain word-level granular text;

[0098] 3) Based on the word segmentation results, construct character and word index dictionaries and the dictionary according to the index position, and save the sentences, annotations, and dictionaries as files;

[0099] 4) Add long words related to febrile diseases from the word segmentation results after each sentence text (manual filtering can be performed), reconstructing the input sequence of sentence texts. Each original sentence text is reconstructed to form a new text sequence. Add two positional codes to each token in the new text sequence, namely the aforementioned head and tail absolute positional codes.

[0100] For example, the text is "Typhoid enters through the pores of the skin". After word segmentation, the common words related to warm diseases, "typhoid" and "pores of the skin", are extracted and added to the original text. "Shang" is the 1st character of the original text, "Han" is the 2nd character of the original text, "Mao" is the 4th character of the original text, and "Qiao" is the 5th character of the original text. Absolute position numbers are assigned to each character / word. The specific results are as Figure 2 .

[0101] 5) Generate the index input required for the BERT preprocessing model through dictionaries. Extract syntactic and semantic information through the BERT preprocessing layer, and output the distributed representations of each character and word (both are tokens) in the new text sequence.

[0102] Suppose the indexes of the seven characters "Typhoid enters through the pores of the skin" and the two words "typhoid" and "pores of the skin" are 0, 1, 2, 3, 4, 5, 6, 7, 8. First, represent them as one-hot vectors, such as "Shang" being [1, 0, 0,...]. Input each character token and word token into BERT to obtain 128-dimensional distributed representations, denoted as x1, x2, x3.....

[0103] 6) Use the fused position encoding algorithm to assign a parameterized encoding that combines position information to each character and word, so that the head absolute position encoding and tail absolute position encoding of each character and word are respectively incorporated into the corresponding distributed representations, thereby forming a distributed representation with position encoding. The fused position encoding algorithm is implemented by the above linear transformation formula:

[0104]

[0105] The specific linear transformation process will not be elaborated here. Since the learnable parameters θ for linear transformation of different two-dimensional elements are not shared and are learned independently, and the head absolute position encoding and tail absolute position encoding of the token need to be used as the position information n respectively, and the linear transformation is performed independently on each group of two-dimensional elements in the distributed representation. Therefore, each character token will actually have 64 learnable parameters θ, and each word token will actually have 128 learnable parameters θ.

[0106] Suppose: The distributed representation of "Han" is [a, b, c, d,....]. After transformation, the representation incorporating absolute position information is [a cos 2θ1 - b sin 2θ1, a sin 2θ1 + b cos 2θ1, c cos 2θ2 - d sin 2θ2, c sin 2θ2 + dcos 2θ2,......].

[0107] 7) Input the vector representations and position encodings of characters and words into the Transformer-based network, and the network structure incorporates relative position information to obtain the text encoding;

[0108] First through W q W k W v Find two sets of q, k, v for each token:

[0109] q = xW q ,k=xW k v=xW v

[0110] Then, through a self-attention mechanism, relative positional information is introduced, while simultaneously establishing information between characters / words. First, four sub-attentions are calculated:

[0111]

[0112]

[0113]

[0114]

[0115] The overall attention can be achieved by concatenating or simplified to adding the four sub-attentions. In this example, we use the addition method, that is:

[0116] Z = ∑Attention

[0117] After obtaining Z, it is sent to the next module, namely the fully connected layer. This fully connected layer has two layers. The first layer uses ReLU activation function, and the second layer uses a linear activation function, which can be expressed as:

[0118] FFN(Z) = max(0, ZW1+b1)W2+b2

[0119] 8) Input the output of the fully connected layer into the conditional random field model to obtain the entity recognition result, that is, the entity classification label corresponding to each token.

[0120] The above steps 5) to 8) constitute the framework of the entity recognition model. Using the sample data obtained in steps 1) to 4) as training data, the model can be trained and predicted. Thus, the model construction is completed.

[0121] 9) The trained entity recognition model can be used to identify ancient books on febrile diseases. Relevant ancient texts can be input, and after word segmentation and vectorization, the output will include the symptoms, location, medications, and treatment methods.

[0122] The following experiments are conducted based on the above method, and the results are presented. To demonstrate the effectiveness of this method, both this method and two general algorithms, lattice LSTM and Flat lattice Transformers, were used. Multiple experiments were conducted on two general datasets and a dataset of ancient texts on febrile diseases, and only the best results were displayed to demonstrate the superiority of this invention. Specific results are shown in Table 1, where the performance metrics selected are precision (pre), recall (rec), and F-score (f).

[0123] Table 1

[0124]

[0125] The results show that our method achieves excellent results on general datasets. Furthermore, it significantly outperforms the other two algorithms on the Wenbing Ancient Texts dataset, demonstrating the effectiveness of incorporating fusion positional encoding into the network.

[0126] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A named entity recognition method for ancient books on febrile diseases, characterized in that, include: S1: Obtain ancient texts on febrile diseases, perform entity annotation on the text of sentences in the ancient texts on febrile diseases, and obtain entity annotation information; S2: The text of sentences in the ancient book on febrile diseases is segmented using a word segmentation tool to obtain word-level granular text. S3: Based on the word segmentation results of the ancient texts on febrile diseases, construct dictionaries and lexicons for characters and words respectively; S4: Add the words obtained from the word segmentation of the text to the end of each sentence in the ancient book on febrile diseases, thereby reconstructing a new text sequence. Then, add two absolute position codes, one for the beginning and one for the end, to each character in the original text and each newly added word in the new text sequence. S5: Construct an entity recognition model consisting of a BERT pre-trained model, a fusion positional encoding module, a Transformer layer, and a CRF layer. In this model, the new text sequence is first processed using the dictionary to generate the index input required by the BERT preprocessing model. Then, after the BERT preprocessing model extracts syntactic and semantic information at a shallow level, it outputs a distributed representation of each character and word in the new text sequence. Next, within the fusion positional encoding model, each character and word in the new text sequence is assigned a positional encoding with learnable parameters, integrating the head absolute positional encoding and tail absolute positional encoding of each character and word into their respective distributed representations, thus forming a distributed representation with positional encoding. Then, in the Transformer layer, the distributed representation of each character and word in the new text sequence with positional encoding is input into the Transformer network, where relative positional information is incorporated into the network structure to obtain the label probability space for each character and word. Finally, the sentence encoding result is input into the Conditional Random Field model to obtain the entity recognition result. S6: Train the entity recognition model using the data obtained from the preprocessing of S1~S4, and use the trained entity recognition model to perform entity recognition on the ancient text of febrile diseases to be recognized. S4 specifically includes the following sub-steps: Step S41: For each sentence in the ancient book on febrile diseases, add the words from the word segmentation results to the end of the original sentence text in their respective order in the original sentence text sequence to form a new text sequence; Step S42: Assign two absolute position codes, the head and the tail, to each character of the original sentence text in the new text sequence. The head absolute position code and the tail absolute position code of each character are both the absolute position of the character in the original sentence text. Step S43: Give two absolute position codes, the beginning and the end, to each word newly added at the end of the original sentence text in the new text sequence. The absolute position code of the beginning of each word is the absolute position of the first character of the word in the original sentence text, and the absolute position code of the end of each word is the absolute position of the last character of the word in the original sentence text. The fusion position encoding algorithm executed in the fusion position encoding model is as follows: In the new text sequence, each character and word is represented as a token. The distributed representation of each token is divided into pairs of adjacent dimensions. For each token's distributed representation, the absolute position codes of the token's head and tail are used as position information. Each pair of two-dimensional elements in the distributed representation is then independently processed with learnable parameters. The linear transformation formula for any two dimensions [x,y] is as follows: → = + In the formula: n represents the positional information used for the current linear transformation, which is either the head absolute position code or the tail absolute position code of the token; and the learnable parameters for linear transformation of different two-dimensional elements. They do not share resources; each learns independently. For each token's distributed representation, after linear transformation of all dimensions, two distributed representations X with positional encodings are obtained. The distributed representation X with its header absolute positional encoding is obtained by linear transformation using the token's header absolute positional encoding as the positional information. head After linear transformation using the absolute position code of the token's tail as position information, a distributed representation X with a tail absolute position code is obtained. tail .

2. The named entity recognition method for ancient books on febrile diseases as described in claim 1, characterized in that, In step S1, entity annotation is performed on the ancient text of febrile diseases using annotation tools or manual annotation to obtain annotation information. The annotation information is decomposed into sentences and annotations, and the sentences and annotations are stored in a sentence list and an annotation list respectively.

3. The named entity recognition method for ancient books on febrile diseases as described in claim 1, characterized in that, In step S2, the word segmentation tool jieba is used to decompose the text of sentences in ancient books on febrile diseases into word-level representations.

4. The named entity recognition method for ancient books on febrile diseases as described in claim 1, characterized in that, In step S3, the information of the statement, entity annotation, and dictionary is saved as a JSON file.

5. The named entity recognition method for ancient books on febrile diseases as described in claim 1, characterized in that, In the new text sequence output by the BERT preprocessing model, the distributed representation of each character and word is 128-dimensional.

6. The named entity recognition method for ancient books on febrile diseases as described in claim 1, characterized in that, In the Transformer layer, the distributed representation of each character and word in the new text sequence with positional encoding is input into the Transformer network. The relative positional information is incorporated into the network structure to obtain the label probability space of each character and word. This specifically includes the following sub-steps: S51: Input the distributed representation X with position encoding into the Transformer network, and perform a linear transformation on the distributed representation X of each token: X , X , X In the formula: , , These represent the query, key, and value in the attention mechanism, respectively, and when X=X head hour , , They are respectively , , When X=X tail hour , , They are respectively , , ; S52: For any two tokens in the new text sequence, first calculate four sub-attentions: , = ) , = ) , = ) , = ) In the formula: , These represent the m-th token. , ,and , These represent the nth token. , , , ; The four sub-attentions are then concatenated or superimposed to obtain the total attention Z; S53: After obtaining the total attention Z between each pair of tokens in the new text sequence, feed it into the fully connected layer; The fully connected layer has two layers. The first layer uses ReLU activation function, and the second layer uses a linear activation function. The output of the fully connected layer is expressed as follows: FFN(Z)= + ) + In the formula: max represents the operation of taking the maximum value. and For two learnable weights, and Indicates bias; S54: The output of the fully connected layer is passed through the decoding module of Transformer to obtain the label probability space of each token in the new text sequence.

Images