A method and system for named entity recognition based on a geology pre-trained model

Abstract

The application provides a named entity recognition method and system based on a geological pre-training model, comprising the following steps: constructing a feature adapter, adapting and fusing a Chinese character feature vector set and a text semantic vector set through the feature adapter to obtain a strengthened text semantic vector set; constructing a knowledge fusioner, fusing a knowledge semantic vector set and the strengthened text semantic vector set through the knowledge fusioner to obtain a fused semantic vector set; and performing parameter optimization on the geological pre-training model through the fused semantic vector set and four pre-training tasks to obtain a final geological pre-training model. The application proposes a feature adapter, which integrates Chinese character features into the geological pre-training model in the pre-training stage and assists the model in learning semantics through Chinese character features; and proposes a knowledge fusioner, which integrates a geological knowledge graph into the geological pre-training model to enhance the knowledge of the geological pre-training model.

Description

Technical Field

[0001] This invention relates to the field of geological text recognition, and in particular to a named entity recognition method and system based on a geological pre-trained model. Background Technology

[0002] With the deepening of data-driven innovation, the knowledge mining and utilization of the ever-increasing volume of text data has become more urgent and important. Named Entity Recognition (NER) is one of the important tasks in Natural Language Processing (NLP), aiming to identify and classify entities from text. In geological NER, entities to be identified include minerals, rocks, strata, geological structures, and geological ages. Pre-trained Language Models (PLMs) are a crucial foundation for efficient and accurate named entity recognition. Using PLMs, complex textual knowledge can be accurately and completely mapped to a vector space, obtaining semantically rich representations that provide vector support for feature learning in downstream models (such as convolutional neural networks).

[0003] Current implementations of NER can be summarized into three categories: rule-based, traditional machine learning, and deep learning. Rule-based methods suffer from low efficiency and poor scalability due to the need for extensive domain knowledge and manual intervention. Machine learning methods, on the other hand, suffer from poor overall performance due to the requirement for extensive manual annotation and weak generalization ability. Deep learning methods based on PLMS are currently the mainstream approach.

[0004] However, a crucial foundation for feature learning in downstream models (such as BiLSTM-CRF) is the semantic richness of the representation vectors. This is essential for NER in vertical domains. Therefore, corresponding domain-specific pre-trained models have been trained to address this issue in many vertical domains, such as BioBERT in biomedicine, ClinicalBERT in clinical medicine, SCIBERT in science, FinBERT in finance, and GeoBERT in geography.

[0005] Furthermore, the strength of a pre-trained model's representational ability largely depends on the knowledge it learns during the pre-training phase. The aforementioned domain-specific pre-trained models incorporate unstructured knowledge from the text, while the Jaket model proposed by Yu et al. and SMedBERT proposed by Zhang et al. incorporate knowledge graphs, thus advancing the development of pre-trained models. However, pre-trained models require comprehensive knowledge for text representation, including the glyphs and pronunciation features of Chinese characters, semantic features of the text, and structured knowledge from the knowledge base. Existing models do not consider the impact of Chinese character features on text recognition, resulting in low accuracy in text recognition. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention provides a named entity recognition method based on a geological pre-trained model, comprising the following steps:

[0007] S1: Obtain the geological named entity recognition corpus, geological knowledge graph and geological text corpus, and construct a Chinese character feature corpus through the geological text corpus;

[0008] S2: Represent the geological knowledge map using a graph neural network to obtain a set of knowledge semantic vectors;

[0009] S3: Represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors;

[0010] S4: Represent the geological text corpus using the Transformer encoder to obtain a set of text semantic vectors;

[0011] S5: Construct a feature adapter to adapt and fuse the set of Chinese character feature vectors and the set of text semantic vectors to obtain an enhanced set of text semantic vectors;

[0012] S6: Construct a knowledge fusion engine to fuse the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set;

[0013] S7: The parameters of the geological pre-training model are fine-tuned by fusing the semantic vector set with four pre-training tasks to obtain the final geological pre-training model.

[0014] S8: Combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

[0015] Preferably, the Chinese character feature vectors in the set of Chinese character feature vectors include: pinyin feature vectors, radical feature vectors, and stroke feature vectors.

[0016] Preferably, step S5 specifically includes:

[0017] S51: Let A denote the text semantic vector of the α-th Chinese character. α The feature vector of pinyin is denoted as P. α The feature vector of the radical is denoted as Y. α The stroke feature vector is denoted as B. α P α and B α The expression is:

[0018] P α =(P α1 ,P α2 ,P α3 )

[0019] B α = (B α1 ,......,B αβ ,......,B αm )

[0020] Where α is the character number, β is the stroke number, m is the maximum number of strokes, and P... α1 Let P be the initial consonant vector of the α-th Chinese character. α2 Let P be the vowel vector of the α-th Chinese character. α3 Let be the tone vector of the α-th Chinese character;

[0021] S52: Combine the text semantic vector with the pinyin feature vector P respectively. α and stroke feature vector B α A nonlinear transformation is performed to calculate the pinyin feature parameter V. Pαβ and stroke feature parameters V Bαβ ;

[0022] S53: Calculate and obtain the pinyin feature vector P α The correlation M of the α-th Chinese character Pα and stroke feature vector B α The correlation M of the α-th Chinese character Bα ;

[0023] S54: Via V Pαβ V Bαβ M Pα and M Bα Calculate and obtain the pinyin weighted feature vector H pα And stroke-weighted feature vector H Bα The calculation formula is:

[0024]

[0025]

[0026] S55: H pα H Bα Y α and A α The enhanced text semantic vector is obtained by fusion, and the calculation formula is as follows:

[0027] G α =H pα +Y α +H Bα +A α

[0028] Among them, G α Let α be the enhanced text semantic vector of the αth Chinese character.

[0029] Preferably, the pinyin feature parameter V Pαβ and stroke feature parameters V Bαβ The calculation formula is:

[0030] V Pαβ =W p2 (tanh(W P1 P αβ +b P1 ))+b p2

[0031] V Bαβ =W B2 (tanh(W B1 B αβ +b B1 ))+b B2

[0032] Among them, W P1 W B1 These are the weight matrices for the first transformation of pinyin and stroke count, respectively, with dimensions of [missing information]. W p2 W B2 These are the weight matrices for the second transformation of pinyin and strokes, respectively, with dimensions of [missing information]. γ、 These represent the word embedding size and hidden layer size of BERT, respectively; b P1 and b p2 b is the bias parameter for the pinyin in the two transformations; B1 and b B2 The offset parameter of the stroke in the two transformations.

[0033] Preferably, the correlation M Pα and correlation M Bα The calculation formula is:

[0034]

[0035]

[0036] Among them, V Pα =(V pα1 V pα2 V pα3 V Bα =(V Bα1 ,......,V Bαβ ,......,V Bαm ), W attnP and W attnB These are bilinear weight matrices for pinyin and stroke count, respectively.

[0037] Preferably, step S6 specifically includes:

[0038] S61: The enhanced text semantic vector is denoted as G. i via G i The obtained text is denoted as The dimension is G is a real matrix; i The corresponding knowledge semantic vector is denoted as Where C represents the number of candidate words corresponding to each word in the knowledge graph, i is the word number in the text, j is the element number in the geological knowledge, and dim K Let K be the dimension of the knowledge semantic vector.

[0039] S62: Calculate G i and Corresponding first attention score The calculation formula is:

[0040]

[0041] Among them, U A This is the first intermediate vector that can be trained;

[0042] S63: Introducing Identifier Vectors Represented as:

[0043]

[0044] Among them, G [CLS] Represents Token [CLS] The vector, This is the second intermediate vector that can be trained;

[0045] G is obtained through attention mechanism calculation. i and Second attention score The calculation formula is:

[0046]

[0047] in, This is the third intermediate vector that can be obtained through training;

[0048] S64: Through and Calculate the weight of external knowledge With the weight of the identifier vector The calculation formula is:

[0049]

[0050] S65: Through and Calculate the aggregation parameter N i The calculation formula is:

[0051]

[0052] S66: Through N i and G i Calculate and obtain the fused semantic vector F i The calculation formula is:

[0053]

[0054] Preferably, the four pre-training tasks include: mask prediction model training, next sentence prediction training, entity classification training, and relationship prediction training.

[0055] A named entity recognition system based on a geological pre-trained model includes:

[0056] The corpus construction module is used to acquire a geological named entity recognition corpus, a geological knowledge graph, and a geological text corpus, and to construct a Chinese character feature corpus using the geological text corpus;

[0057] The knowledge semantic vector acquisition module is used to represent the geological knowledge map through a graph neural network and obtain a set of knowledge semantic vectors.

[0058] The Chinese character feature vector acquisition module is used to represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors;

[0059] The text semantic vector acquisition module is used to represent the geological text corpus through the Transformer encoder to obtain a set of text semantic vectors;

[0060] The text semantic vector enhancement module is used to construct a feature adapter, which adapts and fuses the set of Chinese character feature vectors with the set of text semantic vectors to obtain an enhanced set of text semantic vectors.

[0061] The vector fusion module is used to build a knowledge fusion engine, which fuses the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set.

[0062] The parameter tuning module is used to tune the parameters of the geological pre-training model by fusing the semantic vector set with four pre-training tasks, so as to obtain the final geological pre-training model.

[0063] The recognition model training module is used to combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

[0064] A storage device that stores instructions and data for implementing the named entity recognition method based on a geological pre-training model.

[0065] A named entity recognition device based on a geological pre-trained model includes: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the named entity recognition method based on the geological pre-trained model.

[0066] The present invention has the following beneficial effects:

[0067] 1. This invention proposes a feature adapter that integrates Chinese character features into a geological pre-training model during the pre-training stage. By using Chinese character features to assist the model in learning semantics, the training effect of the model is improved.

[0068] 2. This invention proposes a knowledge fusion device that integrates geological knowledge graphs into geological pre-training models, thereby enhancing the knowledge of the geological pre-training models and improving their representation capabilities.

[0069] 3. This invention combines the final geological pre-trained model with the BiLSTM-CRF network, further improving the accuracy of geological text recognition. Attached Figure Description

[0070] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0071] Figure 2 Structure diagram of the geological named entity recognition model;

[0072] The implementation, functional features, and advantages of this invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0073] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0074] Reference Figure 1 This invention provides a named entity recognition method based on a geological pre-trained model, comprising the following steps:

[0075] S1: Obtain the geological named entity recognition corpus, geological knowledge graph and geological text corpus, and construct a Chinese character feature corpus through the geological text corpus;

[0076] S2: Represent the geological knowledge map using a graph neural network to obtain a set of knowledge semantic vectors;

[0077] S3: Represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors;

[0078] S4: Represent the geological text corpus using the Transformer encoder to obtain a set of text semantic vectors;

[0079] S5: Construct a feature adapter to adapt and fuse the set of Chinese character feature vectors and the set of text semantic vectors to obtain an enhanced set of text semantic vectors;

[0080] S6: Construct a knowledge fusion engine to fuse the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set;

[0081] S7: The parameters of the geological pre-training model are fine-tuned by fusing the semantic vector set with four pre-training tasks to obtain the final geological pre-training model.

[0082] S8: Combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

[0083] Furthermore, the geological text corpus in step S1 has the following three sources: (1) Chinese geological reports. Geological reports are rich in descriptive texts about the properties, distribution, and relationships of elements such as minerals, rocks, and geological structures, and the content is concentrated. Therefore, this invention collects a large number of geological reports and designs an extraction program to automatically extract them. (2) Abstracts of geological journal articles. This invention collects abstracts of 34 geological journal articles. The characteristic of this part is that there are fewer elements in the text that interfere with model training, and after extraction, it can be directly used for training without complicated data cleaning. (3) Abstracts of geoscience dissertations. Abstracts of geoscience dissertations are also included in the corpus of this invention. This invention collects abstracts of geoscience dissertations from 161 universities and research institutions to further expand the corpus. In the process of extracting text, this invention ignores table information, image information, and other non-geological or non-sentence content. In order to ensure the fluency of sentences and the high weight of geological content, this invention deletes incoherent sentences in the corpus and converts special symbols such as serial numbers, marks, and values ​​in the text into the letter N. Through the above preprocessing, a high-quality geological text corpus is constructed.

[0084] Furthermore, geological texts contain both geological terminology and general domain information, and BERT has already demonstrated its superiority in general domains. Therefore, this invention uses BERT to initialize the parameters of a geological pre-trained model, thereby further incorporating geological features and knowledge into the parameters trained in the general domain, effectively enhancing the geological pre-trained model's ability to represent geological texts in a comprehensive and effective manner.

[0085] Further, the Chinese character feature vectors in the Chinese character feature vector set include: pinyin feature vectors, radical feature vectors, and stroke feature vectors.

[0086] Specifically, based on the geological text corpus, a corresponding Chinese character feature corpus can be constructed. The features contained in Chinese characters are divided into three parts: pinyin, radicals, and strokes. Combining Chinese character features can effectively improve the performance of the model. For example, the Chinese characters "士" and "土" have similar structures, and combining pinyin can assist the model in learning the correct semantics of Chinese characters. At the same time, many Chinese characters are pictographs, such as "火" and "山". Incorporating the structural features of Chinese characters into the geological pre-training model can enhance the representation ability of the geological pre-training model.

[0087] Further, for multiple features of Chinese characters, the present invention proposes a feature adapter, which is added between the encoders of each layer of Transformer in BERT to complete the adaptation and fusion of semantics and Chinese character features during the training process, as shown in step S5;

[0088] Step S5 is specifically as follows:

[0089] S51: Denote the text semantic vector of the α-th Chinese character as A α , the pinyin feature vector as P α , the radical feature vector as Y α , the stroke feature vector as B α , P α and B α The expressions of are:

[0090] P α =(P α1 ,P α2 ,P α3 )

[0091] B α ​​​​​​​​​​​​​​​​​​​​​​​​​​​

[0094] S53: Calculate and obtain the pinyin feature vector P α The correlation M of the α-th Chinese character Pα and stroke feature vector B α The correlation M of the α-th Chinese character Bα ;

[0095] S54: Via V Pαβ V Bαβ M Pα and M Bα Calculate and obtain the pinyin weighted feature vector H pα And stroke-weighted feature vector H Bα The calculation formula is:

[0096]

[0097]

[0098] S55: H pα H Bα Y α and A α The enhanced text semantic vector is obtained by fusion, and the calculation formula is as follows:

[0099] G α =H pα +Y α +H Bα +A α

[0100] Among them, G α Let α be the enhanced text semantic vector of the αth Chinese character.

[0101] Furthermore, in the feature adapter, in order to align the pinyin feature vector and the stroke feature vector with the semantic vector respectively, a non-linear transformation is required on the pinyin feature vector and the stroke feature vector;

[0102] Pinyin feature parameter V Pαβ and stroke feature parameters V Bαβ The calculation formula is:

[0103] V Pαβ =W p2 (tanh(W P1 P αβ +b P1 ))+b p2

[0104] V Bαβ =W B2 (tanh(W B1 B αβ +bB1 ))+b B2

[0105] Among them, W P1 W B1 These are the weight matrices for the first transformation of pinyin and stroke count, respectively, with dimensions of [missing information]. W p2 W B2 These are the weight matrices for the second transformation of pinyin and strokes, respectively, with dimensions of [missing information]. γ、 These represent the word embedding size and hidden layer size of BERT, respectively; b P1 and b p2 b is the bias parameter for the pinyin in the two transformations; B1 and b B2 The offset parameter of the stroke in the two transformations.

[0106] Furthermore, since the contribution of each feature vector to the Chinese character is different, an attention mechanism is introduced to calculate the correlation of each feature vector with respect to the Chinese character.

[0107] Correlation M Pα and correlation M Bα The calculation formula is:

[0108]

[0109]

[0110] Among them, V Pα =(V pα1 V pα2 V pα3 V Bα =(V Bα1 ,......,V Bαβ ,......,V Bαm ), W attnP and W attnB These are bilinear weight matrices for pinyin and stroke count, respectively.

[0111] Furthermore, in order to effectively integrate the knowledge semantic vector of the geological knowledge graph with the text semantic vector based on Chinese character feature enhancement, this invention designs a knowledge fusion machine to perform the fusion operation, as shown in step S6;

[0112] Step S6 is as follows:

[0113] S61: The enhanced text semantic vector is denoted as G. i via G i The obtained text is denoted as The dimension is G is a real matrix;i The corresponding knowledge semantic vector is denoted as Where C represents the number of candidate words corresponding to each word in the knowledge graph, i is the word number in the text, j is the element number in the geological knowledge, and dim K Let K be the dimension of the knowledge semantic vector.

[0114] S62: Calculate G i and Corresponding first attention score The calculation formula is:

[0115]

[0116] Among them, U A This is the first intermediate vector that can be trained;

[0117] S63: Introducing Identifier Vectors Represented as:

[0118]

[0119] Among them, G [CLS] Represents Token [CLS] The vector, This is the second intermediate vector that can be trained;

[0120] G is obtained through attention mechanism calculation. i and Second attention score The calculation formula is:

[0121]

[0122] in, This is the third intermediate vector that can be obtained through training;

[0123] S64: Through and Calculate the weight of external knowledge With the weight of the identifier vector The calculation formula is:

[0124]

[0125] S65: Through and Calculate the aggregation parameter N i The calculation formula is:

[0126]

[0127] S66: Through N iand G i Calculate and obtain the fused semantic vector F i The calculation formula is:

[0128]

[0129] Furthermore, the four pre-training tasks include: mask prediction model training, next sentence prediction training, entity classification training, and relationship prediction training.

[0130] Specifically, after obtaining a set of fused semantic vectors rich in multifaceted features and knowledge, the geological pre-trained model is trained sequentially using the fused semantic vector set to perform mask prediction model (MLM), next sentence prediction (NSP), entity classification, and relation prediction training. The model is pre-trained from four levels: word, sentence, entity, and relation, respectively, to obtain the final geological pre-trained model (GeoBERT-KE).

[0131] Furthermore, compared to general-domain pre-trained models, the final geological pre-trained model (GeoBERT-KE) effectively reduces semantic loss, achieves more accurate semantic representation, and provides richer semantic information in the representation of geological text. To further enhance text recognition accuracy, this invention constructs a neural network structure combining the final geological pre-trained model with BiLSTM-CRF, namely GeoBERT-KE-BiLSTM-CRF, as follows: Figure 2 As shown;

[0132] GeoBERT-KE is used to map the semantics of the text input to the system to a high-dimensional space vector to complete text encoding. BiLSTM takes the text encoding result as input to learn the dependencies on the observation sequence, and finally uses CRF to learn the relationship of the state sequence and obtain the answer. BiLSTM consists of a forward and a backward Long Short-Term Memory (LSTM) network. This structure can learn the features of geological entities by combining the geological text context, and can output a score for each label. The score is input into CRF, which is used to add some constraints learned in training to the final predicted label to ensure the effectiveness of the prediction result.

[0133] The neural network structure was fully trained using a geological named entity recognition corpus to obtain a geological named entity recognition model. The entities in the text were labeled using the prediction results of the geological named entity recognition model, and the recognition results were presented in a visual manner and saved.

[0134] This invention provides a named entity recognition system based on a geological pre-trained model, comprising:

[0135] The corpus construction module is used to acquire a geological named entity recognition corpus, a geological knowledge graph, and a geological text corpus, and to construct a Chinese character feature corpus using the geological text corpus;

[0136] The knowledge semantic vector acquisition module is used to represent the geological knowledge map through a graph neural network and obtain a set of knowledge semantic vectors.

[0137] The Chinese character feature vector acquisition module is used to represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors;

[0138] The text semantic vector acquisition module is used to represent the geological text corpus through the Transformer encoder to obtain a set of text semantic vectors;

[0139] The text semantic vector enhancement module is used to construct a feature adapter, which adapts and fuses the set of Chinese character feature vectors with the set of text semantic vectors to obtain an enhanced set of text semantic vectors.

[0140] The vector fusion module is used to build a knowledge fusion engine, which fuses the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set.

[0141] The parameter tuning module is used to tune the parameters of the geological pre-training model by fusing the semantic vector set with four pre-training tasks, so as to obtain the final geological pre-training model.

[0142] The recognition model training module is used to combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

[0143] The present invention provides a storage device that stores instructions and data for implementing the named entity recognition method based on a geological pre-training model.

[0144] This invention provides a named entity recognition device based on a geological pre-trained model, comprising: a processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the named entity recognition method based on the geological pre-trained model.

[0145] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0146] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In the unit claims listing several devices, several of these devices may be embodied by the same hardware item. The use of the terms first, second, and third, etc., does not indicate any order and can be interpreted as identifiers.

[0147] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A named entity recognition method based on a geological pre-trained model, characterized in that, Including the following steps: S1: Obtain the geological named entity recognition corpus, geological knowledge graph and geological text corpus, and construct a Chinese character feature corpus through the geological text corpus; S2: Represent the geological knowledge map using a graph neural network to obtain a set of knowledge semantic vectors; S3: Represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors; the Chinese character feature vectors in the set of Chinese character feature vectors include: pinyin feature vectors, radical feature vectors, and stroke feature vectors; S4: Represent the geological text corpus using the Transformer encoder to obtain a set of text semantic vectors; S5: Construct a feature adapter to adapt and fuse the set of Chinese character feature vectors and the set of text semantic vectors to obtain an enhanced set of text semantic vectors; S6: Construct a knowledge fusion engine to fuse the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set; Step S6 is as follows: S61: Enhanced text semantic vector denoted as ,pass The obtained text is denoted as , The dimension is A real matrix; The corresponding knowledge semantic vector is denoted as , ;in, This represents the number of candidate words corresponding to each word in the knowledge graph. The word number in the text. These are the element designations used in geological knowledge. Knowledge semantic vector The dimension; S62: Calculated and Corresponding first attention score The calculation formula is: in, This is the first intermediate vector that can be trained; S63: Introducing Identifier Vectors , is represented as: in, express The vector, This is the second intermediate vector that can be trained; Obtained through attention mechanism calculation and Second attention score The calculation formula is: ； in, This is the third intermediate vector that can be obtained through training; S64: Through and Calculate the weight of external knowledge With the weight of the identifier vector The calculation formula is: S65: Through and Calculate the aggregation parameters The calculation formula is: S66: Through and Calculate and obtain the fused semantic vector The calculation formula is: S7: The parameters of the geological pre-training model are fine-tuned by fusing the semantic vector set with four pre-training tasks to obtain the final geological pre-training model. S8: Combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

2. The named entity recognition method based on a geological pre-trained model according to claim 1, characterized in that, Step S5 is as follows: S51: The first The text semantic vector of each Chinese character is denoted as . The pinyin feature vector is denoted as The eigenvector of the radical is denoted as The stroke feature vector is denoted as , and The expression is: in, For Chinese characters, Here, is the stroke number, and m is the maximum number of strokes. For the first The initial consonant vector of each Chinese character For the first The vowel vector of each Chinese character For the first The tone vector of each Chinese character; S52: Combine the text semantic vector with the pinyin feature vector respectively. and stroke feature vector Perform a nonlinear transformation to calculate the pinyin feature parameters. and stroke feature parameters ; S53: Calculate and obtain the Pinyin feature vector For the first Correlation of individual Chinese characters and stroke feature vectors For the first Correlation of individual Chinese characters ; S54: Through , , and Calculate and obtain the weighted feature vector of Pinyin Stroke-weighted feature vector The calculation formula is: S55: Will , , and The enhanced text semantic vector is obtained by fusion, and the calculation formula is as follows: in, For the first Enhanced text semantic vectors for each Chinese character.

3. The named entity recognition method based on a geological pre-trained model according to claim 2, characterized in that, Pinyin feature parameters and stroke feature parameters The calculation formula is: in, , These are the weight matrices for the first transformation of pinyin and stroke count, respectively, with dimensions of [missing information]. ; , These are the weight matrices for the second transformation of pinyin and strokes, respectively, with dimensions of [missing information]. ; , These represent the word embedding size and hidden layer size of BERT, respectively. and This represents the bias parameter for the pinyin during the two transformations; and The offset parameter of the stroke in the two transformations.

4. The named entity recognition method based on a geological pre-trained model according to claim 2, characterized in that, Correlation and correlation The calculation formula is: in, , , and These are bilinear weight matrices for pinyin and stroke count, respectively.

5. The named entity recognition method based on a geological pre-trained model according to claim 1, characterized in that, The four pre-training tasks include: mask prediction model training, next sentence prediction training, entity classification training, and relationship prediction training.

6. A named entity recognition system based on a geological pre-training model, used to implement the named entity recognition method based on a geological pre-training model as described in any one of claims 1-5, characterized in that, include: The corpus construction module is used to acquire a geological named entity recognition corpus, a geological knowledge graph, and a geological text corpus, and to construct a Chinese character feature corpus using the geological text corpus; The knowledge semantic vector acquisition module is used to represent the geological knowledge map through a graph neural network and obtain a set of knowledge semantic vectors. The Chinese character feature vector acquisition module is used to represent the Chinese character feature corpus using BERT to obtain a set of Chinese character feature vectors; The text semantic vector acquisition module is used to represent the geological text corpus through the Transformer encoder to obtain a set of text semantic vectors; The text semantic vector enhancement module is used to construct a feature adapter, which adapts and fuses the set of Chinese character feature vectors with the set of text semantic vectors to obtain an enhanced set of text semantic vectors. The vector fusion module is used to build a knowledge fusion engine, which fuses the knowledge semantic vector set with the enhanced text semantic vector set to obtain a fused semantic vector set. The parameter tuning module is used to tune the parameters of the geological pre-training model by fusing the semantic vector set with four pre-training tasks, so as to obtain the final geological pre-training model. The recognition model training module is used to combine the final geological pre-trained model with the BiLSTM-CRF neural network structure and train it fully on the geological named entity recognition corpus to obtain the geological named entity recognition model.

7. A storage device, characterized in that: The storage device stores instructions and data to implement the named entity recognition method based on a geological pre-trained model as described in any one of claims 1 to 5.

8. A named entity recognition device based on a geological pre-trained model, characterized in that: include: A processor and a storage device; the processor loads and executes instructions and data in the storage device to implement the named entity recognition method based on a geological pre-trained model as described in any one of claims 1 to 5.

Images