Geological entity recognition method and device based on deep learning, and terminal

Abstract

The invention relates to the technical field of geological text recognition, and particularly discloses a geological entity recognition method and device based on deep learning, and a terminal, and the method comprises the steps: obtaining geological text data, and constructing a corpus according to the geological text data; marking geological entities in the corpus to generate a data set; training a neural network model based on the data set to obtain a trained entity recognition model; and performing geological entity recognition on the geological text data through the trained entity recognition model. According to the method, a clear corpus is constructed through labeling according to related geological text data, a foundation is laid for subsequent neural network model training, meanwhile, training is performed based on the labeled corpus, the obtained entity recognition model can more accurately recognize the geological entity from the text data, the accuracy is higher, and the method is suitable for popularization and application. And the working efficiency of geological entity recognition is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of geological text recognition, and in particular to a geological entity recognition method, device and terminal based on deep learning. Background Art

[0002] With the continuous enrichment of knowledge in different fields, the huge amount of information urgently needs to be transformed from a scattered form of existence attached to physical objects to a centralized, data-based storage form. Knowledge graph technology provides a convenient and efficient solution to this problem. The named entity recognition task is a basic task of NLP (Natural Language Processing) work, and it is also a prerequisite for the construction of knowledge graphs. It plays a core role in the construction of knowledge graphs. In the existing geological entity recognition, with the development of deep learning technology, this efficient recognition technology has gradually become the main method of current NLP research. However, due to the complexity of the data in the original geological field data and the lack of specific annotations, the named entity recognition error is large and the accuracy is not high, which brings certain limitations. Summary of the invention

[0003] In order to solve the problems of complex geological data and low entity recognition accuracy in existing geological entity recognition, the present invention provides a geological entity recognition method, device and terminal based on deep learning.

[0004] In order to achieve the above-mentioned object of the invention, the present invention provides the following technical solutions:

[0005] In a first aspect, the present invention provides a method for geological entity recognition based on deep learning, the method comprising:

[0006] Acquiring geological text data, and constructing a corpus based on the geological text data;

[0007] annotating geological entities in the corpus to generate a data set;

[0008] Training a neural network model based on the data set to obtain a trained entity recognition model;

[0009] The geological entity recognition is performed on the geological text data through the trained entity recognition model.

[0010] According to a specific implementation, in the above method, constructing a corpus based on the geological text data includes:

[0011] Converting the geological text data into a text document;

[0012] Screening out noise data in the text document;

[0013] The filtered text documents are segmented to generate a corpus.

[0014] According to a specific implementation, in the above method, labeling the geological entities in the corpus includes:

[0015] constructing labels based on the categories of the geological entities;

[0016] The geological entities are labeled using the BIO labeling method according to the labels.

[0017] According to a specific implementation, in the above method, the annotated corpus is converted into a format in which each line contains a character and a corresponding label to generate a data set.

[0018] According to a specific implementation, in the above method, the categories of geological entities include engineering types, geological characteristics, surveying techniques, surveying tools, geological data, samples and experiments.

[0019] According to a specific implementation, in the above method, the entity recognition model adopts a BERT-BiLSTM-CRF model, and the training of the BERT-BiLSTM-CRF model includes:

[0020] Dividing the data set into a training set, a validation set, and a test set;

[0021] Use precision and recall as evaluation indicators;

[0022] Multiple groups of parameter combinations of the BERT-BiLSTM-CRF model are set, the BERT-BiLSTM-CRF model is trained based on the training set, the parameter combination with the highest evaluation index is obtained from the BERT-BiLSTM-CRF model based on the validation set, and the BERT-BiLSTM-CRF model is tested according to the test set to obtain a trained entity recognition model.

[0023] According to a specific implementation, in the above method, the parameter combination includes batch size and number of iterations.

[0024] According to a specific implementation, in the above method, the evaluation index adopts the harmonic mean of precision and recall.

[0025] In a second aspect, the present invention provides a geological entity recognition device based on deep learning, comprising:

[0026] Data acquisition module, used to obtain geological text data;

[0027] A corpus processing module, used for constructing a corpus based on the geological text data;

[0028] A data annotation module, used to annotate the geological entities in the corpus and generate a data set;

[0029] A model training module, used to train a neural network model based on the data set to obtain a trained entity recognition model;

[0030] The entity recognition module is used to perform geological entity recognition on geological text data through the trained entity recognition model.

[0031] In a third aspect, the present invention provides a geological entity identification terminal based on deep learning, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and execute the computer program so that the device executes the method described in any one of claims 1 to 8.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] The present invention constructs a clear corpus through annotation based on relevant geological text data, which lays a foundation for subsequent neural network model training. At the same time, based on the annotated corpus for training, the entity recognition model obtained by the present invention can more accurately identify geological entities from text data with higher accuracy, thereby improving the work efficiency of geological entity recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a flow chart of a geological entity recognition method based on deep learning provided in an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of a text conversion process provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of the marking entity rule provided by an embodiment of the present invention;

[0037] Figure 4 A schematic diagram of a partial example of an annotated corpus provided by an embodiment of the present invention;

[0038] Figure 5 A schematic diagram of the BiLSTM model structure provided in an embodiment of the present invention;

[0039] Figure 6 A schematic diagram of the structure of an entity recognition model provided by an embodiment of the present invention;

[0040] Figure 7 A schematic diagram of the LSTM model structure provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The present invention is further described in detail below in conjunction with test examples and specific implementation methods. However, this should not be understood as the scope of the above subject matter of the present invention being limited to the following embodiments, and all technologies realized based on the content of the present invention belong to the scope of the present invention.

[0042] Please refer to Figure 1 , which shows a schematic flow chart of a geological entity recognition method based on deep learning provided by an embodiment of the present invention, the method comprising:

[0043] S1. Acquire geological text data, and construct a corpus based on the geological text data.

[0044] S2. Annotating geological entities in the corpus to generate a data set.

[0045] S3. Training a neural network model based on the data set to obtain a trained entity recognition model.

[0046] S4. Perform geological entity recognition on geological text data through the trained entity recognition model.

[0047] It can be understood that the embodiment of the present invention constructs a clear corpus through annotation based on relevant geological text data, which lays the foundation for subsequent neural network model training. At the same time, based on the annotated corpus for training, the entity recognition model obtained by the present invention can more accurately identify geological entities from text data with higher accuracy, thereby improving the work efficiency of geological entity recognition.

[0048] Each step is introduced and explained in detail below.

[0049] Further, as described in S1 above, geological text data is obtained, and a corpus is constructed based on the geological text data. The geological text data includes the main content of paper data in the geological field. In the embodiment of the present invention, the paper data is converted into a text in PDF format by photographing and scanning. Further, the database is constructed based on the geological text data, including:

[0050] S11, converting the geological text data into a text document.

[0051] In the embodiment of the present invention, OCR recognition technology is used to convert the content of geological text data in PDF format into a text document for storage.

[0052] S12: Screen out noise data in the text document.

[0053] This step mainly screens out English, mathematical formulas and corresponding mathematical symbols in geological text materials, retaining only text content related to geological entities to further improve the clarity of the corpus.

[0054] Specifically, please refer to Figure 2 , which shows a schematic diagram of the text conversion process provided by an embodiment of the present invention. Specifically, in a possible implementation method, the embodiment of the present invention recognizes the geological text data converted into PDF through API calling OCR, and then converts it into a text document. OCR (Optical Character Recognition) is a recognition technology that can recognize and extract text information from image files and convert it into editable and searchable data. In the OCR recognition process of geological text data, it specifically includes: recognizing text lines in the image and segmenting them, and further segmenting the text lines into separate characters or words. Extracting features from the characters in the image, such as shape, edge, texture, etc., and then matching the extracted features with the feature library of known characters to recognize the characters. The recognized text is output as a file in the format of TXT, PDF, etc. Through these steps, OCR technology can convert geological text data in PDF format into editable and searchable text data, greatly improving the efficiency and accuracy of data processing.

[0055] S13 segments the text documents after being screened out to generate a corpus.

[0056] This step specifically includes checking and determining the sentence characteristics of the collected text, selecting appropriate segmentation standards, segmenting by hand or program, and saving the segmented text in the form of a text document to obtain a corpus.

[0057] Further, as described in S2 above, the geological entities in the corpus are annotated to generate a data set, which specifically includes:

[0058] S21. Constructing labels based on the categories of the geological entities.

[0059] This step is used to organize and determine the text logic in the corpus, and determine the category of geological entities based on the text logic of the corpus. The geological entity categories include engineering types, geological characteristics, surveying techniques, surveying tools, geological data, samples and experiments. At the same time, a label that is easy to extract is constructed for each category. For example, the label of "engineering type" is "PC" and the label of "geological characteristics" is "GF". The label is only a schematic. In the actual implementation process, the label that can distinguish each type of geological entity can be used as a part of the embodiment of the present invention.

[0060] S22, labeling geological entities using BIO labeling method according to the labels.

[0061] Please refer to Figure 3 , which shows a schematic diagram of labeling entity rules provided by an embodiment of the present invention.

[0062] S23. Convert the annotated corpus into a format where each line contains one character and a corresponding label to generate a data set.

[0063] Please refer to Figure 4 , which shows a schematic diagram of a partial example of an annotated corpus provided by an embodiment of the present invention. Among them, two adjacent sentences are separated by a line jump and are not annotated, and the corresponding labels of the unannotated data are all "O".

[0064] Further, as described in S3 above, a neural network model is trained based on the data set to obtain a trained entity recognition model.

[0065] In a possible implementation, the entity recognition model adopts a BERT-BiLSTM-CRF model. The full names of BERT, BiLSTM, and CRF in the BERT-BiLSTM-CRF model are Bidirectional Encoder Representation from Transformers, Bi-directional Long Short-Term Memory, and Conditional Random Field; its common Chinese translation is Transformer-based bidirectional encoder representation, bidirectional long short-term memory, and conditional random field; Transformer is a proprietary name for a classic neural network-based machine translation model. Please refer to Figure 5 , which is a schematic diagram of the BiLSTM model structure provided by an embodiment of the present invention. Considering the close contextual connection of Chinese text, in order to obtain richer information from the data, the BiLSTM model is used to replace the LSTM model. The BiLSTM model inputs the sequence information into the module of the LSTM model through both forward and reverse directions, and combines the forward hidden layer and the backward hidden layer by adding a reverse LSTM unit to obtain two different vector representations of the input information at the current moment in the recursive operation, and combines them into a vector representation of the input information at the current moment, which can access the previous information and the next information, so that the forward and reverse bidirectional information of the sentence can be obtained, and the connection between the sentences can be found from the context.

[0066] Specifically, the S3 specifically includes:

[0067] S31, dividing the data set into a training set, a validation set and a test set.

[0068] S32. Use precision and recall as evaluation indicators.

[0069] S33. Set multiple groups of parameter combinations for the BERT-BiLSTM-CRF model, train the BERT-BiLSTM-CRF model based on the training set, obtain the parameter combination with the highest evaluation index from the BERT-BiLSTM-CRF model based on the validation set, test the BERT-BiLSTM-CRF model according to the test set, and obtain a trained entity recognition model.

[0070] Please refer to Figure 6 , which shows a schematic diagram of the entity recognition model structure provided by an embodiment of the present invention. Specifically, the BERT-BiLSTM-CRF model is a three-layer structure. First, the BERT pre-training model collects the vectors of the feature representation layer of the input geological entity words, and inputs the collected features into the deep learning model Bidirectional Long Short-Term Memory Model (BiLSTM) for deep learning of the full-text feature information. Finally, the output of the BiLSTM model is input into the Conditional Random Field (CRF) model for processing, and the global optimal sequence is obtained based on the label, thereby outputting the prediction result that meets the maximum possibility.

[0071] The BERT model is a bidirectional encoding model based on the self-attention mechanism, which means that it can more fully capture the relevance of the context in the text and the correlation information between sentences. In addition, the transformer uses position embedding to add time information to respond to the problem that the self-attention mechanism cannot extract time features. Therefore, the input of the BERT model contains rich information about the entire sentence and the words and their positions.

[0072] like Figure 7 As shown in the figure, the LSTM model structure contains three "gate" structures: forget gate, which represents the retention ratio of prior information, takes a value of (0, 1), and uses the sigmoid activation function; input gate, which represents the degree of absorption of current information, takes a value of (0, 1), and uses the sigmoid activation function; output gate, which represents the probability of output data, takes a value of (-1, 1), and uses the sigmoid activation function.

[0073] Step 1: For a certain time t, the output h of the previous time t-1 t-1 and the input X at the current time t t The function expression result first passes through a forget gate structure to calculate its score f t As mentioned above, f t The value range of is (0, 1), which is used to judge whether the previous information should be forgotten and the degree of forgetting. t When it is 1, the previous information is completely retained; ft When it is between 0 and 1, part of the prior information is retained, f t The larger the value, the more it retains; when f t When it is 0, the previous information is completely forgotten, that is, it is no longer retained. t The function expression is as follows:

[0074] f t =ξ(M fx X t +M fh h t-1 +d f ),

[0075] Among them, f t is the value of the forget gate, M fx is the input X at time t t The corresponding weight matrix, M fh is the hidden layer output h at time t-1 t-1 The corresponding weight matrix, d f is the bias vector of the forget gate, ξ is the sigmoid activation function, and its function expression is shown as follows:

[0076]

[0077] Step 2: For time t, the output h of the previous time t-1 t-1 and the input X at the current time t t The function expression result passes through an input gate structure to calculate its score As mentioned above The value range of is (0, 1), which is used to determine whether the current information should be updated and the update degree. When it is 1, the current information is completely retained; When it is between 0 and 1, part of the current information is retained. The larger it is, the more it retains; when When it is 0, the current information is completely forgotten, that is, it is not updated. Input gate value The function expression is as follows:

[0078]

[0079] in, is the value of the input gate, M ix is the input X at time t t The weight matrix, M ih is the hidden layer output h at time t-1 t-1 The weight matrix, d i is the bias vector of the input gate, and ξ is the sigmoid activation function.

[0080] Step 3: For time t, the output h of the previous time t-1 t-1 and the input X at the current time t t The function expression result passes through a tanh layer to calculate the state C at time t t , its function expression is as follows:

[0081] C t =tanhM ix (M cx X t +M ch h t-1 +d c ),

[0082] Among them, M ix is the input gate input X at time t t The weight matrix, M cx is the input X at time t t The weight matrix, M ch is the hidden layer output h at time t-1 t-1 The weight matrix, d c is the bias vector of the input gate, and tanh is the tanh activation function.

[0083] Step 4: Change the state C at time t-1 in the second and third steps t-1 Multiply by the value of the forget gate at time t f t , plus the value of the input gate at time t Multiply by the state C at time t t , get the latest status I t , the specific formula is as follows:

[0084]

[0085] Step 5: For time t, the output h of the previous time t-1 t-1 and the input X at the current time t t The function expression result passes through an output gate structure to calculate its score O t . Output gate value O t The function expression is as follows:

[0086] O t =ξ(M ox X t +M oh h t-1 +d o ),

[0087] Among them, O t is the value of the input gate, M ox is the input X at time tt The weight matrix, M oh is the hidden layer output h at time t-1 t-1 The weight matrix, d o is the bias vector of the input gate, and ξ is the sigmoid activation function.

[0088] Then, the latest state I at time t t After the tanh activation function is introduced, the output gate value O at time t t The output of the LSTM unit is obtained by performing a dot multiplication operation, as shown in the following formula:

[0089] h t =O t θtanh(I t ).

[0090] For a given sentence X in a text containing geological entities, the word vector set of each word in the sentence is represented as X = {x1, x2, …, x n}, and the output label sequence Y = {y1, y2, ..., y n}, where the specific function of softmax processing is shown as follows:

[0091]

[0092] where Q is the matrix of scores m*k output by the attention mechanism, where m is the length of the input sentence and k is the number of labels with different entity types. represents the yth word of the i-th word i Finally, the final decoding is performed by the Viterbi algorithm to obtain the predicted label sequence with the highest score, as shown in the following formula:

[0093] y * =arg max(score(X,y)).

[0094] Furthermore, in an embodiment of the present invention, the above-mentioned precision is defined as the proportion of correctly predicted samples among all samples predicted as correct by all models, the above-mentioned recall is defined as the proportion of correctly predicted samples among all samples that should be correctly predicted, and the F1 value is defined as the harmonic mean of the precision and recall rates, and the F1 value is used as an evaluation indicator.

[0095] Furthermore, multiple groups of batch_size and epochs parameter values ​​are set, the BERT-BiLSTM-CRF model is trained according to the training set obtained in step S31, and the batch_size and epochs parameter value combination with the highest evaluation index is selected from the validation set obtained in step S31 to obtain the optimal model parameters, that is, to obtain the entity recognition model.

[0096] Specifically, as described in S4 above, geological entity recognition is performed on geological text data using a trained entity recognition model.

[0097] By using the trained entity recognition model, the input geological text data can be output with the text data of the geological entity annotated according to the annotation tags provided by the embodiment of the present invention, thereby improving the work efficiency of geological entity recognition.

[0098] On the other hand, an embodiment of the present invention further provides a geological entity recognition device based on deep learning, comprising:

[0099] Data acquisition module, used to obtain geological text data;

[0100] A corpus processing module, used for constructing a corpus based on the geological text data;

[0101] A data annotation module, used to annotate the geological entities in the corpus and generate a data set;

[0102] A model training module, used to train a neural network model based on the data set to obtain a trained entity recognition model;

[0103] The entity recognition module is used to perform geological entity recognition on geological text data through the trained entity recognition model.

[0104] On the other hand, an embodiment of the present invention also provides a geological entity recognition terminal based on deep learning, comprising a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and execute the computer program so that the device performs any of the above-mentioned geological entity recognition methods based on deep learning.

[0105] In the embodiment of the present invention, the processor may be an integrated circuit chip having the ability to process signals. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0106] The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The processor reads the information in the storage medium and completes the steps of the above method in combination with its hardware.

[0107] The storage medium may be a memory, which may be, for example, a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memory.

[0108] Among them, the non-volatile memory can be a read-only memory (ROM), a programmable ROM (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0109] The volatile memory may be a random access memory (RAM) which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0110] The storage media described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memory.

[0111] It should be understood that the device disclosed in the embodiments of the present application can be implemented in other ways. For example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the communication connection between units can be through some interfaces, servers or indirect coupling or communication connection, which can be electrical or other forms.

[0112] In addition, each functional module in the embodiment of the present invention may be integrated into one processing unit, each module may exist physically separately, or two or more modules may be integrated into one processing unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit.

[0113] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program code.

[0114] Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way for easy understanding.

[0115] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A geological entity recognition method based on deep learning, characterized in that: The method comprises: Acquiring geological text data, and constructing a corpus based on the geological text data; annotating geological entities in the corpus to generate a data set; Training a neural network model based on the data set to obtain a trained entity recognition model; The geological entity recognition is performed on the geological text data through the trained entity recognition model.

2. A geological entity recognition method based on deep learning according to claim 1, characterized in that: A corpus is constructed based on the geological text data, including: Converting the geological text data into a text document; Screening out noise data in the text document; The filtered text documents are segmented to generate a corpus.

3. A geological entity recognition method based on deep learning according to claim 1, characterized in that: Annotate the geological entities in the corpus, including: constructing labels based on the categories of the geological entities; The geological entities are labeled using the BIO labeling method according to the labels.

4. A geological entity recognition method based on deep learning according to claim 3, characterized in that: Convert the annotated corpus into a format where each line contains one character and the corresponding label to generate a dataset.

5. A geological entity recognition method based on deep learning according to claim 3, characterized in that: The categories of geological entities include engineering types, geological features, survey techniques, survey tools, geological data, samples and experiments.

6. A geological entity recognition method based on deep learning according to claim 1, characterized in that: The entity recognition model adopts a BERT-BiLSTM-CRF model, and the training of the BERT-BiLSTM-CRF model includes: Dividing the data set into a training set, a validation set, and a test set; Use precision and recall as evaluation indicators; Multiple groups of parameter combinations of the BERT-BiLSTM-CRF model are set, the BERT-BiLSTM-CRF model is trained based on the training set, the parameter combination with the highest evaluation index is obtained from the BERT-BiLSTM-CRF model based on the validation set, and the BERT-BiLSTM-CRF model is tested according to the test set to obtain a trained entity recognition model.

7. A geological entity recognition method based on deep learning according to claim 6, characterized in that: The parameter combination includes batch size and number of iterations.

8. A method for geological entity recognition based on deep learning according to claim 6, characterized in that: The evaluation index adopts the harmonic mean of precision and recall.

9. A geological entity recognition device based on deep learning, characterized in that: include: Data acquisition module, used to obtain geological text data; A corpus processing module, used for constructing a corpus based on the geological text data; A data annotation module, used to annotate the geological entities in the corpus and generate a data set; A model training module, used to train a neural network model based on the data set to obtain a trained entity recognition model; The entity recognition module is used to perform geological entity recognition on geological text data through the trained entity recognition model.

10. A geological entity recognition terminal based on deep learning, characterized in that: The device comprises a processor and a memory, wherein the memory is used to store a computer program, and the processor is used to call and execute the computer program, so that the device executes the method according to any one of claims 1 to 8.

Images