A method and system for joint extraction of geological text entity relations
By combining technical means of BERT, Bi-LSTM, GCN and CRF models, the complexity of entity relationship extraction in the geological field is solved, efficient information extraction is achieved, labor costs and time are saved, and data utilization efficiency is improved.
Patent Information
- Application Number
- CN202210066022.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2042-01-20
AI Technical Summary
In the geological field, the extraction of entity relationships has problems such as complex relationship types, variable syntax, redundant texts and overlapping entity relationships, which leads to the traditional methods being time-consuming and labor-intensive and ineffective.
BERT pre-trained model, bidirectional long and short-term memory neural network Bi-LSTM, graph convolutional neural network GCN and CRF models are used to achieve efficient extraction of geological text entity relationships through data cleaning, dependency analysis, word vector representation and label training.
It effectively solves the problem of complex and changeable semantic relationships in geological texts, provides efficient information extraction methods, saves time and cost of manually extracting features, and improves data utilization efficiency.
Smart Images

Figure CN114548088B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of geological text analysis, and in particular to a method and system for jointly extracting geological text entity relationships. Background Art
[0002] In the era of big data, with the advancement of theoretical knowledge and computer technology, massive amounts of data have been generated in geological surveys and work. For these large amounts of unstructured text data, automatic information extraction has become the main means of integrating classified information.
[0003] Entity relationship extraction is the main task of information extraction, so many scholars have begun to study the extraction of geological entity relationships. However, there are still many problems in the extraction of entity relationships in the geological field, such as complex relationship types between entities, complex and changeable syntax, redundancy of geological texts, and complex entity overlap or relationship overlap between text triples.
[0004] For traditional entity relationship extraction methods, manually training corpus in geological entity relationship extraction will consume a lot of time and manpower, and it is difficult to achieve good results.
[0005] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention
[0006] In order to solve the above technical problems, the present invention provides a method for jointly extracting geological text entity relationships, comprising the steps of:
[0007] S1: Acquire geological text data, perform data cleaning, sentence segmentation and word segmentation operations on the geological text data, and obtain a sentence text set;
[0008] S2: classify entities, determine the relationship categories between entities, and filter the sentence text set according to the relationship categories to obtain a filtered sentence text set;
[0009] S3: performing word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector of each sentence; labeling each sentence by the entity and the relationship category to obtain a word label of each sentence; performing dependency analysis on each sentence by a dependency parser to obtain a dependency relationship tree; dividing the filtered sentence text set into a training sample set and a test sample set;
[0010] S4: Obtain an extraction model, the extraction model comprising: a BERT pre-trained model, a bidirectional long short-term memory neural network Bi-LSTM, a graph convolutional neural network GCN and a CRF model; input the word vector of each sentence in the training sample set into the BERT pre-trained model to obtain a continuous embedding representation of each word in the context;
[0011] S5: inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step;
[0012] S6: obtaining an adjacency matrix of the relationship graph between words through the dependency tree, inputting the adjacency matrix of the relationship graph between words and the hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN, and obtaining a word vector representation containing rich neighborhood information and semantic features;
[0013] S7: Input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained;
[0014] S8: Input the test sample set into the trained extraction model to obtain triples of the relationship between entities in the test sample set.
[0015] Preferably, step S2 specifically comprises:
[0016] S21: setting the categories of entities to be extracted and determining the categories of relationships between entities, wherein the categories of entities include: rock mass, landmark area, geological structure and stratum;
[0017] S22: The relationship categories between the entities include: inclusion relationship, direction relationship and position relationship;
[0018] S23: remove sentences that do not belong to the relationship category from the sentence text set to obtain the filtered sentence text set.
[0019] Preferably, in step S3, each sentence is labeled by the entity and the relationship category, and the word label of each sentence is obtained as follows:
[0020] The annotation styles of the word labels of each sentence are: <"entity",[{"type":"E-type","start":ES,"end":EE}]>,<"relation",[{"type":"R-type","head":RS,"tail":RE}]>, where entity represents the name of the entity, E-type represents the category of the entity, ES represents the start position index of the entity in the sentence, Loc_2 represents the end position index of the entity in the sentence, R-type represents the type of relation word corresponding to the entity, RS and RE represent the head entity and tail entity indexes corresponding to the relation word, respectively.
[0021] Preferably, step S4 is specifically:
[0022] S41: word vector h of the sentence N Expressed as: h N =W T +W S +W P ;W T represents the word embedding matrix, W S represents the sentence embedding matrix, W P represents the position embedding matrix;
[0023] S42: Input the word vector of the sentence into the BERT pre-training model for calculation to obtain an output sentence vector;
[0024] S43: The hidden layer state of the output sentence vector is transformed N times through the transformer encoder of the BERT pre-trained model. The specific formula is: α =Trans(h α-1 ), α∈[1,N]; where α represents the number of conversion layers, h α represents the hidden layer state of the output sentence vector when the αth layer is converted, Trans represents the conversion function of the transformer encoder, and N represents the total number of conversions;
[0025] S44: After converting the hidden layer state of the output sentence vector N times, a continuous embedding representation of each word connection context is obtained.
[0026] Preferably, step S5 is specifically:
[0027] S51: Performing feature learning from back to front on the continuous embedding representation of the word connection context through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as:
[0028]
[0029] Where t represents the calculation time, Represents the feature vector obtained from the backward feature learning at time t, represents the unit state from back to front at time t, LSTM represents the feature extraction function, x t represents the information input at time t, represents the feature vector obtained from the backward feature learning at time t-1, Represents the unit state from back to front at time t-1;
[0030] S52: Performing feature learning from front to back on the continuous embedding representation of the context of each word through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as:
[0031]
[0032] in, Represents the feature vector obtained from the forward-to-backward feature learning at time t, represents the unit state from front to back at time t, Represents the feature vector obtained from the forward-to-backward feature learning at time t-1, Represents the unit state from front to back at time t-1;
[0033] S53: By splicing and Obtain the hidden layer state vector containing context information of each word at the current time step at time t, expressed as:
[0034] Preferably, step S6 specifically comprises:
[0035] S61: Acquire all nodes in the dependency tree, and send the feature information of each node to neighboring nodes after transformation;
[0036] S62: Aggregate the feature information of neighboring nodes of each node to obtain a node feature information set;
[0037] S63: performing nonlinear transformation on the node feature information set to obtain a nonlinear node feature information set;
[0038] S64: Input the nonlinear node feature information set and the hidden layer state vector containing context information of the word at the current time step into the graph convolutional neural network GCN, and calculate to obtain the word vector representation containing rich neighborhood information and semantic features. The specific formula is expressed as follows:
[0039]
[0040] Among them, N(v) represents the set of the nearest nodes of node v, u represents the node number in the N(v) set, and l represents the number of layers of the graph convolutional neural network GCN. represents the word vector representation of node v at the lth level, which contains rich neighborhood information and semantic features. ReLU represents the activation function, and W l-1 The adjacency matrix representing the relationship graph between words at level l-1, Represents the hidden layer state vector containing context information of the word of node u at the l-1th level at the current time step, b (l-1)Represents the deviation of the l-1th level.
[0041] Preferably, step S7 specifically includes:
[0042] S71: Calculate and obtain the label score corresponding to each word vector;
[0043] S72: Calculate the probability of obtaining the label sequence of the sentence;
[0044] S73: Calculate the probability of obtaining the label of each word vector in the sentence.
[0045] A geological text entity relationship joint extraction system, comprising:
[0046] A sentence text set acquisition module is used to acquire geological text data, perform data clearing and sentence segmentation operations on the geological text data, and obtain a sentence text set;
[0047] A screening module, used to classify entities, determine the relationship categories between entities, and screen the sentence text set according to the relationship categories to obtain a screened sentence text set;
[0048] A data annotation module is used to perform word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector of each sentence; annotate each sentence by the entity and the relationship category to obtain a word label of each sentence; perform dependency analysis on each sentence by a dependency parser to obtain a dependency relationship tree; and divide the filtered sentence text set into a training sample set and a test sample set;
[0049] A semantic learning module, used for learning text word vectors, inputting the word vectors of each sentence in the training sample set into the BERT pre-training model to obtain a continuous embedding representation of each word in relation to the context;
[0050] A deep semantic learning module, used for inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step;
[0051] A semantic region feature extraction module, used to obtain an adjacency matrix of a word relationship graph through the dependency tree, input the adjacency matrix of the word relationship graph and a hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN, and obtain a word vector representation containing rich neighborhood information and semantic features;
[0052] A word label calibration module is used to input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained;
[0053] The triple acquisition module is used to input the test sample set into the trained extraction model to obtain the triples of the relationship between entities in the test sample set.
[0054] The present invention has the following beneficial effects:
[0055] The entity relationship joint extraction method based on graph convolutional neural network can effectively solve the problems of complex and changeable semantic relationships and the existence of a large number of polysemy in geological texts. It provides an efficient information extraction method for the geological field, enables the effective utilization of a large amount of unstructured data, and can automatically obtain models by training a large amount of data without the need for manual feature extraction, saving a lot of time and labor costs. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 This is a flow chart of a method according to an embodiment of the present invention;
[0057] Figure 2 Example of input representation for the BERT pre-trained model;
[0058] Figure 3 This is a system structure diagram of an embodiment of the present invention;
[0059] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0061] Reference Figure 1 The present invention provides a method for jointly extracting geological text entity relationships, comprising the steps of:
[0062] S1: Acquire geological text data, perform data cleaning, sentence segmentation and word segmentation operations on the geological text data, and obtain a sentence text set;
[0063] Specifically, the geological text data that need to extract entity and relationship categories are cleaned, and non-text content such as geological maps and tables are deleted; secondly, the cleaned geological text data is sentence-segmented, and the geological text data is divided into a series of sentences; then the Chinese Jieba word segmentation tool is used to segment these sentences, and relevant geological dictionaries can be introduced during word segmentation to avoid word segmentation errors; then the geological entity recognition method is used to identify and extract the entities and relationship categories between entities in the geological text data, and the set of all sentences containing two or more entities is taken as the sentence text set;
[0064] S2: classify entities, determine the relationship categories between entities, and filter the sentence text set according to the relationship categories to obtain a filtered sentence text set;
[0065] S3: performing word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector of each sentence; labeling each sentence by the entity and the relationship category to obtain a word label of each sentence; performing dependency analysis on each sentence by a dependency parser to obtain a dependency relationship tree; dividing the filtered sentence text set into a training sample set and a test sample set;
[0066] Specifically, the screened sentence text set is divided into a training sample set and a test sample set in a 1:1 ratio;
[0067] S4: Obtain an extraction model, the extraction model comprising: a BERT pre-trained model, a bidirectional long short-term memory neural network Bi-LSTM, a graph convolutional neural network GCN and a CRF model; input the word vector of each sentence in the training sample set into the BERT pre-trained model to obtain a continuous embedding representation of each word in the context;
[0068] S5: inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step;
[0069] S6: obtaining an adjacency matrix of the relationship graph between words through the dependency tree, inputting the adjacency matrix of the relationship graph between words and the hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN, and obtaining a word vector representation containing rich neighborhood information and semantic features;
[0070] S7: Input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained;
[0071] S8: Input the test sample set into the trained extraction model to obtain triples of the relationship between entities in the test sample set.
[0072] Specifically, assume that there are the following geological text materials:
[0073] Dark grey limestone lenses (or quasi-layered) are found in the lower and top slates of the third member of the Nieru Formation, and iron nodules are found locally. A small amount of bivalve and ammonite fossils are found in the limestone at the top. In the lower part, there are trough-mold sedimentary structures, slip and gravity-induced entrained bedding, and in the upper part, there are heavy-load-mold sedimentary structures, and a large number of Bauma sequences are developed.
[0074] The geological entities that can be extracted from the geological text data include "Nieru Formation", "slate", "limestone", "lens", "sedimentary structure", "ammonite", "trough mold", "heavy load mold", etc. The relationship between them is judged pairwise, so there are 28 groups of entity pairs with relationship categories that need to be judged, such as "Nieru Formation" and "slate", "limestone" and "ammonite", etc., which are not listed here one by one.
[0075] After adopting the method of the present invention, the label probability of the current sentence can be calculated according to the two labeled entity pairs, such as "Nieru Group" and "Slate", and the maximum possible output label can be obtained. Finally, it can be determined that "lower part and top part" is the relational word between "Nieru Group" and "Slate", forming the <Nieru Group, lower part and top part, Slate> triple;
[0076] When there is no relationship between entity pairs, such as "slot model" and "heavy load model", the present invention will determine whether the score exceeds a threshold that there is no relationship between the entity pairs, that is, Unknown, and form a <slot model, Unknown, heavy load model> triple.
[0077] In this embodiment, step S2 is specifically as follows:
[0078] S21: setting the categories of entities to be extracted and determining the categories of relationships between entities, wherein the categories of entities include: rock mass, landmark area, geological structure and stratum; these four entities are categories that can be intuitively expressed in geological space;
[0079] S22: The relationship categories between the entities include: inclusion relationship, direction relationship and position relationship;
[0080] S23: remove sentences that do not belong to the relationship category from the sentence text set to obtain the filtered sentence text set.
[0081] In this embodiment, in step S3, each sentence is labeled by the entity and the relationship category, and the word label of each sentence is obtained as follows:
[0082] A combination of manual and automatic annotation methods is used to annotate the sentences in the screened sentence text set one by one;
[0083] The annotation styles of the word labels of each sentence are: <"entity",[{"type":"E-type","start":ES,"end":EE}]>,<"relation",[{"type":"R-type","head":RS,"tail":RE}]>, where entity represents the name of the entity, E-type represents the category of the entity, ES represents the start position index of the entity in the sentence, Loc_2 represents the end position index of the entity in the sentence, R-type represents the type of relation word corresponding to the entity, RS and RE represent the head entity and tail entity indexes corresponding to the relation word, respectively.
[0084] In this embodiment, in step S4, the BERT pre-trained model is used as the word vector embedding layer to encode context information, wherein the word vector of the sentence as the input can not only obtain the semantic features of the words, but also learn the deep semantics at the sentence level through the superimposed input vector sequence;
[0085] Step S4 is specifically as follows:
[0086] S41: Figure 2 As shown, the word vector h of the sentence N Expressed as: h N =W T +W S +W P ;W T represents the word embedding matrix, W S represents the sentence embedding matrix, W P represents the position embedding matrix;
[0087] S42: Input the word vector of the sentence into the BERT pre-training model for calculation to obtain an output sentence vector;
[0088] S43: The hidden layer state of the output sentence vector is transformed N times through the transformer encoder of the BERT pre-trained model. The specific formula is: α =Trans(h α-1 ), α∈[1,N]; where α represents the number of conversion layers, h α represents the hidden layer state of the output sentence vector when the αth layer is converted (that is, the context information of the sentence in the αth layer), Trans represents the conversion function of the transformer encoder, and N represents the total number of conversions;
[0089] The input of the transformer encoder passes through the Self-attention layer, which can effectively capture the dependency between words through the self-attention mechanism;
[0090] S44: After converting the hidden layer state of the output sentence vector N times, a continuous embedding representation of each word connection context is obtained.
[0091] In this embodiment, step S5 uses a bidirectional long short-term memory neural network Bi-LSTM to further extract context features from the continuous embedding representation of the word connection context obtained in step S4, and learns the features of the sentence from front to back and from back to front to obtain a hidden layer state vector containing context information for each word at the current time step; and prepares data for the input layer of the graph convolutional neural network GCN in step S6;
[0092] Step S5 is specifically as follows:
[0093] S51: Performing feature learning from back to front on the continuous embedding representation of the word connection context through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as:
[0094]
[0095] Where t represents the calculation time, Represents the feature vector obtained from the backward feature learning at time t, represents the unit state from back to front at time t, LSTM represents the feature extraction function, x t represents the information input at time t, represents the feature vector obtained from the backward feature learning at time t-1, Represents the unit state from back to front at time t-1;
[0096] S52: Performing feature learning from front to back on the continuous embedding representation of the context of each word through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as:
[0097]
[0098] in, Represents the feature vector obtained from the forward-to-backward feature learning at time t, represents the unit state from front to back at time t, Represents the feature vector obtained from the forward-to-backward feature learning at time t-1, Represents the unit state from front to back at time t-1;
[0099] S53: By splicing and Obtain the hidden layer state vector containing context information of each word at the current time step at time t, expressed as:
[0100] In this embodiment, based on the adjacency matrix of the word relationship graph generated by the dependency tree obtained in step S3, and using the hidden layer state vector containing context information of the word at the current time step output in step S5, more accurate semantic information is obtained;
[0101] Step S6 is specifically as follows:
[0102] S61: Acquire all nodes in the dependency tree, and send the feature information of each node to neighboring nodes after transformation;
[0103] S62: Aggregate the feature information of neighboring nodes of each node to obtain a node feature information set;
[0104] S63: performing nonlinear transformation on the node feature information set to obtain a nonlinear node feature information set;
[0105] S64: Input the nonlinear node feature information set and the hidden layer state vector containing context information of the word at the current time step into the graph convolutional neural network GCN, and calculate to obtain the word vector representation containing rich neighborhood information and semantic features. The specific formula is expressed as follows:
[0106]
[0107] Among them, N(v) represents the set of the nearest nodes of node v, u represents the node number in the N(v) set, and l represents the number of layers of the graph convolutional neural network GCN. represents the word vector representation of node v at the lth level, which contains rich neighborhood information and semantic features. ReLU represents the activation function, and W l-1 The adjacency matrix representing the relationship graph between words at level l-1, Represents the hidden layer state vector containing context information of the word of node u at the l-1th level at the current time step, b (l-1) Represents the deviation of the l-1th level.
[0108] In this embodiment, in terms of the output recognition entity layer, the CRF model is used to introduce the dependency relationship between entity labels to obtain the probability of each label;
[0109] Step S7 is specifically as follows:
[0110] S71: Calculate and obtain the label score corresponding to each word vector;
[0111] S72: Calculate the probability of obtaining the label sequence of the sentence;
[0112] S73: Calculate the probability of obtaining the label of each word vector in the sentence;
[0113] Specifically, the transfer matrix is used to avoid the unreasonable situation that the first word of the entity tag is not B or different categories appear in one entity tag, thereby reducing the error rate of entity tag recognition.
[0114] refer to Figure 3 The present invention provides a geological text entity relationship joint extraction system, comprising:
[0115] A sentence text set acquisition module is used to acquire geological text data, perform data clearing and sentence segmentation operations on the geological text data, and obtain a sentence text set;
[0116] A screening module, used to classify entities, determine the relationship categories between entities, and screen the sentence text set according to the relationship categories to obtain a screened sentence text set;
[0117] A data annotation module is used to perform word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector of each sentence; annotate each sentence by the entity and the relationship category to obtain a word label of each sentence; perform dependency analysis on each sentence by a dependency parser to obtain a dependency relationship tree; and divide the filtered sentence text set into a training sample set and a test sample set;
[0118] A semantic learning module, used for learning text word vectors, inputting the word vectors of each sentence in the training sample set into the BERT pre-training model to obtain a continuous embedding representation of each word in relation to the context;
[0119] A deep semantic learning module, used for inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step;
[0120] A semantic region feature learning module, used to obtain an adjacency matrix of a relationship graph between words through the dependency tree, and input the adjacency matrix of the relationship graph between words and a hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN to obtain a word vector representation containing rich neighborhood information and semantic features;
[0121] A word label calibration module is used to input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained;
[0122] The triple acquisition module is used to input the test sample set into the trained extraction model to obtain the triples of the relationship between entities in the test sample set.
[0123] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.
[0124] The serial numbers of the embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments. In a unit claim that lists several means, several of these means may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order and these words may be interpreted as identifiers.
[0125] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.
Claims
1. A method for joint extraction of geological text entity relations, characterized in that: Includes steps: S1: Acquire geological text data, perform data cleaning, sentence segmentation and word segmentation operations on the geological text data, and obtain a sentence text set; S2: classify entities, determine the relationship categories between entities, and filter the sentence text set according to the relationship categories to obtain a filtered sentence text set; S3: performing word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector of each sentence; labeling each sentence by the entity and the relationship category to obtain a word label of each sentence; Perform dependency analysis on each sentence through a dependency parser to obtain a dependency tree; Dividing the screened sentence text set into a training sample set and a test sample set; S4: Obtain an extraction model, the extraction model comprising: a BERT pre-trained model, a bidirectional long short-term memory neural network Bi-LSTM, a graph convolutional neural network GCN and a CRF model; input the word vector of each sentence in the training sample set into the BERT pre-trained model to obtain a continuous embedding representation of each word in the context; S5: inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step; S6: obtaining an adjacency matrix of the relationship graph between words through the dependency tree, inputting the adjacency matrix of the relationship graph between words and the hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN, and obtaining a word vector representation containing rich neighborhood information and semantic features; S7: Input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained; S8: Input the test sample set into the trained extraction model to obtain triples of the relationship between entities in the test sample set.
2. The method for extracting geological text entity relationships according to claim 1, characterized in that: Step S2 is specifically as follows: S21: setting the categories of entities to be extracted and determining the categories of relationships between entities, wherein the categories of entities include: rock mass, landmark area, geological structure and stratum; S22: The relationship categories between the entities include: inclusion relationship, direction relationship and position relationship; S23: remove sentences that do not belong to the relationship category from the sentence text set to obtain the filtered sentence text set.
3. The method for extracting geological text entity relationships according to claim 1, characterized in that: In step S3, each sentence is labeled by the entity and the relationship category, and the word label of each sentence is obtained as follows: The annotation styles of the word tags of each sentence are: <"entity", [{"type":"E-type","start":ES,"end":EE}]>, <"relation", [{"type":"R-type","head":RS,"tail":RE}]>, where entity represents the name of the entity, E-type represents the category of the entity, ES represents the start position index of the entity in the sentence, Loc_2 represents the end position index of the entity in the sentence, R-type represents the type of relation word corresponding to the entity, RS and RE represent the head entity and tail entity indexes corresponding to the relation word, respectively.
4. The method for extracting geological text entity relationships according to claim 1, characterized in that: Step S4 is specifically as follows: S41: word vector h of the sentence N Expressed as: h N =W T +W S +W P ;W T represents the word embedding matrix, W S represents the sentence embedding matrix, W P represents the position embedding matrix; S42: Input the word vector of the sentence into the BERT pre-training model for calculation to obtain an output sentence vector; S43: The hidden layer state of the output sentence vector is transformed N times through the transformer encoder of the BERT pre-trained model. The specific formula is: α =Trans(h α-1 ), α∈[1,N]; where α represents the number of conversion layers, h α represents the hidden layer state of the output sentence vector when the αth layer is converted, Trans represents the conversion function of the transformer encoder, and N represents the total number of conversions; S44: After converting the hidden layer state of the output sentence vector N times, a continuous embedding representation of each word connection context is obtained.
5. The method for extracting geological text entity relationships according to claim 1, characterized in that: Step S5 is specifically as follows: S51: Performing feature learning from back to front on the continuous embedding representation of the word connection context through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as: Where t represents the calculation time, Represents the feature vector obtained from the backward feature learning at time t, represents the unit state from back to front at time t, LSTM represents the feature extraction function, xt represents the information input at time t, represents the feature vector obtained from the backward feature learning at time t-1, Represents the unit state from back to front at time t-1; S52: Performing feature learning from front to back on the continuous embedding representation of the context of each word through the bidirectional long short-term memory neural network Bi-LSTM, the specific formula is expressed as: in, Represents the feature vector obtained from the forward-to-backward feature learning at time t, represents the unit state from front to back at time t, Represents the feature vector obtained from the forward-to-backward feature learning at time t-1, Represents the unit state from front to back at time t-1; S53: By splicing and Obtain the hidden layer state vector containing context information of each word at the current time step at time t, expressed as:
6. The method for joint extraction of geological text entity relations according to claim 1, characterized in that: Step S6 is specifically as follows: S61: Acquire all nodes in the dependency tree, and send the feature information of each node to neighboring nodes after transformation; S62: Aggregate the feature information of neighboring nodes of each node to obtain a node feature information set; S63: performing nonlinear transformation on the node feature information set to obtain a nonlinear node feature information set; S64: Input the nonlinear node feature information set and the hidden layer state vector containing context information of the word at the current time step into the graph convolutional neural network GCN, and calculate to obtain the word vector representation containing rich neighborhood information and semantic features. The specific formula is expressed as follows: Among them, N(v) represents the set of the nearest nodes of node v, u represents the node number in the N(v) set, and l represents the number of layers of the graph convolutional neural network GCN. represents the word vector representation of node v at the lth level, which contains rich neighborhood information and semantic features. ReLU represents the activation function, and W l-1 The adjacency matrix representing the relationship graph between words at level l-1, Represents the hidden layer state vector containing context information of the word of node u at the l-1th level at the current time step, b (l-1) Represents the deviation of the l-1th level.
7. The method for joint extraction of geological text entity relations according to claim 1, characterized in that: Step S7 is specifically as follows: S71: Calculate and obtain the label score corresponding to each word vector; S72: Calculate the probability of obtaining the label sequence of the sentence; S73: Calculate the probability of obtaining the label of each word vector in the sentence.
8. A geological text entity relationship joint extraction system, characterized in that: include: A sentence text set acquisition module is used to acquire geological text data, perform data clearing and sentence segmentation operations on the geological text data, and obtain a sentence text set; A screening module, used to classify entities, determine the relationship categories between entities, and screen the sentence text set according to the relationship categories to obtain a screened sentence text set; A data annotation module is used to perform word-level segmentation on each sentence in the filtered sentence text set to obtain a word vector for each sentence; annotate each sentence using the entity and the relationship category to obtain a word label for each sentence; Perform dependency analysis on each sentence through a dependency parser to obtain a dependency tree; Dividing the screened sentence text set into a training sample set and a test sample set; A semantic learning module, used for learning text word vectors, inputting the word vectors of each sentence in the training sample set into the BERT pre-training model to obtain a continuous embedding representation of each word in relation to the context; A deep semantic learning module, used for inputting the continuous embedding representation of the context of each word into the bidirectional long short-term memory neural network Bi-LSTM to extract context features, and obtaining a hidden layer state vector containing context information of each word at the current time step; A semantic region feature learning module, used to obtain an adjacency matrix of a relationship graph between words through the dependency tree, and input the adjacency matrix of the relationship graph between words and a hidden layer state vector containing context information of each word at the current time step into the graph convolutional neural network GCN to obtain a word vector representation containing rich neighborhood information and semantic features; A word label calibration module is used to input the word vector representation containing rich neighborhood information and semantic features and the word labels of each sentence into the CRF model for training. When the probability of each label is obtained, it indicates that the extraction model training is completed, and a trained extraction model is obtained; The triple acquisition module is used to input the test sample set into the trained extraction model to obtain the triples of the relationship between entities in the test sample set.
Citation Information
Patent Citations
Entity relationship joint extraction method and system
CN111160008A
Triple information extraction method and device, electronic equipment and storage medium
CN113590784A
Pre-trained model and fine-tuning technology-based medical text relationship extraction method
WO2020211275A1
