Autonomous navigation ship emergency event case entity extraction model construction method and device

Abstract

The entity extraction method based on deep learning does not need to construct and screen features, can automatically learn data features, has good generalization ability, can greatly improve entity recognition precision, improves named entity recognition accuracy, realizes autonomous navigation ship emergency knowledge structuring, and has good application prospects. And meanwhile, the informatization and intelligentization levels in the aspect of management are effectively improved. According to the technical scheme, the method comprises the following steps that S1, emergency corpus data of the autonomous navigation ship are collected, and a training set is obtained according to the collected emergency corpus data of the autonomous navigation ship; s2, a BERT-BiLSTM-Att-CRF extraction model is constructed, and the BERT-BiLSTM And S3, training the BERT-BiLSTM-Att-CRF extraction model by using a training set to obtain an entity extraction model for the emergency event case of the autonomous navigation ship. The invention further discloses an extraction model construction device.

Description

Technical Field

[0001] The present invention relates to the field of ship emergency decision-making, and in particular to a method and a device for constructing an entity extraction model for emergency case of autonomously sailing ships. Background Art

[0002] Autonomous navigation is an emerging technology in the field of ships. The navigation process involves multiple systems and multiple links. The failure of any system will bring risks to the navigation of ships. Since autonomous navigation technology is still in the development stage, the autonomous navigation of ships needs to be carried out under the monitoring of monitoring personnel. After the system alarm of an emergency occurs, it is necessary to take over manually and take emergency measures. Human risk factors must also be considered when responding to emergencies. Therefore, with the help of computer intelligent emergency decision-making assistance, the influence of human risk factors in responding to emergencies can be avoided to a certain extent. The historical cases of maritime emergencies include rich decision-making knowledge and historical experience. At the same time, the emergency plan also includes the emergency decision-making plan under a certain emergency scenario. Therefore, historical cases and emergency plans are used as the knowledge basis for intelligent emergency decision-making assistance to output emergency decision-making plans. However, historical cases of emergencies are mostly stored in text form, and information such as specific measures taken to deal with emergencies cannot be effectively conveyed. There is a phenomenon of unstructured or low degree of structure, and it is impossible to deeply apply historical maritime emergency information. At the same time, the pre-made emergency plans are mostly in the form of text with fixed content format, and the query and retrieval efficiency is poor. Therefore, the knowledge graph method is used to convert unstructured text data such as historical cases and emergency plans into structured data for storage and use.

[0003] The knowledge graph is composed of entities and the relationships between them. The process of building a knowledge graph involves knowledge extraction, knowledge storage, knowledge fusion, and application analysis. Knowledge extraction refers to the technology of extracting knowledge from data of different structures and sources using entity extraction and relationship extraction, which is the basis for building a knowledge graph. Therefore, entity extraction is very important in the process of building a knowledge graph. Entity extraction, also known as named entity recognition, aims to automatically identify entities with specific meanings from texts and mark them as corresponding types to extract entities. Entity extraction is performed through entity recognition technology, and the relationship between entities is extracted using relationship extraction technology to convert unstructured data into structured triples. If the entities in the text cannot be correctly identified and extracted, it will also affect the subsequent extraction of the relationship between entities, and the knowledge graph cannot be correctly constructed. At the same time, entity extraction can help ensure the accuracy and completeness of the entity information in the knowledge graph, and improve the quality of the knowledge graph.

[0004] Entity extraction can be roughly divided into rule-based and dictionary-based entity extraction methods, traditional machine learning-based entity extraction methods, and deep learning-based entity extraction methods. Rule-based and dictionary-based entity extraction methods mainly perform entity boundary detection based on feature design rules and regular expressions, and then identify entities based on dictionary matching; however, rules and dictionaries need to be constructed manually, and the results are highly dependent on rules and dictionaries. Rules and dictionaries need to be updated or adjusted for different entity types.

[0005] The entity extraction method based on traditional machine learning mainly uses models such as support vector machines, hidden Markov models and conditional random fields to extract data features, and recognize entities through classifier training. Compared with the entity extraction method based on rules and dictionaries, it does not require manual construction and adjustment of rules and dictionaries, and can achieve automatic learning of features, but it is not easy to learn deep and complex features of the data. Summary of the invention

[0006] The present invention solves the problems that in the current traditional entity extraction of emergency cases of autonomous sailing ships, the rules and dictionaries in the entity extraction method based on rules and dictionaries need to be manually constructed, the results are highly dependent on the rules and dictionaries, and the rules and dictionaries need to be updated or adjusted for different entity types; and the entity extraction method based on traditional machine learning is not easy to learn the deep and complex features of data. The present invention provides an entity extraction method based on deep learning, which does not require the construction and screening of features, can automatically learn data features, has good generalization ability, can greatly increase the accuracy of entity recognition, improve the accuracy of named entity recognition and realize the structuring of emergency knowledge of autonomous sailing ships, and at the same time effectively improves the informationization and intelligentization level of management, thereby providing an entity extraction model method and device for emergency cases of autonomous sailing ships.

[0007] The technical solution of the present invention is:

[0008] A method for constructing an entity extraction model for autonomous navigation emergency cases, characterized in that it comprises the following steps:

[0009] S1: Collecting ship emergency corpus data, the emergency corpus data is formed by collecting traditional ship navigation collision accident data and emergency plans in recent years to form a corpus, including multiple historical cases and key information corresponding to emergency plans, and obtaining a training set based on the collected autonomous navigation ship emergency corpus data;

[0010] S2: Construct a BERT-BiLSTM-Att-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to fine-tune the input corpus data, match the input data with semantic dynamic word vectors, and output the first layer results; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att is the Self-Attention model layer. The Self-Attention model layer learns the results of the BiLSTM layer, gives different weights to the context information of each part according to the degree of attention, learns deeper context sequence features, and outputs the third layer results; (4) CRF model layer: used to decode the third layer results and output the corresponding annotation results of all the texts in the emergency corpus data. The key information and non-key information have different annotations.

[0011] S3: Use the training set to train the BERT-BiLSTM-Att-CRF extraction model to obtain an entity extraction model for emergency cases of autonomous ships.

[0012] The step S1 also includes the following steps: obtaining a verification set based on the emergency corpus data of the autonomous navigation ship, and modifying the constructed BERT-BiLSTM-Att-CRF extraction model based on the verification set.

[0013] The step S1 further includes the following steps: the key information includes the type of ship, the cause of the accident, and the weather conditions, etc., and the training set and the validation set are annotated with text sequences, and the text sequences are annotated with the type of ship, the cause of the accident, and the weather conditions, etc. 。

[0014] The training set and validation set are annotated using the BIO method, where B (Begin) indicates that the word is at the beginning of a named entity, I (Inside) indicates that the word is in a named entity, and O (Outside) indicates that the word does not belong to a named entity. Each word is annotated into one of three states.

[0015] The step S1 also includes: obtaining a test set according to the autonomous navigation ship emergency corpus data.

[0016] The step S3 also includes the following steps: using the extraction model to extract the test set to obtain extraction cases.

[0017] The data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1. The training set, the validation set and the test set respectively include different individual case information and emergency information, that is, key information of ship types, accident causes and weather conditions in different cases.

[0018] The steps of the Bert model layer matching the input data with dynamic word vectors with semantics are as follows: input a training set text sequence, add [CLS] and [SEP] labels at the beginning and end of the text sequence respectively, obtain a position vector, a sentence vector and a word vector corresponding to each word, merge them and input them into a bidirectional transformer network to obtain a context-related vector representation of a single word, and average or max-pool all word vectors to obtain a sequence vector of semantic features.

[0019] The BiLSTM model layer is composed of a forward LSTM and a backward LSTM, and both the forward LSTM and the backward LSTM include a forget gate, an input gate, an output gate, and a memory unit;

[0020] Forget gate calculation formula: f t =σ(W f [h t-1 , x t ]+b f );

[0021] Input gate calculation formula: i t =σ(W i [h t-1 , x t ]+b i ),

[0022] Unit status update calculation formula:

[0023] Output gate calculation formula: o t =σ(W o [h t-1 , x t ]+b o ),

[0024] Among them, σ represents the sigmoid function, the expression is W f represents the forget gate weight matrix; W i represents the input gate weight matrix; W c represents the unit state matrix; W o represents the output gate weight matrix; h t-1 Indicates the output value at the previous moment; x t represents the current unit input vector; b f represents the bias parameter of the forget gate; b i represents the input gate bias parameter; b c represents the unit state bias parameter; b o Represents the output gate bias parameter;

[0025] The forward LSTM and backward LSTM calculation process formulas are the same, the difference is that the forward LSTM processes the input sequence from left to right, and the backward LSTM processes the input sequence from right to left. The forward LSTM calculates Backward LSTM calculation The context vector is obtained by concatenating the hidden layer state sequence information output by the forward LSTM and the backward LSTM Output hidden layer sequence (h1, h2, h3, h4, ..., h t ).

[0026] A fully connected layer is set between the BiLSTM model layer and the Self-Attention model layer to perform linear transformation on the feature vector to achieve dimensionality reduction. The output of the BiLSTM layer is used as the input of the fully connected layer to perform the following calculations:

[0027] X=hW FC +b FC ;

[0028] Where h represents the output vector of BiLSTM; W FC represents the weight matrix of the fully connected layer, whose dimension is D×d, where d is the number of entity labels. D is the dimension of the output vector of BiLSTM; b FC Represents the bias vector of the fully connected layer.

[0029] The output of the fully connected layer is used as the input of the Self-Attention model layer. The Self-Attention model layer assigns attention weights to the hidden layer vectors of each time step. The input of the Self-Attention model layer is X = [h1,h2,h3,......,h n ], where n∈[1,t], X represents a set of input sequences, and for each input sequence, by comparing it with the weight matrix W q , W k , W v Multiply them to get the query vector Q, key vector K and value vector V respectively. Use the multi-head attention mechanism to calculate, set the number of heads num_head and the hidden layer vector dimension hidden_dim, split Q (query), K (key value) and V (value) according to the number of heads, and perform the following calculation for each head:

[0030]

[0031] Where Q, K, and V represent the query matrix, key matrix, and value matrix respectively; d k Represents the vector dimension of each head,

[0032] Use the scaled dot product as the attention scoring function, and then concatenate the multi-head attention calculation results to get the output vector X = [h att1 ,h att2 ,h att3 ,......,h attn ].

[0033] The decoding process of the CRF model layer in step S2 includes: using the sequence vector output by the Self-Attention layer as the input of the CRF layer, and the CRF model layer learns the input sequence vector to obtain a state transfer matrix and a transmission probability matrix. The transmission probability matrix represents the probability of converting the feature vector at each position into a label. The matrix will capture the emission law of the label at each position and can be calculated from the output of the Self-Attention model layer. The state transfer matrix represents the probability of transferring from one label to another. The matrix will capture the transfer law between labels. The CRF layer state transfer probability calculation formula is as follows:

[0034]

[0035]

[0036] in, Indicates that the jth word is in the yth j The scores under the labels, represents the probability of transferring from the jth label to the j+1th label, represents the true label, Y X Represents the set of all tags;

[0037] The probabilities of all paths are calculated through the forward algorithm, that is, the probabilities of all paths from the starting position of the sequence to each position whose label is a given label are calculated, and the tag sequence with the largest probability is selected as the output tag sequence, which is the entity extraction result obtained according to the model prediction.

[0038] Step S3 also includes the following process: using the training set data to train the model, when the number of iterations reaches the specified number or the loss value meets the set conditions, the model training is terminated; the accuracy (Precision), recall (Recall) and F1 value of the model are obtained through the validation set as evaluation indicators; the calculation formulas for the accuracy, recall and F1 value evaluation indicators are as follows:

[0039]

[0040]

[0041]

[0042] Where TP represents the number of correctly identified entities, FP represents the number of incorrectly identified entities, and FN represents the number of entities that were not correctly identified;

[0043] If the evaluation index value is low, it is necessary to adjust hyperparameters such as the number of iterations or learning rate to make the model reach the optimal state. Finally, the test set entity extraction is used to evaluate the performance and generalization ability of the model.

[0044] The device for constructing an entity extraction model for an autonomous navigation emergency case is characterized by comprising the following modules: a corpus data collection module (11), a training set module (12), a BERT-BiLSTM-Att-CRF extraction model construction module (13) and a training extraction model module (14);

[0045] The corpus data collection module (11) is used to collect emergency corpus data of autonomous navigation ships, and the training set module (12) is used to obtain a training set based on the collected emergency corpus data of autonomous navigation ships;

[0046] Constructing a BERT-BiLSTM-Att-CRF extraction model module (13) is used to construct a BERT-BiLSTMAtt-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to pre-train the input data, match the dynamic word vector with semantics to the input data, and output the first layer result; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att, i.e., Self-Attention layer, gives different weights to the context information of each part according to the degree of attention, and learns deeper context sequence features; (4) CRF model layer: used to decode the third layer results and output the results;

[0047] The training extraction model module (14) is used to train the BERT-BiLSTM-Att-CRF extraction model using a training set to obtain an entity extraction model for emergency cases of autonomous navigation ships.

[0048] It also includes a verification set module (15), which is used to obtain a verification set based on the emergency corpus data of the autonomous navigation ship, and to modify the constructed BERT-BiLSTM-Att-CRF extraction model based on the verification set.

[0049] It also includes a test set module (16), which is used to obtain a test set based on the autonomous navigation ship emergency corpus data, and to extract the test set using the extraction model to obtain extraction cases.

[0050] The effects of the present invention are:

[0051] The entity extraction method of the emergency case of autonomous navigation ships includes the following steps:

[0052] S1: Collecting ship emergency corpus data, the emergency corpus data is formed by collecting traditional ship navigation collision accident data and emergency plans in recent years to form a corpus, including multiple historical cases and key information corresponding to emergency plans, and obtaining a training set based on the collected autonomous navigation ship emergency corpus data;

[0053] S2: Construct a BERT-BiLSTM-Att-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to pre-train the input corpus data, match the input data with semantic dynamic word vectors, and output the first layer results; (2) BiLSTM model layer: used to learn the contextual features of the first layer results and output the second layer results; (3) Att, or Self-Attention model layer, gives different weights to the contextual information of each part according to the degree of attention, and learns deeper context sequence features; (4) CRF model layer: used to decode the third layer results and output the results. The output results will return the corresponding annotations of all texts and texts, which are reflected in the output of the last layer of the model (CRF layer). The key information and non-key information have different annotations. For example: Northeast wind level 6-7 along the coast of Zhangzhou. Putting the above words into the trained model can output the labels corresponding to each of the above words. The labels of key information will be B-, I-, and non-key information will be O (i.e., the labels of northeast wind level 6-7 are B-weather conditions, I-weather conditions..., I-weather conditions;

[0054] S3: Use the training set to train the BERT-BiLSTM-Att-CRF extraction model to obtain an entity extraction model for emergency cases of autonomous ships.

[0055] The present invention constructs a BERT-BiLSTM-Att-CRF extraction model, obtains the vector representation of the text through the deep neural network in the model, and automatically learns the deep hidden features of the data. Each layer of the BERT-BiLSTM-Att-CRF model can learn the corresponding hidden features: the hidden features learned by the BERT layer include the word vector containing semantic information corresponding to each word in the sequence paragraph; the hidden features learned by the BiLSTM layer include the context feature expression in the local context; the hidden features learned by the Att layer include the weights allocated to different parts of the sequence, that is, different weights are given to the context information of each part according to the degree of attention, and deeper context sequence features are learned; the hidden features learned by the CRF layer include weights for optimizing the label sequence. Other data can also obtain the hidden features of each layer corresponding to their own data through the BERT-BiLSTM-Att-CRF model. Realize the recognition and extraction of entities. The entity extraction method based on deep learning does not need to construct and screen features, can automatically learn data features, and does not need to manually construct feature vectors. The main reason for being able to automatically learn data features is that the deep learning model can learn to automatically extract features end-to-end, and no artificial design of features is required in the process. Other data can also learn the features corresponding to their own data through deep neural networks, without the need for artificially designed features, and have good generalization capabilities.

[0056] The entity extraction method for emergency cases of autonomous navigation ships provided by the present invention, compared with the traditional BiLSTM-CRF model, adopts the BERT pre-trained language model to understand the semantics and contextual relationships in the text, generates richer and more accurate semantic representations, and greatly increases the accuracy of entity recognition. The BERT model layer is essentially a coding layer, which generates a corresponding word vector for each word in the text. Before the emergence of the BERT model, the encoding of the BiLSTM-CRF model was implemented by word embedding technologies such as Word2vec, and the word vector generated by it only contained the content of the meaning of the word itself, which was a static word vector independent of the context. In comparison, the BERT model uses a combination of word, sentence and position embedding and a Transformer network to understand the left and right contexts of the word, generate word embeddings containing contextual information, and generate different word vector representations for the same word, effectively solving the problem that polysemy cannot be achieved in Word2Vec.

[0057] Compared with the Bert-BiLSTM-CRF model, the entity extraction method for emergency cases of autonomous navigation ships provided by the present invention simultaneously introduces the Self-Attention model to further learn the potential semantic correlation between contextual information and improve the accuracy of named entity recognition.

[0058] Although emergency plans are mostly in the form of text with fixed content formats, they often return information-related paragraphs when queried, and it is impossible to directly obtain the key information in a certain situation. However, the model of claim 1 can directly extract the key information required in each paragraph from the text as an entity, which can greatly increase the accuracy and efficiency of entity recognition and the accuracy of named entity recognition, and at the same time effectively improve the level of informationization and intelligence in management.

[0059] The key information includes the type of ship, the cause of the accident and the weather conditions. The training set and the validation set are annotated with text sequences, and the text sequences are annotated with the type of ship, the cause of the accident and the weather conditions. According to existing literature, different types of ships have different operating performances. The type of ship, severe weather conditions and mechanical equipment failures in the cause of the accident are key information in historical accident cases. Annotating the type of ship, the cause of the accident and the weather conditions can accurately extract these entities to construct a knowledge graph for subsequent accident emergency response. By searching for reference emergency measures for emergency response through the type of ship, the cause of the accident and the weather conditions.

[0060] The method for extracting entity of autonomous ship emergency cases provided by the present invention can process the autonomous ship emergency knowledge corpus, extract relevant entities to construct the autonomous ship emergency domain knowledge graph, apply the autonomous ship emergency domain knowledge graph as the knowledge basis for emergency decision-making of emergencies, realize the structuring of autonomous ship emergency knowledge, and effectively improve the information and intelligent level of management.

[0061] The method for extracting entity of emergency cases of autonomous navigation ships provided by the present invention is suitable for application in the work of extracting entity of emergency decision-making plans of autonomous navigation ships based on knowledge graph.

[0062] The present invention will be further described below in conjunction with the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 A flowchart of the entity extraction method for emergency cases of autonomous ships;

[0064] Figure 2 This is a schematic diagram of the BERT-BiLSTM-Att-CRF model structure;

[0065] Figure 3 This is the structural block diagram of the entity extraction device for emergency cases of autonomous ships. DETAILED DESCRIPTION

[0066] Figure 1 In the present invention, a method for constructing an entity extraction model for autonomous navigation emergency cases includes the following steps:

[0067] S1: Collecting ship emergency corpus data, the emergency corpus data is formed by collecting traditional ship navigation collision accident data and emergency plans in recent years to form a corpus, including multiple historical cases and key information corresponding to emergency plans, and the collected autonomous navigation ship emergency corpus data is used to obtain a training set;

[0068] S2: Construct a BERT-BiLSTM-Att-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to pre-train the input data, match the input data with semantic dynamic word vectors, and output the first layer results; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att, or Self-Attention model layer, the Self-Attention model layer learns the results of the BiLSTM layer (i.e., the second layer results), gives different weights to the context information of each part according to the degree of attention, learns deeper context sequence features, and outputs the third layer results; (4) CRF model layer: used to decode the third layer results (see Figure 2 ), outputting the corresponding annotation results of all characters in the emergency corpus data, wherein the key information and the non-key information have different annotations;

[0069] S3: Use the training set to train the BERT-BiLSTM-Att-CRF extraction model to obtain an entity extraction model for emergency cases of autonomous ships.

[0070] The step S1 further includes the following steps: the key information includes the type of ship, the cause of the accident and the weather conditions, and the training set and the verification set are annotated with text sequences, and the text sequences are annotated with the type of ship, the cause of the accident and the weather conditions. The training set and the verification set are annotated with the BIO method, where B (Begin) indicates that the word is at the beginning of the named entity, I (Inside) indicates that the word is in the named entity, and O (Outside) indicates that the word does not belong to the named entity. Each word is annotated with one of three states.

[0071] Step S1 also includes: obtaining a test set according to the autonomous navigation ship emergency corpus data 。

[0072] Step S3 also includes the following steps: using the extraction model to extract the test set to obtain extraction cases.

[0073] The data set is divided into a training set, a validation set and a test set in a ratio of 8:1:1. The training set, the validation set and the test set respectively include different individual case information and emergency information, that is, key information such as ship types, accident causes and weather conditions of different cases.

[0074] The training set and the validation set are annotated with text sequences. The annotated named entity types include: ship type, accident cause, and weather conditions. The annotation method used is the BIO annotation method, where B (Begin) indicates that the word is at the beginning of the named entity, I (Inside) indicates that the word is in the named entity, and O (Outside) indicates that the word does not belong to the named entity. Each word is annotated into one of three states.

[0075] The BERT-BiLSTM-Att-CRF model in step 2 specifically includes the Bert layer, BILSTM layer, Self-Attention layer, and CRF layer.

[0076] The steps for the Bert layer in step S2 to obtain the dynamic word vector of the input sequence semantics are as follows: input the training set text sequence, add special tags [CLS] and [SEP] at the beginning and end of the text sequence respectively, obtain the position vector, sentence vector and the word vector corresponding to each word, merge them and input them into the bidirectional transformer network to obtain the context-related vector representation of a single word, and average or max-pool all word vectors to obtain a sequence vector with rich semantic features.

[0077] In step S2, the BiLSTM layer learns context features, and uses the sequence vector output by the Bert layer as the input of the BiLSTM layer. The BiLSTM layer consists of a forward LSTM and a backward LSTM. Both the forward LSTM and the backward LSTM include a forget gate, an input gate, an output gate, and a memory unit.

[0078] Forget gate calculation formula: f t =σ(W f [h t-1 , x t ]+b f );

[0079] Input gate calculation formula: i t =σ(W i [h t-1 , x t ]+b i ),

[0080] Unit status update calculation formula:

[0081] Output gate calculation formula: o t =σ(W o [h t-1 , x t ]+b o ),

[0082] Among them, σ represents the sigmoid function, the expression is W f represents the forget gate weight matrix; W i represents the input gate weight matrix; W c represents the unit state matrix; W o represents the output gate weight matrix; h t-1 Indicates the output value at the previous moment; x t represents the current unit input vector; b f represents the bias parameter of the forget gate; b i represents the input gate bias parameter; b c represents the unit state bias parameter; b o Represents the output gate bias parameter.

[0083] The forward LSTM and backward LSTM calculation process formulas are the same, the difference is that the forward LSTM processes the input sequence from left to right, and the backward LSTM processes the input sequence from right to left. The forward LSTM calculates Backward LSTM calculation The context vector is obtained by concatenating the hidden layer state sequence information output by the forward LSTM and the backward LSTM Output hidden layer sequence (h1, h2, h3, h4, ..., h t ).

[0084] In step S2, the fully connected layer performs linear transformation on the feature vector to achieve dimensionality reduction. The output of the BiLSTM layer is used as the input of the fully connected layer to perform the following calculations:

[0085] X=hW FC +b FC ;

[0086] Where h represents the output vector of BiLSTM; W FC represents the weight matrix of the fully connected layer, whose dimension is D×d, where d is the number of entity labels. D is the dimension of the output vector of BiLSTM; b FC Represents the bias vector of the fully connected layer.

[0087] In step 2, the Self-Attention layer gives different weights to the context information of each part according to the degree of attention, and learns deeper context sequence features. The output of the fully connected layer is used as the input of the Self-Attention layer, and the attention weight is given to the hidden layer vector of each time step. The input of the Self-Attention layer is X = [h1,h2,h3,......,h n], where n∈[1,t], X represents a set of input sequences, and for each input sequence, by comparing it with the weight matrix W q , W k , W v Multiply them to get the query vector Q, key vector K and value vector V. Use the multi-head attention mechanism to calculate, set the number of heads num_hea and the hidden layer vector dimension hidden_dim, split Q (query), K (key value) and V (value) according to the number of heads, and perform the following calculations on each head:

[0088]

[0089] Where Q, K, and V represent the query matrix, key matrix, and value matrix respectively; d k Represents the vector dimension of each head,

[0090] Use the scaled dot product as the attention scoring function, and then concatenate the multi-head attention calculation results to get the output vector X = [h att1 ,h att2 ,h att3 ,......,h attn ].

[0091] In step S2, the CRF layer decodes the named entity label sequence and optimizes it to avoid unreasonable labels. The sequence vector output by the Self-Attention layer is used as the input of the CRF layer. The CRF layer needs to learn the input sequence vector to obtain the state transfer matrix and the emission probability matrix. The emission probability matrix represents the probability of converting the feature vector at each position into a label. This matrix will capture the emission law of the label at each position and can be calculated from the output of the Self-Attention layer. The state transfer matrix represents the probability of transferring from one label to another. This matrix will capture the transfer law between labels. The CRF layer state transition probability calculation formula is as follows:

[0092]

[0093] in, Indicates that the jth word is in the yth j The scores under the labels, represents the probability of transferring from the jth label to the j+1th label, represents the true label, Y X Represents the set of all tags.

[0094] The probabilities of all paths are calculated through the forward algorithm, that is, the probabilities of all paths from the starting position of the sequence to each position whose label is a given label are calculated, and the tag sequence with the largest probability is selected as the output tag sequence, which is the entity extraction result obtained according to the model prediction.

[0095] The step S3 also includes the following process: the model is trained using the training set data, and the model training is terminated when the number of iterations reaches the specified number or the loss value meets the set conditions. The accuracy (Precision), recall (Recall) and F1 value of the model are obtained through the validation set as evaluation indicators. The calculation formulas for the accuracy, recall and F1 value evaluation indicators are as follows:

[0096]

[0097]

[0098]

[0099] Where TP represents the number of entities correctly identified, FP represents the number of entities incorrectly identified, and FN represents the number of entities not correctly identified.

[0100] If the evaluation index value is low, it is necessary to adjust hyperparameters such as the number of iterations or learning rate to make the model reach the optimal state. Finally, the test set entity extraction is used to evaluate the performance and generalization ability of the model.

[0101] Figure 3 In the present invention, the autonomous navigation emergency case entity extraction model construction device is composed of the following modules: a corpus data collection module 11, a training set module 12, a BERT-BiLSTM-Att-CRF extraction model construction module 13 and a training extraction model module 14;

[0102] The corpus data collection module 11 is used to collect emergency corpus data of autonomous navigation ships, and the training set module 12 is used to obtain a training set based on the collected emergency corpus data of autonomous navigation ships;

[0103] Constructing the BERT-BiLSTM-Att-CRF extraction model module 13 is used to construct the BERT-BiLSTMAtt-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to pre-train the input data, match the dynamic word vector with semantics for the input data, and output the first layer result; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att, i.e., Self-Attention layer, gives different weights to the context information of each part according to the degree of attention, and learns deeper context sequence features; (4) CRF model layer: used to decode the third layer results and output the results;

[0104] The training extraction model module 14 is used to train the BERT-BiLSTM-Att-CRF extraction model using a training set to obtain an entity extraction model for emergency cases of autonomous navigation ships.

[0105] It also includes a verification set module 15, which is used to obtain a verification set based on the emergency corpus data of the autonomous navigation ship, and to modify the constructed BERT-BiLSTM-Att-CRF extraction model based on the verification set.

[0106] The technical solution provided by the present invention is further described in detail above through several specific implementation modes in order to highlight the advantages and benefits of the technical solution provided by the present invention. However, the several specific implementation modes described above are not intended to be used as limitations on the present invention. Any reasonable modification and improvement of the present invention, combination of implementation modes and equivalent substitution within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

[0107] In the description of this specification, only the preferred embodiments of the present invention are provided, and the scope of the rights of the present invention cannot be limited by this. In addition, the description of the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, in the absence of mutual contradiction, a person skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples. In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" can explicitly or implicitly include at least one of the features.

[0108] A person of ordinary skill in the art can understand that all or part of the steps carried by the method of the above-mentioned embodiment can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium, and the program, when executed, includes one or a combination of the steps of the method embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

Claims

1. A method for constructing an entity extraction model for autonomous navigation emergency cases, characterized by: The following steps are involved: S1: Collecting ship emergency corpus data, the emergency corpus data is formed by collecting traditional ship navigation collision accident data and emergency plans in recent years to form a corpus, including multiple historical cases and key information corresponding to emergency plans, and obtaining a training set based on the collected autonomous navigation ship emergency corpus data; S2: Build a BERT-BiLSTM-Att-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to fine-tune the input corpus data, match the input data with semantic dynamic word vectors, and output the first layer results; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att, or Self-Attention model layer, is used to learn the results of the BiLSTM layer, give different weights to the context information of each part according to the degree of attention, learn deeper context sequence features, and output the third layer results; (4) CRF model layer: used to decode the third layer results and output the corresponding annotation results of all the characters in the emergency corpus data, and the key information and non-key information have different annotations. S3: Use the training set to train the BERT-BiLSTM-Att-CRF extraction model to obtain an entity extraction model for emergency cases of autonomous ships.

2. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 1 is characterized in that The step S1 also includes the following steps: obtaining a verification set based on the emergency corpus data of the autonomous navigation ship, and modifying the constructed BERT-BiLSTM-Att-CRF extraction model based on the verification set.

3. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 2 is characterized in that The step S1 also includes the following steps: the key information includes the type of ship, the cause of the accident and the weather conditions, etc., and the training set and the verification set are annotated with text sequences, and the text sequences are annotated with the type of ship, the cause of the accident and the weather conditions, etc.

4. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 3 is characterized by: The training set and validation set are annotated using the BIO method, where B (Begin) indicates that the word is at the beginning of a named entity, I (Inside) indicates that the word is in a named entity, and O (Outside) indicates that the word does not belong to a named entity. Each word is annotated into one of three states.

5. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 4 is characterized in that The step S1 also includes: obtaining a test set according to the autonomous navigation ship emergency corpus data 。 6. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 5 is characterized by: The step S3 also includes the following steps: using the extraction model to extract the test set to obtain extraction cases.

7. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 6 is characterized by: The data set is divided into a training set, a validation set and a test set in a ratio of 8:1:

1. The training set, the validation set and the test set respectively include different individual case information and emergency information, that is, key information of ship types, accident causes and weather conditions in different cases.

8. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 7 is characterized by: The steps of the Bert model layer matching the input data with dynamic word vectors with semantics are as follows: input a training set text sequence, add [CLS] and [SEP] labels at the beginning and end of the text sequence respectively, obtain a position vector, a sentence vector and a word vector corresponding to each word, merge them and input them into a bidirectional transformer network to obtain a context-related vector representation of a single word, and average or max-pool all word vectors to obtain a sequence vector of semantic features.

9. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 8, characterized in that The BiLSTM model layer is composed of a forward LSTM and a backward LSTM, and both the forward LSTM and the backward LSTM include a forget gate, an input gate, an output gate, and a memory unit; Forget gate calculation formula: ft = σ (Wf [ht-1, xt] + bf); Input gate calculation formula: i t =σ(W i [h t-1 , x t ]+b i ), Unit status update calculation formula: Output gate calculation formula: o t =σ(W o [h t-1 , x t ]+b o ), Among them, σ represents the sigmoid function, the expression is W f represents the forget gate weight matrix; W i represents the input gate weight matrix; W c represents the unit state matrix; W o represents the output gate weight matrix; h t-1 Indicates the output value at the previous moment; x t represents the current unit input vector; b f represents the bias parameter of the forget gate; b i represents the input gate bias parameter; b c represents the unit state bias parameter; b o Represents the output gate bias parameter; The forward LSTM and backward LSTM calculation process formulas are the same, the difference is that the forward LSTM processes the input sequence from left to right, and the backward LSTM processes the input sequence from right to left. The forward LSTM calculates Backward LSTM calculation The context vector is obtained by concatenating the hidden layer state sequence information output by the forward LSTM and the backward LSTM Output hidden layer sequence (h1, h2, h3, h4, ..., h t ).

10. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 9, characterized in that A fully connected layer is set between the BiLSTM model layer and the Self-Attention model layer to perform linear transformation on the feature vector to achieve dimensionality reduction. The output of the BiLSTM layer is used as the input of the fully connected layer to perform the following calculations: X=hW FC +b FC ; Where h represents the output vector of BiLSTM; W FC represents the weight matrix of the fully connected layer, whose dimension is D×d, where d is the number of entity labels. D is the dimension of the output vector of BiLSTM; b FC Represents the bias vector of the fully connected layer.

11. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 10, characterized in that The output of the fully connected layer is used as the input of the Self-Attention model layer. The Self-Attention model layer assigns attention weights to the hidden layer vectors of each time step. The input of the Self-Attention model layer is X = [h1,h2,h3,......,h n ], where n∈[1,t], X represents a set of input sequences, and for each input sequence, by comparing it with the weight matrix W q , W k , W v Multiply them to get the query vector Q, key vector K and value vector V respectively. Use the multi-head attention mechanism to calculate, set the number of heads num_hea and the hidden layer vector dimension hidden_dim, split Q (query), K (key value) and V (value) according to the number of heads, and perform the following calculations on each head: Where Q, K, and V represent the query matrix, key matrix, and value matrix respectively; d k Represents the vector dimension of each head, Use the scaled dot product as the attention scoring function, and then concatenate the multi-head attention calculation results to get the output vector X = [h att1 ,h att2 ,h att3 ,......,h attn ].

12. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 11, characterized in that The decoding process of the CRF model layer in step S2 includes: using the sequence vector output by the Self-Attention layer as the input of the CRF layer, and the CRF model layer learns the input sequence vector to obtain a state transfer matrix and a transmission probability matrix. The transmission probability matrix represents the probability of converting the feature vector at each position into a label. The matrix will capture the emission law of the label at each position and can be calculated from the output of the Self-Attention model layer. The state transfer matrix represents the probability of transferring from one label to another. The matrix will capture the transfer law between labels. The CRF layer state transfer probability calculation formula is as follows: in, Indicates that the jth word is in the yth j The scores under the labels, represents the probability of transferring from the jth label to the j+1th label, represents the true label, Y X Represents the set of all tags; The probabilities of all paths are calculated through the forward algorithm, that is, the probabilities of all paths from the starting position of the sequence to each position whose label is a given label are calculated, and the tag sequence with the largest probability is selected as the output tag sequence, which is the entity extraction result obtained according to the model prediction.

13. The method for constructing an entity extraction model for autonomous navigation emergency cases according to claim 12, characterized in that Step S3 also includes the following process: using the training set data to train the model, when the number of iterations reaches the specified number or the loss value meets the set conditions, the model training is terminated; the accuracy (Precision), recall (Recall) and F1 value of the model are obtained through the validation set as evaluation indicators; the calculation formulas for the accuracy, recall and F1 value evaluation indicators are as follows: Where TP represents the number of correctly identified entities, FP represents the number of incorrectly identified entities, and FN represents the number of entities that were not correctly identified; If the evaluation index value is low, it is necessary to adjust hyperparameters such as the number of iterations or learning rate to make the model reach the optimal state. Finally, the test set entity extraction is used to evaluate the performance and generalization ability of the model.

14. Autonomous navigation emergency case entity extraction model construction device, characterized by It is composed of the following modules: a corpus data collection module (11), a training set module (12), a BERT-BiLSTM-Att-CRF extraction model construction module (13) and a training extraction model module (14); The corpus data collection module (11) is used to collect emergency corpus data of autonomous navigation ships, and the training set module (12) is used to obtain a training set based on the collected emergency corpus data of autonomous navigation ships; Constructing a BERT-BiLSTM-Att-CRF extraction model module (13) is used to construct a BERT-BiLSTMAtt-CRF extraction model, which consists of four model layers: (1) BERT model layer: used to pre-train the input data, match the dynamic word vector with semantics to the input data, and output the first layer result; (2) BiLSTM model layer: used to learn the context features of the first layer results and output the second layer results; (3) Att, i.e., Self-Attention layer, gives different weights to the context information of each part according to the degree of attention, and learns deeper context sequence features; (4) CRF model layer: used to decode the third layer results and output the results; The training extraction model module (14) is used to train the BERT-BiLSTM-Att-CRF extraction model using a training set to obtain an entity extraction model for emergency cases of autonomous navigation ships.

15. The autonomous navigation emergency case entity extraction model building device according to claim 14 is characterized in that It also includes a verification set module (15), which is used to obtain a verification set based on the emergency corpus data of the autonomous navigation ship, and to modify the constructed BERT-BiLSTM-Att-CRF extraction model based on the verification set.

16. The autonomous navigation emergency case entity extraction model building device according to claim 15 is characterized in that It also includes a test set module (16), which is used to obtain a test set based on the autonomous navigation ship emergency corpus data, and to extract the test set using the extraction model to obtain extraction cases.

Images