Maritime affair text entity and relation joint extraction method based on deep reasoning algorithm
By using a deep inference algorithm to process the entity starting point detection, nested entity span recognition, and relation reasoning of maritime texts in parallel, the problem of multi-level nested structures and domain adaptability in maritime texts is solved, and efficient and accurate entity and relation extraction is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- DALIAN MARITIME UNIVERSITY
- Filing Date
- 2026-01-07
- Publication Date
- 2026-04-10
AI Technical Summary
Existing technologies cannot effectively handle multi-level nested structures, have low reasoning efficiency, and poor domain adaptability when processing maritime texts, resulting in insufficient accuracy in extracting entities and relationships from maritime texts.
We employ a deep inference algorithm-based approach, which involves parallel execution of three subtasks: entity origin detection, nested entity span recognition, and entity relationship reasoning. We utilize a non-autoregressive Transformer encoder and a joint loss function for end-to-end training, sharing the same encoder to eliminate error propagation and improve adaptability.
It achieves efficient recognition of nested entities in maritime texts, improves inference speed by 20 to 40 times, and achieves state-of-the-art extraction accuracy on complex datasets, significantly reducing false negative and false positive rates.
Smart Images
Figure CN121835913A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of maritime text entity and relation extraction, in particular, especially relates to a maritime text entity and relation joint extraction method based on a deep reasoning algorithm. BACKGROUND
[0002] Maritime text entity and relation extraction technology is an important research direction in the field of natural language processing (NLP), and its goal is to automatically identify key entities (such as ships, equipment, accident types, etc.) and their mutual relationships from unstructured text in the maritime field (such as accident investigation reports, administrative punishment decisions, etc.). Traditional methods usually use sequence labeling or classification models, but these technologies face many challenges when processing maritime texts. Maritime texts have complex syntactic structures, dense professional terms, and deep semantic dependency levels, for example, "the left oil tank in the engine room of the Far Ocean No. 9" involves a multi-layer nested structure of "ship-part-subpart", and existing models often have difficulty effectively capturing such semantic relationships. In addition, maritime regulatory data is large in size and requires real-time performance, further increasing the difficulty of technical implementation.
[0003] Currently, maritime text entity and relation extraction technology is mainly divided into three categories: pipeline extraction method, joint extraction method based on generation, and joint extraction method based on span. The pipeline method (such as BiLSTM-CRF) first identifies entities and then predicts relationships, but it has error propagation problems and cannot handle nested entities. The joint extraction method based on generation (such as SPN, Tanl) directly generates triples through the Seq2Seq framework, although the accuracy is improved, but its autoregressive decoding method leads to low reasoning efficiency, which is difficult to meet the processing needs of large-scale data. The joint extraction method based on span (such as PL-Marker, UniRel) solves the nesting problem by enumerating possible span combinations, but the computational complexity increases quadratically, which is inefficient for long text processing. In addition, existing technologies generally lack the ability to adapt to maritime professional terms and ambiguous semantics, for example, the subtle difference between "collision" and "touch" in legal terms is ignored, resulting in insufficient accuracy of the extraction results. In summary, existing technologies cannot balance multi-level semantic modeling, efficient reasoning, and domain adaptability, and there is an urgent need for a more optimized solution. SUMMARY
[0004] According to the above-mentioned existing technology in processing maritime texts, it is difficult to simultaneously solve the technical problems of multi-level nesting, low reasoning efficiency, and poor domain adaptability, and a maritime text entity and relation joint extraction method based on a deep reasoning algorithm is provided.
[0005] The technical means adopted by the present application are as follows: A maritime text entity and relation joint extraction method based on a deep reasoning algorithm comprises: S1, perform word segmentation on the input maritime document text to obtain a Token sequence, and map the Token sequence to a context vector sequence using a non-autoregressive Transformer encoder fine-tuned on maritime corpus; S2, construct a deep inference model, and perform an entity starting point detection subtask, a nested entity span identification subtask, and an entity relationship inference subtask in parallel, wherein: perform the entity starting point detection subtask: perform binary classification on each position in the context vector sequence in parallel to obtain the confidence that the position is an entity starting point, and add positions with confidence greater than a threshold to a candidate starting point set; perform the nested entity span identification subtask: for each potential end position in the sequence, backtrack the candidate starting point set within a local window constrained by a nested depth parameter to obtain a candidate span set; perform multi-classification on each candidate span in parallel to obtain its confidence of belonging to each entity type, and output a final entity set according to the nested depth parameter; perform the entity relationship inference subtask: pair all valid entities in the final entity set two by two, and perform multi-label classification in parallel to obtain a confidence set that the entity pair belongs to multiple maritime relationship types at the same time, and filter the final relationship triple set from the confidence set according to a confidence threshold; S3, use a joint loss function to train all learnable parameters in step S2 end-to-end, so that the entity starting point detection, nested entity span identification, and entity relationship inference subtasks share the same encoder and calibrate each other to eliminate error propagation and improve maritime field adaptability.
[0006] Further, step S1 includes: S11, define an entity type set ={ship, location, part, accident type, cause, environment}; S12, define a relationship type set ={collision, located, belong, cause, occur}; S13, divide the text into Token sequences after cleaning , and set the maximum sequence length N; S14, map the Token sequence as an input sequence to a deep context vector sequence , wherein , is the hidden layer dimension.
[0007] Further, in step S2, the entity starting point detection subtask is performed, specifically including: the context vector of each position in the sequence Make independent judgments, predict whether the position is the starting point of an entity, define a prediction function As follows:
[0008] In the above formula, represents a feedforward neural network for starting point detection, and the formula represents that if the network output is greater than 0, the position is determined as the starting point of a potential entity; Make a binary classification prediction for each position in the sequence , and filter all possible entity starting positions in parallel, as follows:
[0009] In the above formula, represents the probability that the position is determined as the starting point of an entity; represents a Sigmoid activation function; represents a learnable weight matrix; represents a learnable scalar bias term; Set a threshold value, when is greater than the first threshold value, then mark the position as a candidate starting point and add it to the candidate starting point set.
[0010] Further, in step S2, the nested entity span recognition subtask is performed, including introducing a nested depth constraint parameter , constructing a search strategy based on a dynamic window, for each potential end position in the sequence, the model only searches for potential starting positions within a limited local context window , and the relative position features between them , to determine whether they constitute an entity of type , specifically including: Define the candidate starting point set of the potential end position as follows:
[0011] Define the prediction function as follows:
[0012] In the above formula, represents the starting point context vector; represents the end point context vector; represents the relative position encoding vector of the starting point and the end point; This represents a feedforward neural network that integrates the start context vector, the end context vector, and the relative position encoding vectors of the start and end points; For the starting point and the end point The pairing is used to calculate its entity type. The probability of is given by the following formula:
[0013] In the above formula, Indicates span Determined as an entity type The probability distribution vector; Represents the learnable weight matrix; This indicates vector concatenation; This represents a learnable bias vector.
[0014] Further, in step S2, the entity relationship reasoning subtask is performed, specifically including: For any two identified entities and It performs aggregation operations on features within the entity span and performs parallel inference on the entity. and Does a relationship exist between them, i.e., is the definition of a relationship prediction function? as follows:
[0015] In the above formula, This represents a feature aggregation function used to extract key semantic information across the entire entity span, rather than relying solely on boundary markers; This indicates the set confidence threshold. Pair all valid entities in the output into a relation classification network and calculate the entity pairs. There is a relationship The confidence score is calculated using the following formula:
[0016] In the above formula, Represents the learnable weight matrix; Indicates the entity Pooling is performed on all vectors within the span; Indicates the entity Pooling is performed on all vectors within the span; This represents a learnable scalar bias term.
[0017] Furthermore, in step S3, the joint loss function used is as follows:
[0018] In the above formula, represents a joint loss function; , , respectively correspond to the cross-entropy loss functions of the three sub-tasks of entity start point detection, span identification and relationship inference; , , is a task balance hyperparameter, used to adjust the weight of each sub-task.
[0019] Compared with the prior art, the present application has the following advantages: 1. The present application has excellent nested entity processing capability. By introducing the nearest left boundary matching mechanism, it can effectively identify the common nested entities (such as ship component attribution) in maritime documents, which is difficult to achieve by traditional sequence labeling methods.
[0020] 2. The present application uses a non-autoregressive architecture, and the three sub-tasks of entity start point detection, nested entity span identification and entity relationship inference can be processed in parallel. Compared with the mainstream autoregressive model (such as ASP), the inference speed of the present application on maritime long text is increased by about 20 to 40 times (for example, more than 600 samples per second per card).
[0021] 3. The method of the present application achieves the SOTA (State-of-the-Art) level in the relationship extraction task (such as a significant improvement in F1 value on complex data sets), effectively reducing the omission rate and the false detection rate. BRIEF DESCRIPTION OF DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.
[0023] Figure 1 is the flowchart of the method of the present application.
[0024] Figure 2 is the schematic diagram of the three sub-task modules of entity start point detection, nested entity span identification and entity relationship inference of the present application.
[0025] Figure 3 is the schematic diagram of the nested entity identification principle provided by the embodiment of the present application.
[0026] Figure 4 is the comparison of the inference throughput of the method of the present application and the traditional regression method.
[0027] Figure 5The accuracy and efficiency of the method of the present application and prior art are compared.
[0028] Figure 6 The ablation experiment results of the present application.
[0029] Figure 7 The inference speed of different models on the maritime data set is compared.
[0030] Figure 8 The performance stability of the method of the present application under different thresholds is analyzed.
[0031] Figure 9 The performance of the method of the present application and the performance of the prior art on the maritime data set are compared. DETAILED DESCRIPTION
[0032] In order for those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.
[0033] It should be noted that the terms "include" and "have" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device containing a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.
[0034] As shown in Figure 1 , the present application provides a maritime text entity and relationship joint extraction method based on a deep reasoning algorithm, comprising: S1, performing word segmentation on the input maritime document text to obtain a Token sequence, and mapping the Token sequence to a context vector sequence using a non-autoregressive Transformer encoder fine-tuned on maritime corpus; S2, as shown in Figure 2 , a deep reasoning model is constructed, and an entity starting point detection subtask, a nested entity span recognition subtask and an entity relationship reasoning subtask are executed in parallel, wherein: The entity starting point detection subtask is executed: binary classification is performed on each position in parallel on the context vector sequence to obtain the confidence that the position is an entity starting point, and positions with confidence greater than a threshold are added to a candidate starting point set; performing a nested entity span identification subtask: for each potential end position in the sequence, backtracking the set of candidate start points within a local window constrained by a nesting depth parameter to obtain a set of candidate spans; performing multi-classification on each candidate span in parallel to obtain its confidence in belonging to each entity type, and outputting a final entity set according to the nesting depth parameter; performing an entity relation reasoning subtask: pairing all valid entities in the final entity set two by two, and performing multi-label classification in parallel to obtain a confidence set that the entity pair belongs to multiple maritime relation types at the same time, and screening a final relation triple set from the confidence set according to a confidence threshold; S3, using a joint loss function to perform end-to-end training on all learnable parameters in step S2, so that the entity start point detection, nested entity span identification and entity relation reasoning three subtasks share the same encoder and calibrate each other to eliminate error propagation and improve maritime field adaptability.
[0035] In specific implementation, as a preferred embodiment of the present application, taking processing of text data in maritime accident investigation reports as an example, 2000 accident investigation reports and administrative punishment decision documents published by the maritime bureau are collected in step S1, and the preprocessing process includes: S11, defining an entity type set ={Ship, Location, Part, Accident, Reason, Environment}; S12, defining a relation type set ={Collided_With, Located_At, Has_Part, Caused_By, Happen_At}; S13, cleaning and cutting the text into a Token sequence , and setting the maximum sequence length N=512; S14, mapping the Token sequence as an input sequence into a deep context vector sequence , wherein , is the hidden layer dimension (such as 1024). In this embodiment, the contextual encoding uses a pre-trained language model DeBERTa-v3-large as a backbone network.
[0036] In specific implementation, as a preferred embodiment of the present application, in step S2, the entity start point detection subtask is performed, specifically including: performing multi-classification on the context vector of each position in the sequence independently judge whether the position is the start point of an entity, define a prediction function As follows:
[0037] In the above formula, represents a feedforward neural network for start point detection, and the formula represents that if the network output is greater than 0, the position is determined as the start point of a potential entity; For each position in the sequence , a binary classification prediction is performed, and all possible entity start positions are screened in parallel, and the formula is as follows:
[0038] In the above formula, represents the probability output of the position being determined as the start point of an entity; represents a Sigmoid activation function; represents a learnable weight matrix; represents a learnable scalar bias term; A threshold is set, and when is greater than a first threshold, the position is marked as a candidate start point and is added to a candidate start point set. In this embodiment, the first threshold is 0.5.
[0039] In specific implementation, as a preferred embodiment of the present application, in step S2, a nested entity span recognition subtask is performed, including introducing a nested depth constraint parameter , constructing a search strategy based on a dynamic window, for each potential end position in the sequence, the model only searches for a potential start position in a limited local context window and relative position features between the two , and determines whether they constitute an entity of type , specifically including: Defining a candidate start point set of the potential end position is as follows:
[0040] Defining a prediction function As follows:
[0041] In the above formula, represents a start point context vector; represents an end point context vector; a relative position encoding vector representing the relative position of the start point and the end point; a feedforward neural network representing the fusion of the start point context vector, the end point context vector, and the relative position encoding vector of the start point and the end point; for a pair of start point and end point , the probability that it belongs to an entity type is calculated, and the formula is as follows:
[0042] In the above formula, denotes the span whose probability distribution vector is determined as an entity type ; denotes a learnable weight matrix; denotes vector splicing; denotes a learnable bias vector. In this embodiment, = 3.
[0043] In this embodiment, in order to solve the problem of insufficient multi-level semantic dependency modeling, a dynamic span matching mechanism based on a start point is cited. Unlike traditional flat labels, a nesting depth parameter is introduced to allow the model to review multiple potential starting points before the current position. Principle: as shown in Figure 3 , for a nested structure like the port side of 'Far Sea No. 9', the model first identifies that both "Far" and "Left" are entity start points in parallel; then when scanning "side", it can pair it with the nearest start point "left" through depth reasoning, while pairing "No." with the start point "Far". Effect: this mechanism can accurately capture the multi-level nested relationships such as "whole-part" and "geographical location-specific water area" in maritime texts, without losing semantic dependencies. The model makes an independent judgment on the context vector of each position in the sequence to predict whether the position is the starting point of an entity.
[0044] In specific implementation, as a preferred embodiment of the present application, in step S2, the entity relationship reasoning subtask is performed, specifically including: for any two identified entities and , an aggregation operation (Aggregation) is performed on the features within the entity span, and it is inferred in parallel whether there is a relationship between the entities and , i.e. defining the relationship prediction function as follows:
[0045] In the above formula, denotes a feature aggregation function that extracts key semantic information within the whole entity span, rather than relying on boundary tokens only; denotes a set confidence threshold; in the embodiment, the above formula supports multi-label classification, that is, multiple semantic relations can exist between a pair of entities at the same time.
[0046] All valid entities in the output are paired, input into the relationship classification network, and the confidence score of the entity pair has a relationship is calculated, and the formula is as follows:
[0047] In the above formula, denotes a learnable weight matrix; denotes a pooling operation on all vectors within the span of the entity ; denotes a pooling operation on all vectors within the span of the entity ; denotes a learnable scalar bias term.
[0048] In specific implementation, as a preferred embodiment of the present application, in view of the data distribution characteristics of entity-intensive and relation-sparse in maritime texts, in order to balance the gradient contributions of entity start point detection, span identification and relationship reasoning in the training process, and prevent simple tasks from dominating the loss function, a weighted joint loss function is constructed in step S3 as follows:
[0049] In the above formula, denotes a joint loss function; , , respectively correspond to the cross-entropy loss of the three sub-tasks of entity start point detection, span identification and relationship reasoning; , , are task balancing hyperparameters, used to adjust the weights of each sub-task. By jointly optimizing the loss function, the model can use high-level relationship semantic information to reverse-calibrate the bottom-level entity recognition results, thereby effectively improving the overall extraction performance in complex maritime contexts.
[0050] In this embodiment, in view of the poor adaptability in the maritime field, the application adopts a feature sharing and joint optimization strategy. Principle: the three sub-tasks of entity start point detection, span identification and relationship reasoning share the same deep context representation generated by the Transformer encoder fine-tuned on maritime corpus. During training, a joint loss function is used for end-to-end optimization. Effect: the shared features enable the model to utilize relationship information to reverse-calibrate the results of entity recognition (for example, through the relationship of "collision", it is inferred that the subject must be "ship" rather than "company"), thereby effectively processing ambiguous semantics and professional terms in maritime texts, and significantly improving the adaptability and robustness in the vertical field.
[0051] As shown in Figure 4 , the method of the application has a high throughput, adopts a non-autoregressive architecture, and all three reasoning sub-task steps can be processed in parallel. Compared with mainstream autoregressive models (such as ASP), the reasoning speed of the method on maritime long texts is increased by about 20 to 40 times (for example, more than 600 samples per second per card).
[0052] As shown in Figure 5 , 6 , they are comprehensive experimental effect diagram and ablation experiment results respectively, from Figure 5 , it can be seen that the horizontal axis is the model category, and the BiLSTM-CRF, ASP and the method of the application are in turn; the vertical axis is the F1 percentage. The bar chart shows that the F1 of the joint entity and relationship extraction of the method of the application on the complex maritime dataset reaches 88.0%, which is 6.8 percentage points higher than the strongest autoregressive baseline ASP, and the experiment proves that the deep reasoning architecture reaches the SOTA (State-of-the-Art) level on the relationship extraction task (such as a significant increase in F1 value on complex datasets), effectively reducing the missed detection rate and false detection rate.
[0053] As shown in Figure 7 , 8 , 9, they are the reasoning speed comparison of different models on the maritime dataset, the performance stability analysis of the method of the application under different threshold values, and the performance comparison of the method of the application and the existing mainstream technology on the maritime dataset, from the figures, it can be seen that compared with the autoregressive model, the method of the application does not need serial decoding, and all calculations can be completed in parallel on the Transformer encoder. Experiments show that while maintaining the SOTA level of accuracy (such as F1 value on complex datasets exceeding 71.9%), the reasoning throughput of the architecture is increased by more than 20 times (600+ samples per second per card), perfectly solving the contradiction between efficiency and accuracy.
[0054] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions recorded in the above embodiments can be modified, or some or all of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.
Claims
1. A method for jointly extracting maritime text entities and relations based on deep reasoning algorithms, characterized in that, include: S1. Segment the input maritime official document text to obtain a token sequence, and use a non-autoregressive Transformer encoder finely tuned by the maritime corpus to map the token sequence into a context vector sequence. S2. Construct a deep reasoning model and execute the entity origin detection subtask, nested entity span recognition subtask, and entity relationship reasoning subtask in parallel, wherein: Perform the entity origin detection subtask: perform binary classification on each position in parallel on the context vector sequence, obtain the confidence that the position is the entity origin, and add the positions with confidence greater than the threshold to the candidate origin set; Perform the nested entity span identification subtask: For each potential end position in the sequence, backtrack the candidate start point set within a local window constrained by the nesting depth parameter to obtain a candidate span set; perform multi-classification in parallel for each candidate span to obtain its confidence level belonging to each entity type, and output the final entity set according to the nesting depth parameter; Perform the entity relationship reasoning subtask: pair up all valid entities in the final entity set, perform multi-label classification in parallel to obtain the confidence set of the entity pair belonging to multiple maritime relationship types, and filter out the final relationship triplet set from the confidence set according to the confidence threshold. S3. Use a joint loss function to train all learnable parameters in step S2 end-to-end, so that the three sub-tasks of entity origin detection, nested entity span recognition and entity relationship reasoning share the same encoder and calibrate with each other, so as to eliminate error propagation and improve the adaptability in the maritime field.
2. The method for joint extraction of maritime text entities and relations based on deep reasoning algorithm according to claim 1, characterized in that, Step S1 includes: S11. Define the entity type collection ={Vessel, Location, Part, Accident Type, Cause, Environment}; S12, Define the set of relation types ={collision, located at, belong to, cause, occur at}; S13. After cleaning the text, segment it into token sequences. And set the maximum sequence length N; S14. The token sequence is used as an input sequence and mapped to a deep context vector sequence. ,in , For the hidden layer dimension.
3. The method for joint extraction of maritime text entities and relations based on deep reasoning algorithm according to claim 1, characterized in that, In step S2, the entity starting point detection subtask is executed, which specifically includes: Context vector for each position in the sequence Perform an independent judgment to predict whether the location is the starting point of the entity, and define a prediction function. ,as follows: In the above formula, This represents a feedforward neural network used for origin detection. The formula indicates that if the network output is greater than 0, the position is determined. The starting point for potential entities; For each position in the sequence Perform binary classification prediction and filter all possible starting positions of entities in parallel, as shown in the following formula: In the above formula, Indicates position Output the probability of being identified as the origin of an entity; This represents the Sigmoid activation function; Represents the learnable weight matrix; Represents a learnable scalar bias term; Set a threshold, when If the value is greater than the first threshold, then mark the position. Select a candidate starting point and add it to the candidate starting point set.
4. The method for joint extraction of maritime text entities and relations based on deep reasoning algorithm according to claim 1, characterized in that, In step S2, the nested entity span recognition subtask is performed, including introducing nesting depth constraint parameters. A search strategy based on dynamic windows is constructed for each potential end position in the sequence. The model only backtracks to search for potential starting positions within a limited local context window. and the relative positional characteristics between the two. Determine whether they constitute type . The entities specifically include: Define potential end position Candidate starting point set as follows: Define prediction function ,as follows: In the above formula, Represents the starting context vector; Represents the endpoint context vector; A vector representing the relative position of the starting and ending points; This represents a feedforward neural network that integrates the start context vector, the end context vector, and the relative position encoding vectors of the start and end points; For the starting point and the end point The pairing is used to calculate its entity type. The probability of is given by the following formula: In the above formula, Indicates span Determined as an entity type The probability distribution vector; Represents the learnable weight matrix; This indicates vector concatenation; This represents a learnable bias vector.
5. The method for joint extraction of maritime text entities and relations based on deep reasoning algorithm according to claim 1, characterized in that, In step S2, the entity relationship reasoning subtask is executed, which specifically includes: For any two identified entities and It performs aggregation operations on features within the entity span and performs parallel inference on the entity. and Does a relationship exist between them, i.e., is the definition of a relationship prediction function? as follows: In the above formula, This represents a feature aggregation function used to extract key semantic information across the entire entity span, rather than relying solely on boundary markers; This indicates the set confidence threshold. Pair all valid entities in the output into a relation classification network and calculate the entity pairs. There is a relationship The confidence score is calculated using the following formula: In the above formula, Represents the learnable weight matrix; Indicates the entity Pooling is performed on all vectors within the span; Indicates the entity Pooling is performed on all vectors within the span; This represents a learnable scalar bias term.
6. The method for joint extraction of maritime text entities and relations based on deep reasoning algorithm according to claim 1, characterized in that, In step S3, the joint loss function used is as follows: In the above formula, Represents the joint loss function; , , The cross-entropy loss functions correspond to the three sub-tasks of entity origin detection, span recognition, and relationship reasoning, respectively. , , This is a task balancing hyperparameter used to adjust the weights of each subtask.