Accident element extraction model training method, extraction method, content generation method

Abstract

The present application relates to the accident element extraction model training method, extraction method, content generation method. In the training process, for each extraction entity word, the character range expansion method is adopted based on the character range in the text sequence to obtain at least one expansion entity word corresponding to each extraction entity word. For each expansion entity word, its corresponding explanation definition text is searched in the knowledge base, and each expansion entity word is input into the trained first large model after being matched with its corresponding explanation definition text to obtain the judgment result of whether each expansion entity word belongs to the accident element entity type of the corresponding extraction entity word. The judgment result factor is integrated into the subsequent loss value calculation process. That is, the "knowledge search large model verification feature enhancement network" is added for training, and the accuracy and recall rate of the extraction of the corresponding element content corresponding entity word task are better.

Description

Technical Field

[0001] Several embodiments of this specification relate to the field of accident element content extraction technology, specifically to an accident element extraction model training method, extraction method, and content generation method. Background Technology

[0002] In numerous fields such as industrial production, transportation, and public safety, accident reports are crucial documents for recording and analyzing the process, causes, and consequences of accidents. To effectively manage and utilize this information and reduce the occurrence of similar accidents in the future, it is currently urgent to extract key elements from the text sequence of accident reports. These elements typically include, but are not limited to: the specific location of the accident (accident site), the direct or indirect factors leading to the accident (cause of the accident), the time of the accident, the duration of the accident (accident duration), the response and resolution measures taken (accident handling methods), and the direct or indirect financial losses caused by the accident (economic losses).

[0003] Accurately and comprehensively extracting these elements is fundamental for subsequent in-depth accident analysis and experience summarization. By conducting correlation analysis on the extracted elements, patterns of accident occurrence, high-risk stages, and effective prevention or handling strategies can be identified, forming valuable accident experience summaries. These summary reports can provide reference for decision-makers, managers, and frontline workers in relevant fields, helping them identify potential risks, optimize operational processes, and improve safety measures, thereby effectively avoiding or reducing the occurrence of similar accidents in subsequent activities and ensuring personnel and property safety.

[0004] To address the task of extracting the aforementioned elements, several solutions have been proposed in existing technologies. For example, some studies have employed global pointer networks (GPNs) for information extraction. GPNs learn the relationship between any two positions in a text sequence, attempting to directly locate the start and end positions containing the target element, theoretically capable of capturing long-distance dependencies. However, practical experience shows that existing GPN-based extraction schemes still suffer from insufficient accuracy and recall in real-world applications. This performance deficiency limits the feasibility of summarizing high-quality accident experience based on the extraction results and makes it difficult to fully meet the needs of in-depth mining and effective utilization of accident information in practical applications. Summary of the Invention

[0005] This specification provides an embodiment of a method for training an accident element extraction model, an extraction method, and a content generation method.

[0006] The technical solution is as follows:

[0007] This specification provides an embodiment of a method for training an accident element extraction model, including:

[0008] Input text sequences into the BERT network encoding layer;

[0009] The BERT network encoding layer processes the text sequence to output a vector sequence, and inputs the vector sequence into the global pointer network layer;

[0010] The global pointer network layer obtains the score matrix corresponding to each accident element entity type. The score matrix corresponding to each accident element entity type includes the predicted score of the initial entity word belonging to that accident element entity type under different position ranges, and extracts at least one extracted entity word belonging to each accident element entity type.

[0011] For each extracted entity word, a method of expanding the character range based on the character range in the text sequence is adopted to obtain at least one expanded entity word corresponding to each extracted entity word.

[0012] For each expanded entity word, its corresponding explanation and definition text is searched in the knowledge base. Each expanded entity word is then paired with its corresponding explanation and definition text and input into the trained first model to obtain the judgment result of whether each expanded entity word belongs to the accident element entity type of its corresponding extracted entity word.

[0013] Based on the judgment result corresponding to each expanded entity word and the accident element entity type corresponding to each of all extracted entity words, all text characters in the text sequence are used as nodes to construct graph networks corresponding to multiple accident element entity types, and the adjacency matrix of the graph network corresponding to multiple accident element entity types is obtained.

[0014] The loss function value is calculated based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type.

[0015] Based on the calculated loss function values, the parameters of the global pointer network layer and the BERT network coding layer are optimized.

[0016] Repeat the above steps until the preset requirements are met to obtain the feature extraction model.

[0017] As a preferred embodiment, the global pointer network layer obtains the score matrix corresponding to each accident element entity type. Each score matrix for each accident element entity type includes the predicted scores for the initial entity words belonging to that accident element entity type under different positional ranges. At least one extracted entity word corresponding to each accident element entity type is also extracted, including:

[0018] The two linear layers in the global pointer network layer obtain multiple start position vectors and multiple end position vectors corresponding to each accident element entity type based on the input vector sequence.

[0019] Iterates through all possible combinations of start and end positions. For each accident element entity type, it takes the start and end position vectors corresponding to each position combination and injects position information through the rotation position encoding network layer in the global pointer network layer. This process yields the score matrix corresponding to each accident element entity type. The score matrix for each accident element entity type includes the predicted score of the initial entity words belonging to that accident element entity type under the position range corresponding to different position combinations. Based on the score matrix corresponding to each accident element entity type, it extracts at least one entity word belonging to each accident element entity type from the initial entity words under all different position ranges.

[0020] As a preferred embodiment, each extracted entity word corresponds to at least three expanded entity words, and among the at least three expanded entity words, there is at least one expanded entity word that expands forward only relative to the beginning of the character range of the extracted entity word, at least one expanded entity word that expands backward only relative to the end of the character range of the extracted entity word, and at least one expanded entity word that expands forward both relative to the beginning of the character range of the extracted entity word and relative to the end of the character range of the extracted entity word.

[0021] As a preferred embodiment, the calculation of the loss function value based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type includes:

[0022] Based on the score matrix corresponding to each accident element entity type and the extraction of words corresponding to each accident element entity type, the first loss value is obtained.

[0023] The second loss value is obtained based on the graph network adjacency matrix corresponding to each of the multiple accident element entity types, the multiple start position vectors corresponding to each accident element entity type, and the multiple end position vectors.

[0024] The loss function value is calculated based on the first loss value and the second loss value.

[0025] As a preferred embodiment, the BERT network coding layer processes the text sequence to output a vector sequence that includes the character vector corresponding to each text character in the text sequence, as well as the aggregated representation feature vector of the entire text sequence.

[0026] The training method also includes:

[0027] The aggregated representation feature vector of the entire text sequence is input into the industry domain classification layer, and the industry domain classification layer outputs the industry domain classification result based on the aggregated representation feature vector;

[0028] The loss function value is also calculated based on a third loss value obtained from the industry domain classification result and the real domain classification result;

[0029] The parameter optimization of the global pointer network layer and the BERT network coding layer based on the calculated loss function value includes:

[0030] Based on the calculated loss function values, the parameters of the global pointer network layer, the BERT network encoding layer, and the industry domain classification layer are optimized.

[0031] The feature extraction model also includes a parameter-optimized industry domain classification layer connected to the parameter-optimized BERT network encoding layer.

[0032] As a preferred embodiment, the formula for calculating the first loss value is as follows:

[0033] ;

[0034] in, This represents the first loss value. Indicates the total number of entity types of accident elements. Indicates the starting position and end position The corresponding initial entity word belongs to the first Predicted scores for various accident element entity types This indicates that among all initial entity words, those considered not to belong to the first... The set of start and end positions of entity words extracted for each type of accident element entity. This indicates that among all initial entity words, it is considered to belong to the first... The set of start and end positions of the extracted entity words for each type of accident element entity.

[0035] As a preferred option:

[0036] ;

[0037] in, Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the starting position and end position The rotation matrix corresponding to the character range between them. This indicates transpose.

[0038] As a preferred option:

[0039] ;

[0040] ;

[0041] in, This represents the second loss value. Indicates the total number of entity types of accident elements. Indicates the first Type loss value corresponding to the entity type of each accident element. Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the first The graph network adjacency matrix corresponding to the entity types of accident elements. express The Middle Line number The elements in the column, and The value is the first Starting position in the graph network corresponding to the entity type of each accident element Corresponding text characters and end positions The weights of the edges between corresponding text characters. It is a matrix, and Each column in , It is also a matrix, and Each column , yes diagonal matrix, The elements on the diagonal are diagonal elements, All other elements are 0. Indicates calculation A function of the trace of a matrix.

[0042] Secondly, embodiments of this specification provide an extraction method, including:

[0043] Obtain the target text sequence;

[0044] Using the element extraction model trained by the method described in the first aspect of the above embodiments, at least one target extracted entity word corresponding to each accident element entity type in the target text sequence is extracted.

[0045] Thirdly, embodiments of this specification provide a content generation method, including:

[0046] The feature extraction model trained by the method described in the first aspect of the above embodiments extracts features from all historical text sequences in the historical text sequence set, so as to extract the extraction entity word set corresponding to each historical text sequence. Each extraction entity word set includes at least one extraction entity word belonging to each accident feature entity type.

[0047] Based on the extracted entity word sets corresponding to each historical text sequence, statistical analysis of accident element association is performed, and the results of the statistical analysis of accident element association are stored in the database.

[0048] Retrieve user interaction task instructions;

[0049] Based on the content of user interaction task instructions, obtain the statistical analysis results of accident elements related to the content of user interaction task instructions in the database;

[0050] The user interaction task instruction content and the statistical analysis results of related accident elements in the database are assembled and input into the second major model, and the decision content is output.

[0051] Fourthly, embodiments of this specification provide an electronic device, including a processor and a memory; the processor is connected to the memory; the memory is used to store executable program code; the processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to perform the steps described in the first aspect of the above embodiments.

[0052] Fifthly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the steps described in the first aspect of the above embodiments.

[0053] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0054] During training, for each extracted entity word, a method of expanding its character range based on its position in the text sequence is used to obtain at least one expanded entity word for each extracted entity word. For each expanded entity word, its corresponding explanatory definition text is searched in the knowledge base. Each expanded entity word is then paired with its corresponding explanatory definition text and input into the trained first-level model to determine whether each expanded entity word belongs to the same accident element entity type as its corresponding extracted entity word. This judgment result is incorporated into the subsequent loss calculation. In other words, a "feature enhancement network validated by a knowledge search model" is added during training. Compared to the element extraction model trained without this network, this model exhibits better accuracy and recall in extracting entity words corresponding to the content of relevant elements.

[0055] The system adds an industry category classification function, which allows the extracted information to be matched with industry category information, thereby enabling better subsequent statistical analysis of accident elements.

[0056] The system extracts accident element entities and performs statistical analysis on their correlations, storing the results in a database. When a user inputs an interactive task command, the system retrieves the related statistical analysis results from the database, assembles them, and inputs them into the second main model to generate the required accident analysis and decision support report. Therefore, in addition to its accident element entity extraction function, it also generates accident analysis and decision support reports. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is a flowchart illustrating a training method for an accident element extraction model provided in the embodiments of this specification.

[0059] Figure 2 This is a flowchart illustrating a content generation method provided in an embodiment of this specification.

[0060] Figure 3 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0061] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings.

[0062] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0063] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0064] Reference Figure 1 As shown, Figure 1 A flowchart illustrating a method for training an accident element extraction model, provided in one embodiment of this specification, may include at least:

[0065] Step 102: Input the text sequence into the BERT network encoding layer.

[0066] Step 104: The BERT network encoding layer processes the text sequence to output a vector sequence and inputs the vector sequence into the global pointer network layer.

[0067] BERT (Bidirectional Encoder Representations from Transformers) is a highly efficient pre-trained natural language processing (NLP) model. Its core feature lies in utilizing the Transformer architecture and employing a bidirectional training method to understand the context of language. During training, the BERT model learns bidirectional feature representations of words across all network layers, fully absorbing contextual information from both the left and right sides of words. It has been successfully applied in tasks such as sentence inference, question-answering matching, named entity recognition, and information extraction.

[0068] In the BERT model, the last layer of the network outputs a sequence of vectors encoded by the BERT network. The encoded vector at the marker position with the symbol [CLS] can be used as the aggregate representation feature of the entire sequence and can be used for classification tasks (Note: classification tasks will be described in more detail later, but this content is introduced here for the sake of convenience).

[0069] The input text sequence for the BERT network coding layer is Its length is The [CLS] and [SEP] symbols are added to the beginning and end of each sentence, respectively. Each input character in the input text sequence is processed through a word embedding layer to obtain a vector sequence. The vector corresponding to each character is obtained by adding the word embedding vector, the separator embedding vector, and the positional encoding embedding vector. The first word of each sequence is the embedding vector of the special symbol [CLS] used for classification. BERT processes the input sequence... Encoding is performed. The final encoded vector sequence output is... .

[0070] Vector sequence encoded by BERT network The input to the global pointer network layer yields the predicted score for the span of each accident element entity type. The encoded vector corresponding to the [CLS] position is then input to the industry classification layer to obtain the industry category results involved in the accident text (Note: Industry classification will be described in detail below; it is mentioned here for ease of explanation).

[0071] Step 106: The global pointer network layer obtains the score matrix corresponding to each accident element entity type. The score matrix corresponding to each accident element entity type includes the predicted score of the initial entity word belonging to the accident element entity type under different position ranges. Based on the score matrix corresponding to each accident element entity type, at least one extracted entity word belonging to each accident element entity type is extracted from the initial entity words under all different position ranges (Note: For example, when the predicted score of the initial entity word belonging to the accident element entity type is greater than the preset value, it is determined to be an extracted entity word belonging to the accident element entity type).

[0072] Among them, the entity type of accident element can be, but is not limited to, "accident type", "accident initiator", "accident location", "accident duration", etc.

[0073] Step 108: For each extracted entity word, the character range is expanded based on the character range in the text sequence to obtain at least one expanded entity word corresponding to each extracted entity word.

[0074] Step 110: For each expanded entity word, search the knowledge base to obtain its corresponding explanation and definition text, and input each expanded entity word with its corresponding explanation and definition text into the trained first model to obtain the judgment result of whether each expanded entity word belongs to the accident element entity type of its corresponding extracted entity word.

[0075] Step 112: Based on the judgment results corresponding to each expanded entity word and the accident element entity type to which each extracted entity word belongs, all text characters in the text sequence are used as nodes to construct graph networks corresponding to multiple accident element entity types. In the graph network corresponding to each accident element entity type, the edge weight between two text characters in the same entity word belonging to that accident element entity type is set to 1 and the edge weight between the remaining text characters is set to 0. Based on the graph networks corresponding to multiple accident element entity types, the adjacency matrix of the graph network corresponding to each of the multiple accident element entity types is obtained.

[0076] Step 114: Calculate the loss function value based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type.

[0077] Step 116: Based on the calculated loss function value, optimize the parameters of the global pointer network layer and the BERT network coding layer.

[0078] Step 118: Repeat the above steps until the preset requirements are met to obtain the feature extraction model, which includes a connected parameter-optimized BERT network encoding layer and a parameter-optimized global pointer network layer.

[0079] Understandably, during training, for each extracted entity word, a method is used to expand its character range based on the character range in the text sequence to obtain at least one expanded entity word corresponding to each extracted entity word. Furthermore, for each expanded entity word, its corresponding explanatory definition text is searched in the knowledge base. Each expanded entity word, combined with its corresponding explanatory definition text, is then input into the trained first large-scale model to determine whether each expanded entity word belongs to the same accident element entity type as its corresponding extracted entity word. This judgment result is incorporated into the subsequent loss value calculation. In other words, a "feature enhancement network validated by the knowledge search large-scale model" is added during training. Compared to the element extraction model trained without this "feature enhancement network validated by the knowledge search large-scale model," this model exhibits better accuracy and recall in the task of extracting entity words corresponding to the content of relevant elements.

[0080] In several embodiments of this specification, the global pointer network layer obtains the score matrix corresponding to each accident element entity type. Each score matrix for each accident element entity type includes the predicted scores for the initial entity words belonging to that accident element entity type under different positional ranges. Based on the score matrix corresponding to each accident element entity type, at least one extracted entity word belonging to that accident element entity type is extracted from all the initial entity words under different positional ranges, including:

[0081] The two linear layers in the global pointer network layer obtain multiple start position vectors and multiple end position vectors corresponding to each accident element entity type based on the input vector sequence.

[0082] Iterates through all possible combinations of start and end positions. For each accident element entity type, it takes the start and end position vectors corresponding to each position combination and injects position information through the rotation position encoding network layer in the global pointer network layer. This process yields the score matrix corresponding to each accident element entity type. The score matrix for each accident element entity type includes the predicted score of the initial entity words belonging to that accident element entity type under the position range corresponding to different position combinations. Based on the score matrix corresponding to each accident element entity type, it extracts at least one entity word belonging to each accident element entity type from the initial entity words under all different position ranges.

[0083] The Global Pointer Network (GPN) utilizes the concept of global normalization, considering the start and end positions of accident element entities from a global perspective, thus treating the span of the first and last combination of accident element entities as a whole. As a decoding method based on span classification, for any text sentence, the GPN constructs an upper triangular matrix to traverse all valid spans.

[0084] For any single sentence text, the global pointer network constructs an upper triangular matrix to traverse all possible spans of accident elements and calculates the score for each span to be classified as an element of a certain type.

[0085] First, the vector sequence encoded by the BERT network layers is... Inputting two linear layers yields the start and end position vectors for each accident element entity type. This is represented as:

[0086] (11)

[0087] (12)

[0088] ;

[0089] In the formula: Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the first End position corresponding to each accident element entity type End position vector below and The first The entity first matrix parameters corresponding to the entity type of the accident element and the first Entity tail matrix parameters corresponding to the entity type of each accident element; and For the first Two bias terms corresponding to the entity type of each accident element; Indicates the length of the vector sequence.

[0090] By iterating through all combinations of different start and end positions, the entity score for each accident element entity type is calculated using the following formula. For each accident element entity type's start position vector... and the ending position vector After the position information is injected through the rotation position encoding network layer, the starting position is... and end position The vocabulary composed of the character range is the accident element entity type. The predicted score is:

[0091] ;

[0092] in, Indicates the starting position and end position The corresponding initial entity word belongs to the first Predicted scores for various accident element entity types Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the starting position and end position The rotation matrix corresponding to the character range between them. This indicates transpose.

[0093] in:

[0094] ;

[0095] ;

[0096] ;

[0097] In the formula: The parameters to be solved (specifically, solved using gradient descent via the Adam optimization solver). and Regarding location Rotation position encoding will calculate a rotation matrix. (Note: Regarding location) Similarly); rotation position encoding generates a rotation matrix for each position, and satisfies .

[0098] In several embodiments of this specification, the calculation of the loss function value based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type includes:

[0099] Based on the score matrix corresponding to each accident element entity type and the extraction of words corresponding to each accident element entity type, the first loss value is obtained.

[0100] The second loss value is obtained based on the graph network adjacency matrix corresponding to each of the multiple accident element entity types, the multiple start position vectors corresponding to each accident element entity type, and the multiple end position vectors.

[0101] The loss function value is calculated based on the first loss value and the second loss value.

[0102] The formula for calculating the first loss value is as follows:

[0103] ;

[0104] in, This represents the first loss value. Indicates the total number of entity types of accident elements. Indicates the starting position and end position The corresponding initial entity word belongs to the first Predicted scores for various accident element entity types This indicates that among all initial entity words, those considered not to belong to the first... The set of start and end positions of entity words extracted for each type of accident element entity. This indicates that among all initial entity words, it is considered to belong to the first... The set of start and end positions of the extracted entity words for each type of accident element entity.

[0105] in, is the base of the natural logarithm. It is approximately equal to 2.71828.

[0106] To improve the recall and precision of entity extraction for accident elements, this invention designs a method to add several word windows before and after the entity words in the extraction results from the global pointer network layer, such as the start position and end position being... and entity words We added an expansion window with a size range of 2 or less before and after, generating a larger set of expanded full words:

[0107] ;

[0108] in Indicates the position from the beginning of the original text. To the end position The meanings of the string fragments between them can be deduced from the meanings of other elements, such as: Indicates the position from the beginning of the original text. To the end position The string fragments between.

[0109] Furthermore, in several embodiments of this specification, each extracted entity word corresponds to at least three expanded entity words, and among the at least three expanded entity words, there is at least one expanded entity word that expands forward only relative to the beginning of the character range of the extracted entity word, at least one expanded entity word that expands backward only relative to the end of the character range of the extracted entity word, and at least one expanded entity word that expands forward both relative to the beginning of the character range of the extracted entity word and relative to the end of the character range of the extracted entity word.

[0110] That is, for each extracted entity word, at least the above three modes were expanded to improve the recall and precision of the extracted event element entities.

[0111] Furthermore, these expanded entity words are input into the local knowledge base system and knowledge search engine to obtain the explanatory definition text for each expanded entity word. Next, each expanded entity word, along with its corresponding explanatory definition text, is input into the trained first model to obtain a judgment result on whether each expanded entity word belongs to the entity type of the accident element to which its corresponding extracted entity word belongs.

[0112] It should be explained that the first major model is pre-trained. Its training process requires a pre-prepared training set. Each training sample in the training set contains an entity word, its corresponding explanatory definition text, and the actual accident element entity type to which the entity word belongs. Subsequently, the entity word and its corresponding explanatory definition text are input into the first major model to output the predicted accident element entity type. A loss value is then calculated based on the predicted accident element entity type and the actual accident element entity type to which the entity word belongs, and this loss value is used to train the first major model. Further details will not be elaborated upon here.

[0113] Furthermore, based on the judgment results corresponding to each expanded entity word and the accident element entity type to which each extracted entity word belongs, all text characters in the text sequence are used as nodes to construct graph networks corresponding to multiple accident element entity types. In the graph network corresponding to each accident element entity type, the edge weight between two text characters in the same entity word belonging to that accident element entity type is set to 1, and the edge weight between all other text characters is set to 0. Based on the graph networks corresponding to multiple accident element entity types, the adjacency matrix of the graph network corresponding to each of the multiple accident element entity types is obtained.

[0114] Therefore, we have the first A graph network adjacency matrix was established for each type of accident element entity. Graph network adjacency matrix Each element in The possible values ​​are as follows:

[0115] ;

[0116] in, Indicates the position from the beginning of the original text. To the end position The string fragments between. To determine as the first The set of accident element entity words for each type of accident element entity. This includes those considered to belong to the first The entity words extracted from the accident element entity types also include those considered by the first major model to belong to the second type. Expanded entity terms for various accident element entity types.

[0117] The formula for calculating the second loss value is as follows:

[0118] ;

[0119] ;

[0120] in, This represents the second loss value. Indicates the total number of entity types of accident elements. Indicates the first Type loss value corresponding to the entity type of each accident element. Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the first The graph network adjacency matrix corresponding to the entity types of accident elements. express The Middle Line number The elements in the column, and The value is the first Starting position in the graph network corresponding to the entity type of each accident element Corresponding text characters and end positions The weights of the edges between corresponding text characters. It is a matrix, and Each column in , It is also a matrix, and Each column , yes diagonal matrix, The elements on the diagonal are diagonal elements, All other elements are 0. Indicates calculation A function of the trace of a matrix.

[0121] In several embodiments of this specification, the BERT network coding layer processes the text sequence to output a vector sequence that includes a character vector corresponding to each text character in the text sequence, as well as an aggregated representation feature vector of the entire text sequence (Note: i.e., the one mentioned above). );

[0122] The training method also includes:

[0123] The aggregated representation feature vector of the entire text sequence is input into the industry domain classification layer, and the industry domain classification layer outputs the industry domain classification result based on the aggregated representation feature vector;

[0124] The loss function value is also calculated based on a third loss value obtained from the industry domain classification result and the real domain classification result;

[0125] The parameter optimization of the global pointer network layer and the BERT network coding layer based on the calculated loss function value includes:

[0126] Based on the calculated loss function values, the parameters of the global pointer network layer, the BERT network encoding layer, and the industry domain classification layer are optimized.

[0127] The feature extraction model also includes a parameter-optimized industry domain classification layer connected to the parameter-optimized BERT network encoding layer.

[0128] The industry classification layer consists of a fully connected layer, and the objective function for the classification task is the predicted industry classification result. And the real domain classification results The cross-entropy loss function between them.

[0129] The formula for calculating the third loss value is as follows:

[0130] ;

[0131] That is, an industry category classification function has been added, which can match the extracted information with industry information, thereby enabling better subsequent statistical analysis of accident elements.

[0132] The multi-task global pointer network proposed in this study extracts accident element entities and classifies accident events by industry domain. The objective function for the overall training and optimization of the multi-task global pointer network is to minimize the following loss function:

[0133] .

[0134] It should be noted that in the above training optimization process, multiple text sequences can be input in batches for each training round to improve training efficiency.

[0135] This specification also provides an extraction method, including:

[0136] Obtain the target text sequence;

[0137] The element extraction model trained using the method described in the above embodiments extracts at least one target extracted entity word corresponding to each accident element entity type in the target text sequence.

[0138] Reference Figure 2 As shown in the embodiments of this specification, a content generation method is also provided, the method including:

[0139] The feature extraction model trained by the method described in the above embodiments is used to extract features from all historical text sequences in the historical text sequence set, so as to extract the corresponding extraction entity word set for each historical text sequence. Each extraction entity word set includes at least one extraction entity word belonging to each accident element entity type.

[0140] Based on the extracted entity word sets corresponding to each historical text sequence, statistical analysis of accident element association is performed, and the results of the statistical analysis of accident element association are stored in the database.

[0141] Retrieve user interaction task instructions;

[0142] Based on the content of user interaction task instructions, obtain the statistical analysis results of accident elements related to the content of user interaction task instructions in the database;

[0143] The user interaction task instruction content and the statistical analysis results of related accident elements in the database are assembled and input into the second major model, and the decision content is output.

[0144] It is understood that the results of the statistical analysis of the correlation of the accident elements may include, but are not limited to, for example:

[0145] In 2024, there were 15 fire accidents, including 2 fire accidents involving cotton textiles, 6 fire accidents involving alcohol, and 7 fire accidents involving carelessly discarded cigarette butts.

[0146] In 2023, there were 9 fire accidents in the construction industry, 6 fire accidents in the metallurgical industry, 5 fire accidents in the textile industry, and so on.

[0147] Once the above-mentioned accident element correlation statistical analysis results are available, if the user interaction task instruction is: "Please help me write a decision report on fire prevention," then the relevant accident element correlation statistical analysis results such as "In 2024, there were 15 fire accidents, including 2 fire accidents involving cotton textiles, 6 fire accidents involving alcohol, and 7 fire accidents involving carelessly discarded cigarette butts" and "In 2023, there were 9 fire accidents in the construction industry, 6 fire accidents in the metallurgical industry, and 5 fire accidents in the textile industry" can be assembled with the user interaction task instruction and input into the second major model, and the decision content can be output.

[0148] Furthermore, the database can also provide a statistical analysis result query interface, through which the statistical analysis results can be statistically analyzed and visualized in various dimensions.

[0149] This involves extracting accident element entities, performing statistical analysis on the correlations between these entities, and storing the results in a database. When a user inputs an interactive task command, the system retrieves the related statistical analysis results from the database, assembles them, and inputs them into the second main model to generate the necessary decision report. Therefore, in addition to its accident element entity extraction function, it also has the function of generating decision reports.

[0150] It should be noted that the statistical analysis results of accident elements related to the user's interactive task instructions can be used as natural language prompts and input as contextual information into the second model. The second model can then generate the decision-making content required for the task, such as an accident analysis decision report, based on the input statistical results, contextual information, and the user's task instructions.

[0151] It should be noted that the second major model can be, but is not limited to, models such as the Tongyi Qianwen model, GPT-4, Wenxin Yiyan model, and Deepseek model.

[0152] Please see Figure 3 The diagram shown is a structural schematic of an electronic device provided in an embodiment of this specification.

[0153] like Figure 3 As shown, the electronic device 300 may include at least one processor 301, at least one network interface 304, a user interface 303, a memory 305, and at least one communication bus 302.

[0154] The communication bus 302 can be used to realize the connection and communication of the above components.

[0155] The user interface 303 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0156] The network interface 304 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0157] The processor 301 may include one or more processing cores. The processor 301 connects to various parts within the electronic device 300 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the processor 301 may be implemented using at least one hardware form selected from DSP, FPGA, and PLC. The processor 301 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 301.

[0158] The memory 305 may include RAM or ROM. Optionally, the memory 305 may include a non-transitory computer-readable medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned processor 301. As a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface module, and model training, extraction, or content generation applications. The processor 301 may be used to call the model training, extraction, or content generation programs stored in the memory 305 and execute the steps of the model training method, extraction method, or content generation method mentioned in the foregoing embodiments.

[0159] This specification also provides a computer-readable storage medium storing instructions that, when executed on a computer or processor, cause the computer or processor to perform one or more steps in the embodiments of the above-described model training method, extraction method, or content generation method. If the constituent modules of the above-described electronic device are implemented as software functional units and sold or used as independent products, they can be stored in the computer-readable storage medium.

[0160] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).

[0161] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks. Unless otherwise specified, the technical features of this embodiment and its implementation can be combined arbitrarily.

[0162] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Any modifications and improvements made by those skilled in the art to the technical solutions of this specification without departing from the spirit of this specification should fall within the protection scope defined by the claims of this specification.

Claims

1. A training method for an accident element extraction model, characterized in that, include: Input text sequences into the BERT network encoding layer; The BERT network encoding layer processes the text sequence to output a vector sequence, and inputs the vector sequence into the global pointer network layer; The global pointer network layer obtains the score matrix corresponding to each accident element entity type. The score matrix corresponding to each accident element entity type includes the predicted score of the initial entity word belonging to that accident element entity type under different position ranges, and extracts at least one extracted entity word belonging to each accident element entity type. For each extracted entity word, a method of expanding the character range based on the character range in the text sequence is adopted to obtain at least one expanded entity word corresponding to each extracted entity word. For each expanded entity word, its corresponding explanation and definition text is searched in the knowledge base. Each expanded entity word is then paired with its corresponding explanation and definition text and input into the trained first model to obtain the judgment result of whether each expanded entity word belongs to the accident element entity type of its corresponding extracted entity word. Based on the judgment result corresponding to each expanded entity word and the accident element entity type corresponding to each of all extracted entity words, all text characters in the text sequence are used as nodes to construct graph networks corresponding to multiple accident element entity types, and the adjacency matrix of the graph network corresponding to multiple accident element entity types is obtained. The loss function value is calculated based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type. Based on the calculated loss function values, the parameters of the global pointer network layer and the BERT network coding layer are optimized. Repeat the above steps until the preset requirements are met to obtain the feature extraction model.

2. The accident element extraction model training method according to claim 1, characterized in that, The global pointer network layer obtains the score matrix corresponding to each accident element entity type. Each score matrix for each accident element entity type includes the predicted score of the initial entity word belonging to that accident element entity type under different positional ranges. It also extracts at least one entity word belonging to each accident element entity type, including: The two linear layers in the global pointer network layer obtain multiple start position vectors and multiple end position vectors corresponding to each accident element entity type based on the input vector sequence. Iterates through all possible combinations of start and end positions. For each accident element entity type, it takes the start and end position vectors corresponding to each position combination and injects position information through the rotation position encoding network layer in the global pointer network layer. This process yields the score matrix corresponding to each accident element entity type. The score matrix for each accident element entity type includes the predicted score of the initial entity words belonging to that accident element entity type under the position range corresponding to different position combinations. Based on the score matrix corresponding to each accident element entity type, it extracts at least one entity word belonging to each accident element entity type from the initial entity words under all different position ranges.

3. The accident element extraction model training method according to claim 1, characterized in that, Each extracted entity word corresponds to at least three expanded entity words, and among the at least three expanded entity words, there is at least one expanded entity word that expands forward only relative to the beginning of the character range of the extracted entity word, at least one expanded entity word that expands backward only relative to the end of the character range of the extracted entity word, and at least one expanded entity word that expands forward both relative to the beginning of the character range of the extracted entity word and relative to the end of the character range of the extracted entity word.

4. The accident element extraction model training method according to claim 1, characterized in that, The loss function value is calculated based on the score matrix corresponding to each accident element entity type, the word extraction status corresponding to each accident element entity type, the graph network adjacency matrix corresponding to each of the multiple accident element entity types, and the multiple start position vectors and multiple end position vectors corresponding to each accident element entity type, including: Based on the score matrix corresponding to each accident element entity type and the extraction of words corresponding to each accident element entity type, the first loss value is obtained. The second loss value is obtained based on the graph network adjacency matrix corresponding to each of the multiple accident element entity types, the multiple start position vectors corresponding to each accident element entity type, and the multiple end position vectors. The loss function value is calculated based on the first loss value and the second loss value.

5. The accident element extraction model training method according to claim 4, characterized in that, The BERT network encoding layer processes the text sequence to output a vector sequence that includes the character vector corresponding to each text character in the text sequence, as well as the aggregated representation feature vector of the entire text sequence. The training method also includes: The aggregated representation feature vector of the entire text sequence is input into the industry domain classification layer, and the industry domain classification layer outputs the industry domain classification result based on the aggregated representation feature vector; The loss function value is also calculated based on a third loss value obtained from the industry domain classification result and the real domain classification result; The parameter optimization of the global pointer network layer and the BERT network coding layer based on the calculated loss function value includes: Based on the calculated loss function values, the parameters of the global pointer network layer, the BERT network encoding layer, and the industry domain classification layer are optimized. The feature extraction model also includes a parameter-optimized industry domain classification layer connected to the parameter-optimized BERT network encoding layer.

6. The accident element extraction model training method according to claim 4, characterized in that, The formula for calculating the first loss value is as follows: ； in, This represents the first loss value. Indicates the total number of entity types of accident elements. Indicates the starting position and end position The corresponding initial entity word belongs to the first Predicted scores for various accident element entity types This indicates that among all initial entity words, those considered not to belong to the first... The set of start and end positions of entity words extracted for each type of accident element entity. This indicates that among all initial entity words, it is considered to belong to the first... The set of start and end positions of the extracted entity words for each type of accident element entity.

7. The accident element extraction model training method according to claim 6, characterized in that: ； in, Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the starting position and end position The rotation matrix corresponding to the character range between them. This indicates transpose.

8. The accident element extraction model training method according to claim 4, characterized in that, The formula for calculating the second loss value is as follows: ； ； in, This represents the second loss value. Indicates the total number of entity types of accident elements. Indicates the first Type loss value corresponding to the entity type of each accident element. Indicates the starting position The corresponding rotation matrix, Indicates the end position The corresponding rotation matrix, Indicates the first The starting position corresponding to the type of accident element entity The starting position vector below, Indicates the first End position corresponding to each accident element entity type The ending position vector below, Indicates the first The graph network adjacency matrix corresponding to the entity types of accident elements. express The Middle Line number The elements in the column, and The value is the first Starting position in the graph network corresponding to the entity type of each accident element Corresponding text characters and end positions The weights of the edges between corresponding text characters. It is a matrix, and Each column in , It is also a matrix, and Each column , yes diagonal matrix, The elements on the diagonal are diagonal elements, All other elements are 0. Indicates calculation A function of the trace of a matrix.

9. An extraction method, characterized in that, include: Obtain the target text sequence; The element extraction model trained using the method described in any one of claims 1 to 8 extracts at least one target extracted entity word belonging to each accident element entity type in the target text sequence.

10. A content generation method, characterized in that, include: The feature extraction model trained by any one of claims 1 to 9 is used to extract features from all historical text sequences in the historical text sequence set, so as to extract the corresponding extraction entity word set for each historical text sequence. Each extraction entity word set includes at least one extraction entity word belonging to each accident element entity type. Based on the extracted entity word sets corresponding to each historical text sequence, statistical analysis of accident element association is performed, and the results of the statistical analysis of accident element association are stored in the database. Retrieve user interaction task instructions; Based on the content of user interaction task instructions, obtain the statistical analysis results of accident elements related to the content of user interaction task instructions in the database; The user interaction task instruction content and the statistical analysis results of related accident elements in the database are assembled and input into the second major model, and the decision content is output.

Images