A method for extracting military news information by incorporating common sense reasoning and human-computer interaction

Abstract

This application discloses a method for extracting military news information by incorporating common sense reasoning and human-computer interaction, comprising the following steps: designing a pre-extraction method for event information in military news; summarizing common sense about military news events and designing corresponding event common sense representation methods, constructing a military news event common sense base, using the common sense base to correct and supplement the pre-extraction results, and designing a human-computer interaction method to refine the event extraction results. First, existing military news event patterns are fine-tuned, and pre-extraction results are obtained using deep learning technology. Then, relevant military event common sense is introduced, corresponding common sense representation methods are constructed, and methods for further correcting and supplementing the pre-extraction results using common sense reasoning are implemented, achieving a data and knowledge dual-driven model with deep learning technology. Finally, the extraction results are refined through human-computer interaction to improve the high accuracy of the extraction results, providing a guarantee for subsequent tasks using the event information.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing, and in particular relates to a method for extracting military news information by incorporating common sense reasoning and human-computer interaction. Background Technology

[0002] The focus of information warfare is no longer on personnel allocation and weaponry, but on how to quickly acquire and process diverse battlefield information. As combat activities become increasingly complex, the resulting military information is becoming increasingly massive and exists mostly in the form of unstructured or semi-structured text data. Therefore, how to transform unstructured military text into structured information that can be processed by computers for rapid review and use by relevant personnel is of significant practical importance.

[0003] Military news primarily records military operations and events, documenting changes in battlefield situation and deployment adjustments, making it highly valuable for analysis. Military news event extraction, as a domain-specific task, demands greater accuracy and interpretability compared to ordinary event text extraction due to its unique application environment. Interpretability means that the extracted results follow rules or facts, conforming to human logical thinking, to better serve downstream tasks. Currently, most military information extraction techniques employ deep learning methods, primarily based on underlying technologies such as domain segmentation, part-of-speech tagging, relation extraction, and event classification. Fewer studies utilize template matching-based methods. While deep learning technology has developed rapidly and is widely used, its "black box" nature in implementation leads to a lack of interpretability in the results.

[0004] To address the aforementioned shortcomings and problems, this paper introduces common sense and human-computer interaction to meet the interpretability and accuracy requirements of military news event extraction, achieving a data- and knowledge-driven extraction process. Utilizing human supervision and error correction of the extraction results not only improves the accuracy of the final extraction but also promotes the optimization and updating of deep learning and common sense models. Summary of the Invention

[0005] The purpose of this invention is to provide a method for extracting military news information by incorporating common sense reasoning and human-computer interaction. This method focuses on the task of event information extraction, utilizes a deep learning model to pre-extract event information, and leverages common sense and human-computer interaction technologies to improve the quality of the extraction results. It can be used to solve the problem of data structuring in military news.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A method for extracting military news information that incorporates common-sense reasoning and human-computer interaction, the method comprising the following steps:

[0008] S1: Design an event information pre-extraction model for military news;

[0009] S2: Introduce common sense to complete the post-processing of the event extraction results;

[0010] S3: Introduce human-computer interaction to refine the results of event extraction.

[0011] Figure 1 This demonstrates the overall process of the present invention;

[0012] Regarding step S1, in this embodiment of the invention, after summarizing and classifying the military activities appearing in military news documents, referring to and fine-tuning the event representation framework extracted by ACE events, event types such as navigation, standby, and patrol, as well as event roles and their corresponding event element types such as event subject, event area, time, start time, and end time, are summarized.

[0013] Step S11: Design the corresponding event representation framework and perform data annotation. The structured definition of event representation is... In the formula, Type represents the event type; Trigger represents the event trigger word, which is the clearest word to express the occurrence of an event; Argument represents the event element, which is a certain entity in the event; Role represents the role that the event element plays in a certain event, which depends on the existence of a specific Argument. However, the number of types of event elements and the number of types of element roles in an event are not necessarily the same. Different Arguments may correspond to the same Role, and the same Argument may correspond to different Roles. The data annotation needs to include the event element type, event role type, and event type.

[0014] Step S12: Use a deep learning model to identify basic entities, event trigger words, and relation trigger words. The model used for identification is BERT+BiLSTM+CRF. The word vectors of BERT are dynamically generated according to the sentence. After BERT encodes the characters and sentence vectors, they are fed into BiLSTM to learn sequence information. Then, CRF is used to predict the labels and identify event elements and event trigger words.

[0015] Step S13: Based on the number of trigger words, the role classification and event classification are divided into single-event sentences and multi-event sentences for processing, and the role classification and event classification tasks are completed. The role classification task and the labeling task for single-event sentences are performed simultaneously. This is because in a single-event sentence, one event element can only correspond to one event role, and the event role can be directly labeled during labeling. However, in a multi-event sentence, one event element plays different roles in different types of events. Therefore, during classification, this paper combines the event trigger words identified in the multi-event sentences with the event elements. Specifically, the output of the multi-event sentence in the identification task stage needs to be transformed into event element vectors, event element category vectors, trigger word vectors, event element and trigger word position vectors, and sentence vectors. These feature vectors are used as input features of the role classification model and classified using a feedforward neural network to obtain the event roles and event types.

[0016] In step S2, common sense post-processing is performed on the event results, which mainly includes:

[0017] Step S21: Construct a common knowledge base, organize all military common knowledge that can be used for military news event extraction tasks, and divide this common knowledge into military event common knowledge, and further divide it into general event common knowledge, event type common knowledge, event trigger word common knowledge, and event role common knowledge. Among them, general event common knowledge is mainly common knowledge related to event definition, while event type common knowledge, event trigger word common knowledge, and event role common knowledge are common knowledge used for military news event extraction tasks, representing the characteristics of different event elements;

[0018] Step S22: Perform ASP coding on military events by defining different event coding rules to encode basic event elements such as event type, trigger words, and event elements;

[0019] Step S23: Perform ASP encoding on common military event information. This converts the text-based common military event information into an encoding format that conforms to ASP syntax and can be used by the ASP solver to reason about the common military event information. The main purpose of this is to use common sense to reason out the abnormal results contained in the deep learning-based military news event extraction model.

[0020] Step S24: Obtain the reasoning conclusion on the extraction results through common sense reasoning. Combine the event pre-extraction results in step S22 and the common sense base in step S23, and use the ASP reasoning engine to correct and supplement the event pre-extraction results, thus realizing the common sense post-processing process.

[0021] In step S3, human-computer interaction is used to supervise and correct the extraction results in step S2, and the final event extraction results are output, mainly including:

[0022] Step S31: Quantitative methods are used to filter out abnormal information. Uncertainty joint sampling values ​​are obtained by calculating ME (Maximize Entropy) sampling and BvSB (Best vs. Second Best) sampling. In multi-class classification problems, if there are classes y∈{1,…,k}, meaning there are k classes, and the model is valid for the i-th test sample Y... i The output probability value is p i =[p i1 ,p i2 ,…p ik Then, the sampling calculation method for ME and BvSB of the i-th test sample is as follows:

[0023]

[0024]

[0025] When the two strategies are combined, a strategy weight β∈(0,1) is used, and the uncertainty joint sample value is calculated using β·f(y)+(1-β)·g(y), where f(y) and g(y) represent the sample values ​​calculated by the two strategies, respectively. After calculating the sample values, anomaly information is finally filtered by setting a threshold δ. If the sample value is greater than the threshold, it is judged as anomaly information.

[0026] Step S32: Use qualitative methods to filter abnormal information. Utilize the common sense reasoning results obtained in step S2 to determine the results of events that may cause problems as abnormal information.

[0027] Step S33, abnormal information saving and display: Save the abnormal event extraction results determined in steps S31 and S32 to a specific JSON format file. For each event extraction result, the event type and event element have corresponding candidates in the candidates attribute, representing other possible categories determined by the model, and give the output probability of the corresponding category, i.e. confidence level, and display it to the relevant interaction personnel.

[0028] Step S34: Relevant personnel complete the correction of human-computer interaction information;

[0029] Step S341: Delete redundant event elements;

[0030] Step S342: Add the confirmed event elements;

[0031] Step S343: Modify the erroneous event elements;

[0032] Step S344: Repeat steps S341 to S343 until the extraction result cannot be further modified.

[0033] Compared to existing technologies, the advantages of this invention are as follows: This technical solution utilizes common sense regarding military events, leveraging this common sense to achieve a dual-driven data and knowledge extraction process. This ensures that the extracted military text information conforms to common sense standards, helping to solve the "black box" problem of deep learning, making the extraction results interpretable, and further improving their confidence level through result correction. Introducing human-computer interaction technology in the final stage of the event extraction process, utilizing human supervision and error correction of the extraction results, not only improves the accuracy of the final extraction results but also promotes the optimization and updating of deep learning models and common sense models. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the overall process of the present invention;

[0035] Figure 2 This is a flowchart of the event information pre-extraction process of the present invention;

[0036] Figure 3 This is a flowchart illustrating the post-processing of common knowledge in this invention.

[0037] Figure 4 This is a flowchart of the human-computer interaction result refinement process of the present invention. Detailed Implementation

[0038] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0039] Example 1: See Figures 1-4 A method for extracting military news information that incorporates common-sense reasoning and human-computer interaction, the method comprising the following steps:

[0040] S1: Design an event information pre-extraction model for military news;

[0041] S2: Introduce common sense to complete the post-processing of the event extraction results;

[0042] S3: Introduce human-computer interaction to refine the results of event extraction.

[0043] Step S1: Summarize and categorize the military activities appearing in military news documents to form a corresponding event representation framework.

[0044] Step S2 involves manually analyzing military news to construct specific common knowledge related to military operations in the reports. ASP (Answer Set Programming) is then used to store this common knowledge in a structured manner in a database. Combined with the event pre-extraction results from Step S1, the ASP solver is used to correct and supplement the event pre-extraction results, thus realizing the post-processing of common knowledge.

[0045] Step S3: Use human-computer interaction to supervise and correct the extraction results in step S3, and output the final results as structured information for use by relevant tasks.

[0046] Step S1, the process is as follows: Figure 2 As shown, it includes:

[0047] Step S11: Design the corresponding event representation framework and perform data annotation. The structured definition of event representation is... In the formula, Type represents the event type; Trigger represents the event trigger word, which is the clearest word to express the occurrence of an event; Argument represents the event element, which is a certain entity in the event; Role represents the role that the event element plays in a certain event, which depends on the existence of a specific Argument. However, the number of types of event elements and the number of types of element roles in an event are not necessarily the same. Different Arguments may correspond to the same Role, and the same Argument may correspond to different Roles. The data annotation needs to include the event element type, event role type, and event type.

[0048] Step S12: Use a deep learning model to identify basic entities, event trigger words, and relation trigger words. The model used for identification is BERT+BiLSTM+CRF. The word vectors of BERT are dynamically generated according to the sentence. After BERT encodes the characters and sentence vectors, they are fed into BiLSTM to learn sequence information. Then, CRF is used for label prediction.

[0049] Step S13: Based on the number of trigger words, role classification and event classification are divided into single-event sentences and multi-event sentences for processing, and the role classification and event classification tasks are completed. The role classification task for single-event sentences is performed simultaneously with the annotation task. This is because in a single-event sentence, one event element can only correspond to one event role, and the event role can be directly labeled during annotation. However, in multi-event sentences, one event element plays different roles in different types of events. Therefore, this paper combines the event trigger words identified in multi-event sentences with the event elements during classification. Specifically, the output of the multi-event sentence recognition task needs to be transformed into event element vectors, event element category vectors, trigger word vectors, event element and trigger word position vectors, and sentence vectors. These feature vectors are used as input features of the role classification model and classified using a feedforward neural network.

[0050] Step S2, the process is as follows Figure 3 As shown, it includes:

[0051] Step S21: Construct a common knowledge base, organize all military common knowledge that can be used for military news event extraction tasks, and refer to this common knowledge as military event common knowledge. This common knowledge is divided into general event common knowledge, event type common knowledge, event trigger word common knowledge, and event role common knowledge. General event common knowledge mainly relates to event definitions, while event type common knowledge, event trigger word common knowledge, and event role common knowledge are specific to military news event extraction tasks, representing the characteristics of different event elements. Some examples of this common knowledge are shown in Table 1.

[0052] Table 1: General Knowledge of Some Military Events

[0053]

[0054]

[0055] Step S22 involves ASP coding of military events. This is achieved by defining different event coding rules to encode basic event elements such as event type, trigger words, and event elements. For example, the predicate encoding representing the roles of "event subject" and "target object" is as follows:

[0056] contain_subject(E,S)

[0057] contain_object(E,O)

[0058] Where S represents the event subject, O represents the target object, E represents the event, contain_subject(E,S) means that event E contains the event subject S, and contain_object(E,O) means that event E contains the target object O. All military event ASP codes are shown in Table 2.

[0059] Table 2 ASP Coding Rules for Military Events

[0060] Basic elements of an event ASP coding event event(E) Basic event types event_type(ET) Event type is_type_of(E,ET) Event trigger words contain_trigger(E,T) Relationship trigger words contain_relation_trigger(E,RT) Basic event elements element Contains event elements contain_element(E, Element) main body of the event contain_subject(E,S) target object contain_object(E,O) time contain_time(E,T) Start time contain_starting_time(E,ST) End time contain_ending_time(E,ET) Event Area contain_region(E,R) starting point contain_start(E,S) end contain_end(E,E)

[0061] Step S23: Perform ASP encoding on the common knowledge of military events, converting the text-formatted common knowledge of military events into an encoding format that conforms to ASP syntax and can be used by the ASP solver to reason about the common knowledge of military events, as shown in Table 3.

[0062] Table 3. Coding Methods for Common Military Event Knowledge

[0063]

[0064]

[0065] The primary purpose of using military events is to leverage common sense reasoning to identify anomalous results within the deep learning-based military news event extraction model. Table 4 summarizes the ASP codes involved in anomalous results across all common sense reasoning conclusions in this invention.

[0066] Table 4 Common Conclusions in ASP Coding for Military Events

[0067]

[0068]

[0069] Step S24: Obtain the reasoning conclusion on the extraction results through common sense reasoning. Combining the event pre-extraction results in step S22 and the common sense in the common sense base in step S23, the ASP reasoning engine is used to correct and supplement the event pre-extraction results, realizing the common sense post-processing process.

[0070] Step S3, the process is as follows Figure 4 As shown, it includes:

[0071] Step S31: Quantitative methods are used to filter out abnormal information. Uncertainty joint sampling values ​​are obtained by calculating ME (Maximize Entropy) sampling and BvSB (Best vs. Second Best) sampling. In multi-class classification problems, if there are classes y∈{1,…,k}, meaning there are k classes, and the model is valid for the i-th test sample Y... i The output probability value is p i =[p i1 ,p i2 ,…p ik Then, the sampling calculation method for ME and BvSB of the i-th test sample is as follows:

[0072]

[0073]

[0074] When the two strategies are combined, a strategy weight β∈(0,1) is used, and the uncertainty joint sample value is calculated using β·f(y)+(1-β)·g(y), where f(y) and g(y) represent the sample values ​​calculated by the two strategies, respectively. After calculating the sample values, anomaly information is finally filtered by setting a threshold δ. If the sample value is greater than the threshold, it is judged as anomaly information.

[0075] Step S32: Use qualitative methods to filter abnormal information. Utilize the common sense reasoning results obtained in step S2 to determine the results of events that may cause problems as abnormal information.

[0076] Step S33, Exception Information Saving and Display: The exception event extraction results determined in steps S31 and S32 are saved to a specific JSON format file. For each event extraction result, the event type and event element have corresponding candidates stored in the `candidates` attribute, representing other possible categories determined by the model, and providing the output probability (confidence level) for the corresponding category, which is then displayed to relevant users. This invention retains candidates in the stored results and uses these candidates to assist user interaction. Because users need to modify certain elements when adjusting and correcting the event extraction results, displaying all possible candidates can speed up this process. When users determine that the event elements in the current extraction result are incorrect, they can directly select other content from the relevant candidates.

[0077] Step S34, human-computer interaction information correction, including:

[0078] Step S341: Delete redundant event elements;

[0079] Step S342: Add the confirmed event elements;

[0080] Step S343: Modify the erroneous event elements;

[0081] Step S344: Repeat steps S341 to S343 until the extraction result cannot be further modified.

[0082] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0083] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the specific technologies of the present invention, such as deep learning models and common knowledge content. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for extracting military news information by incorporating common-sense reasoning and human-computer interaction, characterized in that, The method includes the following steps: S1: Design an event information pre-extraction model for military news; S2: Introduce common sense to complete the post-processing of the event extraction results; S3: Introduce human-computer interaction to refine the event extraction results; In step S3, human-computer interaction is used to supervise and correct the extraction results in step S2, and the final event extraction results are output, mainly including: Step S31: Quantitative methods are used to screen outlier information. The value of the joint sampling of uncertainty is obtained by calculating ME sampling and BvSB sampling. In multi-class classification problems, if the multi-class classification problem has categories... That is, it can be divided into Class, and the model for the first test samples The output probability value is So for the first The ME and BvSB sampling calculation methods for each test sample are as follows: When combining two strategies, a single strategy weight is used. and utilize Calculate the joint sampled values ​​with uncertainty, where and These represent the sampled values ​​calculated using the two strategies, respectively. After calculating the sampled values, a threshold is finally set. Perform anomaly information filtering; if the sampled value is greater than the threshold, it is judged as anomaly information. Step S32: Use qualitative methods to filter abnormal information. Utilize the common sense reasoning results obtained in step S2 to determine the results of events that may cause problems as abnormal information. Step S33: Saving and Displaying Abnormal Information. The abnormal event extraction results determined in steps S31 and S32 are saved to a specific JSON format file. For each event extraction result, the event type and event element have corresponding candidates stored in the `candidates` attribute, representing other possible categories determined by the model. The output probability of the corresponding category, i.e., the confidence level, is given and displayed to relevant interaction personnel. Step S34, human-computer interaction information correction, mainly includes: Step S341: Delete redundant event elements; Step S342: Add the confirmed event elements; Step S343: Modify the erroneous event elements; Step S344: Repeat steps S341 to S343 until the extraction result cannot be further modified.

2. The method for extracting military news information by incorporating common sense reasoning and human-computer interaction as described in claim 1, characterized in that, In step S1, military action events appearing in military news are summarized and categorized to form corresponding event representation frameworks. A deep learning model is then used to pre-extract these events, as detailed below: Step S11: Design the corresponding event representation framework and perform data annotation. The structured definition of event representation is... In the formula, Type represents the event type; Trigger represents the event trigger word, which is the clearest word to express the occurrence of an event; Argument represents the event element, which is a certain entity in the event; Role represents the role that the event element plays in a certain event, which depends on the existence of a specific Argument. However, the number of types of event elements and the number of types of element roles in an event are not necessarily the same. Different Arguments may correspond to the same Role, and the same Argument may correspond to different Roles. The data annotation needs to include the event element type, event role type, and event type. Step S12: Use a deep learning model to identify basic entities, event trigger words, and relation trigger words. The model used for identification is BERT+BiLSTM+CRF. The word vectors of BERT are dynamically generated according to the sentence. After BERT encodes the characters and sentence vectors, they are fed into BiLSTM to learn sequence information. Then, CRF is used for label prediction. Step S13: Based on the number of trigger words, the role classification and event classification are divided into single-event sentences and multi-event sentences for processing, and the role classification and event classification tasks are completed. The role classification task for single-event sentences is carried out simultaneously with the labeling task. Specifically, the output of multi-event sentences in the recognition task stage needs to be converted into event element vectors, event element category vectors, trigger word vectors, event element and trigger word position vectors, and sentence vectors. These feature vectors are used as input features of the role classification model and classified using a feedforward neural network.

3. The method for extracting military news information by incorporating common sense reasoning and human-computer interaction as described in claim 1, characterized in that, In step S2, common sense post-processing is performed on the event results, which mainly includes: Step S21: Construct a common knowledge base, organize all military common knowledge that can be used for military news event extraction tasks, and divide this common knowledge into military event common knowledge, and further divide it into general event common knowledge, event type common knowledge, event trigger word common knowledge, and event role common knowledge. Among them, general event common knowledge is mainly common knowledge related to event definition, while event type common knowledge, event trigger word common knowledge, and event role common knowledge are common knowledge used for military news event extraction tasks, representing the characteristics of different event elements. Step S22: Perform ASP coding on military events by defining different event coding rules to encode event elements such as event type, trigger words, and event elements. Step S23: Perform ASP encoding on common military event information. This converts the text-based common military event information into an encoding format that conforms to ASP syntax and can be used by the ASP solver to reason about the common military event information. The main purpose of this is to use common sense to reason out the abnormal results contained in the deep learning-based military news event extraction model. Step S24: Obtain the reasoning conclusion on the extraction results through common sense reasoning. Combine the event pre-extraction results in step S22 and the common sense base in step S23, and use the ASP reasoning engine to correct and supplement the event pre-extraction results, thus realizing the common sense post-processing process.

Images