Man-machine mixed situation fusion and display method

Abstract

The invention belongs to the field of intelligence analysis, and relates to a situation fusion calculation method based on intelligence data and a calculation process. The invention provides a method for performing association analysis and fusion analysis by utilizing intelligence and algorithms under the guidance of a user and the assistance of various algorithms of a computer, then performing situation display and situation prediction, and updating the situation in real time. The method comprises the following steps: (10) identifying information key information; (20) performing target category information reasoning by using a Bayesian network; (30) performing target category compilation information reasoning by using a D-S evidence theory; (40) carrying out multi-source target information integration and duplicate and false removal; (50) target situation prediction and our task display; and (60) target situation updating iteration and situation integration reporting.

Description

Technical Field

[0001] The present invention belongs to the field of intelligence analysis and is a method and calculation process for fusion and display of situation based on intelligence. The present invention provides a method for performing correlation analysis and fusion analysis using intelligence and algorithms under the guidance of a user and with the assistance of multiple computer algorithms, and then performing situation display and situation prediction, and updating the situation in real time. Background Art

[0002] The intelligence analysis process of traditional command and control systems is generally handled by intelligence processing personnel, and most of them do not use intelligent algorithms to assist in analysis. The advantages of automated and intelligent intelligence analysis and processing systems in assisting users in making decisions are very obvious. These intelligent systems can not only use intelligent algorithms such as machine learning to process massive amounts of data and process graphic data into formatted data for storage; they can also use intelligent recommendation algorithms to select the most urgent, important, and critical information from massive amounts of data; they can also use display systems to display the enemy and our information generated by the system in real time on the situation interface. This effectively assists users in analyzing the current situation of the red and blue sides, and provides data support for users' analysis and decision-making.

[0003] Compared with the "unmanned" intelligent command and control system that does not require intelligence analysis and processing, the human-machine hybrid situation fusion display system also has great advantages. On the one hand, the limitation of intelligent technologies such as machine learning is that it also learns knowledge from Dalian's data and cases, and then makes decisions based on this knowledge. Therefore, when faced with real scenarios with huge data, complex situations, and no boundaries in the rules of engagement, intelligent agents and intelligent systems will show a lack of creativity. On the other hand, excessive reliance on machines and software systems as the core of the system will also create a system security crisis. Once the core command and control system of the system is destroyed or abnormally controlled, the consequences are unimaginable.

[0004] Therefore, it is necessary to study the human-machine hybrid situation fusion display system under the guidance of users and with computer intelligent decision-making assistance. Summary of the invention

[0005] In view of the problems that the situation compilation process is too complicated, the data is too large, and there are many participants, the present invention proposes a human-machine hybrid intelligence-based situation fusion and display method and calculation process. This effectively assists users in analyzing the red and blue side situations, provides various data support and situation analysis diagram support for user decision-making, and thus achieves the purpose of improving user decision-making ability.

[0006] The present invention implements a human-machine hybrid situation fusion and display method through the following technical scheme, comprising the following steps:

[0007] (10) Identification of key intelligence information.

[0008] (20) Use Bayesian networks to infer target category information.

[0009] (30) Use DS evidence theory to compile information reasoning for target classes.

[0010] (40) Integration and deduplication of multi-source target information, i.e. display of enemy and friendly force icons (status).

[0011] (50) Target situation prediction and our mission display (i.e. display of enemy and our situation).

[0012] (60) Target situation update iteration and situation integration reporting. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 The steps of the method of the present invention are shown in FIG.

[0014] Figure 2 Detailed flow chart of the method of the present invention

[0015] Figure 3 The enemy and our strength icon display of the present invention

[0016] Figure 4 The enemy and our situation diagram of the present invention

[0017] Figure 5 The enemy and our situation map after the update iteration of the present invention Specific implementation methods

[0019] (10) Identification of key intelligence information.

[0020] The steps for identifying key intelligence information in (10) above are as follows:

[0021] (11) Data preprocessing

[0022] According to the analysis of the annotated data files, the annotation information related to the knowledge extraction task is extracted, which is mainly divided into text information and labe1 labels. The labe1 label is the label annotated by BIO. The training data is organized to provide a data basis for downstream model training.

[0023] (12) Text vectorization

[0024] Commonly used text vectorization methods include one-hot encoding, N-Gram model, and Word2Vec model. However, the word vectors encoded by these methods are all static word vectors, which cannot solve the problem of polysemy in extraction tasks. This solution uses the BERT pre-trained model for word vector encoding, and uses the multi-layer bidirectional transformer encoder of the BERT model to encode the word context information to obtain a dynamically changing word vector representation, effectively solving the problem of polysemy in extraction tasks.

[0025] (13) Entity Extraction

[0026] This solution treats knowledge extraction as a sequence labeling task and processes it according to the sequence labeling process. The input is a text sentence, which is processed by the knowledge extraction model and outputs a sentence sequence label.

[0027] Label prediction is performed on the words in the text, and each word has a corresponding label. Because there are cross-word trigger words, this patent uses the "BIO" annotation mode to perform sequence annotation on the text. Among them, "0" represents a non-entity word, "B-Attack" represents the starting position of an entity of type "Attack", and "T-Attack" represents other positions of an entity of type "Attack". The entity type is indicated after the labels "B" and "I".

[0028] The structure of the knowledge extraction model mainly consists of the following parts: vector representation layer, trigger word recognition output layer.

[0029] BERT word vector representation: In order to solve the problem of polysemy, the BERT pre-trained model is used for vector representation. The BERT model multi-layer bidirectional Transformers encoder is used to generate dynamic word vectors. For a given sentence, the vector representation x of each word in the sentence is obtained through the above BERT pre-trained model. i ∈R d , where d represents the dimension of the word vector. A sentence of length n is represented as X∈R n×d :

[0030] X=[x 1 , x 2 , …, x n ]

[0031] Linear output layer: During training, the target loss function is:

[0032]

[0033] Where x is the true label, y is the predicted label, and K is the correction coefficient. When predicting, the argmax function is used to obtain a set of sequences with the largest overall probability to obtain the predicted label value.

[0034] The BERT model used is the RBT3 model. The model has 3 layers, and the hidden layer dimension is H = 768. The loss function uses the cross entropy cost function, and the optimizer uses the Adam algorithm with a learning rate of 1e-5. The longest sequence length is 60, the batch_size is 12, and the epoch is set to 20. The knowledge extraction experiment is based on the BERT pre-trained model and fine-tuned during parameter training.

[0035] (14) Integration and storage of key intelligence data

[0036] The extracted equipment model, coordinates, geographic target type, discovery time and other element data are formatted and stored in the database.

[0037] (20) Use Bayesian networks to infer target category information.

[0038] The specific steps of the target information query and classification in (20) above are as follows:

[0039] (21) Read key intelligence data and match it with the knowledge base

[0040] Read detailed equipment parameters in the knowledge base according to the equipment model, including maneuvering speed, load, shooting distance, etc. Save detailed information of multiple equipment and associate them with the target ID.

[0041] (22) Infer target information

[0042] Based on the target's multiple equipment information, quantity, number of personnel, intelligence location and other information, the Bayesian network is used to infer the enemy target category and add credibility parameters.

[0043]

[0044] Where f represents the objective function to be optimized; X t ={(x 1 ,y 1 ), (x 2 ,y 2 ),...,(x t ,y t )} represents a data set with known true observation values, x t Represents the tth data, which is the decision vector, which includes the target’s equipment category, model, quantity, personnel data, intelligence location, area, movement direction, etc.

[0045] And y n =f(x n)+ε n is the corresponding real observation value, representing air defense, engineering, fortification support, rear support, artillery, reconnaissance and electronics, communications command, armored infantry, geographic target, chemical defense, aviation, etc. ε n is the corresponding observation error value, also called "noise" or error value;

[0046] p(f) represents the prior probability distribution of f, which is the prior assumption of the target function, that is, the target is the prior probability distribution of a certain type of target in a certain assumption; p(X t |f) represents the likelihood distribution of the observed value; p(f|X t ) is the posterior probability distribution of f, which describes the credibility of the proxy model as an approximation of the objective function after adjusting the prior probability distribution by known observations. That is, the posterior probability distribution of the target is a certain class of targets in the determination assumption.

[0047] (23) Use the interface information to display the inferred target category information and display it for the user to confirm. After confirmation, store it in the database.

[0048] (30) Use DS evidence theory to compile information reasoning for target classes.

[0049] (31) Based on the target category inferred by (23), as well as multiple equipment details, personnel numbers, intelligence locations and other information, the DS evidence theory is used to infer the enemy target organization and formation information, as well as the general parameters of this organization in the knowledge base (including detailed equipment list, capability parameters, personnel information, etc.), and additional credibility parameters are added.

[0050] (32) First, calculate the normalization coefficient P = 1 / (1-K), where K is the evidence conflict factor, which represents the degree of conflict between evidences. The larger the K value, the more serious the conflict between evidences. When K = 1, there is a complete conflict between evidences. The formula is:

[0051]

[0052] Where m k Information representing intelligence, such as m 1 represents the target category, m 2 Represents detailed information such as the target equipment model, m 3 Represents intelligence location, etc.

[0053] Where A i Represents the compilation information of the target, such as A 1 Represents similar enemy organizational units such as "such battalion, such company, such platoon".

[0054] (33) Then, use the formula to calculate the belief function of the D-S fusion result of each target information. For example, the belief function of A is calculated as follows:

[0055]

[0056] In the formula, P is the normalization coefficient; it is assumed that m 1 and m 2 are independent basic probability assignment functions defined in the recognition frame Θ, and by default Θ = {A 1 , A 2 , A 3 , ……};

[0057] When K = 1, P does not exist, and the D-S fusion result formula is meaningless; when 0 < K < 1, use the D-S fusion formula to obtain the credibility of different results Ai with different intelligence information mk as evidence. For a target, integrate multiple intelligence information evidences mk to obtain the most credible result Ai, and then output it.

[0058] (34) Display the speculation results for the user to analyze and confirm. The user can modify the speculation content such as the position and equipment strength of the enemy target based on this information. After confirmation, a result with the highest credibility is given for each target id.

[0059] (40) Integrate and deduplicate multi-source target information, that is, display the enemy and our force icons (states).

[0060] (41) Classify and analyze the large number of targets obtained in (20) according to their positions. Select targets with close positions (using a filtering algorithm) and judge whether their categories are the same. If they are the same, they are considered the same target, and integrated analysis is performed according to multiple sources. Integrate them into one entity target (delete all source entities), attach the equipment information from different sources to the entity (including the intelligence source and the coordinates where the intelligence is generated). The entity includes the general equipment in the knowledge base and is attached with the equipment information from multiple intelligence sources.

[0061] (42) Read all the fused entity targets output in (31), count the intelligence from all sources of this target, use the Kalman filtering algorithm to integrate the coordinates of all intelligence, and output a unique coordinate as the coordinate of the current target.

[0062] (43) Screen out individual targets with relatively large distance intervals that did not participate in the calculation in (31), use the integrity of the information, and then add algorithm analysis and user confirmation to judge the authenticity. If it is true, retain this target; if it is false, delete this target.

[0063] (44) Iteratively judge the coordinate positions of the remaining targets until the fusion judgment of all targets is completed.

[0064] (50) Target situation prediction and our mission display (i.e. display of enemy and our situation).

[0065] (51) Display the icon position in real time according to the mission of our combat force in the system. Display the enemy icon in real time according to the category and coordinates of the enemy target output by (30). Click the enemy or our icon to display the location of our force, equipment, personnel and other attribute data. (Display "status")

[0066] (52) According to the tasks and action instructions of our combat forces in the system, draw relevant plots such as maneuver routes and attack directions. (Showing our "momentum")

[0067] (53) Based on the enemy target attributes and past tasks, the attention mechanism algorithm is used to predict the specific tasks of the enemy entity. The specific predicted tasks of the enemy entity are integrated with the Bayesian algorithm to predict the enemy's overall intentions, and the enemy entity's maneuver routes, attack directions, etc. are plotted. (Showing the enemy's "momentum").

[0068] (54) Supports users to modify the situation map output by the system. Supports modification of enemy position, our position, enemy entity mission, and our entity mission.

[0069] (60) Target situation update iteration and situation integration reporting.

[0070] (61) The user prepares combat missions and reconnaissance missions and executes them.

[0071] (62) Our forces carry out missions, obtain intelligence on enemy targets, and organize and integrate the intelligence into textual information.

[0072] (63) Call (10) in an iterative loop to update the status; at the same time, based on the needs, after confirmation by the user, report the status of this level to the upper-level user; or send the status of this level to the lower-level user.

Claims

1. A human-machine hybrid situation fusion and display method, characterized in that The following steps are involved: (10) Identification of key intelligence information. (20) Use Bayesian networks to infer target category information. (30) Use DS evidence theory to compile information reasoning for target classes. (40) Integration and deduplication of multi-source target information, i.e. display of enemy and friendly force icons (status). (50) Target situation prediction and our mission display (i.e. display of enemy and our situation). (60) Target situation update iteration and situation integration reporting.

2. The method according to claim 1, characterized in that The specific steps of identifying the key intelligence information in (10) are as follows: (11) Data preprocessing According to the analysis of the annotated data files, the annotation information related to the knowledge extraction task is extracted, which is mainly divided into text information and labe1 labels. The labe1 label is the label annotated by BIO. The training data is organized to provide a data basis for downstream model training. (12) Text vectorization Commonly used text vectorization methods include one-hot encoding, N-Gram model, and Word2Vec model. However, the word vectors encoded by these methods are all static word vectors, which cannot solve the problem of polysemy in extraction tasks. This solution uses the BERT pre-trained model for word vector encoding, and uses the multi-layer bidirectional transformer encoder of the BERT model to encode the word context information to obtain a dynamically changing word vector representation, effectively solving the problem of polysemy in extraction tasks. (13) Entity Extraction This solution treats knowledge extraction as a sequence labeling task and processes it according to the sequence labeling process. The input is a text sentence, which is processed by the knowledge extraction model and outputs a sentence sequence label. Label prediction is performed on words in the text, and each word has a corresponding label. Because there are cross-word trigger words, this patent uses the "BIO" annotation mode to perform sequence annotation on the text. Among them, "O" represents a non-entity word, "B-Attack" represents the starting position of an entity of type "Attack", and "T-Attack" represents other positions of an entity of type "Attack". The entity type is indicated after the labels "B" and "I". The structure of the knowledge extraction model mainly consists of the following parts: vector representation layer, trigger word recognition output layer. BERT word vector representation: In order to solve the problem of polysemy, the BERT pre-trained model is used for vector representation. The BERT model multi-layer bidirectional Transformers encoder is used to generate dynamic word vectors. For a given sentence, the vector representation x of each word in the sentence is obtained through the above BERT pre-trained model. i ∈R d , where d represents the dimension of the word vector. A sentence of length n is represented as X∈R n×d : X=[x1,x2,…,x n ] Linear output layer: During training, the target loss function is: Where x is the true label, y is the predicted label, and K is the correction coefficient. When predicting, the argmax function is used to obtain a set of sequences with the largest overall probability to obtain the predicted label value. The BERT model used is the RBT3 model. The model has 3 layers, and the hidden layer dimension is H = 768. The loss function uses the cross entropy cost function, and the optimizer uses the Adam algorithm with a learning rate of 1e-5. The longest sequence length is 60, the batch_size is 12, and the epoch is set to 20. The knowledge extraction experiment is based on the BERT pre-trained model and fine-tuned during parameter training. (14) Integration and storage of key intelligence data The extracted equipment model, coordinates, geographic target type, discovery time and other element data are formatted and stored in the database.

3. The method according to claim 1, characterized in that The specific steps for inferring target category information using the Bayesian network in (20) are as follows: (21) Read the key intelligence data and match the knowledge base Read the detailed equipment parameters in the knowledge base according to the equipment model, including maneuvering speed, loading capacity, shooting range, etc. Save the detailed information of multiple pieces of equipment and associate it with the target id. (22) Infer target information Based on information such as multiple pieces of equipment information, quantity, number of personnel, and intelligence location of the target, use the Bayesian network to infer the enemy target category and attach a credibility parameter. Where f represents the objective function to be optimized; X t ={(x1,y1),(x2,y2),...,(x t ,y t )} represents a data set with known true observation values, x t Represents the tth data, which is the decision vector, which includes the target’s equipment category, model, quantity, personnel data, intelligence location, area, movement direction, etc. And y n =f(x n )+ε n is the corresponding real observation value, representing the front-line equipment, such as drones and unmanned vehicles. n is the corresponding observation error value, also called "noise" or error value; p(f) represents the prior probability distribution of f, which is the prior assumption of the target function, that is, the target is the prior probability distribution of a certain type of target in a certain assumption; p(X t |f) represents the likelihood distribution of the observed value; p(f|X t ) is the posterior probability distribution of f, which describes the credibility of the proxy model as an approximation of the objective function after adjusting the prior probability distribution by known observations. That is, the posterior probability distribution of the target is a certain class of targets in the determination assumption. (23) Use the interface information to display the inferred target category information and display it for the user to confirm. After confirmation, store it in the database.

4. The method according to claim 1, characterized in that The specific steps for inferring target category establishment information using the D-S evidence theory in (30) are as follows: (31) Based on the target category inferred in (23), as well as information such as multiple pieces of detailed equipment information, number of personnel, and intelligence location, use the D-S evidence theory to infer the enemy target establishment and formation information, as well as the general parameters of this formation of troops in the knowledge base (including detailed equipment list, ability parameters, personnel information, etc.), and attach a credibility parameter. (32) First, calculate the normalization coefficient P = 1 / (1 - K), where K is the evidence conflict factor, representing the degree of conflict between evidence. The larger the K value, the more serious the conflict between evidence. When K = 1, the evidence is completely in conflict. Its formula is: Where m k Information representing intelligence, for example, m1 represents the target category, m2 represents detailed information such as the target equipment model, and m3 represents the intelligence location, etc. Where A i Represents the target's organizational information, for example, A1 represents "a certain battalion, a certain company, a certain platoon" or a similar organizational unit of the enemy. (33) Then use the formula to calculate the belief function of the D-S fusion result of each target information. For example, calculate the belief function of A as follows: In the formula, P is the normalization coefficient; where it is assumed that m1 and m2 are independent basic probability assignment functions defined in the recognition frame Θ, and by default: When K = 1, P does not exist, and the D-S fusion result formula is meaningless; when 0 < K < 1, use the D-S fusion formula to obtain the credibility of different results Ai obtained from different intelligence information mk as evidence. For a target, integrate multiple intelligence information evidences mk to obtain the most credible result Ai, and then output it. (34) Display the inferred result for the user to analyze and confirm. The user can modify information such as the position and strength inference content of the enemy target based on this information. After confirmation, give a result with the highest credibility for each target id.

5. The method according to claim 1, characterized in that The specific steps for integrating and removing duplicates and false information of multi-source target information in (40) are as follows: (41) Classify and analyze a large number of targets obtained in (20) according to their positions. Select targets with close positions (using a filtering algorithm) and judge whether their categories are the same. If they are the same, consider them as the same target and conduct integrated analysis based on multiple sources. Integrate them into one entity target (delete all source entities), attach the equipment information from different sources to the entity (including the intelligence source and the coordinates where the intelligence is generated). The entity contains the general equipment in the knowledge base and also attaches the equipment information from multiple intelligence sources. (42) Read all the fused entity targets output in (31), count the intelligence from all sources of this target, use the Kalman filtering algorithm to integrate the coordinates of all intelligence, and output a unique coordinate as the coordinate of the current target. (43) Filter out individual targets with long distance intervals that are not involved in the calculation of (31), and use the completeness of the information plus the algorithm plus the user confirmation to determine the authenticity. If it is true, keep this target; if it is false, delete this target. (44) Iteratively determine the coordinate positions of the remaining targets until the fusion judgment of all targets is completed.

6. The method according to claim 1, characterized in that The specific steps of target situation prediction and our mission display (i.e. displaying the enemy and our situation) in (50) are as follows: (51) Display the icon position in real time according to the mission of our combat force in the system. Display the enemy icon in real time according to the category and coordinates of the enemy target output by (30). Click the enemy or our icon to display the location of our force, equipment, personnel and other attribute data. (Display "status") (52) According to the tasks and action instructions of our combat forces in the system, we draw the maneuver routes, attack directions, etc. (showing our "momentum") (53) Based on the enemy target attributes and past tasks, the attention mechanism algorithm is used to predict the specific tasks of the enemy entity. The specific predicted tasks of the enemy entity are integrated with the Bayesian algorithm to predict the enemy's overall intentions, and the enemy entity's maneuver routes, attack directions, etc. are plotted. (Showing the enemy's "momentum"). (54) Supports users to modify the situation map output by the system. Supports modification of enemy position, our position, enemy entity mission, and our entity mission.

7. The method according to claim 1, characterized in that The specific steps of the target situation update iteration and situation integration reporting (60) are: (61) The user prepares combat missions and reconnaissance missions and executes them. (62) Our forces carry out missions, obtain intelligence on enemy targets, and organize and integrate the intelligence into textual information. (63) Call (10) in an iterative loop to update the status; at the same time, report the status of this level to the upper-level user or send the status of this level to the lower-level user according to the needs.

Images