Battlefield semantic situation rapid generation method based on image analysis

Abstract

The invention discloses a battlefield semantic situation rapid generation method based on image analysis, and belongs to the technical field of battlefield situation generation. The method comprises the following steps: dividing a battlefield area into a plurality of local battlefields; reconnaissance is carried out on each local battlefield area; generating a multi-source heterogeneous image database; performing time-space relationship unification on the multi-source heterogeneous image database; constructing a battlefield situation element composition system; constructing a battlefield situation element extraction technology system; carrying out adaptive extraction on battlefield situation elements; generating a local battlefield situation; a global battlefield situation is generated based on the local battlefield situation. The invention provides a battlefield situation element extraction framework based on deep learning, designs a battlefield situation generation method based on deep learning, and provides a conflict situation fusion method and a battlefield region division method based on an attention mechanism and a complete space-time relationship unification mode. A whole set of battlefield semantic situation adaptive generation method is constructed, and decision-making assistance can be provided for direct commanders and fighters.

Description

Technical Field

[0001] The present invention belongs to the technical field of battlefield situation construction, and in particular refers to a method for quickly generating battlefield semantic situation based on image analysis, which can be used for quickly generating battlefield semantic situation and providing support for combat command. Background Art

[0002] Unmanned combat is an important development trend of intelligent future wars. Unmanned / unmanned collaboration and manned / unmanned collaboration are the main modes of future combat. The combat coordination capability of unmanned platforms is the key to winning future wars. Under complex battlefield conditions, unmanned combat platforms face harsh electromagnetic combat environments. Network communications between unmanned platforms and between manned / unmanned platforms are limited by non-real-time and narrow bandwidth, and cannot support real-time exchange of large-capacity information. The battlefield situation can comprehensively reflect the real-time status of the entire battlefield environment, the deployment of enemy and our forces, combat operations and other information, and can quickly respond and infer the enemy's strategic action deployment. Moreover, the amount of information is small, which can match the harsh communication environment of the battlefield. Therefore, a correct understanding of the battlefield situation is a prerequisite for unmanned platforms to achieve combat coordination.

[0003] In unmanned combat, image data is important intelligence data and an important supplement to electromagnetic intelligence data. However, the image data obtained from the battlefield is often multi-source heterogeneous image data, and there is a serious problem of mismatch between time and space. Therefore, based on the multi-source heterogeneous image data of the battlefield, the semantic comprehensive situation construction technology of complex battlefields with unified time and space relationships is studied to achieve a semantic unified representation of the battlefield situation, which is conducive to battlefield commanders to grasp the current battlefield situation in real time, improve decision-making efficiency, and provide support for unmanned combat.

[0004] Previous studies have mostly divided situation construction into multiple modules: data fusion, system construction, estimation algorithm, expert knowledge base, algorithm evaluation, etc. The modules are separated and not organically combined. Some modules are based on manually designed rules, which will seriously affect the effectiveness of the entire system. Moreover, the situation assessment results are mostly some situation classifications, such as: attack, defense, retreat, etc., and the logical relationship between the core targets on the battlefield cannot be known. These have affected the further application of situation construction in modern warfare. Summary of the invention

[0005] In view of this, the present invention proposes a method for quickly generating semantic battlefield situation based on image analysis. The method obtains the local battlefield situation based on image analysis, and generates the global battlefield situation based on the local battlefield situation through an attention mechanism.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] A method for quickly generating battlefield semantic situation based on image analysis includes the following steps:

[0008] Step 1, dividing the battlefield area into multiple local battlefields;

[0009] Step 2: Conduct reconnaissance of each local battlefield area;

[0010] Step 3, generating a multi-source heterogeneous image database;

[0011] Step 4, unify the spatiotemporal relationship of the multi-source heterogeneous image database;

[0012] Step 5: Construct a battlefield situation element composition system;

[0013] Step 6: Construct a battlefield situation factor extraction technology system;

[0014] Step 7, adaptively extracting battlefield situation elements;

[0015] Step 8, generating a local battlefield situation;

[0016] Step 9, generating the global battlefield situation based on the local battlefield situation.

[0017] Furthermore, in step 1, one or more of the following rules are used to divide the battlefield area:

[0018] 1a) If there are mountains or rivers in the battlefield area, they will be regarded as the local battlefield boundary;

[0019] 1b) If there are multiple administrative regions within the battlefield area, the administrative region will be directly used as the local battlefield area;

[0020] 1c) If multiple local battlefields have been formed in the battlefield area, the current local battlefield will be directly used as the regional division result;

[0021] 1d) Use geometric figures to divide the battlefield area;

[0022] 1e) Clustering the combat targets in the battlefield area based on the existing intelligence, and using the clustering results as the basis for regional division;

[0023] 1f) Divide local areas based on past battle examples.

[0024] Furthermore, in step 2, the method of conducting reconnaissance on each local battlefield area includes:

[0025] 2a) Use drones to conduct reconnaissance of local battlefields;

[0026] 2b) Use manned aircraft to conduct reconnaissance of local battlefields;

[0027] 2c) Use satellites to conduct reconnaissance of local battlefields.

[0028] Furthermore, the multi-source heterogeneous image database described in step 3 may include optical images, infrared images, SAR images, and optical remote sensing images.

[0029] Furthermore, the specific method of step 4 is:

[0030] 4a) Divide the local battlefield into multiple key areas based on previous battle examples, important buildings or target statistics in the area;

[0031] 4b) Divide the time into multiple time periods by evenly dividing the time periods or extracting hot time periods;

[0032] 4c) Based on the results of local battlefield area division and time period division, a multi-source heterogeneous data directory is constructed;

[0033] 4d) putting images located in the same local area into the same directory;

[0034] 4f) Put the images in the same time period into the same directory.

[0035] Furthermore, the battlefield situation element composition system described in step 5 includes:

[0036] 5a) Force deployment and combat capability, including: enemy, our and friendly force strength, battle sequence, force deployment, and force losses;

[0037] 5b) Environmental factors, including traffic conditions, climate and weather, and geographical environment, are described as follows:

[0038] ① Transportation conditions: highway, railway, aviation, and waterway;

[0039] ② Meteorology and climate: weather forecast, airport weather, sea waves, sea breeze;

[0040] ③Geographical environment: topography, river systems, and important landmarks.

[0041] Furthermore, the battlefield situation element extraction technology system described in step 6 includes:

[0042] 6a) Through target detection, intelligent target detection is realized for images, combat targets are extracted, and important markers are intelligently identified to extract information of important markers;

[0043] 6b) Intelligent classification of image scenes through scene classification;

[0044] 6c) Accurately identify the image through object recognition and identify the specific category of a certain area of ​​the image.

[0045] Furthermore, the specific method of step 7 is:

[0046] 7a) According to the actual scenario, deep learning models of target detection, scene classification, and object recognition are selected to extract battlefield elements;

[0047] 7b) Use the battlefield intelligence collected in advance to classify and count the extracted situational elements to obtain the ownership of the combat targets, troop deployment and losses, and battlefield environment.

[0048] Furthermore, the specific method of step 8 is:

[0049] By adopting deep neural networks and fitting training of battlefield data, on the one hand, the automatic selection of battlefield situation elements is realized, and on the other hand, the generation of battlefield situation is realized by following the processing method of classification problems.

[0050] Furthermore, the specific method of step 9 is:

[0051] By introducing the attention mechanism, we can automatically assign corresponding weight values ​​to different local battlefield situations to measure the contribution of local battlefield situations to the understanding of the global battlefield situation, thereby generating the global battlefield situation. The global battlefield situation generation method based on the attention mechanism is described as follows:

[0052] α=(α1,…,α N )

[0053] α=softmax{σ[W2ReLU(W1·evident)]}

[0054] α1+…+α N =1

[0055] Among them, α represents the weight vector obtained by the Attention module, σ represents the sigmoid function, ReLU represents the nonlinear activation function, softmax represents the softmax function, W1 and W2 are the parameters of the Attention module, and evident represents the local battlefield situation vector;

[0056] Multiply the weight vector by the local battlefield situation vector to obtain a weighted local battlefield situation vector;

[0057] Take the maximum value of the corresponding elements of the weighted local battlefield situation vector to obtain the global battlefield situation vector.

[0058] Compared with the prior art, the present invention has the following advantages:

[0059] 1. The battlefield situation is complex and changeable, including a variety of image data types, and situation factors generally include environmental characteristics, object types, object dynamics and other elements, which are difficult to extract. The traditional manually designed situation factor extraction method lacks theoretical guidance and is time-consuming and labor-intensive. The present invention proposes a battlefield situation factor extraction architecture based on deep learning, which does not need to rely on expert experience for manual situation factor extraction, realizes adaptive extraction of battlefield situation factors, and improves the efficiency of battlefield situation factor extraction.

[0060] 2. Most of the existing situation generation methods need to meet many assumptions and are usually only applicable to a single scenario. The present invention proposes a battlefield situation generation method based on deep learning, which fits the mapping relationship between battlefield situation elements and the current battlefield situation. It does not need to rely on fixed rules to generate battlefield situations, realizes adaptive generation of battlefield situations, and improves the efficiency of battlefield situation generation.

[0061] 3. The battlefield multi-source heterogeneous image data has a serious problem of discretization of temporal and spatial relations, and battlefield situation conflicts are difficult to avoid. Considering that different local battlefield situations contribute differently to the global battlefield situation, a conflict situation fusion method based on the attention mechanism is proposed, which can perform self-learning and updating according to the battlefield data, and improve the accuracy of battlefield situation generation results.

[0062] 4. The present invention proposes complete guiding principles for the division of battlefield areas, including topography, administrative divisions, geometric figures, existing intelligence, previous battle examples, etc., and the global battlefield can be divided into multiple local battlefields by referring to one or more of these principles.

[0063] 5. Aiming at the problem of discrete spatiotemporal relationship of multi-source heterogeneous data in battlefield, the present invention proposes a complete unified spatiotemporal relationship method, including key area selection, time period division and other methods.

[0064] 6. The present invention constructs a complete set of battlefield semantic situation adaptive generation methods, which do not require human participation in the intermediate process and can quickly generate battlefield semantic situation.

[0065] 7. Through the method of the present invention, the original image information can be abstracted into semantic information, which can achieve exponential compression of information, facilitate information transmission, and the situation information has a high semantic level, providing decision-making assistance for direct commanders and fighters. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is the overall flow chart of the present invention.

[0067] Figure 2 Classify battlefield situation elements.

[0068] Figure 3 Extract the plan diagram for battlefield situation elements.

[0069] Figure 4 Generate a scenario map for the local battlefield situation.

[0070] Figure 5 Generate a schematic diagram of the global battlefield situation based on the attention mechanism. DETAILED DESCRIPTION

[0071] The technical solution and effects of the present invention are further described in detail below in conjunction with the accompanying drawings.

[0072] Reference Figure 1 A method for quickly generating battlefield semantic situation based on image analysis includes the following steps:

[0073] Step 1: Divide the battlefield area into multiple local battlefields. The specific division method is as follows:

[0074] 1a) If there are typical landforms such as mountains or rivers in the battlefield area, they can be used as the local battlefield boundary;

[0075] 1b) If there are multiple administrative regions within the battlefield area, the administrative region can be directly used as the local battlefield area;

[0076] 1c) If multiple local battlefields have been formed in the battlefield area, the current local battlefield can be directly used as the regional division result;

[0077] 1d) Use geometric shapes such as rectangles and hexagons to directly divide the area;

[0078] 1e) Cluster the combat targets in the region based on the available intelligence and use the clustering results as the basis for regional division;

[0079] 1f) Based on past battle examples, this area is divided into local areas

[0080] Step 2: Conduct reconnaissance of each local battlefield area. The specific method is as follows:

[0081] 2a) Use drones to conduct reconnaissance of local battlefields;

[0082] 2b) Use manned aircraft to conduct reconnaissance of local battlefields;

[0083] 2c) Use satellites to conduct reconnaissance of local battlefields.

[0084] Step 3, generate a multi-source heterogeneous image database, including: light, infrared, SAR, optical remote sensing and other images.

[0085] Step 4: Unify the spatiotemporal relationship of the multi-source heterogeneous image database. The specific method is as follows:

[0086] 4a) Divide the local battlefield into multiple key areas, with specific guiding principles such as previous battle examples, important buildings, and target statistics in the area;

[0087] 4b) Divide the time into multiple time periods; the specific guiding principles are: even division of time periods, extraction of hot time periods (based on target statistics within the time period), etc.

[0088] 4c) Construct a multi-source heterogeneous data directory based on the local battlefield area division results and time period division results;

[0089] 4d) putting images located in the same local area into the same directory;

[0090] 4f) Put the images in the same time period into the same directory.

[0091] Step 5: Construct a battlefield situation element composition system, such as Figure 2 As shown, including:

[0092] 5a) Force deployment and combat capability, including: enemy, our and friendly force strength, battle sequence, force deployment, and force losses;

[0093] 5b) Environmental factors, including traffic conditions, climate and weather, and geographical environment, are described as follows:

[0094] ① Transportation conditions: highway, railway, aviation, and waterway;

[0095] ② Meteorology and climate: weather forecast, airport weather, sea waves, sea breeze;

[0096] ③Geographical environment: topography, river systems, and important landmarks.

[0097] Step 6: Construct a battlefield situation factor extraction technology system, including:

[0098] 6a) Target detection: Realize intelligent target detection of images, extract combat targets, and intelligently identify important markers to extract information of important markers;

[0099] 6b) Scene classification realizes intelligent classification of image scenes, which can be used for climate and meteorological classification, geographical environment classification, etc.

[0100] 6c) Object recognition enables accurate recognition of images and identifies the specific category of a certain area in the image, such as bridges, railways, etc.

[0101] Step 7, adaptively extract battlefield situation elements, such as Figure 3 As shown, the specific method is:

[0102] 7a) According to the actual scenario, select appropriate deep learning models such as target detection, scene classification, and object recognition to extract battlefield elements;

[0103] 7b) Finally, use the battlefield intelligence collected in advance to classify and count the extracted situational elements, such as the owner of the combat target (enemy, our side, friendly side), troop deployment and losses, battlefield environment, etc.

[0104] Step 8: Generate a local battlefield situation, such as Figure 4 As shown, the specific method is:

[0105] By adopting deep neural networks and fitting training of battlefield data, on the one hand, the automatic selection of battlefield situation elements is realized, and on the other hand, the processing method of classification problems is followed to realize the generation of battlefield situation, so as to avoid introducing human factors into the model as much as possible.

[0106] Step 9, generate the global battlefield situation based on the local battlefield situation, such as Figure 5 As shown, the specific method is:

[0107] By introducing the attention mechanism, we can automatically assign corresponding weight values ​​to different local battlefield situations to measure the contribution of local battlefield situations to the understanding of the global battlefield situation, thereby generating the global battlefield situation. The global battlefield situation generation method based on the attention mechanism is described as follows:

[0108] α=(α1,…,α N )

[0109] α=softmax{σ[W2ReLU(W1·evident)]}

[0110] α1+…+α N =1

[0111] Among them, α represents the weight vector obtained by the Attention module, σ represents the sigmoid function, ReLU represents the nonlinear activation function, softmax represents the softmax function, W1 and W2 are the parameters of the Attention module (which can be updated through iteration), evident represents the local battlefield situation vector, and the final weight vector is multiplied by the local battlefield situation vector to obtain the weighted local battlefield situation vector. The maximum value of the corresponding elements of the weighted local battlefield situation vector is taken to obtain the global battlefield situation vector.

[0112] The effect of this method can be further illustrated by the following simulation experiment:

[0113] 1. Experimental conditions and methods

[0114] The hardware platform is: Intel Xeon E5-2678 v332GB;

[0115] Software platform: Linux Ubuntu 18.04LTS;

[0116] Programming language: Python 3.6;

[0117] Deep learning framework: Pytorch 1.6.0.

[0118] Experimental method: This method.

[0119] 2. Simulation content and results

[0120] Construct a multi-source image heterogeneous database based on Figure 3 The proposed method adaptively extracts battlefield elements based on Figure 4 The proposed method adaptively fits the battlefield situation. Figure 5 The method given generates local battlefield situation diagrams and global battlefield situation diagrams.

[0121] The present invention is aimed at unmanned combat. It utilizes multi-source heterogeneous data obtained at different times and locations from various types of unmanned equipment and payloads on the battlefield, and realizes adaptive extraction of battlefield situation elements based on a situation element extraction framework based on a deep neural network. The situation generation method based on deep learning introduces an attention mechanism, and realizes the generation of complex battlefield situations with unified time and space relationships by unifying time nodes and spatial positions.

[0122] The present invention proposes a battlefield situation element extraction architecture based on deep learning, designs a battlefield situation generation method based on deep learning, proposes a conflict situation fusion method based on attention mechanism, proposes a battlefield area division method, proposes a complete unified way of time and space relationship, and constructs a complete set of battlefield semantic situation adaptive generation methods, thereby abstracting the original image information into semantic information, which can achieve exponential compression of information and facilitate information transmission. The situation information has a high semantic level, providing decision-making assistance for direct commanders and fighters.

Claims

1. A method for rapidly generating battlefield semantic situation based on image analysis, characterized in that: The steps include: Step 1, dividing the battlefield area into multiple local battlefields; Step 2: Conduct reconnaissance of each local battlefield area; Step 3, generating a multi-source heterogeneous image database; Step 4, unify the spatiotemporal relationship of the multi-source heterogeneous image database; Step 5: Construct a battlefield situation element composition system; Step 6: Construct a battlefield situation factor extraction technology system; Step 7, adaptively extracting battlefield situation elements; Step 8, generating a local battlefield situation; Step 9, generating the global battlefield situation based on the local battlefield situation.

2. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: In step 1, use one or more of the following rules to divide the battlefield area: 1a) If there are mountains or rivers in the battlefield area, they will be regarded as the local battlefield boundary; 1b) If there are multiple administrative regions within the battlefield area, the administrative region will be directly used as the local battlefield area; 1c) If multiple local battlefields have been formed in the battlefield area, the current local battlefield will be directly used as the regional division result; 1d) Use geometric figures to divide the battlefield area; 1e) Clustering the combat targets in the battlefield area based on the existing intelligence, and using the clustering results as the basis for regional division; 1f) Divide local areas based on past battle examples.

3. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: In step 2, the methods for conducting reconnaissance on each local battlefield area include: 2a) Use drones to conduct reconnaissance of local battlefields; 2b) Use manned aircraft to conduct reconnaissance of local battlefields; 2c) Use satellites to conduct reconnaissance of local battlefields.

4. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: The multi-source heterogeneous image database described in step 3 may include optical images, infrared images, SAR images, and optical remote sensing images.

5. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1 is characterized in that: The specific method of step 4 is: 4a) Divide the local battlefield into multiple key areas based on previous battle examples, important buildings or target statistics in the area; 4b) Divide the time into multiple time periods by evenly dividing the time periods or extracting hot time periods; 4c) Based on the results of local battlefield area division and time period division, a multi-source heterogeneous data directory is constructed; 4d) putting images located in the same local area into the same directory; 4f) Put the images in the same time period into the same directory.

6. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: The battlefield situation elements composition system described in step 5 includes: 5a) Force deployment and combat capability, including: enemy, our and friendly force strength, battle sequence, force deployment, and force losses; 5b) Environmental factors, including traffic conditions, climate and weather, and geographical environment, are described as follows: ① Transportation conditions: highway, railway, aviation, and waterway; ② Meteorology and climate: weather forecast, airport weather, sea waves, sea breeze; ③Geographical environment: topography, river systems, and important landmarks.

7. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: The battlefield situation element extraction technology system described in step 6 includes: 6a) Through target detection, intelligent target detection is realized for images, combat targets are extracted, and important markers are intelligently identified to extract information of important markers; 6b) Intelligent classification of image scenes through scene classification; 6c) Accurately identify the image through object recognition and identify the specific category of a certain area of ​​the image.

8. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1 is characterized in that: The specific method of step 7 is: 7a) According to the actual scenario, deep learning models of target detection, scene classification, and object recognition are selected to extract battlefield elements; 7b) Use the battlefield intelligence collected in advance to classify and count the extracted situational elements to obtain the ownership of the combat targets, troop deployment and losses, and battlefield environment.

9. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: The specific method of step 8 is: By adopting deep neural networks and fitting training of battlefield data, on the one hand, the automatic selection of battlefield situation elements is realized, and on the other hand, the generation of battlefield situation is realized by following the processing method of classification problems.

10. The method for rapidly generating battlefield semantic situation based on image analysis according to claim 1, characterized in that: The specific method of step 9 is: By introducing the attention mechanism, we can automatically assign corresponding weight values ​​to different local battlefield situations to measure the contribution of local battlefield situations to the understanding of the global battlefield situation, thereby generating the global battlefield situation. The global battlefield situation generation method based on the attention mechanism is described as follows: a=(a1,…,a N ) a=softmax{s[W2ReLU(W1·evident)]} α1+…+a N ＝1 Among them, α represents the weight vector obtained by the Attention module, s represents the sigmoid function, ReLU represents the nonlinear activation function, softmax represents the softmax function, W1 and W2 are the parameters of the Attention module, and evident represents the local battlefield situation vector; Multiply the weight vector by the local battlefield situation vector to obtain a weighted local battlefield situation vector; Take the maximum value of the corresponding elements of the weighted local battlefield situation vector to obtain the global battlefield situation vector.

Images