Method, device and equipment for determining task target based on situation awareness and probability decision, and medium
By combining data processing from optical cameras and infrared thermal imagers, a dynamic knowledge graph is constructed and Bayesian network reasoning is performed, solving the target identification and decision-making problem of loitering munitions in complex areas and achieving higher intelligence and adaptability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- AVIC (CHENGDU) UAS CO LTD
- Filing Date
- 2026-05-22
- Publication Date
- 2026-06-19
AI Technical Summary
Existing loitering munitions' automatic target identification technology cannot distinguish the target environment in complex and ever-changing areas, resulting in insufficient decision-making intelligence and adaptability, making them prone to tactical traps and unable to handle uncertainties in front-end perception.
Real-time data of the target area is acquired using optical cameras and infrared thermal imagers. Multimodal feature maps are extracted through convolutional neural networks. A dynamic knowledge graph is constructed by combining it with a pre-set prior knowledge base and mapped to a target Bayesian network for probabilistic reasoning to determine the final task objective.
It significantly improves perception range and decision-making accuracy in complex environments, reduces the probability of accidental injury and falling into tactical traps, and makes the decision-making process transparent and traceable.
Smart Images
Figure CN122242782A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of aviation technology, and in particular to a method, apparatus, equipment and medium for determining mission objectives based on situational awareness and probabilistic decision-making. Background Technology
[0002] As a smart munition integrating ISR (Intelligence, Surveillance, and Reconnaissance) and precision strike capabilities, the core intelligence of loitering munitions lies in the "seeing-understanding-decision" chain. Currently, the technological implementation of this chain heavily relies on deep learning-based Automatic Target Recognition (ATR) and multi-factor scoring-based decision-making models. Since step A only provides isolated target information, the evaluation model in step B is completely unaware of the target's tactical environment. For example, it cannot distinguish between "a target parked in the wilderness" and "a target parked near a school," nor can it identify whether the target is under the cover of an enemy air defense system. This cognitive model is highly susceptible to making decisions that fall into tactical traps (attacking high-value decoys); furthermore, it cannot handle the inherent uncertainties of front-end perception (such as low confidence due to target camouflage and obstruction), making the decision-making unable to adapt to the complex and ever-changing real-time situation in the region, resulting in a severe lack of intelligence and adaptability.
[0003] As can be seen from the above, how to enable loitering munitions to make decisions on the optimal mission objectives in complex and ever-changing environments is a problem that urgently needs to be solved. Summary of the Invention
[0004] In view of this, the purpose of this invention is to provide a method, apparatus, device, and medium for determining mission targets based on situational awareness and probabilistic decision-making, enabling loitering munitions to make decisions regarding optimal mission targets in complex and variable environments. The specific solution is as follows: Firstly, this application provides a mission target determination method based on situational awareness and probabilistic decision-making, applied to an airborne embedded computing platform for loitering munitions, comprising: Real-time data of the target area is acquired using optical cameras and infrared thermal imagers on the loitering munition. Features are extracted from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map. The multimodal feature map is then input into the target deep learning model to obtain each recognition result and the corresponding recognition confidence. A graph rule is determined using a pre-defined prior knowledge base, and a dynamic knowledge graph is constructed based on the recognition results and the recognition confidence level, using the graph rule. The dynamic knowledge graph is mapped to a target Bayesian network, and a preset inference algorithm is used to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision output; Based on the reasoning results, a multi-objective decision ranking is performed, the final task objective is determined using the ranking results, the target reasoning path corresponding to the final task objective is determined, and a corresponding decision report is generated based on the target reasoning path.
[0005] Optionally, the real-time data of the target area acquired by the optical camera and infrared thermal imager on the loitering munition, and the feature extraction of the real-time data using a convolutional neural network to construct a corresponding multimodal feature map, includes: The loitering munition uses an optical camera on its loitering munition to capture real-time visible light images of the target area, and uses an infrared thermal imager on its loitering munition to capture real-time infrared images of the target area. The visible light real-time image and the infrared real-time image are respectively denoised, non-uniformity corrected and image registered to obtain the processed visible light image and the processed infrared image; The first semantic feature corresponding to the processed visible light image and the second semantic feature corresponding to the processed infrared image are extracted using the target network structure in a parallel convolutional neural network. A multimodal feature map is determined based on the first semantic feature and the second semantic feature and using a cross-modal fusion module.
[0006] Optionally, the step of determining the multimodal feature map based on the first semantic feature and the second semantic feature and utilizing the cross-modal fusion module includes: A cross-modal fusion module is constructed based on the structure of the Transformer encoder, and the first cross-attention weight corresponding to the first semantic feature and the second cross-attention weight corresponding to the second semantic feature are determined by the cross-modal fusion module. Construct the corresponding multimodal feature map based on the first cross-attention weight and the second cross-attention weight.
[0007] Optionally, the step of inputting the multimodal feature map into the target deep learning model to obtain each recognition result and the corresponding recognition confidence score includes: A target deep learning model is constructed based on a graph neural network or DETR architecture, and the target deep learning model is used to identify the multimodal feature map to obtain the target entity and the first confidence level corresponding to the target entity, the entity attribute and the second confidence level corresponding to the entity attribute, the entity relationship and the third confidence level corresponding to the entity relationship.
[0008] Optionally, the step of determining the graph rules using a preset prior knowledge base, and constructing a dynamic knowledge graph based on the recognition results and the recognition confidence level using the graph rules, includes: Graph rules, including node types and relationship types, are determined using a pre-defined prior knowledge base; Based on the graph rules, the target node type corresponding to the target entity is determined, and the target relationship type corresponding to the entity relationship is determined using the graph rules; Nodes are determined using the target entity and the target node type, and edges are determined based on the entity relationship and the target relationship type; A dynamic knowledge graph is constructed based on the nodes, the confidence scores corresponding to the nodes, the edges, and the confidence scores corresponding to the edges.
[0009] Optionally, the step of mapping the dynamic knowledge graph to a target Bayesian network and using a preset inference algorithm to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region includes: Determine the initial Bayesian network, map the dynamic knowledge graph to the bottom-level nodes of the initial Bayesian network, and derive the intermediate-level nodes from the bottom-level nodes. The intermediate layer nodes are derived to obtain the top layer node, and the target Bayesian network is determined based on the bottom layer node, the intermediate layer node, and the top layer node. Probabilistic reasoning is performed on the target Bayesian network using an exact reasoning algorithm or an approximate reasoning algorithm to obtain the expected utility value and risk quantification index of each potential mission target of the loitering munition in the target area.
[0010] Optionally, the step of performing multi-objective decision ranking based on the reasoning results, determining the final task objective using the obtained ranking results, determining the target reasoning path corresponding to the final task objective, and generating a corresponding decision report based on the target reasoning path includes: The strike score corresponding to each potential mission target is determined using the strike expected utility value, the risk quantification index, and the preset risk aversion coefficient; The strike scores are sorted from highest to lowest, and the potential mission target with the highest strike score is determined as the final mission target based on the sorting results. The probabilistic reasoning process corresponding to the final task objective is backtracked to obtain the objective reasoning path, and a corresponding decision report is generated based on the objective reasoning path.
[0011] Secondly, this application provides a mission target determination device based on situational awareness and probabilistic decision-making, applied to an airborne embedded computing platform for a loitering munition, comprising: The feature extraction module is used to acquire real-time data of the target area based on the optical camera and infrared thermal imager on the loitering munition, extract features from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map, and input the multimodal feature map into the target deep learning model to obtain each recognition result and the corresponding recognition confidence. The knowledge graph construction module is used to determine graph rules using a preset prior knowledge base, and to construct a dynamic knowledge graph based on the recognition results and the recognition confidence and using the graph rules. The network reasoning module is used to map the dynamic knowledge graph into a target Bayesian network and use a preset reasoning algorithm to perform probabilistic reasoning on the target Bayesian network to obtain the reasoning results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision outputs; The decision report generation module is used to sort multi-objective decisions based on the reasoning results, determine the final task objective using the obtained sorting results, determine the target reasoning path corresponding to the final task objective, and generate a corresponding decision report based on the target reasoning path.
[0012] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is used to execute the computer program to implement the aforementioned task target determination method based on situational awareness and probabilistic decision-making.
[0013] Fourthly, this application provides a computer-readable storage medium for storing a computer program, wherein the computer program, when executed by a processor, implements the aforementioned task target determination method based on situational awareness and probabilistic decision-making.
[0014] This application acquires real-time data of the target area using an optical camera and an infrared thermal imager on a loitering munition. It then uses a convolutional neural network to extract features from the real-time data to construct a corresponding multimodal feature map. This multimodal feature map is input into a target deep learning model to obtain various recognition results and corresponding recognition confidence levels. A pre-defined prior knowledge base is used to determine graph rules. Based on the recognition results and recognition confidence levels, and using the graph rules, a dynamic knowledge graph is constructed. This dynamic knowledge graph is mapped to a target Bayesian network, and a pre-defined inference algorithm is used to perform probabilistic inference on the target Bayesian network to obtain inference results corresponding to each potential mission objective of the loitering munition in the target area. The target Bayesian network includes bottom-level nodes representing observational evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision outputs. Based on the inference results, multi-objective decision ranking is performed. The obtained ranking results are used to determine the final mission objective, and a target inference path corresponding to the final mission objective is determined. A corresponding decision report is generated based on the target inference path.
[0015] As can be seen from the above, this application utilizes an optical camera to capture the details of the target's shape and an infrared thermal imager to capture thermal signals. Combining these features significantly expands the effective perception range in complex environments. A deep learning model is used to identify the feature maps, obtaining the target entities and their relationships. A dynamic knowledge graph is constructed using graph rules, mapping the graph to a three-layer target Bayesian network. A pre-defined inference algorithm is used to perform bottom-up probabilistic inference on the target Bayesian network, and the final mission objective is determined based on the inference results. In this way, by tracing back the probabilistic inference process corresponding to the final mission objective, a corresponding decision report is generated, making the decision-making process transparent and traceable, greatly reducing the probability of accidental injury or falling into tactical traps in complex environments. Attached Figure Description
[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0017] Figure 1 This is a flowchart of a task target determination method based on situational awareness and probabilistic decision-making disclosed in this application; Figure 2 This application discloses a specific method for determining task objectives based on situational awareness and probabilistic decision-making. Figure 3 This is a schematic diagram illustrating the execution of one of the final mission objectives disclosed in this application; Figure 4 This is a schematic diagram of a task target determination device based on situational awareness and probabilistic decision-making disclosed in this application; Figure 5 This is a structural diagram of an electronic device disclosed in this application. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Currently, existing technologies include Automatic Target Recognition (ATR) based on deep learning and decision-making models based on multi-factor scoring. However, these are prone to making decisions that fall into tactical traps (attacking high-value false targets). Furthermore, they cannot handle the inherent uncertainties in front-end perception (such as low confidence due to target camouflage or occlusion), making the decisions unable to adapt to complex and ever-changing real-time regional situations, resulting in severe deficiencies in intelligence and adaptability. To address this, this application provides a task target determination method based on situational awareness and probabilistic decision-making. By retrospectively analyzing the probabilistic reasoning process corresponding to the final task target, a corresponding decision report is generated, making the decision-making process transparent and traceable, significantly reducing the probability of friendly fire or falling into tactical traps in complex environments.
[0020] See Figure 1 As shown, this invention discloses a mission target determination method based on situational awareness and probabilistic decision-making, applied to an airborne embedded computing platform for a loitering munition, comprising: Step S11: Based on the optical camera and infrared thermal imager on the loitering munition, real-time data of the target area is acquired. The real-time data is then used to extract features from the data using a convolutional neural network to construct a corresponding multimodal feature map. The multimodal feature map is then input into the target deep learning model to obtain each recognition result and the corresponding recognition confidence.
[0021] In this embodiment, the airborne embedded computing platform of the loitering munition includes at least a multispectral imaging sensor (such as a visible light camera or an infrared thermal imager), a system-on-chip (SoC) or an onboard computer, memory, and a preset control interface. The software system runs on a real-time operating system (such as VxWorks) or a high-performance Linux system. The loitering munition's optical camera captures real-time visible light images of the target area, and the loitering munition's infrared thermal imager captures real-time thermal signal images of the target area, i.e., real-time infrared images. Noise reduction is applied to these images, and non-uniformity correction is performed on the infrared images to make the thermal signal display more accurate. The positions of the visible light real-time images and the infrared real-time images are precisely aligned to obtain processed images. Then, the target network structure in a target number of parallel convolutional neural networks is used to extract high-level semantic features from the processed images. These semantic features are input into a cross-modal fusion module based on an attention mechanism to fuse the semantic features and obtain a multimodal feature map. The target number can be determined according to actual conditions; the target network structure can be ResNet-50 (Residual Network).
[0022] Specifically, the process of acquiring real-time data of the target area using an optical camera and an infrared thermal imager on a loitering munition, and extracting features from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map, includes: capturing a real-time visible light image of the target area using the optical camera on the loitering munition, and capturing a real-time infrared image of the target area using the infrared thermal imager on the loitering munition; performing denoising, non-uniformity correction, and image registration on the real-time visible light image and the real-time infrared image, respectively, to obtain a processed visible light image and a processed infrared image; extracting a first semantic feature corresponding to the processed visible light image and a second semantic feature corresponding to the processed infrared image using the target network structure in a parallel convolutional neural network; and determining the multimodal feature map based on the first semantic feature and the second semantic feature using a cross-modal fusion module.
[0023] It is understood that the cross-modal fusion module can be structured as a Transformer encoder. The cross-modal fusion module determines the cross-attention weights corresponding to each semantic feature, and fuses the semantic features based on these cross-attention weights to obtain a multimodal feature map. For example, in a smoke environment, the model automatically enhances the weights of infrared features. Specifically, determining the multimodal feature map based on the first and second semantic features using the cross-modal fusion module includes: constructing a cross-modal fusion module based on the Transformer encoder structure, and using the cross-modal fusion module to determine the first cross-attention weight corresponding to the first semantic feature and the second cross-attention weight corresponding to the second semantic feature; constructing the corresponding multimodal feature map based on the first and second cross-attention weights.
[0024] Furthermore, the target deep learning model employs a graph neural network or a modified DETR (DEtectionTransformer, an end-to-end target detection model) architecture, and utilizes the target deep learning model to identify the multimodal feature map to obtain target entities, entity attributes, entity relationships, and the confidence scores of each identification result; the target entities are all potential task targets in the target region; the entity attributes are the states corresponding to each potential task target; and the entity relationships are the association relationships between the potential task targets. Specifically, the step of inputting the multimodal feature map into the target deep learning model to obtain each identification result and the corresponding identification confidence score includes: constructing a target deep learning model based on a graph neural network or DETR architecture, and using the target deep learning model to identify the multimodal feature map to obtain the target entity and its corresponding first confidence score, the entity attribute and its corresponding second confidence score, and the entity relationship and its corresponding third confidence score.
[0025] Step S12: Determine the graph rules using a preset prior knowledge base, and construct a dynamic knowledge graph based on the recognition results and the recognition confidence level using the graph rules.
[0026] In this embodiment, a pre-defined prior knowledge base is used to determine graph rules including node types and relationship types. The node types can include three categories: potential mission targets, facilities, and geographical regions. The pre-defined prior knowledge base can be a professional knowledge database stored in advance on the airborne embedded computing platform. The target entities are identified as nodes, and the entity relationships are identified as edges. The nodes and edges are classified and connected based on the node types and relationship types to obtain a dynamic knowledge graph centered on the potential mission targets. It is worth mentioning that the dynamic knowledge graph is updated based on the new recognition results for each new frame of sensor data collected.
[0027] Specifically, the step of determining graph rules using a preset prior knowledge base, and constructing a dynamic knowledge graph based on the recognition results and the recognition confidence level using the graph rules, includes: determining graph rules including node types and relationship types using a preset prior knowledge base; determining the target node type corresponding to the target entity based on the graph rules, and determining the target relationship type corresponding to the entity relationship using the graph rules; determining nodes using the target entity and the target node type, and determining edges based on the entity relationship and the target relationship type; and constructing a dynamic knowledge graph based on the nodes, the confidence level corresponding to the nodes, the edges, and the confidence level corresponding to the edges.
[0028] Step S13: Map the dynamic knowledge graph into a target Bayesian network, and use a preset inference algorithm to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision output.
[0029] In this embodiment, an initial Bayesian network is determined based on a preset prior knowledge base. This initial Bayesian network is a three-layer directed acyclic structure. The bottom-layer nodes can be obtained by mapping the dynamic knowledge graph. For example, in the dynamic knowledge graph, the confidence level of "edge (A, neighbor, school) 0.8" is transformed into a probability of 0.8 for the bottom-layer node NearCivilianArea_A being "true". The intermediate-layer nodes are derived from the observational evidence of the bottom-layer nodes, such as the risk of collateral damage and the survival probability of the mission objective. The top-layer nodes are derived from the evaluation factors of the intermediate-layer nodes. If the computing power of the airborne embedded computing platform meets the preset sufficient conditions, a precise inference algorithm is used to perform probabilistic inference on the target Bayesian network. The precise inference algorithm can be a connection tree algorithm. If the preset sufficient conditions are not met, an approximate inference algorithm is used to perform probabilistic inference on the target Bayesian network.
[0030] Specifically, the step of mapping the dynamic knowledge graph to a target Bayesian network and using a preset inference algorithm to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region includes: determining an initial Bayesian network, mapping the dynamic knowledge graph to the bottom-level nodes of the initial Bayesian network, and deriving the bottom-level nodes to obtain intermediate-level nodes; deriving the intermediate-level nodes to obtain top-level nodes; determining a target Bayesian network based on the bottom-level nodes, the intermediate-level nodes, and the top-level nodes; and using an exact inference algorithm or an approximate inference algorithm to perform probabilistic inference on the target Bayesian network to obtain the expected utility value and risk quantification index for each potential mission target of the loitering munition in the target region.
[0031] Step S14: Based on the reasoning results, perform multi-objective decision ranking, use the obtained ranking results to determine the final task objective, determine the target reasoning path corresponding to the final task objective, and generate a corresponding decision report based on the target reasoning path.
[0032] In this embodiment, the strike score corresponding to each potential mission target is determined using the strike expected utility value, the risk quantification index, and the preset risk aversion coefficient. The corresponding formula is as follows: ; in, The expected utility value of the strike; To preset the risk aversion coefficient; The risk quantification index is used. After obtaining the strike scores corresponding to each potential mission target, the strike scores are arranged from high to low, and the highest score is determined as the final mission target. Then, the probabilistic reasoning process of the target Bayesian network corresponding to the final mission target is traced back to obtain the target reasoning path, and a structured decision report is generated based on the target reasoning path; for example, mission target A is recommended. The main basis is: high value (confidence 85%) and low risk of collateral damage (confidence 78%). Note that the mission target is located at the edge of the region, and the success probability is 65%. In advanced autonomous mode, if the preset strike conditions are met, the final mission target is struck using the preset control interface; the preset strike conditions are that the expected utility value of the final mission target exceeds the target safety threshold and the risk quantification index is lower than the target risk threshold; the target safety threshold and the target risk threshold can be determined according to the actual situation.
[0033] Specifically, the step of performing multi-objective decision ranking based on the reasoning results, determining the final task objective using the obtained ranking results, determining the target reasoning path corresponding to the final task objective, and generating a corresponding decision report based on the target reasoning path includes: determining the strike score corresponding to each of the potential task objectives using the strike expected utility value, the risk quantification index, and the preset risk aversion coefficient; ranking the strike scores in descending order, and determining the potential task objective with the highest strike score as the final task objective based on the obtained ranking results; backtracking the probabilistic reasoning process corresponding to the final task objective to obtain the target reasoning path, and generating a corresponding decision report based on the target reasoning path.
[0034] As can be seen from the above, this application utilizes an optical camera to capture the details of the target's shape and an infrared thermal imager to capture thermal signals. Combining these features significantly expands the effective perception range in complex environments. A deep learning model is used to identify the feature maps, obtaining the target entities and their relationships. A dynamic knowledge graph is constructed using graph rules, mapping the graph to a three-layer target Bayesian network. A pre-defined inference algorithm is used to perform bottom-up probabilistic inference on the target Bayesian network, and the final mission objective is determined based on the inference results. In this way, by tracing back the probabilistic inference process corresponding to the final mission objective, a corresponding decision report is generated, making the decision-making process transparent and traceable, greatly reducing the probability of accidental injury or falling into tactical traps in complex environments.
[0035] As can be seen from the above embodiments, this application determines the final mission target by performing probabilistic reasoning on the target Bayesian network, so as to achieve precise strike on the final mission target. Therefore, the process of determining the final mission target by performing probabilistic reasoning on the target Bayesian network is described.
[0036] Combination Figure 2 and Figure 3 As shown, this embodiment of the invention discloses a specific method for determining mission objectives based on situational awareness and probabilistic decision-making, applied to an airborne embedded computing platform for a loitering munition, comprising: In this embodiment, a real-time visible light image of the target area is captured using an optical camera on the loitering munition, and a real-time infrared image of the target area is captured using an infrared thermal imager on the same munition. Noise reduction, non-uniformity correction, and image registration are performed on each image to obtain a processed image. Semantic features of the processed image are extracted based on the target network structure in a convolutional neural network, and a multimodal feature map is determined using a cross-modal fusion module. The multimodal feature map is then identified based on the target deep learning model to obtain the target entity, entity attributes, entity relationships, and the confidence scores corresponding to each identification result. A graph rule, including node type and relationship type, is determined using a preset prior knowledge base. The target entity is identified as a node, and the entity relationship is identified as an edge. A dynamic knowledge graph is constructed based on the graph rule, the nodes, and the edges.
[0037] Understandably, after obtaining the dynamic knowledge graph, an initial Bayesian network is determined based on a preset prior knowledge base. The dynamic knowledge graph is then mapped to the bottom-level nodes of the initial Bayesian network. The bottom-level nodes are deduced to obtain intermediate-level nodes, and the intermediate-level nodes are deduced to obtain top-level nodes. Subsequently, a three-layer target Bayesian network is constructed based on each node. A target inference algorithm is determined based on the computing power of the airborne embedded computing platform. The target inference algorithm is then used to perform probabilistic inference on the target Bayesian network to obtain the expected utility value and risk quantification index of each potential mission target of the loitering munition in the target area.
[0038] Furthermore, after obtaining the expected utility value of the strike and the risk quantification index, the strike score corresponding to each potential mission target is determined by combining a preset risk avoidance coefficient. Based on the strike scores, the potential mission targets are ranked from highest to lowest, and the one with the highest score is determined as the final mission target. Then, the probabilistic reasoning process of the target Bayesian network corresponding to the final mission target is traced back to obtain the target reasoning path, and a decision report is generated based on the target reasoning path. In supervised mode, the decision report is sent back to the target command for final adjudication. If the adjudication result is approval, the corresponding strike operation is performed; if the adjudication result is rejection, the process jumps to the step of acquiring real-time data of the target area based on the optical camera and infrared thermal imager on the loitering munition, and the mission target is replanned. In autonomous mode, it is determined whether the expected utility value of the final mission target exceeds the target safety threshold and the risk quantification index is lower than the target risk threshold. If so, the final mission target is struck using a preset control interface.
[0039] As can be seen from the above, this application utilizes optical cameras and infrared thermal imagers to acquire photoelectric and infrared images, and employs deep learning models to identify multimodal feature maps constructed from these images to obtain target entities and their relationships. Then, a dynamic knowledge graph is constructed by combining graph rules, mapping the graph to a three-layer target Bayesian network. Probabilistic reasoning is performed on the target Bayesian network, and the final mission target is determined based on the reasoning results. In this way, by tracing back the probabilistic reasoning process corresponding to the final mission target, a corresponding decision report is generated, and the decision on whether to engage the final mission target is based on the decision report, reducing the probability of friendly fire or falling into tactical traps in complex environments.
[0040] Accordingly, see Figure 4 As shown, this application also provides a mission target determination device based on situational awareness and probabilistic decision-making, applied to an airborne embedded computing platform for a loitering munition, comprising: The feature extraction module 11 is used to acquire real-time data of the target area based on the optical camera and infrared thermal imager on the loitering munition, extract features from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map, and input the multimodal feature map into the target deep learning model to obtain each recognition result and the corresponding recognition confidence. Knowledge graph construction module 12 is used to determine graph rules using a preset prior knowledge base, and to construct a dynamic knowledge graph based on the recognition results and the recognition confidence and using the graph rules; The network reasoning module 13 is used to map the dynamic knowledge graph into a target Bayesian network and use a preset reasoning algorithm to perform probabilistic reasoning on the target Bayesian network to obtain the reasoning results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision outputs; The decision report generation module 14 is used to perform multi-objective decision ranking based on the reasoning results, determine the final task objective using the obtained ranking results, determine the target reasoning path corresponding to the final task objective, and generate a corresponding decision report based on the target reasoning path.
[0041] In some specific embodiments, the feature extraction module 11 may specifically include: The image capturing unit is used to capture real-time visible light images of the target area using an optical camera on the loitering munition, and to capture real-time infrared images of the target area using an infrared thermal imager on the loitering munition. The image registration unit is used to perform denoising, non-uniformity correction and image registration on the real-time visible light image and the real-time infrared image respectively, so as to obtain the processed visible light image and the processed infrared image. The feature extraction unit is used to extract the first semantic feature corresponding to the processed visible light image and the second semantic feature corresponding to the processed infrared image using the target network structure in the parallel convolutional neural network. The feature map determination submodule is used to determine a multimodal feature map based on the first semantic feature and the second semantic feature and by utilizing the cross-modal fusion module.
[0042] In some specific implementations, the feature map determination submodule may specifically include: The weight determination unit is used to construct a cross-modal fusion module based on the structure of the Transformer encoder, and to use the cross-modal fusion module to determine the first cross-attention weight corresponding to the first semantic feature and the second cross-attention weight corresponding to the second semantic feature, respectively. The feature map construction unit is used to construct a corresponding multimodal feature map based on the first cross-attention weight and the second cross-attention weight.
[0043] In some specific embodiments, the feature extraction module 11 may specifically include: The feature map recognition unit is used to construct a target deep learning model based on a graph neural network or DETR architecture, and to use the target deep learning model to recognize the multimodal feature map to obtain the target entity and the first confidence level corresponding to the target entity, the entity attribute and the second confidence level corresponding to the entity attribute, the entity relationship and the third confidence level corresponding to the entity relationship.
[0044] In some specific embodiments, the knowledge graph construction module 12 may specifically include: The graph rule determination unit is used to determine graph rules, including node type and relation type, using a preset prior knowledge base; The relationship type determination unit is used to determine the target node type corresponding to the target entity based on the graph rules, and to determine the target relationship type corresponding to the entity relationship using the graph rules; An edge determination unit is used to determine nodes using the target entity and the target node type, and to determine edges based on the entity relationship and the target relationship type; The graph construction unit is used to construct a dynamic knowledge graph based on the nodes, the confidence levels corresponding to the nodes, the edges, and the confidence levels corresponding to the edges.
[0045] In some specific embodiments, the network inference module 13 may specifically include: The bottom-level node derivation unit is used to determine the initial Bayesian network, map the dynamic knowledge graph to the bottom-level nodes of the initial Bayesian network, and derive the bottom-level nodes to obtain the intermediate-level nodes. The target network determination unit is used to deduce the intermediate layer nodes to obtain the top layer nodes, and determine the target Bayesian network based on the bottom layer nodes, the intermediate layer nodes, and the top layer nodes. The network probabilistic inference unit is used to perform probabilistic inference on the target Bayesian network using an exact inference algorithm or an approximate inference algorithm to obtain the expected utility value and risk quantification index of each potential mission target of the loitering munition in the target area.
[0046] In some specific embodiments, the decision report generation module 14 may specifically include: The strike score determination unit is used to determine the strike score corresponding to each of the potential mission targets by using the strike expected utility value, the risk quantification index and the preset risk avoidance coefficient; The final target determination unit is used to sort the strike scores in descending order and determine the potential mission target with the highest strike score as the final mission target based on the sorting result. The report generation unit is used to backtrack the probabilistic reasoning process corresponding to the final task objective to obtain the target reasoning path, and generate a corresponding decision report based on the target reasoning path.
[0047] Furthermore, embodiments of this application also disclose an electronic device, Figure 5 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content of the diagram should not be construed as limiting the scope of this application. The electronic device 20 may specifically include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 stores a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the task target determination method based on situational awareness and probabilistic decision-making disclosed in any of the foregoing embodiments. Furthermore, the electronic device 20 in this embodiment may specifically be an electronic computer.
[0048] In this embodiment, the power supply 23 is used to provide operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and external devices, and the communication protocol it follows can be any communication protocol applicable to the technical solution of this application, and is not specifically limited here; the input / output interface 25 is used to acquire external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.
[0049] In addition, the memory 22, as a carrier for resource storage, can be a read-only memory, random access memory, disk or optical disk, etc. The resources stored thereon can include operating system 221, computer program 222, etc., and the storage method can be temporary storage or permanent storage.
[0050] The operating system 221 is used to manage and control the various hardware devices on the electronic device 20 and the computer program 222, which may be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program capable of performing the task target determination method based on situational awareness and probabilistic decision-making disclosed in any of the foregoing embodiments, the computer program 222 may further include computer programs capable of performing other specific tasks.
[0051] Furthermore, this application also discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, it implements the aforementioned task target determination method based on situational awareness and probabilistic decision-making. Specific steps of this method can be found in the corresponding content disclosed in the foregoing embodiments, and will not be repeated here.
[0052] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to in the method section.
[0053] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0054] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.
[0055] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0056] The technical solutions provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for determining task objectives based on situational awareness and probabilistic decision-making, characterized in that, Airborne embedded computing platforms for loitering munitions include: Real-time data of the target area is acquired using optical cameras and infrared thermal imagers on the loitering munition. Features are extracted from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map. The multimodal feature map is then input into the target deep learning model to obtain each recognition result and the corresponding recognition confidence. A graph rule is determined using a pre-defined prior knowledge base, and a dynamic knowledge graph is constructed based on the recognition results and the recognition confidence level, using the graph rule. The dynamic knowledge graph is mapped to a target Bayesian network, and a preset inference algorithm is used to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision output; Based on the reasoning results, a multi-objective decision ranking is performed, the final task objective is determined using the ranking results, the target reasoning path corresponding to the final task objective is determined, and a corresponding decision report is generated based on the target reasoning path.
2. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 1, characterized in that, The method involves acquiring real-time data of the target area using an optical camera and an infrared thermal imager mounted on a loitering munition, and then using a convolutional neural network to extract features from the real-time data to construct a corresponding multimodal feature map, including: The loitering munition uses an optical camera on its loitering munition to capture real-time visible light images of the target area, and uses an infrared thermal imager on its loitering munition to capture real-time infrared images of the target area. The visible light real-time image and the infrared real-time image are respectively denoised, non-uniformity corrected and image registered to obtain the processed visible light image and the processed infrared image; The first semantic feature corresponding to the processed visible light image and the second semantic feature corresponding to the processed infrared image are extracted using the target network structure in a parallel convolutional neural network. A multimodal feature map is determined based on the first semantic feature and the second semantic feature and using a cross-modal fusion module.
3. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 2, characterized in that, The step of determining a multimodal feature map based on the first semantic feature and the second semantic feature and using a cross-modal fusion module includes: A cross-modal fusion module is constructed based on the structure of the Transformer encoder, and the first cross-attention weight corresponding to the first semantic feature and the second cross-attention weight corresponding to the second semantic feature are determined by the cross-modal fusion module. Construct the corresponding multimodal feature map based on the first cross-attention weight and the second cross-attention weight.
4. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 1, characterized in that, The step of inputting the multimodal feature map into the target deep learning model to obtain each recognition result and the corresponding recognition confidence score includes: A target deep learning model is constructed based on a graph neural network or DETR architecture, and the target deep learning model is used to identify the multimodal feature map to obtain the target entity and the first confidence level corresponding to the target entity, the entity attribute and the second confidence level corresponding to the entity attribute, the entity relationship and the third confidence level corresponding to the entity relationship.
5. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 4, characterized in that, The step of determining graph rules using a preset prior knowledge base, and constructing a dynamic knowledge graph based on the recognition results and the recognition confidence level using the graph rules, includes: Graph rules, including node types and relationship types, are determined using a pre-defined prior knowledge base; Based on the graph rules, the target node type corresponding to the target entity is determined, and the target relationship type corresponding to the entity relationship is determined using the graph rules; Nodes are determined using the target entity and the target node type, and edges are determined based on the entity relationship and the target relationship type; A dynamic knowledge graph is constructed based on the nodes, the confidence scores corresponding to the nodes, the edges, and the confidence scores corresponding to the edges.
6. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 1, characterized in that, The step of mapping the dynamic knowledge graph to a target Bayesian network and using a preset inference algorithm to perform probabilistic inference on the target Bayesian network to obtain the inference results corresponding to each potential mission target of the loitering munition in the target region includes: Determine the initial Bayesian network, map the dynamic knowledge graph to the bottom-level nodes of the initial Bayesian network, and derive the intermediate-level nodes from the bottom-level nodes. The intermediate layer nodes are derived to obtain the top layer node, and the target Bayesian network is determined based on the bottom layer node, the intermediate layer node, and the top layer node. Probabilistic reasoning is performed on the target Bayesian network using an exact reasoning algorithm or an approximate reasoning algorithm to obtain the expected utility value and risk quantification index of each potential mission target of the loitering munition in the target area.
7. The task objective determination method based on situational awareness and probabilistic decision-making according to claim 6, characterized in that, The process of ranking multi-objective decisions based on the reasoning results, determining the final task objective using the ranking results, determining the target reasoning path corresponding to the final task objective, and generating a corresponding decision report based on the target reasoning path includes: The strike score corresponding to each potential mission target is determined using the strike expected utility value, the risk quantification index, and the preset risk aversion coefficient; The strike scores are sorted from highest to lowest, and the potential mission target with the highest strike score is determined as the final mission target based on the sorting results. The probabilistic reasoning process corresponding to the final task objective is backtracked to obtain the objective reasoning path, and a corresponding decision report is generated based on the objective reasoning path.
8. A task target determination device based on situational awareness and probabilistic decision-making, characterized in that, Airborne embedded computing platforms for loitering munitions include: The feature extraction module is used to acquire real-time data of the target area based on the optical camera and infrared thermal imager on the loitering munition, extract features from the real-time data using a convolutional neural network to construct a corresponding multimodal feature map, and input the multimodal feature map into the target deep learning model to obtain each recognition result and the corresponding recognition confidence. The knowledge graph construction module is used to determine graph rules using a preset prior knowledge base, and to construct a dynamic knowledge graph based on the recognition results and the recognition confidence and using the graph rules. The network reasoning module is used to map the dynamic knowledge graph into a target Bayesian network and use a preset reasoning algorithm to perform probabilistic reasoning on the target Bayesian network to obtain the reasoning results corresponding to each potential mission target of the loitering munition in the target region; the target Bayesian network includes bottom-level nodes representing observation evidence, intermediate-level nodes representing evaluation factors, and top-level nodes representing decision outputs; The decision report generation module is used to sort multi-objective decisions based on the reasoning results, determine the final task objective using the obtained sorting results, determine the target reasoning path corresponding to the final task objective, and generate a corresponding decision report based on the target reasoning path.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the task target determination method based on situational awareness and probabilistic decision-making as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Used to store a computer program, wherein the computer program, when executed by a processor, implements the task target determination method based on situational awareness and probabilistic decision-making as described in any one of claims 1 to 7.