Air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation

Abstract

The invention discloses an air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation, and relates to the technical field of intelligent detection, and the method comprises the steps: carrying out the networking initialization through a plurality of detection nodes of an airspace, the ground and a sea area, asynchronously collecting the multi-modal original sensor data with a space-time label, and obtaining an original sensor data flow; performing time synchronization and space registration processing on the original sensor data stream to obtain multi-source observation data after space-time alignment; and inputting the multi-source observation data after space-time alignment into a physical information neural network compensator for compensation correction, and outputting physically credible compensated data by the physical information neural network compensator according to a built-in physical model constraint. According to the method, through dynamic fusion of data compensation of physical model constraint and knowledge graph enhancement, the accuracy of multi-source heterogeneous data perception and the reliability of situation generation in a complex environment are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection technology, and in particular to an unmanned air, ground and sea detection method based on Bayesian fusion and deep learning compensation. Background Technology

[0002] Integrated air-ground-sea collaborative detection networks represent an important development direction for fields such as environmental monitoring and security defense. Their core lies in fusing multi-source sensor data from heterogeneous unmanned platforms to generate a unified situation map. Existing technologies typically employ a hierarchical processing architecture, where each platform first preprocesses and estimates the state of its local sensor data before transmitting the results to a fusion center for data association and fusion based on classical algorithms. This approach lays the foundation for the comprehensive utilization of multi-source information, but its effectiveness is highly dependent on the accuracy of local perception and the degree of matching between the pre-set statistical model and the actual environment.

[0003] However, when dealing with complex and dynamic cross-domain detection scenarios, existing technologies have inherent limitations in sensing data quality and fusing prior knowledge. Sensor observation errors caused by complex physical environments are difficult to be fully corrected by traditional compensation methods based on ideal assumptions, resulting in inherent biases in the fused input data. The prior knowledge in traditional fusion algorithms is mostly statically set and cannot dynamically respond to changes in the detection context, limiting its ability to perform intelligent error correction and reasoning when data quality is poor, thus restricting the accuracy and reliability of the global situation map. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides an unmanned air, ground, and sea detection method based on Bayesian fusion and deep learning compensation to solve the problem of insufficient reliability of global situational awareness caused by inherent data bias and static prior knowledge.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides an unmanned air, ground and sea detection method based on Bayesian fusion and deep learning compensation, which includes initializing a network by using multiple detection nodes in the airspace, ground and sea areas, and asynchronously collecting multimodal raw sensor data with spatiotemporal labels to obtain raw sensor data streams.

[0008] The raw sensor data stream is time-synchronized and spatially registered to obtain spatiotemporally aligned multi-source observation data.

[0009] The spatiotemporally aligned multi-source observation data is input into the physical information neural network compensator for compensation and correction. The physical information neural network compensator outputs physically reliable compensated data based on the built-in physical model constraints.

[0010] Local target perception and confidence estimation are performed based on physically reliable compensated data to obtain local perception results with confidence.

[0011] The fusion center receives local perception results with confidence and queries the online knowledge graph to obtain contextual information to generate dynamic prior probabilities. It then combines the Bayesian fusion algorithm to perform collaborative reasoning calculations and obtain a globally unified collaborative detection situation map.

[0012] The globally unified collaborative detection situation map is transmitted to the command and control center via data link. The command and control center then visualizes the received globally unified collaborative detection situation map, forming a complete air-ground-sea collaborative detection system.

[0013] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the method includes the following steps: initializing a network using multiple detection nodes in the airspace, ground, and sea, and asynchronously acquiring multimodal raw sensor data with spatiotemporal labels to obtain the raw sensor data stream:

[0014] By synchronizing clocks based on a unified timing signal, airspace detection nodes, ground detection nodes, and marine detection nodes, and exchanging their respective geographic coordinates and coordinate system information, a unified spatiotemporal reference is established.

[0015] Based on clock synchronization and coordinate unification, airspace detection nodes, ground detection nodes, and sea area detection nodes are interconnected according to preset communication protocols and network topology to form a collaborative detection network.

[0016] Based on the collaborative detection network, airspace detection nodes, ground detection nodes, and sea area detection nodes independently begin to collect multimodal sensor readings of optical images, infrared radiation, radar radio frequency signals, and acoustic echoes according to their respective sampling frequencies and working cycles.

[0017] Each time a sensor reading is generated, a unified timestamp from the collaborative detection network and the positioning and attitude information of the airspace detection node, ground detection node, and sea area detection node are automatically added to form a multimodal raw sensor reading with spatiotemporal tags.

[0018] Multimodal raw sensor readings with spatiotemporal tags are continuously generated from airspace detection nodes, ground detection nodes, and marine detection nodes, and are collected through a collaborative detection network to form raw sensor data.

[0019] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the method includes the following steps: performing time synchronization and spatial registration processing on the original sensor data stream to obtain spatiotemporally aligned multi-source observation data.

[0020] Based on the original sensor data stream, time interpolation is performed according to the timestamp information attached to the data packet to unify the sensor readings onto a common time axis and generate time-synchronized multi-source sensor readings.

[0021] The time-synchronized multi-source sensor readings are transformed using the positioning and attitude information of the detection node attached to each reading, and the multi-source sensor readings are converted to a unified geographic coordinate system to generate spatially registered multi-source sensor readings.

[0022] The spatially registered multi-source sensor readings are processed through data association to match and bundle different modal readings for the same geographic entity at the same time, forming spatiotemporally aligned multi-source observation data.

[0023] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the method includes: inputting spatiotemporally aligned multi-source observation data into a physical information neural network compensator for compensation and correction; the physical information neural network compensator outputs physically reliable compensated data based on built-in physical model constraints, comprising the following steps:

[0024] The spatiotemporally aligned multi-source observation data is input into the input layer of the physical information neural network compensator, which converts the spatiotemporally aligned multi-source observation data into a tensor format required for internal processing by the physical information neural network compensator.

[0025] The spatiotemporally aligned tensor of multi-source observation data after format conversion is transformed layer by layer in the forward propagation path of the physical information neural network compensator.

[0026] Based on the physical model constraints built into the physical information neural network compensator, and using the physical equations describing the atmospheric refraction of electromagnetic waves, the output layer of the physical information neural network compensator generates physically reliable compensated data.

[0027] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the method includes the following steps: Local target perception and confidence estimation are performed based on physically reliable compensated data to obtain local perception results with confidence levels.

[0028] The physically reliable compensated data is input into the target detection algorithm, which processes the physically reliable compensated data, identifies and locates potential targets, and generates a preliminary perception list of target locations and categories.

[0029] The preliminary perception list of target location and category is output to the confidence estimation algorithm, which calculates a confidence score for each target item in the preliminary perception list of target location and category based on the contrast between target features and background, the integrity of target shape, and the historical performance of the sensor.

[0030] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the fusion center receives local perception results with confidence levels and queries an online knowledge graph to obtain contextual information to generate dynamic prior probabilities. It then performs collaborative reasoning calculations using a Bayesian fusion algorithm to obtain a globally unified collaborative detection situation map, including the following steps:

[0031] The fusion center receives local sensing results with confidence levels from airspace, ground, and sea area detection nodes, and temporarily stores the local sensing results with confidence levels in the data buffer.

[0032] The fusion center queries the online knowledge graph, which retrieves and provides relevant contextual information based on the target location and time information contained in the local perception results with confidence.

[0033] The fusion center dynamically adjusts the probability of different target categories appearing in the local perception results with confidence levels based on contextual information provided by an online knowledge graph, generating dynamic prior probabilities that reflect the current situation.

[0034] The fusion center inputs the local perception results with confidence and the dynamic prior probability into the Bayesian fusion algorithm. The Bayesian fusion algorithm performs collaborative reasoning calculation and outputs the best estimate of the confirmed target and the fused state. After formatting, it generates a globally unified collaborative detection situation map.

[0035] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the method involves transmitting a globally unified collaborative detection situation map to the command and control center via a data link, including the following steps:

[0036] The globally unified collaborative detection situation map is submitted to the data link protocol encapsulation process, which converts the globally unified collaborative detection situation map into data packets in the data link transmission format.

[0037] Data packets in data link transmission format transmitted via radio channels are received by the receiving terminal of the command and control center. The receiving terminal of the command and control center decapsulates the received data packets in data link transmission format to restore a globally unified collaborative detection situation map.

[0038] As a preferred embodiment of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation described in this invention, the command and control center visualizes the received globally unified collaborative detection situation map to form a complete air-ground-sea collaborative detection system, including the following steps:

[0039] The command and control center analyzes the received globally unified collaborative detection situation map and extracts the target trajectory, attributes, and confidence information.

[0040] The target trajectory, attributes, and confidence information are mapped to the corresponding geographic coordinates on the electronic map. The target trajectory, attributes, and confidence information mapped to the electronic map are then used by the graphics rendering engine to generate visual symbols.

[0041] The visual symbols generated by the graphics rendering engine are overlaid on the electronic map base map and composited on the display screen. The final image composited on the display screen obtains a complete air-ground-sea collaborative detection situation.

[0042] In a second aspect, the present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, it implements any step of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation as described in the first aspect of the present invention.

[0043] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation as described in the first aspect of the present invention.

[0044] The beneficial effects of this invention are as follows: A raw sensor data stream is formed by initializing a network of multiple air, ground, and sea detection nodes and asynchronously acquiring multimodal data with spatiotemporal labels; spatiotemporally aligned multi-source observation data is obtained after time synchronization and spatial registration processing; the observation data is compensated and corrected using the physical model constraints built into the physical information neural network compensator, outputting physically reliable compensated data; local target perception and confidence estimation are performed based on the data to generate local perception results with confidence; the fusion center generates dynamic prior probabilities by combining contextual information provided by an online knowledge graph, performs collaborative reasoning through a Bayesian fusion algorithm, and obtains a globally unified collaborative detection situation map, which is transmitted to the command and control center via a data link for visualization, realizing air-ground-sea collaborative detection. The entire process, through data compensation constrained by physical models and dynamic fusion enhanced by knowledge graphs, effectively improves the accuracy of multi-source heterogeneous data perception and the reliability of situation generation in complex environments. Attached Figure Description

[0045] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the accompanying drawings without creative effort.

[0046] Fig. 1 This is a flowchart of an unmanned air, ground, and sea detection method based on Bayesian fusion and deep learning compensation.

[0047] Fig. 2 This is a diagram illustrating data aggregation. Detailed Implementation

[0048] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0049] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0051] Reference Figs. 1-2This is one embodiment of the present invention, which provides an unmanned air, ground, and sea detection method based on Bayesian fusion and deep learning compensation, including the following steps: S1. Initialize the network by using multiple detection nodes in the airspace, ground and sea areas, and asynchronously collect multimodal raw sensor data with spatiotemporal labels to obtain the raw sensor data stream.

[0052] S1. Initialize the network by using multiple detection nodes in the airspace, ground and sea areas, and asynchronously collect multimodal raw sensor data with spatiotemporal labels to obtain the raw sensor data stream.

[0053] S1.1. The airspace detection nodes, ground detection nodes, and marine detection nodes synchronize their clocks based on a unified timing signal and exchange their respective geographic coordinates and coordinate system information to establish a unified spatiotemporal reference.

[0054] Furthermore, the airspace detection nodes, ground detection nodes, and marine detection nodes receive unified timing signals from the Global Navigation Satellite System or Network Time Protocol (NAT) server. Each detection node calibrates its internal clock counter based on this signal to ensure that the time reference of all detection nodes is consistent. Subsequently, the airspace detection nodes broadcast their geographic coordinates and elevation information based on the WGS-84 coordinate system via data link, the ground detection nodes report their position data based on the UTM coordinate system, and the marine detection nodes provide their positioning results based on the ECEF coordinate system. All coordinate information is uniformly converted to the geocentric map coordinate system through a standard coordinate transformation algorithm, thereby establishing a unified spatiotemporal reference covering the airspace, ground, and sea.

[0055] S1.2 Based on clock synchronization and coordinate unification, the airspace detection nodes, ground detection nodes and sea area detection nodes are interconnected according to the preset communication protocol and network topology to form a collaborative detection network.

[0056] Furthermore, based on the airspace detection nodes, ground detection nodes, and sea area detection nodes that have completed clock synchronization and coordinate unification, neighbor discovery and link establishment are carried out sequentially according to pre-configured communication protocols such as TCP / IP sockets or custom UDP packet formats, and according to star or mesh network topology. The airspace detection nodes, as temporary aggregation points, send connection requests to the ground detection nodes and sea area detection nodes. Direct communication links are also established between the ground detection nodes and neighboring sea area detection nodes. Finally, all detection nodes complete bidirectional communication verification, forming a collaborative detection network capable of data exchange and routing.

[0057] S1.3 Based on the collaborative detection network, the airspace detection node, ground detection node and sea area detection node independently start to collect multi-modal sensor readings of optical images, infrared radiation, radar radio frequency signals and acoustic echoes according to their respective sampling frequencies and working cycles.

[0058] Furthermore, after the collaborative detection network is established, the airspace detection nodes periodically acquire high-resolution optical images and infrared radiation data at a sampling frequency of 30 Hz, the ground detection nodes acquire millimeter-wave radar radio frequency signals at a sampling frequency of 20 Hz, and the sea area detection nodes acquire acoustic echo data from multibeam sonar at a sampling frequency of 10 Hz. Each detection node independently starts its sensors according to its own working cycle and reads the raw voltage values ​​output by the analog-to-digital converter, generating a continuous multimodal sensor reading sequence.

[0059] S1.4. When each sensor reading is generated, the unified timestamp of the collaborative detection network and the positioning and attitude information of the airspace detection node, ground detection node and sea area detection node are automatically added to form a multimodal raw sensor reading with spatiotemporal tags.

[0060] Furthermore, whenever a sensor reading is generated, such as when an image frame exposure is completed at an airspace detection node or a sonar pulse echo is recorded at a sea area detection node, a unified millisecond-level precision timestamp from the collaborative detection network is automatically marked onto the reading via a high-precision clock chip. The GPS positioning and IMU attitude angle of the airspace detection node, the inertial navigation system output of the ground detection node, or the differential GPS position and compass heading of the sea area detection node at the time of the reading are recorded. The positioning and attitude information is packaged as metadata with the sensor reading itself to form a multimodal raw sensor reading data packet with spatiotemporal tags.

[0061] S1.5. Multimodal raw sensor readings with spatiotemporal labels are continuously generated from airspace detection nodes, ground detection nodes, and sea area detection nodes, and are collected through a collaborative detection network to form raw sensor data.

[0062] Furthermore, multimodal raw sensor readings with spatiotemporal tags are continuously generated from airspace detection nodes, ground detection nodes, and marine detection nodes. Data packets are transmitted to data processing nodes in the network through the routing function of the cooperative detection network, following a preset convergence path. Data from airspace detection nodes is forwarded through air relay links, while data from ground and marine detection nodes are converged through ground wireless gateways or offshore buoy base stations. All data streams are ultimately aggregated to a designated data receiving endpoint and merged into a time-series continuous raw sensor data stream according to timestamp order.

[0063] S2. Perform time synchronization and spatial registration processing on the raw sensor data stream to obtain spatiotemporally aligned multi-source observation data.

[0064] S2.1 Based on the original sensor data stream, time interpolation processing is performed according to the timestamp information attached to the data packet to unify the sensor readings onto a common time axis and generate time-synchronized multi-source sensor readings.

[0065] Furthermore, the raw sensor data stream is fed into a time-aligned buffer. Based on the precise timestamp information attached to each data packet, linear interpolation or spline interpolation algorithms are used to process the asynchronous sensor readings from different detection nodes. For any set fusion time point, the nearest reading samples before and after that time point are selected, and their interpolation weights are calculated. This allows the high-frequency image data from the airspace detection nodes, the mid-frequency radar signals from the ground detection nodes, and the low-frequency sonar readings from the sea detection nodes to be uniformly interpolated onto a common time axis with equal intervals, generating a time-synchronized multi-source sensor reading sequence.

[0066] S2.2 After time synchronization, the multi-source sensor readings are transformed using the positioning and attitude information of the detection node attached to each reading. The multi-source sensor readings are then converted to a unified geographic coordinate system to generate spatially registered multi-source sensor readings.

[0067] Furthermore, the time-synchronized multi-source sensor readings are transmitted to the coordinate transformation engine. This engine reads the positioning and attitude information of the detection nodes accompanying each reading, including the latitude, longitude, altitude, and attitude angle of the airspace detection nodes, the planar coordinates and orientation of the ground detection nodes, and the three-dimensional coordinates and heading of the sea area detection nodes. By applying the direction cosine matrix transformation, the spatial position of all readings is transformed from the local sensor coordinate system of each detection node to a pre-set unified geographic coordinate system, such as the geocentric map coordinate system, to ensure that all observation data have a consistent spatial reference benchmark and generate spatially registered multi-source sensor readings.

[0068] S2.3 After spatial registration, the multi-source sensor readings are processed through data association to match and bundle different modal readings for the same geographic entity at the same time, forming spatiotemporally aligned multi-source observation data.

[0069] Furthermore, the spatially registered multi-source sensor readings enter the data association process, employing the nearest neighbor association algorithm or joint probabilistic data association method, with spatial location under a unified geographic coordinate system and timestamps on a common time axis as the main association criteria. The spatial distance between different modal readings at the same fusion time is determined to originate from the same geographic entity when the distance is less than a set threshold and the timestamps are consistent. Subsequently, the optical characteristics of the airspace detection nodes, the radar cross-section characteristics of the ground detection nodes, and the acoustic characteristics of the marine detection nodes are bundled at the feature level to form a comprehensive observation unit containing multimodal attributes. The final output is spatiotemporally aligned multi-source observation data.

[0070] S3. The spatiotemporally aligned multi-source observation data is input into the physical information neural network compensator for compensation and correction. The physical information neural network compensator outputs physically reliable compensated data based on the built-in physical model constraints.

[0071] S3.1 Input the spatiotemporally aligned multi-source observation data into the input layer of the physical information neural network compensator. The input layer converts the spatiotemporally aligned multi-source observation data into a tensor format required for internal processing by the physical information neural network compensator.

[0072] Furthermore, the spatiotemporally aligned multi-source observation data is fed into the input layer of the physical information neural network compensator. The input layer reshapes and standardizes the spatiotemporally aligned multi-source observation data according to the preset dimensional specifications, converting the matrix structure containing multi-channel time-series information into a tensor format with three axes: batch size, sequence length, and feature dimension, to meet the computational requirements of subsequent neural network layers and complete the data format conversion.

[0073] S3.2 The spatiotemporally aligned multi-source observation data tensor after format conversion is transformed layer by layer in the forward propagation path of the physical information neural network compensator.

[0074] Furthermore, the spatiotemporally aligned multi-source observation data tensor after format conversion enters the forward propagation path of the physical information neural network compensator. This path consists of alternating fully connected layers, convolutional layers, and activation function layers. The tensor first undergoes linear transformation and dimensionality enhancement through fully connected layers, then local spatiotemporal features are extracted through convolutional layers, and finally the expressive power of the model is introduced through nonlinear activation functions. This layer-by-layer linear and nonlinear transformation process progresses sequentially, mapping the initial tensor into a high-dimensional feature representation.

[0075] S3.3 Based on the physical model constraints built into the physical information neural network compensator, the output layer of the physical information neural network compensator generates physically reliable compensated data using physical equations that describe the atmospheric refraction of electromagnetic waves.

[0076] The total loss function is expressed as follows:

[0077]

[0078] in, The total loss function for training the physical information neural network compensator, This represents the number of samples used in a single training iteration. For the first The true value of each sample For the first Physically reliable compensated data for each sample. For hyperparameters, For the first The refractive index predicted by the network for each sample. It is a constant. For the first Atmospheric pressure environmental parameters for each sample For the first Absolute temperature environmental parameters for each sample. This is the wet term coefficient. For the first Water vapor pressure environmental parameters for each sample This is the sample index.

[0079] Furthermore, based on the physical model constraints built into the physical information neural network compensator, the total loss function used in the training process is composed of two weighted parts: data reconstruction loss and physical consistency loss. The data reconstruction loss calculates the mean square error between the physically reliable compensated data output by the network and the true observation value. The physical consistency loss requires that the refractive index parameter predicted by the intermediate layer of the network must conform to the theoretical value calculated by the atmospheric pressure environmental parameter, absolute temperature environmental parameter, and water vapor pressure environmental parameter through the classical electromagnetic wave refraction formula. By minimizing these two losses simultaneously through the backpropagation algorithm, the physical information neural network compensator is forced to strictly abide by physical laws while learning complex mapping relationships, thereby generating compensated data at the output layer that is both fitted and physically reliable.

[0080] S4. Perform local target perception and confidence estimation based on physically reliable compensated data to obtain local perception results with confidence.

[0081] S4.1 The physically reliable compensated data is input into the target detection algorithm. The target detection algorithm processes the physically reliable compensated data, identifies and locates potential targets, and generates a preliminary perception list of target locations and categories.

[0082] The object detection expression is:

[0083] ;

[0084] in, For target detection mapping function, To perceive the total number of targets, For the index of the perceived target. For the first Type identifier of an entity, For the first The state vector of each entity, For the first Credibility estimation of individual entity detection.

[0085] Furthermore, the physically reliable compensated data is input into the object detection algorithm. The object detection algorithm first extracts multi-scale feature maps of the physically reliable compensated data through a convolutional neural network. Then, it applies a region proposal network to the feature maps to generate candidate target regions. Next, it performs category classification and bounding box regression on each candidate region. Finally, it outputs a set of triples containing the target category identifier, the location state vector, and the initial detection confidence estimate, forming a preliminary perception list of the target location and category.

[0086] S4.2 Output the preliminary perception list of target location and category to the confidence estimation algorithm. The confidence estimation algorithm calculates a confidence score for each target item in the preliminary perception list of target location and category based on the contrast between target features and background, the integrity of target shape, and the historical performance of the sensor.

[0087] The confidence score expression is:

[0088]

[0089] in, For the first The confidence score of each target item. for, This is a record of historical performance.

[0090] The initial perception list of target location and category is bound to a confidence score, and a confidence score is attached to the target item to form a local perception result with confidence.

[0091] Furthermore, the preliminary perception list of target location and category is transmitted to the confidence estimation algorithm. The confidence estimation algorithm first calculates the gray-level variance ratio of the feature of each target item to the background region as a measure of the contrast between the target feature and the background. Then, it analyzes the continuity of the edge gradient within the target bounding box to evaluate the integrity of the target shape. At the same time, it queries historical performance records to obtain the detection accuracy statistics of similar sensors in similar environments. The algorithm calculates the comprehensive confidence score through a weighted fusion formula and binds the confidence score to each target item in the preliminary perception list of target location and category to form a local perception result with confidence.

[0092] S5. The fusion center receives local perception results with confidence levels and queries the online knowledge graph to obtain contextual information to generate dynamic prior probabilities. It then performs collaborative reasoning calculations using a Bayesian fusion algorithm to obtain a globally unified collaborative detection situation map.

[0093] S5.1 The fusion center receives local sensing results with confidence levels from airspace, ground and sea area detection nodes, and temporarily stores the local sensing results with confidence levels in the data buffer.

[0094] Furthermore, the fusion center continuously receives local sensing result data packets with confidence scores from airspace detection nodes, ground detection nodes, and sea area detection nodes through the data receiving port. Each data packet contains a target type identifier, a state vector, and a confidence score. The data packets are stored in a first-in-first-out data buffer in the order of reception time, waiting for subsequent fusion processing.

[0095] S5.2 The fusion center queries the online knowledge graph. Based on the target location and time information contained in the local perception results with confidence, the online knowledge graph retrieves and provides relevant contextual information.

[0096] Furthermore, the fusion center sends a query request to the online knowledge graph, which carries the target location coordinates and timestamp information contained in the local perception results with confidence. The online knowledge graph matches geographical region attributes, such as sea area type or restricted navigation area, in the spatial database based on the location coordinates, and retrieves the same activity patterns, such as fishing vessel operation patterns, from the historical records based on the timestamps. The retrieved geographical attributes and activity patterns are then returned to the fusion center as contextual information.

[0097] S5.3 The fusion center dynamically adjusts the probability of different target categories appearing in the local perception results with confidence based on the context information provided by the online knowledge graph, and generates dynamic prior probabilities that reflect the current situation.

[0098] Furthermore, the fusion center analyzes the contextual information returned by the online knowledge graph. When the contextual information indicates that the current area is a traditional fishing ground, it automatically increases the prior probability weight of the target category as fishing vessel in the local perception results with confidence. When the contextual information indicates that the current time period is a high-frequency passage period for merchant ships, it correspondingly increases the prior probability weight of the merchant ship category. Through this context-based dynamic adjustment, a dynamic prior probability distribution reflecting the real-time situation is generated.

[0099] S5.4 The fusion center inputs the local perception results with confidence and the dynamic prior probability into the Bayesian fusion algorithm. The Bayesian fusion algorithm performs collaborative reasoning calculation and outputs the best estimate of the confirmed target and the fused state. After formatting, it generates a globally unified collaborative detection situation map.

[0100] The optimal estimate of the merged state is expressed as:

[0101] ;

[0102] in, For the first The node is the first The fusion weights for estimating the target state. For the first The optimal state estimate after fusing the target states For probing node indexes, For the first The node and the first Local state estimation associated with each target, For the first The node is the first The fusion weights for estimating the target state. To detect the total number of nodes, The target state;

[0103] Furthermore, the fusion center inputs the local perception results with confidence scores and the dynamic prior probabilities from the data buffer into the Bayesian fusion algorithm. It then associates the data based on the target's spatial location to determine whether observations from different nodes belong to the same target. For each successfully associated target, the fusion weight of each node's observation is calculated, determined by the confidence score in the local perception results with confidence scores and the dynamic prior probabilities. A weighted fusion formula is then used to fuse the multiple associated observation states to obtain the optimal state estimate for each target. Finally, the optimal state estimates of the confirmed targets are encapsulated in a standard data format to generate a globally unified collaborative detection situation map containing target trajectories, attributes, and confidence scores.

[0104] S6. Transmit the globally unified collaborative detection situation map to the command and control center via data link.

[0105] S6.1 Submit the globally unified collaborative detection situation map to the data link protocol encapsulation process. The data link protocol encapsulation process converts the globally unified collaborative detection situation map into data packets in the data link transmission format.

[0106] Furthermore, the data structure of the globally unified collaborative detection situation map is serialized according to the Link-16 message standard or similar protocol specifications, and necessary frame headers, target identifiers, checksums, and encryption fields are added. Then, the serialized data is divided into data blocks that conform to the maximum transmission unit size, and finally assembled into a data packet with complete protocol header information in the data link transmission format.

[0107] S6.2. Data packets in data link transmission format transmitted via radio channel are received by the receiving terminal of the command and control center. The receiving terminal of the command and control center decapsulates the received data packets in data link transmission format and restores the globally unified cooperative detection situation map.

[0108] Furthermore, data packets in the data link transmission format transmitted via radio channels such as the UHF band or satellite communication links are received by the joint tactical information distribution system receiving terminal of the command and control center. The receiving terminal first performs cyclic redundancy check on the data packets in the data link transmission format to verify data integrity, then strips the protocol header and trailer information, decrypts and reassembles the payload data, and finally restores the globally unified cooperative detection situation map data structure that is completely consistent with the sending end through deserialization operation.

[0109] S7. The command and control center visualizes the received global unified collaborative detection situation map, forming a complete air-ground-sea collaborative detection system.

[0110] S7.1 The command and control center analyzes the received globally unified collaborative detection situation map and extracts the target trajectory, attributes, and confidence information.

[0111] Furthermore, the command and control center analyzes the received globally unified collaborative detection situation map. Through the analysis algorithm, it reads the target identifier field, time series location coordinates, target type code, and confidence value stored in the situation map data structure. From the complex data structure, it extracts the latitude and longitude sequence of each target trajectory, target attribute classification, and corresponding confidence assessment information one by one.

[0112] S7.2 Map the target trajectory, attributes, and confidence information to the corresponding geographic coordinates on the electronic map. The target trajectory, attributes, and confidence information mapped to the electronic map are used to generate visual symbols through a graphics rendering engine.

[0113] Furthermore, the target trajectory, attributes, and confidence information are mapped to the corresponding geographic coordinates on the electronic map. Specifically, the extracted target trajectory latitude and longitude coordinates are converted into screen pixel coordinates through map projection transformation. At the same time, a predefined icon library is matched according to the target attribute classification code, and different colors and transparency are mapped according to the confidence value. The target trajectory, attributes, and confidence information mapped onto the electronic map are used by the vector drawing interface of the graphics rendering engine to generate visual symbols composed of icon symbols, trajectory lines, and transparency colors.

[0114] S7.3 The visual symbols generated by the graphics rendering engine are superimposed on the electronic map base map and composited on the display screen. The final image composited on the display screen obtains a complete air-ground-sea collaborative detection situation.

[0115] Furthermore, the visual symbols generated by the graphics rendering engine are overlaid and synthesized with the electronic map base map. The visual symbol layer is fused with the base map layer at the pixel level through the alpha mixing algorithm to ensure that the symbols are clearly visible while preserving the underlying geographic information. The synthesized image frame is output to a high-resolution display screen through the display interface for refresh and rendering. The image rendered on the display screen presents a complete air-ground-sea collaborative detection situation that includes the dynamic trajectories, attribute identifiers, and confidence indicators of all targets.

[0116] This embodiment also provides a computer device applicable to the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation as proposed in the above embodiment.

[0117] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0118] This embodiment also provides a storage medium storing a computer program. When executed by a processor, the program implements the air-ground-sea unmanned reconnaissance method based on Bayesian fusion and deep learning compensation as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0119] In summary, this invention initializes a network of multiple air, ground, and sea detection nodes and asynchronously acquires multimodal data with spatiotemporal labels to form a raw sensor data stream. After time synchronization and spatial registration, it obtains spatiotemporally aligned multi-source observation data. The physical model constraints built into the physical information neural network compensator are used to compensate and correct the observation data, outputting physically reliable compensated data. Based on the data, local target perception and confidence estimation are performed to generate local perception results with confidence. The fusion center combines contextual information provided by an online knowledge graph to generate dynamic prior probabilities, and performs collaborative reasoning through a Bayesian fusion algorithm to obtain a globally unified collaborative detection situation map. This map is transmitted via data link to the command and control center for visualization, realizing collaborative air, ground, and sea detection. The entire process, through data compensation constrained by physical models and dynamic fusion enhanced by knowledge graphs, effectively improves the accuracy of multi-source heterogeneous data perception and the reliability of situation generation in complex environments.

[0120] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. An unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation, characterized by: This includes initializing the network through multiple detection nodes in the airspace, ground, and sea, and asynchronously acquiring multimodal raw sensor data with spatiotemporal labels to obtain the raw sensor data stream; The raw sensor data stream is time-synchronized and spatially registered to obtain spatiotemporally aligned multi-source observation data. The spatiotemporally aligned multi-source observation data is input into the physical information neural network compensator for compensation and correction. The physical information neural network compensator outputs physically reliable compensated data based on the built-in physical model constraints. Local target perception and confidence estimation are performed based on physically reliable compensated data to obtain local perception results with confidence. The fusion center receives local perception results with confidence and queries the online knowledge graph to obtain contextual information to generate dynamic prior probabilities. It then combines the Bayesian fusion algorithm to perform collaborative reasoning calculations and obtain a globally unified collaborative detection situation map. The globally unified collaborative detection situation map is transmitted to the command and control center via data link. The command and control center then visualizes the received globally unified collaborative detection situation map, forming a complete air-ground-sea collaborative detection system.

2. The unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 1, characterized in that: The network is initialized by using multiple detection nodes in the airspace, ground, and sea, and multimodal raw sensor data with spatiotemporal labels is acquired asynchronously to obtain the raw sensor data stream, including the following steps: By synchronizing clocks based on a unified timing signal, airspace detection nodes, ground detection nodes, and marine detection nodes, and exchanging their respective geographic coordinates and coordinate system information, a unified spatiotemporal reference is established. Based on clock synchronization and coordinate unification, airspace detection nodes, ground detection nodes, and sea area detection nodes are interconnected according to preset communication protocols and network topology to form a collaborative detection network. Based on the collaborative detection network, airspace detection nodes, ground detection nodes, and sea area detection nodes independently begin to collect multimodal sensor readings of optical images, infrared radiation, radar radio frequency signals, and acoustic echoes according to their respective sampling frequencies and working cycles. Each time a sensor reading is generated, a unified timestamp from the collaborative detection network and the positioning and attitude information of the airspace detection node, ground detection node, and sea area detection node are automatically added to form a multimodal raw sensor reading with spatiotemporal tags. Multimodal raw sensor readings with spatiotemporal tags are continuously generated from airspace detection nodes, ground detection nodes, and marine detection nodes, and are collected through a collaborative detection network to form raw sensor data.

3. The unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 2, characterized in that: The raw sensor data stream is time-synchronized and spatially registered to obtain spatiotemporally aligned multi-source observation data, including the following steps: Based on the original sensor data stream, time interpolation is performed according to the timestamp information attached to the data packet to unify the sensor readings onto a common time axis and generate time-synchronized multi-source sensor readings. The time-synchronized multi-source sensor readings are transformed using the positioning and attitude information of the detection node attached to each reading, and the multi-source sensor readings are converted to a unified geographic coordinate system to generate spatially registered multi-source sensor readings. The spatially registered multi-source sensor readings are processed through data association to match and bundle different modal readings for the same geographic entity at the same time, forming spatiotemporally aligned multi-source observation data.

4. The unmanned air, ground, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 3, characterized in that: The spatiotemporally aligned multi-source observation data is input into a physical information neural network compensator for compensation and correction. The physical information neural network compensator outputs physically reliable compensated data based on the constraints of a built-in physical model, including the following steps: The spatiotemporally aligned multi-source observation data is input into the input layer of the physical information neural network compensator, which converts the spatiotemporally aligned multi-source observation data into a tensor format required for internal processing by the physical information neural network compensator. The spatiotemporally aligned tensor of multi-source observation data after format conversion is transformed layer by layer in the forward propagation path of the physical information neural network compensator. Based on the physical model constraints built into the physical information neural network compensator, and using the physical equations describing the atmospheric refraction of electromagnetic waves, the output layer of the physical information neural network compensator generates physically reliable compensated data.

5. The unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 4, characterized in that: Local target perception and confidence estimation are performed based on physically reliable compensated data to obtain local perception results with confidence levels, including the following steps: The physically reliable compensated data is input into the target detection algorithm, which processes the physically reliable compensated data, identifies and locates potential targets, and generates a preliminary perception list of target locations and categories. The preliminary perception list of target location and category is output to the confidence estimation algorithm, which calculates a confidence score for each target item in the preliminary perception list of target location and category based on the contrast between target features and background, the integrity of target shape, and the historical performance of the sensor.

6. The unmanned air, ground, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 5, characterized in that: The fusion center receives local perception results with confidence levels and queries the online knowledge graph to obtain contextual information to generate dynamic prior probabilities. It then combines these with a Bayesian fusion algorithm for collaborative reasoning calculation to obtain a globally unified collaborative detection situation map. This process includes the following steps: The fusion center receives local sensing results with confidence levels from airspace, ground, and sea area detection nodes, and temporarily stores the local sensing results with confidence levels in the data buffer. The fusion center queries the online knowledge graph, which retrieves and provides relevant contextual information based on the target location and time information contained in the local perception results with confidence. The fusion center dynamically adjusts the probability of different target categories appearing in the local perception results with confidence based on the context information provided by the online knowledge graph, and generates dynamic prior probabilities that reflect the current situation. The fusion center inputs the local perception results with confidence and the dynamic prior probability into the Bayesian fusion algorithm. The Bayesian fusion algorithm performs collaborative reasoning calculations and outputs the best estimate of the confirmed target and the fused state. After formatting, it generates a globally unified collaborative detection situation map.

7. The unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 6, characterized in that: Transmitting a globally unified collaborative detection situation map to the command and control center via data link includes the following steps: The globally unified collaborative detection situation map is submitted to the encapsulation process of the data link protocol, which then converts the globally unified collaborative detection situation map into data packets in the data link transmission format. Data packets in data link transmission format transmitted via radio channels are received by the receiving terminal of the command and control center. The receiving terminal of the command and control center decapsulates the received data packets in data link transmission format to restore a globally unified collaborative detection situation map.

8. The unmanned air, land, and sea detection method based on Bayesian fusion and deep learning compensation as described in claim 7, characterized in that, The command and control center visualizes the received globally unified collaborative detection situation map, forming a complete air-ground-sea collaborative detection system, including the following steps: The command and control center analyzes the received globally unified collaborative detection situation map and extracts the target trajectory, attributes, and confidence information. The target trajectory, attributes, and confidence information are mapped to the corresponding geographic coordinates on the electronic map. The target trajectory, attributes, and confidence information mapped to the electronic map are then used by the graphics rendering engine to generate visual symbols. The visual symbols generated by the graphics rendering engine are overlaid on the electronic map base map and composited on the display screen. The final image composited on the display screen obtains a complete air-ground-sea collaborative detection situation.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the air-ground-sea unmanned detection method based on Bayesian fusion and deep learning compensation as described in any one of claims 1 to 8.

Images