A space-airborne carried fire source early warning method, system and device

Abstract

The application relates to the technical field of fire monitoring and early warning, and discloses an aerospace airborne fire source early warning method, system and device, which comprises the following steps: constructing and training a neural network detection model based on fire source smoke image data containing a visible light and infrared dual mode; deploying the trained neural network detection model to an embedded computing unit of an aerospace airborne platform; synchronously acquiring real-time image data collected by an image sensor on the airborne platform and environment perception data collected by a non-image sensor; inputting the real-time image data into the neural network detection model to obtain image-based fire source detection confidence; dynamically adjusting the weight of the image-based fire source detection confidence and the environment perception data in decision-making and performing weighted fusion to generate comprehensive early warning confidence; and outputting graded early warning information based on the comprehensive early warning confidence. The application uses a GAM attention mechanism to enhance the model and overcomes the limitations of a single image source in complex scenes such as low light.

Description

Technical Field

[0001] This invention relates to the field of fire monitoring and early warning technology, specifically to an airborne fire source early warning method, system, and equipment. Background Technology

[0002] Fire is a sudden and rapidly spreading disaster that poses a serious threat to life and property. Traditional fire early warning technologies typically rely on deploying various sensors (such as temperature, humidity, smoke, and gas sensors) at fixed locations to trigger alarms by monitoring whether environmental parameters exceed preset thresholds. However, this approach has significant drawbacks:

[0003] 1. Insufficient reliability: Single sensors are susceptible to interference from environmental dust, equipment aging, and space height, which can lead to unstable data acquisition and false alarms or missed alarms.

[0004] 2. Limited coverage: Fixed-point deployment makes it difficult to achieve comprehensive and dynamic monitoring of large areas with complex terrain.

[0005] 3. Low level of intelligence: It lacks the ability to perform in-depth analysis and intelligent fusion of multi-source heterogeneous data, and cannot effectively cope with complex scenarios such as low light, low visibility, and partial occlusion of targets.

[0006] 4. Response delay: The data processing chain is long and the decision-making relies on simple thresholds, making it difficult to meet the needs of rapid early warning of fires.

[0007] With the development of infrared imaging, computer vision, and deep learning technologies, image-based fire detection methods have shown great potential. However, existing deep learning-based fire detection solutions are mostly designed for fixed ground cameras or close-range video, lacking optimized designs for the specific application scenarios of airborne platforms (mobile platforms, long distances, and resource constraints). In particular, how to integrate multimodal image information such as visible light and infrared, and effectively combine it with data from other sensors to achieve real-time, accurate, and adaptive fire source early warning on an airborne embedded platform remains a pressing technical challenge. Summary of the Invention

[0008] This invention aims to overcome the aforementioned shortcomings of existing technologies and provide a space-based airborne fire source early warning method, system, and device. Its purpose is to solve the technical problem of achieving high-precision, high-reliability, and real-time fire source early warning on space-based airborne platforms under long-distance, low-light, and resource-constrained conditions by deeply fusing visible light and infrared dual-modal visual information and combining an innovative neural network model with an adaptive multi-source data fusion strategy.

[0009] This invention is achieved through the following technical solution:

[0010] A space-based airborne fire source early warning method includes:

[0011] Based on fire smoke image data containing both visible light and infrared dual modes, a neural network detection model is constructed and trained. The neural network detection model integrates an attention mechanism module in the feature fusion network to enhance cross-dimensional feature interaction.

[0012] The trained neural network detection model is deployed in the embedded computing unit of the aerospace airborne platform;

[0013] During the early warning process, real-time image data collected by the image sensor on the airborne platform and environmental perception data collected by non-image sensors are acquired simultaneously. The real-time image data is input into the neural network detection model to obtain the image-based fire source detection confidence score. Based on the real-time detection distance between the airborne platform and the suspected fire source, the weights of the image-based fire source detection confidence score and the environmental perception data in the decision-making process are dynamically adjusted and weighted fused to generate a comprehensive early warning confidence score.

[0014] Based on the comprehensive early warning confidence level, graded early warning information is output.

[0015] As an optimization, the attention mechanism module for enhancing cross-dimensional feature interaction is the GAM attention module; the GAM attention module processes the input feature map through sequentially connected channel attention sub-modules and spatial attention sub-modules, thereby enhancing the interaction and expression of cross-dimensional features without reducing the channel dimension.

[0016] As an optimization, when constructing and training the neural network detection model, the CIoU loss function is used to perform regression optimization on the predicted bounding boxes of the neural network detection model. The CIoU loss function comprehensively considers the overlap area between the predicted bounding boxes and the ground truth bounding boxes, the distance between their center points, and the similarity of their aspect ratios.

[0017] As an optimization, adaptive data fusion is performed by dynamically adjusting the weights based on the real-time detection distance. Specifically, based on the comparison result between the real-time detection distance and the preset distance threshold, a piecewise function is used to assign different fusion weight coefficients to different data sources.

[0018] As an optimization, the expression for the piecewise function is: ;

[0019] Where P is the comprehensive early warning confidence level. For confidence scores based on long-distance visible light image data, For confidence levels based on infrared image data, For confidence based on image fusion data, bais represents the confidence level based on environmental sensing data of smoke and temperature and humidity, and bais is the correction coefficient.

[0020] When the detection distance D > 5km, k1 = 0.45, k2 = 0.05, k3 = 0.45, k4 = 0.05;

[0021] When the detection distance D≤5km, k1=0.45, k2=0.15, k3=0.35, k4=0.05.

[0022] As an optimization, based on the comprehensive early warning confidence level, hierarchical early warning information is output, specifically including:

[0023] Set a first threshold and a second threshold that is higher than the first threshold;

[0024] If the overall warning confidence level is lower than the first threshold, then no warning is triggered;

[0025] If the overall warning confidence level reaches or exceeds the first threshold but is lower than the second threshold, then the first-level warning information is output.

[0026] If the overall early warning confidence level reaches or exceeds the second threshold, then the second-level early warning information is output.

[0027] As an optimization, before deploying the trained neural network detection model to the embedded computing unit of the aerospace airborne platform, the method further includes: performing parameter quantization and network pruning on the trained neural network detection model to reduce the computational complexity and storage requirements of the neural network detection model, so that the neural network detection model can adapt to the resource constraints of the embedded computing unit.

[0028] As an optimization, after outputting the graded early warning information, the method further includes: encapsulating the graded early warning information with the corresponding system timestamp, device identification number and trajectory location information to form a structured early warning data packet for reporting and storage.

[0029] This invention also discloses an airborne fire source early warning system, comprising:

[0030] The model building and deployment unit is used to perform the steps of building and training the neural network detection model and deploying the model on the embedded computing unit in the aforementioned method;

[0031] The real-time perception and fusion unit is used to perform the steps in the aforementioned method: synchronously acquiring real-time image data and environmental perception data, performing model inference, and weighted fusion based on dynamic adjustment of weights according to real-time detection distance to generate a comprehensive early warning confidence level.

[0032] The early warning output and processing unit is used to perform the steps of outputting graded early warning information based on comprehensive early warning confidence in the aforementioned method, as well as subsequent data encapsulation, reporting and storage.

[0033] This invention also discloses an airborne fire source early warning device, comprising:

[0034] Memory, which stores computer programs;

[0035] The sensor array includes visible light image sensors, infrared image sensors, and non-image sensors for acquiring environmental parameters;

[0036] The processor, which is an airborne embedded processor, is connected to the memory and the sensor group and is configured to execute the computer program to implement the airborne airborne fire source early warning method as described above.

[0037] A communication module, connected to the processor, is used to report early warning information.

[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0039] This invention effectively overcomes the limitations of a single image source in complex scenarios such as low light, low visibility, and occlusion by fusing visible light and infrared dual-modal images and utilizing the GAM attention mechanism to enhance the model's ability to capture key features, thus significantly improving the accuracy and environmental adaptability of fire source detection.

[0040] This invention innovatively introduces an adaptive multi-source data fusion strategy based on real-time detection distance. This strategy can intelligently adjust the contribution weight of each data source according to the performance characteristics of the sensor at different distances, realizing the complementary advantages and cross-validation of multi-sensor information, thereby greatly suppressing false alarms and missed alarms caused by the failure of a single sensor or environmental interference.

[0041] This invention enables deep learning models to run efficiently on airborne embedded platforms with strictly limited computing, storage, and power resources by performing targeted lightweight processing (quantization and pruning), ensuring low end-to-end latency from data acquisition to early warning output and meeting real-time early warning requirements.

[0042] This invention relies on the mobility and wide field of view of an airborne platform, combined with this intelligent early warning method, to achieve dynamic and blind-spot-free patrol and monitoring of large-scale, complex terrain areas, thus completely solving the fundamental problem of the limited coverage of fixed-point early warning systems.

[0043] The early warning results in this invention are accompanied by a comprehensive confidence score and a clear level classification, making the decision-making process transparent. Simultaneously, complete timestamps, trajectories, and multi-source data records provide a solid data foundation for the review of early warning results, system performance evaluation, and subsequent optimization. Attached Figure Description

[0044] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0045] Figure 1 This is a flowchart of an airborne fire source early warning method according to the present invention;

[0046] Figure 2 A schematic diagram of the GAM attention mechanism;

[0047] Figure 3 This is a schematic diagram of the GAM channel attention module structure;

[0048] Figure 4 This is a schematic diagram of the spatial attention module structure in GAM.

[0049] Figure 5 This is a schematic diagram of the structure of an airborne fire source early warning system according to the present invention;

[0050] Figure 6 This is a schematic diagram of the structure of the device described in this invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0052] This embodiment 1 provides a spaceborne airborne fire source early warning method, such as Figure 1 As shown, steps S1 to S4 are included. Next, the specific technical details of each step will be introduced.

[0053] S1. Based on fire smoke image data containing both visible light and infrared dual modes, a neural network detection model is constructed and trained, wherein the neural network detection model integrates an attention mechanism module for enhancing cross-dimensional feature interaction in the feature fusion network.

[0054] The specific process of step S1 is as follows:

[0055] S101. Construction of a multimodal dataset:

[0056] Visible light images and videos of the fire source and smoke were acquired. Keyframes were extracted from the acquired videos to obtain fire source images. The fire source and smoke targets in all fire source images were labeled to obtain a fire source and smoke dataset. Simultaneously, infrared images and videos containing fire source and smoke were acquired, and the fire source and smoke targets were labeled to obtain an infrared fire source and smoke dataset. This resulted in a set of labeled images containing both visible light and infrared modalities for model training.

[0057] S102, Data Preprocessing and Augmentation:

[0058] Data augmentation was performed on the aforementioned fire source smoke dataset and infrared fire source smoke dataset. The Mixup, Mosaic, and SAT algorithms were used to perform data augmentation preprocessing on the labeled datasets in a uniformly distributed manner, resulting in augmented datasets. This aims to expand data diversity and improve the model's generalization ability across different scenarios.

[0059] S103, Dataset Partitioning and Anchor Frame Calculation:

[0060] The two augmented datasets were divided into training, validation, and test datasets according to a fixed ratio. Then, the K-Means adaptive clustering algorithm was used to calculate the position and size distribution of all bounding boxes in the training dataset, adaptively outputting a set of optimal anchor box values ​​for model initialization, providing prior knowledge that is more closely related to the actual target size for subsequent model training.

[0061] S104. Construction of the neural network detection model:

[0062] A neural network detection model capable of integrating global contextual feature information is constructed. This model generally follows an architecture of "backbone network - feature fusion network (neck) - detection head (head)". Specifically:

[0063] Backbone network selection: Based on the hardware platform conditions of the target deployment (such as computing power and power consumption), select a suitable backbone network (such as CSPDarknet, ResNet, etc.) to extract multi-level feature information from the input image.

[0064] Feature fusion network optimization: A GAM attention mechanism module is added to the feature fusion network (i.e., the "neck") after the backbone network and before the detection head. For example... Figure 2 , Figure 3 , Figure 4 As shown, the GAM module processes the input feature map through sequentially connected channel attention submodules and spatial attention submodules. Its core design is to enhance the interaction and expression of cross-dimensional (channel and spatial) features without reducing the channel dimension, thereby extracting the weight parameters of the feature map, effectively changing the local attention focus of the model, and improving the model's feature extraction and fusion capabilities for key features of fire source smoke.

[0065] Detection head design: In the model head, the bounding box coordinates are predicted and the categories of "fire source / smoke / background" are classified based on the feature map output by the aforementioned network.

[0066] S105. Model Training and Optimization:

[0067] The constructed neural network detection model is iteratively trained using the training dataset. During training, the CIoU_Loss loss function is used to perform regression optimization on the model's predicted bounding boxes, calculating the intersection-union ratio (IU) between the predicted and actual bounding boxes, and removing boxes exceeding a set threshold. The formula for calculating this loss function is:

[0068] ;

[0069] In the formula, A and B are the actual bounding box and the predicted bounding box, respectively. and These represent the center positions of A and B, respectively. represents the Euclidean distance between the center points of the predicted bounding box and the actual bounding box; c represents the diagonal distance between the minimum bounding rectangles of the two boxes; and v is a factor that measures the similarity of the aspect ratios of the two boxes. These are the weighting coefficients affected by v, where w and h are the width and height of the predicted bounding box. and Here, represents the width and height of the actual bounding box. The CIoU loss function comprehensively considers the overlap area (IoU) between the predicted and ground truth bounding boxes, the distance between their center points, and the similarity of their aspect ratios, making bounding box regression more accurate and faster to converge.

[0070] The model parameters are iteratively updated using the backpropagation algorithm, and the training process is monitored using the validation dataset to prevent overfitting, until the model's performance in detecting fire and smoke targets on the validation set reaches a stable and good state.

[0071] S106, Lightweight Model Preprocessing:

[0072] The trained network model was fine-tuned to further improve its performance in specific scenarios. Subsequently, the model's weights were compressed and quantized, converting the original 32-bit floating-point model parameters into 8-bit fixed-point numbers to significantly reduce model size while maintaining high detection accuracy. Furthermore, pruning techniques were used to remove low-contribution or unimportant connections and parameters, further reducing model complexity and computational cost. These optimizations aim to make the model more suitable for airborne embedded deployment environments with limited computing resources, storage space, and communication bandwidth.

[0073] S107, Model Inference and Post-processing:

[0074] After the model is trained and deployed, during actual fire source detection, the model outputs a large number of predicted bounding boxes with confidence scores for the input image. At this point, post-processing of the prediction results using the Non-Maximum Suppression (NMS) method is necessary. The core of NMS is to find predicted boxes with local maxima of confidence near the same target and suppress other non-maximum predicted boxes whose Intersection over Union (IoU) with the highest-scoring box exceeds a set threshold (e.g., 0.5). This step is crucial for eliminating duplicate detections and ensuring that each fire source or smoke target in the final output corresponds to only one optimal bounding box.

[0075] S2. Deploy the trained neural network detection model in the embedded computing unit of the aerospace airborne platform.

[0076] The specific process of step S2 is as follows:

[0077] S201. Model Format Conversion and Adaptation:

[0078] The neural network detection model, after lightweight preprocessing (quantization, pruning) in S1, is converted from its original format (such as PyTorch, TensorFlow) to the inference engine format supported by the target embedded hardware platform. For example, for NVIDIA Jetson platforms, it can be converted to TensorRT format; for Intel Movidius platforms, it can be converted to OpenVINO IR format; and for general-purpose ARM processors, it can be converted to TFLite format. This conversion process typically includes further graph optimization, operator fusion, and acceleration for specific hardware instruction sets to ensure that the model can run efficiently on embedded units.

[0079] S202, Embedded Platform Environment Configuration:

[0080] On the selected spaceborne embedded computing unit (such as a microcontroller, FPGA, ZYNQ, or PYNQ), deploy or burn the corresponding operating system (such as Linux for Embedded Systems) and necessary runtime libraries (such as CUDA, OpenCL, ARM Compute Library). Simultaneously, install the inference engine corresponding to the model format (such as TensorRT Runtime, OpenVINO Runtime, TFLite Interpreter), and configure the driver interfaces with the image sensor, other sensors, and communication modules to prepare a stable hardware and software environment for model operation.

[0081] S203, Model Integration and Software Packaging:

[0082] The converted model file is then integrated into an embedded software application specifically developed for fire source early warning missions. This application needs to implement the following core functional modules:

[0083] 1. Data Acquisition and Synchronization Module: Controls image sensors (visible light, infrared) and other sensors (smoke, temperature and humidity) to acquire data at a preset frequency and performs strict synchronization based on the system timestamp.

[0084] 2. Model Inference Module: Loads and initializes the inference engine and model, receives real-time image data from the sensor, calls the model to perform forward inference, and outputs preliminary detection results (including bounding boxes and class confidence).

[0085] 3. Post-processing and fusion module: This module integrates the NMS post-processing algorithm (as described in S107) to filter the original detection boxes output by the model. Simultaneously, this module includes pre-defined distance-based adaptive data fusion logic as described in S3.

[0086] 4. Early warning decision and output module: Based on the integrated confidence level, it executes the hierarchical early warning rules and controls the communication module to encapsulate and report the early warning results, timestamps, trajectory information, etc. It can also perform status indication or log recording locally.

[0087] S204, Performance Testing and Optimization:

[0088] The packaged software system is deployed and tested on the target embedded hardware. The focus of the testing is on the real-time performance of the model inference (whether the single-frame processing time meets the video stream frame rate requirements), accuracy (whether the detection accuracy in the embedded environment meets the standards), as well as power consumption and stability. Based on the test results, further adjustments to the model's quantization accuracy, multi-threading or pipeline optimization of the software, or fine-tuning of the sensor acquisition parameters may be necessary to ensure that the entire system can operate stably, reliably, and efficiently in a real airborne operating environment.

[0089] S3. During the early warning process, real-time image data collected by the image sensor on the airborne platform and environmental perception data collected by non-image sensors are acquired simultaneously; the real-time image data is input into the neural network detection model to obtain the image-based fire source detection confidence score; based on the real-time detection distance between the airborne platform and the suspected fire source, the weights of the image-based fire source detection confidence score and the environmental perception data in the decision-making are dynamically adjusted and weighted fusion is performed to generate a comprehensive early warning confidence score.

[0090] The specific process of step S3 is as follows:

[0091] S301, Multi-source data synchronous acquisition:

[0092] When the airborne aerospace platform performs a patrol flight early warning mission, the following multi-source data are synchronously triggered and collected based on a unified system timestamp:

[0093] 1. Trajectory and Time Data: Obtain the platform's real-time latitude, longitude, altitude, attitude, and precise system timestamps from the flight control system or integrated navigation system.

[0094] 2. Image sensor data: The onboard long-range visible light image acquisition equipment and infrared image acquisition equipment simultaneously capture images or video streams in the current field of view.

[0095] 3. Environmental sensing data: The smoke concentration, ambient temperature, relative humidity and other parameters in the monitoring area are collected by smoke sensors, temperature and humidity sensors and other non-image sensors.

[0096] S302, Image Composition and Data Verification:

[0097] The long-range visible light image and infrared image simultaneously acquired in S301 are registered and fused to generate a high-quality synthetic image with richer information and clearer details. Subsequently, the synthetic image and the original sensor data are verified and checked, such as whether the image is blurry and whether the sensor values ​​are within the effective range, to ensure the reliability of the data input to the subsequent model and avoid misjudgment due to data anomalies.

[0098] S303, Model Inference and Initial Confidence Acquisition:

[0099] The validated real-time image data (including long-range visible light images, infrared images, and synthesized high-quality images) are input into the deployed neural network detection model (from S2) for inference. The model analyzes each image and outputs the bounding boxes of suspected fire sources or smoke targets and their corresponding category confidence scores. The highest confidence score for each target across all image categories is taken as follows:

[0100] Confidence based on long-distance visible light images ;

[0101] Confidence based on infrared images ;

[0102] Confidence based on high-quality synthetic images ;

[0103] Simultaneously, based on smoke concentration and the degree of temperature and humidity anomalies, confidence levels based on smoke and temperature / humidity data are calculated using predefined mapping rules. .

[0104] S304, Adaptive Weighted Fusion Decision:

[0105] This step is the core innovation of this method. First, using primary and secondary factor analysis, four core detection methods are selected: long-distance visible light images, infrared images, high-quality synthetic images, and smoke temperature and humidity data. Detection distance is identified as the dominant factor influencing the confidence level. Based on this analysis, an adaptive weighted fusion strategy is constructed, using real-time detection distance as the segmentation criterion.

[0106] 1. Distance acquisition: The detection distance D between the airborne platform and the suspected fire source target is estimated in real time by using airborne LiDAR, visual ranging algorithms, or by combining the platform height and the pixel size of the target in the image.

[0107] 2. Dynamic Weight Adjustment and Fusion Calculation: Considering that ranging accuracy and azimuth accuracy both decrease with increasing distance, a custom piecewise function is used to characterize the fused confidence level. Specifically, based on the comparison between distance D and a preset threshold (e.g., 5km), different weighting coefficients are used to weight and sum the four confidence levels mentioned above to calculate the comprehensive early warning confidence level P. Its piecewise function expression is as follows:

[0108] Where bais is the system correction coefficient.

[0109] When the detection distance D > 5km, the weighting coefficients are set as follows: k1=0.45, k2=0.05, k3=0.45, k4=0.05. This setting takes into account that the details of infrared images are relatively blurred at long distances, so it mainly relies on visible light and its fusion results.

[0110] When the detection distance D ≤ 5km, the weighting coefficients are adjusted to: k1 = 0.45, k2 = 0.15, k3 = 0.35, k4 = 0.05. This setting is appropriate because infrared characteristics are significant at close range, hence the weighting is increased accordingly.

[0111] In another preferred embodiment of the present invention, the fusion weight coefficients k1, k2, k3, and k4 are not fixed empirical values, but are adaptively determined during the training phase or before deployment using an optimization algorithm, in order to further improve the fusion performance of the system under different distances and environments. That is, in the piecewise function strategy, the fusion weight coefficients assigned to different detection distance intervals are obtained offline through an optimization algorithm on a validation dataset with distance labels, aiming to maximize the early warning performance index.

[0112] The specific implementation method is as follows:

[0113] 1. Optimization problem:

[0114] Optimization variables: The weight coefficient vector K = [k1,k2,k3,k4] must satisfy k1+k2+k3+k4=1 and ki≥0.

[0115] Objective function: The goal is to maximize the overall early warning performance metric on the validation set. This performance metric can be the F1-Score, the mean average precision (mAP), or a custom metric J that combines the detection rate and the false alarm rate.

[0116] Constraints: Divide the detection distance D into several discrete intervals (e.g., D≤1km, 1km). <D≤3km,3km<D≤5km,D> (5km). Within each distance interval, solve for a set of optimal weighting coefficients. This embodies the core idea of ​​adaptive weighting coefficients based on distance segmentation.

[0117] 2. Implement the optimization algorithm:

[0118] Offline optimization (training phase): This is performed using a labeled validation dataset with distance information. The optimization process can be described as follows:

[0119] For each distance interval In the validation set subset (Distance belongs to) Solve using the data: ,in, , A quantitative evaluation metric can be the F1 score: the harmonic mean of precision and recall, which is a commonly used indicator for comprehensively evaluating the detection capabilities of a model. F1 = 2 * (Precision * Recall) / (Precision + Recall).

[0120] Commonly used optimization algorithms include, but are not limited to:

[0121] Grid Search: Dense sampling is performed in the weight combination space to evaluate the performance of each set of parameters and select the best one.

[0122] Bayesian optimization: By constructing a surrogate model, it efficiently finds the global optimum with fewer evaluations.

[0123] Genetic Algorithm: Simulates the natural selection process and is suitable for multivariable, nonlinear optimization problems.

[0124] Results Consolidation and Application: The optimal weight coefficient set obtained through offline optimization... It is embedded into the system. During the real-time early warning phase (S304), the system determines the interval to which the current detection distance D belongs. Directly call the corresponding optimal weight coefficient Perform fusion computing.

[0125] The weight coefficients obtained through the optimization algorithm have a physical meaning consistent with the basic design principle of this invention (emphasizing visible light at long distances and increasing infrared weight at short distances). For example, the coefficients optimized by the algorithm in the D>5km range are likely to be close to [0.45, 0.05, 0.45, 0.05], thus verifying the rationality of the fixed weight scheme. This method has stronger data-driven and adaptive capabilities. It can automatically learn and adapt to specific sensor performance, regional environmental characteristics (such as the different effects of vegetation and buildings on infrared radiation in different regions), and task requirements (such as a greater emphasis on recall or precision), thereby potentially achieving better fusion results than preset fixed values, enabling the early warning system to have stronger environmental generalization capabilities.

[0126] S305, Early Warning Data Encapsulation:

[0127] The final generated comprehensive early warning confidence score P, the corresponding suspected fire source target location (image coordinates or geographic coordinates), the system timestamp, and the device's unique identification number are associated. Furthermore, the trajectory data of the aircraft (which can be generated by fitting continuous position points to a B-spline curve to create a smooth trajectory, i.e., described by a control point set C of the B-spline, and the node vector of the B-spline is calculated, with input parameters: n+1 control points and k B-splines. Generally, the larger the values ​​of n and k, the smaller the curve fitting error, but the greater the computational complexity and storage space usage. In this embodiment, n is 8 and k is 4), is used to form an early warning estimation component package according to the generated trajectory and the order of time generation. Each early warning estimation component package corresponds to image data at each time and is timestamped for easy data analysis in the early warning assessment model. The specific storage structure of the early warning estimation component package is shown in the table below, facilitating subsequent transmission, storage, and analysis.

[0128] Serial Number Timestamp (year, month, day, hour, minute, second.millisecond.microsecond) Device Identification Number Fire source trajectory early warning data (including confidence level P, location, etc.) id.001 20241009120102.999.58 123456789 XXXXXXXXXXXXX0 id.002 20241009120103.599.58 123456789 XXXXXXXXXXXXX1

[0129] All data in the table are for illustrative purposes only.

[0130] Serial number: Represents the number of the stored data;

[0131] Timestamp: Year, Month, Day, Hour, Minute, Second.Millisecond.Microsecond;

[0132] Device identification number: uses a character combination that meets the system requirements;

[0133] Fire source trajectory early warning data: This is the optimized early warning data.

[0134] S4. Based on the comprehensive early warning confidence level, output graded early warning information.

[0135] The specific process of step S4 is as follows:

[0136] S401, Tiered Early Warning Decision:

[0137] Based on the comprehensive early warning confidence level P calculated in step S304, the early warning level is automatically determined by applying preset threshold rules.

[0138] Level 1 Decision (No Risk): If P < 60%, the system determines that there is no fire source risk in the current monitoring result, does not trigger any warning prompts, and the relevant data can only be recorded as background logs.

[0139] Level 2 Decision (Low Risk / Suspected): If 60% ≤ P < 80%, the system determines that there is a suspected fire source risk. At this time, a Level 1 warning message is output, and a general marker is displayed on the monitoring interface (for example, a yellow exclamation mark or the text label "Suspected Fire Source" is displayed at the corresponding location on the electronic map), and a low-priority prompt sound or message is triggered.

[0140] Level 3 Decision (High Risk / Confirmed): If P ≥ 80%, the system determines that a confirmed fire source exists. At this time, a Level 2 warning message is output, and a special and eye-catching mark is displayed on the monitoring interface (e.g., displaying a red flame icon, a flashing warning box, or a "Fire Source" text label), and a high-priority alarm sound, pop-up notification, or SMS / telephone linkage is triggered.

[0141] S402. Visualization and Interface Presentation of Early Warning Results:

[0142] The tiered early warning results generated by S401 are displayed in real time on a dedicated display screen or command system interface at the ground monitoring center. Presentation methods include, but are not limited to:

[0143] 1. Geographic information overlay: Accurately overlay warning icons (such as yellow / red markers) onto the corresponding geographic locations on the electronic map (converted from the trajectory and location information in S305).

[0144] 2. Details panel display: Clicking the warning icon will expand the details panel, which displays key information such as the real-time image snapshot (visible light / infrared / composite image) that triggered the warning, confidence level value, sensor reading, and timestamp.

[0145] 3. Trajectory and Situation Display: By combining continuous multi-frame early warning data, the movement trajectory of suspected fire sources or the preliminary situation map of fire spread can be drawn on the interface.

[0146] S403, Early Warning Information Reporting and Communication:

[0147] The system uses an onboard communication module (such as 4G / 5G or satellite communication link) to automatically and in real time report the early warning estimation component package encapsulated in S305 and the early warning level determined by S401 to a remote monitoring center server or cloud platform. The reporting protocol must ensure data integrity, real-time performance, and security. When communication conditions are limited, the system has local caching and breakpoint resumption capabilities.

[0148] S404, Multi-dimensional Data Storage and Archiving:

[0149] All data related to early warnings are systematically stored to support post-event review, analysis, and model optimization.

[0150] 1. Warning Event Database: Stores core information of all warning events (including suspected and confirmed events), such as time, location, confidence level, level, and processing status, forming a queryable historical event database.

[0151] 2. Raw data archiving: The system associates and stores the raw image data (visible light, infrared), raw readings from environmental sensors, and complete trajectory data at the moment the warning is triggered, providing a data foundation for in-depth analysis and false alarm investigation.

[0152] 3. System Log Recording: Records system running status, model inference time, communication status and other log information for system health monitoring and maintenance.

[0153] S405, Linkage between Decision Support and Response:

[0154] For high-level alerts (Level 2), the system can automatically or assistedly perform the following actions to improve emergency response efficiency:

[0155] Generate a preliminary fire report: Automatically extract key information and generate a rapid report including location, time, confidence level, and on-site images.

[0156] Trigger preset plans: Based on the warning location and level, automatically associate and prompt preset emergency plans or notify the list of relevant responsible personnel.

[0157] Equipment linkage: It can control the gimbal camera to perform automatic zoom and continuous tracking shooting, or instruct the drone to fly to the target area for close reconnaissance.

[0158] Example 2 discloses an airborne fire source early warning system corresponding to the method described in Example 1. This system is logically divided into three core functional units, which work collaboratively to achieve full automation from model preparation to real-time early warning. Specifically, it includes the following units:

[0159] 1. Model Building and Deployment Unit: This unit is primarily responsible for model preparation and integration during the offline phase, specifically including:

[0160] Dataset Management Module: Used to receive and manage raw fire source smoke images and video data collected by visible light and infrared sensors, and provides functions for data annotation, augmentation (Mixup, Mosaic, SAT, etc.) and partitioning (training / validation / test sets).

[0161] Model Training and Optimization Module: This module provides a training framework for building a neural network detection model integrated with the GAM attention mechanism module, and iteratively training and validating it using the dataset with the CIoU loss function. This module also includes a model lightweighting submodule, responsible for performing parameter quantization (e.g., FP32 to INT8 conversion) and network pruning operations on the trained model.

[0162] Model conversion and packaging module: responsible for converting the lightweight model into a format supported by the target embedded computing unit (such as TensorRT for NVIDIA Jetson, OpenVINO for Intel Movidius), and packaging it together with the core control logic, NMS post-processing algorithm, etc. into a software package that can be directly deployed in the embedded environment.

[0163] 2. Real-time perception and fusion unit: This unit operates on the airborne platform and is the core of online execution of early warning missions. Specifically, it includes:

[0164] Multi-source data synchronous acquisition module: Based on a high-precision system clock, it synchronously triggers and drives hardware such as visible light camera, infrared thermal imager, smoke sensor, temperature and humidity sensor to complete the acquisition of raw data and timestamp alignment.

[0165] Image preprocessing and synthesis module: This module denoises, corrects, and registers the acquired visible light and infrared images, and executes image fusion algorithms to generate high-quality synthesized images. Simultaneously, it verifies the validity of the sensor data.

[0166] Model Inference and Confidence Generation Module: Loads and runs the deployed neural network detection model, performs inference on real-time input visible light images, infrared images, and synthetic images respectively, outputs preliminary detection boxes and corresponding class confidence scores, and generates... , , .

[0167] Adaptive Fusion Decision Module: This is the intelligent hub of the system. This module incorporates the piecewise function fusion strategy. It obtains the current detection distance D in real time and dynamically calls preset weight coefficients (or the optimal coefficients obtained through an optimization algorithm) to adjust the image confidence based on the interval where D is located (e.g., greater than or less than 5km). , , and environmental confidence We perform weighted fusion to calculate the final comprehensive early warning confidence level P.

[0168] 3. Early Warning Output and Processing Unit: This unit is responsible for the generation, management, and human-computer interaction of early warning results, specifically including:

[0169] The graded early warning judgment module receives the comprehensive confidence level P from the fusion decision module and makes a real-time judgment based on preset thresholds (such as 60% or 80%) to determine the early warning level as "no risk", "suspected fire source" (level 1) or "confirmed fire source" (level 2).

[0170] Data encapsulation and communication module: This module encapsulates information such as warning level, confidence level, target location, system timestamp, device ID, and flight trajectory (fitted using B-spline) into a structured warning data packet. This packet is reliably reported to the ground monitoring center via an airborne communication link (4G / 5G / satellite).

[0171] Local storage and display module: The module continuously records raw sensor data, intermediate model inference results, and final warning data packets on the onboard storage device. Simultaneously, it provides local indication of system status and high-level warnings via a simple display screen or status indicator lights.

[0172] System workflow: During mission execution, the real-time perception and fusion unit and the early warning output and processing unit run continuously on the airborne processor, forming a closed loop of "perception-fusion-decision-output". The model building and deployment unit mainly runs on the ground server before the mission or during system upgrades, providing the airborne system with optimized model software packages.

[0173] Example 3 discloses a physical hardware device for implementing the method described in Example 1, such as... Figure 6 As shown, this device is an airborne terminal that can be integrated into aerospace platforms such as drones and airships. Its specific components are as follows:

[0174] 1. Core computing and storage components:

[0175] Processor 302: Employs a high-performance embedded AI processor that meets the requirements of airborne environments (wide temperature range, vibration resistance), such as the NVIDIA Jetson AGX Orin series, Huawei Atlas 200 series, or a self-developed computing card based on FPGA / ZYNQ. This processor is responsible for running all online software modules of the system described in Example 2.

[0176] Memory 301 includes high-speed RAM (for program execution and data caching) and solid-state storage (such as eMMC or NVMe SSD) for storing the device operating system, system software of Embodiment 2, lightweight neural network model parameters, and various types of data generated during the task.

[0177] 2. Multifunctional sensor assembly 304:

[0178] Visible light image sensor: Employs an industrial camera with high resolution, optical zoom, and global shutter capabilities to capture detailed information of distant scenes.

[0179] Infrared image sensor: Employs an uncooled infrared thermal imager with high thermal sensitivity, used to detect areas of abnormal temperature and compensate for the limitations of visible light at night and in smoke.

[0180] Environmental sensing sensors: integrate laser rangefinder (for obtaining accurate detection distance D), smoke particle detection sensor, temperature and humidity composite sensor, etc., to provide multi-dimensional environmental parameters.

[0181] 3. Communication and Interface Components 303:

[0182] Communication module: Integrates 4G / 5G mobile communication module and Beidou / GPS / satellite communication module to ensure reliable two-way transmission of early warning information even in complex terrain and remote areas.

[0183] Device interfaces: Provides a rich set of industrial interfaces, such as CAN bus (for communicating with the flight control system to obtain attitude and trajectory information), Ethernet, RS-485 / 232 (for connecting external sensors), HDMI (for local debugging and display), etc.

[0184] 4. Auxiliary and Support Components:

[0185] Power management module: Provides a wide voltage input (such as 9-36V DC) and provides stable, isolated power to each component, managing device power consumption.

[0186] Structure and heat dissipation: The equipment casing is made of lightweight, high-strength alloy material and is designed with an efficient heat dissipation structure (such as heat dissipation fins and air ducts) to ensure stable operation for a long time in a closed airborne environment.

[0187] Device Operating Mode: After power-on, the processor loads system software from memory and initializes each sensor and communication module. During flight, the device autonomously executes the entire process described in Example 1: data is synchronously collected through the sensor group, the processor performs image processing, model inference, adaptive fusion, and early warning decision-making, and finally reports the results through the communication module. The entire device constitutes an independent intelligent sensing and early warning node.

[0188] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A space-based airborne fire source early warning method, characterized in that, include: Based on fire smoke image data containing both visible light and infrared dual modes, a neural network detection model is constructed and trained. The neural network detection model integrates an attention mechanism module in the feature fusion network to enhance cross-dimensional feature interaction. The trained neural network detection model is deployed in the embedded computing unit of the aerospace airborne platform; During the early warning process, real-time image data collected by the image sensor on the airborne platform and environmental perception data collected by non-image sensors are acquired simultaneously. The real-time image data is input into the neural network detection model to obtain the image-based fire source detection confidence score. Based on the real-time detection distance between the airborne platform and the suspected fire source, the weights of the image-based fire source detection confidence score and the environmental perception data in the decision-making process are dynamically adjusted and weighted fused to generate a comprehensive early warning confidence score. Based on the comprehensive early warning confidence level, graded early warning information is output.

2. The airborne airborne fire source early warning method according to claim 1, characterized in that, The attention mechanism module used to enhance cross-dimensional feature interaction is the GAM attention module; the GAM attention module processes the input feature map through sequentially connected channel attention sub-modules and spatial attention sub-modules, enhancing the interaction and expression of cross-dimensional features without reducing the channel dimension.

3. A spaceborne airborne fire source early warning method according to claim 1 or 2, characterized in that, When constructing and training the neural network detection model, the CIoU loss function is used to perform regression optimization on the predicted bounding boxes of the neural network detection model. The CIoU loss function comprehensively considers the overlap area between the predicted bounding boxes and the ground truth bounding boxes, the distance between the center points, and the similarity of the aspect ratio.

4. The airborne airborne fire source early warning method according to claim 1, characterized in that, Adaptive data fusion is performed by dynamically adjusting weights based on real-time detection distance. Specifically, based on the comparison between the real-time detection distance and a preset distance threshold, a piecewise function is used to assign different fusion weight coefficients to different data sources.

5. The airborne airborne fire source early warning method according to claim 4, characterized in that, The expression for the piecewise function is: ; Where P is the comprehensive early warning confidence level. For confidence scores based on long-distance visible light image data, For confidence levels based on infrared image data, For confidence based on image fusion data, bais represents the confidence level based on environmental sensing data of smoke and temperature and humidity, and bais is the correction coefficient. When the detection distance D > 5km, k1 = 0.45, k2 = 0.05, k3 = 0.45, k4 = 0.05; When the detection distance D≤5km, k1=0.45, k2=0.15, k3=0.35, k4=0.

05.

6. A spaceborne airborne fire source early warning method according to claim 1 or 5, characterized in that, Based on the comprehensive early warning confidence level, hierarchical early warning information is output, specifically including: Set a first threshold and a second threshold that is higher than the first threshold; If the overall warning confidence level is lower than the first threshold, then no warning is triggered; If the overall warning confidence level reaches or exceeds the first threshold but is lower than the second threshold, then the first-level warning information is output. If the overall early warning confidence level reaches or exceeds the second threshold, then the second-level early warning information is output.

7. The airborne airborne fire source early warning method according to claim 1, characterized in that, Before deploying the trained neural network detection model to the embedded computing unit of the aerospace airborne platform, the method further includes: performing parameter quantization and network pruning on the trained neural network detection model to reduce the computational complexity and storage requirements of the neural network detection model, so that the neural network detection model can adapt to the resource constraints of the embedded computing unit.

8. The airborne airborne fire source early warning method according to claim 1, characterized in that, After outputting the graded early warning information, the method further includes: encapsulating the graded early warning information with the corresponding system timestamp, device identification number and trajectory location information to form a structured early warning data packet for reporting and storage.

9. An airborne airborne fire source early warning system, characterized in that, include: The model building and deployment unit is used to perform the steps of building and training the neural network detection model and deploying the model on the embedded computing unit as described in claim 1; The real-time perception and fusion unit is used to perform the steps in claim 1: synchronously acquiring real-time image data and environmental perception data, performing model inference, and weighted fusion based on dynamic adjustment of weights according to real-time detection distance to generate a comprehensive early warning confidence level. The early warning output and processing unit is used to perform the steps of outputting graded early warning information based on the comprehensive early warning confidence level as described in claim 1, as well as subsequent data encapsulation, reporting and storage.

10. An airborne early warning device for airborne fire sources, characterized in that, include: Memory, which stores computer programs; The sensor array includes visible light image sensors, infrared image sensors, and non-image sensors for acquiring environmental parameters; The processor, which is an airborne embedded processor, is connected to the memory and the sensor group and is configured to execute the computer program to implement the airborne fire source early warning method as described in any one of claims 1 to 8; A communication module, connected to the processor, is used to report early warning information.

Images