Unmanned aerial vehicle fire source intelligent identification method and system based on multi-modal perception

Abstract

The application discloses a multi-modal perception-based unmanned aerial vehicle fire source intelligent identification method and system, relates to the technical field of fire source identification, and comprises the following steps: acquiring a visible light image and an infrared image of a region to be monitored, and performing initial positioning of a suspected fire source region; planning an initial path for approaching the suspected fire source region; controlling a sniffing unmanned aerial vehicle, and constructing a dynamic information value map in real time; optimizing a detection path, and controlling the sniffing unmanned aerial vehicle to fly along the optimal detection path; and when the obtained environmental perception data meet preset fire source confirmation conditions, outputting fire source confirmation information and position information. The application solves the technical problem that, in the prior art, after initial positioning in unmanned aerial vehicle fire source detection, there is a lack of dynamic optimization of a detection path, resulting in low fire source identification accuracy and poor position positioning accuracy, and achieves an intelligent closed loop from initial positioning of a suspected fire source region to dynamic detection and accurate confirmation of an unmanned aerial vehicle, thereby improving the technical effects of fire source identification accuracy and position positioning accuracy.

Description

Technical Field

[0001] This invention relates to the field of fire source identification technology, specifically to a method and system for intelligent fire source identification using unmanned aerial vehicles (UAVs) based on multimodal perception. Background Technology

[0002] Traditional fire source detection methods rely heavily on manual inspections or fixed monitoring equipment, which suffer from limited coverage, poor terrain adaptability, and susceptibility to environmental interference, making it difficult to meet the real-time detection needs in large-area and complex scenarios. With the development of drone technology, drones equipped with single sensors are increasingly being used for fire source detection. However, existing technologies often face two major limitations: First, after relying solely on single-modal data for initial location of suspected fire sources, there is a lack of dynamic path optimization capabilities that combine real-time environmental dynamics with multi-source sensing data. This leads to drones frequently flying redundantly in non-critical areas, reducing detection efficiency. Second, the lack of an effective fusion mechanism for multi-modal sensing data makes it difficult to construct a comprehensive fire source association information model, resulting in low accuracy in fire source identification and poor location precision. This hinders the rapid transition from "suspected location" to "precise confirmation," delaying disaster response.

[0003] In existing technologies, the lack of dynamic optimization of the detection path after initial localization in UAV fire source detection leads to technical problems such as low accuracy of fire source identification and poor location positioning. Summary of the Invention

[0004] This application provides a method and system for intelligent identification of fire sources by unmanned aerial vehicles (UAVs) based on multimodal perception. It is used to address the technical problems in existing UAV fire source detection, such as the lack of dynamic optimization of the detection path after initial localization, which leads to low accuracy in fire source identification and poor location positioning.

[0005] In view of the above problems, this application provides a method and system for intelligent identification of fire sources by unmanned aerial vehicles based on multimodal perception.

[0006] The first aspect of this application provides a method for intelligent identification of fire sources from unmanned aerial vehicles (UAVs) based on multimodal perception, the method comprising: The reconnaissance drone acquires visible light and infrared images of the area to be monitored and performs initial location of suspected fire sources. Based on the initial location results and real-time wind direction and speed data, an initial path is planned for the sniffing drone to approach the suspected fire source area. The sniffing drone is controlled to fly along the initial path and uses its onboard multimodal sensors to collect environmental perception data. Combined with shared perception information synchronously received from the reconnaissance drone, a dynamic information value map is constructed in real time. Based on the dynamic information value map, under flight constraints and with the goal of maximizing information collection efficiency, the detection path is optimized using a particle swarm optimization algorithm. The sniffing drone is controlled to fly along the optimal detection path and continue to perform environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, fire source confirmation information and location information are output.

[0007] A second aspect of this application provides an intelligent fire source identification system for unmanned aerial vehicles (UAVs) based on multimodal perception, the system comprising: The fire source suspected area positioning module is used to acquire visible light and infrared images of the area to be monitored through a reconnaissance drone and perform initial positioning of the suspected fire source area; the initial path planning module is used to plan an initial path for the sniffing drone to approach the suspected fire source area based on the initial positioning results and real-time wind direction and speed data; the value map construction module is used to control the sniffing drone to fly along the initial path and use the onboard multimodal sensors to collect environmental perception data, and combine it with the shared perception information synchronously received from the reconnaissance drone to construct a dynamic information value map in real time; the location information output module is used to optimize the detection path through a particle swarm optimization algorithm based on the dynamic information value map, with the goal of maximizing information collection efficiency under flight constraints, and control the sniffing drone to fly along the optimal detection path to continue environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, the fire source confirmation information and location information are output.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: The system acquires visible light and infrared images of the monitored area using a reconnaissance drone and performs initial location of suspected fire sources. Based on the initial location and real-time wind direction and speed data, an initial path is planned for the sniffing drone to approach the suspected fire source area. The sniffing drone is controlled to fly along the initial path, and a dynamic information value map is constructed in real time by combining the shared sensing information received synchronously from the reconnaissance drone. Based on the dynamic information value map, and with the goal of maximizing information acquisition efficiency under flight constraints, the sniffing drone is controlled to fly along the optimal detection path to continue environmental sensing. When the obtained environmental sensing data meets the preset fire source confirmation conditions, fire source confirmation information and location information are output. This achieves an intelligent closed loop from initial location of suspected fire source areas and dynamic detection to accurate confirmation by the drone, improving the technical effect of fire source identification accuracy and location positioning precision. Attached Figure Description

[0009] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A schematic flowchart of a UAV fire source intelligent identification method based on multimodal perception provided in an embodiment of this application; Figure 2 A schematic diagram of the structure of a UAV fire source intelligent identification system based on multimodal perception provided in an embodiment of this application.

[0011] Explanation of reference numerals in the attached diagram: 10 for suspected fire source location module, 20 for initial path planning module, 30 for value map construction module, and 40 for location information output module. Detailed Implementation

[0012] This application provides a method and system for intelligent identification of fire sources by UAVs based on multimodal perception, which addresses the technical problem of low accuracy in fire source identification and poor location positioning due to the lack of dynamic optimization of the detection path after initial positioning in existing UAV fire source detection technologies.

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0014] Example 1, as Figure 1As shown, this application provides a method for intelligent identification of fire sources from unmanned aerial vehicles (UAVs) based on multimodal perception, the method comprising: Step S100: Obtain visible light and infrared images of the area to be monitored using a reconnaissance drone, and perform initial location of the suspected fire source area.

[0015] Specifically, the first step involves deploying reconnaissance drones to conduct full-area patrol flights over the target monitoring area. Utilizing onboard high-definition visible light cameras and infrared thermal imagers, the drones simultaneously collect visible light and infrared images of the area. Visible light images capture intuitive visual features such as smoke and open flames, while infrared images accurately present the temperature distribution of the area, highlighting abnormally hot zones with temperatures exceeding ambient levels. Subsequently, image preprocessing and feature extraction algorithms identify the shape and color of smoke from the visible light images and filter out high-temperature areas exceeding a preset environmental threshold from the infrared images. The results of the two types of image recognition are cross-validated and fused to eliminate misjudgments caused by interference factors such as clouds and reflections. Finally, areas exhibiting abnormally high temperatures and suspected smoke / open flame characteristics are identified as suspected fire source areas. The approximate geographical coordinates and boundaries of these areas are output, completing the initial location of the suspected fire source areas and providing target guidance for subsequent precise detection by sniffing drones.

[0016] Step S200: Based on the initial positioning results and real-time wind direction and speed data, plan an initial path for the sniffing drone to approach the suspected fire source area.

[0017] Specifically, the core target point is the geographical coordinates and boundary of the suspected fire source area. Real-time wind direction and speed data obtained by meteorological monitoring equipment or sensors on drones are simultaneously accessed. For example, if the wind direction is northeast and the wind speed is 3 m / s, the upwind area is avoided first to prevent the sniffing drone from entering areas with high concentrations of smoke, high temperatures, or harmful gas diffusion too early, thus reducing equipment wear and detection interference risks. Then, based on the path planning algorithm, the outer safety boundary of the suspected fire source area is set as the initial approach endpoint according to the principles of "shortest flight distance, optimal detection angle, and lowest environmental interference". At the same time, the terrain features of the area to be monitored are combined, such as avoiding obstacles and complex terrain, and the flight performance parameters of the sniffing drone, such as maximum endurance and flight speed, to generate multiple candidate paths from the drone's take-off and landing point to the outer edge of the suspected fire source area on the map. Finally, the environmental adaptability of the candidate paths is evaluated, and the path along the downwind or crosswind direction that can gradually approach the suspected area and has no obvious obstacles is selected as the initial path. This ensures that the sniffing drone can initially perceive environmental information related to the fire source, such as gas and particulate matter concentration, during flight, while reserving adjustment space for subsequent dynamic path optimization.

[0018] Step S300: Control the sniffing drone to fly along the initial path, and use the onboard multimodal sensors to collect environmental perception data. Combine the shared perception information received synchronously from the reconnaissance drone to construct a dynamic information value map in real time.

[0019] Specifically, the sniffing drone flies according to the planned initial path. Its onboard multimodal sensors, such as gas sensors and particulate matter sensors, simultaneously collect environmental perception data, including key parameters such as the concentration of combustible gases and particulate matter at different locations within the monitored area. All data carries precise detection location information recorded by the positioning device. Simultaneously, the sniffing drone receives shared perception information transmitted in real time from the reconnaissance drone. This information includes the latest visible light images, infrared thermal images, and a global environmental overview of the suspected area. Then, the monitored area is first gridded into multiple equal grid units, and then the location of each grid unit is determined based on the detection location information. Based on existing detection data, combined with gas concentration gradients and particulate matter concentration gradients from environmental perception data, and the probability values ​​of infrared thermal image temperatures corresponding to grid units extracted from shared perception information, the continued detection value of each grid unit is quantitatively analyzed. For example, grid units that have not been sufficiently detected or whose parameters fluctuate abnormally have higher detection value. Finally, the detection value values ​​of each grid unit are marked in the corresponding grid to form an initial information value map. As the sniffing drone continuously collects new data and receives new shared information during its flight, the detection value of each grid in the map is updated in real time, ultimately constructing a dynamically changing information value map that clearly presents the distribution of detection priorities within the region.

[0020] Step S400: Based on the dynamic information value map, with the goal of maximizing information collection efficiency under flight constraints, the detection path is optimized by particle swarm optimization algorithm. The sniffing drone is controlled to fly along the optimal detection path and continue to perform environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, the fire source confirmation information and location information are output.

[0021] Specifically, based on a dynamically updated information value map, the detection value priority of each grid cell within the area to be detected is clearly defined. High-value grids are mostly areas that have not been fully detected, have abnormal gas / particulate matter concentrations, or have high temperature probability values. Simultaneously, flight constraints for the sniffing drone are incorporated, such as endurance limits, maximum flight speed, and obstacle avoidance safety distances. Then, aiming to maximize information collection efficiency per unit time, a particle swarm optimization algorithm is used to optimize the detection path. The drone's flight path within a preset time period is discretized into ordered path points, forming multiple sets of particle position vectors. Using a fitness function, the cumulative information value of each path point across the grid cells is calculated. The group vector is used to evaluate fitness and continuously update particle positions to select the optimal detection path with the highest fitness. Then, the sniffing drone is controlled to fly along the optimal path and continuously collect environmental perception data through multimodal sensors, such as changes in combustible gas concentration and particulate matter concentration in high-value areas. Finally, the newly collected environmental perception data is compared with the preset fire source confirmation conditions in real time, such as combustible gas concentration reaching the combustion threshold, particulate matter concentration conforming to the characteristics of fire smoke, and temperature data matching the heat radiation law of open flame. When the data fully meets the conditions, the fire source confirmation information is immediately output and the precise location information of the fire source obtained by the positioning device is reported simultaneously to complete the fire source detection closed loop.

[0022] In one possible implementation, step S300 further includes: Step S310: The multimodal sensor includes a gas sensor and a particulate matter sensor, and the sniffing drone carries a positioning device.

[0023] Specifically, the multimodal sensor includes gas sensors and particulate matter sensors. The gas sensor can collect real-time concentration data of combustible gases such as methane and propane in the monitored area, capturing characteristic gas concentration changes that may be caused by the combustion of the fire source. The particulate matter sensor can detect the content and distribution of smoke particles in the air, such as PM2.5 and PM10, to help identify signs of smoke diffusion in the early stages of a fire. At the same time, the positioning equipment carried by the sniffing drone, such as GPS positioning modules and Beidou positioning modules, can accurately record the geographical coordinates of each set of environmental data collected by the sensor, ensuring that all environmental perception data can correspond one-to-one with the specific detection location. This provides a reliable location benchmark for subsequent construction of dynamic information value maps, analysis of the detection value of grid units, and final confirmation of the precise location of the fire source.

[0024] In one possible implementation, step S300 further includes: Step S320: The area to be monitored is gridded to construct multiple grid cells.

[0025] Step S330: The environmental sensing data carries detection location information. Based on the detection location information, the shared sensing information, and the environmental sensing data, a continued detection value analysis is performed on the multiple grid cells to generate multiple grid detection values.

[0026] Step S340: Mark the multiple grid detection values ​​to the multiple grid cells to establish the information value map.

[0027] Specifically, the size and specifications of the grid units are determined by combining the actual geographical range of the area to be monitored, such as the latitude and longitude data of the area boundary obtained by reconnaissance drones and the accuracy requirements for fire source detection, such as the need for precise positioning within a 10m x 10m or 5m x 5m area. Then, a spatial gridding algorithm is used to uniformly divide the entire area to be monitored into multiple non-overlapping square or rectangular grid units with clear boundaries according to the set size. Each grid unit is assigned a unique spatial identifier, such as a coordinate code, to ensure that it can accurately correspond to the specific geographical location within the area. At the same time, during the gridding process, basic geographic information within the area is also linked, such as terrain elevation and obstacle distribution. Grid units with undetectable areas, such as tall buildings or dangerous terrain, are specially marked. Finally, multiple grid units that cover the entire area to be monitored, have a clear structure, and are accurately positioned are formed, providing a standardized spatial carrier for subsequent grid-based detection value analysis and information value map construction.

[0028] This study utilizes precise detection location information, including combustible gas concentration and particulate matter concentration, carried by environmental perception data collected by multimodal sensors of sniffing drones. It identifies the grid cells corresponding to each set of environmental data, and then identifies the existing detection characteristics of each grid cell, such as whether basic detection has been completed, and the completeness and timeliness of data collection. Secondly, it extracts the infrared thermal image temperature probability values ​​corresponding to each grid cell from the shared perception information synchronously transmitted by the reconnaissance drone, reflecting the possibility of high-temperature areas within the grid. Subsequently, it comprehensively evaluates the value of continuing detection in each grid cell to obtain more effective information related to fire sources, based on existing detection characteristics, gas concentration gradients and particulate matter concentration gradients in the environmental perception data (larger gradients indicate more drastic parameter changes within the grid, and a higher probability of proximity to a fire source), and temperature probability values ​​in the shared perception information. Finally, through a pre-set value quantification model, it weights and calculates the scores of each evaluation dimension, converting the continued detection value of each grid cell into a specific numerical value, generating the grid detection value corresponding to all grid cells, and providing data support for the subsequent construction of an information value map.

[0029] A correspondence between grid cells and detection value is established. Based on the specific detection value value of each generated grid cell, such as a quantitative score of 0-10, the value is precisely bound to the unique spatial identifier of that grid cell, such as coordinate code, to ensure that each grid cell has a clear corresponding detection value data. Subsequently, a visualization mapping rule is used to present the detection value intuitively. For example, value levels are distinguished by color gradients: high-value grid cells are marked in red, medium-value grid cells in yellow, and low-value grid cells in blue, clearly reflecting the detection priority of each area. At the same time, during the map construction process, special information marked during grid processing, such as warning signs for undetectable areas, is also integrated. The final result is an information value map that covers the entire area to be monitored, has accurate spatial positioning, and whose detection value is intuitively visible. This map can clearly present the distribution of detection necessity for each grid cell, providing direct data support for subsequent path optimization of sniffing drones.

[0030] In one possible implementation, step S330 further includes: Step S331: Based on the detection location information, identify the existing detection features of the sniffing drone for the multiple grid cells.

[0031] Step S332: Based on the environmental perception data, identify the gas concentration gradient and particulate matter concentration gradient of the multiple grid cells.

[0032] Step S333: Extract the temperature probability value from the infrared thermal image corresponding to the multiple grid cells from the shared sensing information.

[0033] Step S334: Combine the existing detection features, the gas concentration gradient, the particulate matter concentration gradient, and the temperature probability value to evaluate the continued detection value of the multiple grid cells and generate the detection value of the multiple grid cells.

[0034] Specifically, the precise detection location information carried in the environmental perception data is extracted, such as the latitude, longitude, and altitude coordinates recorded by the sniffing drone's positioning equipment. These location coordinates are then spatially matched with multiple pre-defined grid units in the area to be monitored to determine the specific grid unit to which each piece of environmental perception data belongs. Subsequently, based on the matching results, the detection-related information of each grid unit is statistically analyzed to identify existing detection features, including whether each grid unit has been covered by the sniffing drone (i.e., whether there is corresponding environmental perception data), the number of times it has been detected (e.g., single detection or multiple repeated detections), the time of the most recent detection, to determine the timeliness of the data, and the completeness of the types of data collected (e.g., only gas concentration data or both gas and particulate matter concentration data were collected). Finally, these features clearly distinguish between undetected grids, incompletely detected grids, data-lagging grids, and fully detected grids, providing a basis for subsequent judgments on whether each grid unit needs further detection.

[0035] From the environmental perception data collected by the multimodal sensors of the sniffing drone, the concentration data of combustible gases corresponding to each detection location, such as the concentration of characteristic fire gases like methane and carbon monoxide, and the concentration data of particulate matter, such as the concentration of PM2.5 and PM10 in smoke, are separated. Combined with the detection location information carried by the data, these concentration data are accurately matched to the corresponding grid cells. Subsequently, for each grid cell, a spatial gradient calculation method is used to calculate the rate of change of gas concentration and particulate matter concentration within the grid cell and between the grid cell and adjacent grid cells, based on the ratio of the concentration difference between adjacent detection points to the distance. The magnitude of the rate of change corresponds to the concentration gradient. The larger the gradient, the more drastic the change in gas and particulate matter concentration within or around the grid cell. Finally, through the above calculations, the gas concentration gradient value and particulate matter concentration gradient value of each grid cell are identified and determined, providing data support for subsequent judgment on whether the grid cell is close to a fire source.

[0036] The shared sensing information synchronously transmitted by the reconnaissance drone includes infrared thermal images of the area to be monitored, which record the temperature distribution within the area in pixel form. Subsequently, a spatial mapping relationship is established between the pixel coordinates of the infrared thermal image and the grid cells of the area to be monitored. Coordinate matching accurately maps each pixel in the infrared thermal image to its corresponding grid cell, ensuring that each grid cell is associated with the corresponding thermal image temperature data. Next, a temperature probability analysis algorithm is used to statistically analyze the temperature values ​​of all pixels within each grid cell, calculating the percentage of pixels whose temperature exceeds a preset fire risk temperature threshold (e.g., 50°C above the ambient temperature). This percentage is converted into the probability that a high-temperature area exists within the grid cell, i.e., a temperature probability value, ranging from 0 to 1. A value closer to 1 indicates a higher probability of high temperature and proximity to a fire source within the grid cell. Finally, the temperature probability values ​​of all grid cells are extracted, providing thermal feature data support for the subsequent comprehensive evaluation of the grid cell's continued detection value.

[0037] A multi-dimensional evaluation system was constructed, using existing detection features, gas concentration gradients, particulate matter concentration gradients, and temperature probability values ​​as core evaluation indicators. Existing detection features are used to determine the basic state of grid detection; grids with no detection or lagging detection data are assigned a high base value score, while grids with sufficient detection have a lower base value. Gas and particulate matter concentration gradients reflect the intensity of changes in fire-related substances within the grid; a larger gradient indicates a greater likelihood of proximity to the fire source, resulting in a higher value score. Temperature probability values ​​reflect the high-temperature risk within the grid; a value closer to 1 indicates a higher correlation between high temperature and the fire source, resulting in a higher value score. Subsequently, based on the weighted impact of each indicator on fire source detection, with temperature probability values ​​and concentration gradients having higher weights than the existing detection feature base score, a weighted summation algorithm was used to calculate the scores for each indicator in each grid unit. Finally, the calculation results were converted into standardized values ​​of 0-1, representing the continued detection value of that grid unit, i.e., the grid detection value. This process generates the detection value for all grid units, providing a quantitative basis for subsequent information value map construction and path optimization.

[0038] In one possible implementation, step S334 further includes: Based on the existing detection features, the value distribution of the multiple grid cells is performed to establish a first value distribution, wherein the initial value is proportional to the time during which any grid region has not been detected.

[0039] Based on the gas concentration gradient and the particulate matter concentration gradient, the degree of change in gas concentration and particulate matter concentration in each grid region is analyzed, and a value distribution is established according to the degree of change to create a second value distribution.

[0040] A third value distribution is established based on the temperature probability value.

[0041] The first value distribution, the second value distribution, and the third value distribution are weighted to generate the plurality of grid detection values.

[0042] Specifically, the detection history of each grid area is analyzed. By sniffing the detection timestamps carried by the UAV environmental perception data, the time when each grid area was most recently detected is determined. Then, the time interval from the current moment to the most recent detection, i.e., the time without detection, is calculated. Subsequently, a value function based on this time interval is established to clarify the positive correlation between the initial value and the time without detection. The longer the time without detection, such as a grid that has not updated its detection data for more than 1 hour while another grid has not been detected for only 10 minutes, the worse the timeliness of the environmental perception data of that grid is, the higher the possibility of information gaps or unknown changes in the environmental state, and the higher the corresponding initial value. Conversely, the shorter the time without detection and the newer the data of the grid, the lower the initial value. Finally, the initial value of each grid area calculated based on the time without detection is uniformly mapped and assigned, forming a first value distribution that covers all grid areas and reflects the data timeliness requirements. This provides basic data support for filling information gaps in the subsequent comprehensive evaluation of the grid detection value.

[0043] For each grid area, the acquired combustible gas concentration data, such as methane and carbon monoxide, and particulate matter concentration data, such as PM2.5 and PM10 in smoke, are extracted from environmental sensing data. Combined with concentration data from within and adjacent grids, the gas concentration gradient and particulate matter concentration gradient of that grid are calculated. Then, the magnitude of the gradient values ​​is used to determine the severity of concentration changes; a larger gradient value indicates a more significant spatial change in gas and particulate matter concentration within or around the grid. For example, if the gas concentration gradient of a grid is much higher than its surroundings, it means that the area may be on a critical path for the diffusion of ignition source gases or close to the center of the ignition source. Next, a positive correlation mapping rule between the severity of change and value is established. That is, the more drastic the concentration change in a grid area, the richer the key information it contains, such as the location of the ignition source and the trend of fire spread. The higher the value of the effective information that can be obtained through further detection, the higher the corresponding value score is assigned. Finally, the value scores calculated based on the severity of concentration changes for all grid areas are uniformly assigned, forming a second value distribution focusing on the dynamic changes of fire-related substances. This provides data support for subsequent comprehensive evaluation of the grid detection value, locking in the dimensions of ignition source-related areas.

[0044] The temperature probability value extracted from shared sensing information for each grid area is clearly defined. This value reflects the probability that a temperature range corresponding to an open flame or high-temperature heat source, significantly higher than the ambient temperature, exists within the grid. The value ranges from 0 to 1, with values ​​closer to 1 indicating a higher probability of a high-temperature area within the grid and a stronger association with a fire source. Subsequently, a direct mapping logic between temperature probability value and value is established: the temperature probability value is used as the core calculation basis for the third value distribution, employing a positive correlation assignment rule. That is, grid areas with higher temperature probability values ​​have greater potential to contain key information such as the location and intensity of the fire source, and the higher the value of the effective information that can be obtained through further detection, thus receiving a higher value score. For example, a grid with a temperature probability value of 0.9, indicating a high probability of high temperature, has a significantly higher value score than a grid with a probability value of 0.2, indicating a low probability of high temperature. Finally, all grid areas are assigned value scores based on their respective temperature probability values, forming a third value distribution focusing on the correlation between thermal characteristics and risk.

[0045] Combining the actual needs of fire detection scenarios with the core functions of each value distribution, the weight ratio of the three is determined. For example, in the initial fire assessment stage, the second value distribution, reflecting changes in gas and particulate matter concentrations and related to the diffusion of fire-causing materials, and the third value distribution, reflecting temperature probability and related to high-temperature fire source characteristics, are more crucial for locating the fire source, and therefore are given a higher weight, such as 40% each. The first value distribution, reflecting the duration of non-detection and related to data timeliness, focuses on filling information gaps and is given a relatively lower weight, such as 20%. If there are a large number of long-term undetected grids in the area to be monitored, the weight of the first value distribution can be dynamically increased. Subsequently, for each grid area, its specific scores in the first, second, and third value distributions are extracted and weighted according to preset weights. Finally, the weighted calculation result of each grid area is used as its final grid detection value, completing the generation of all grid detection values. This ensures that priority is given to areas with significant fire risk characteristics while also taking into account the detection needs of data-gap areas, providing a precise quantitative core basis for subsequent information value map construction and path optimization.

[0046] In one possible implementation, step S334 further includes: Based on the first value distribution, the second value distribution, and the third value distribution, a significance analysis of the gas and particulate matter signals is performed to generate significant indices.

[0047] If the significance index is less than a preset significance threshold, the weights of the first value distribution and the third value distribution are greater than the weight of the second value distribution.

[0048] If the significance index is greater than or equal to a preset significance threshold, the weights of the first value distribution and the third value distribution are less than the weight of the second value distribution.

[0049] Specifically, before weighting the first, second, and third value distributions, the weights of each distribution are dynamically adjusted through signal significance analysis to adapt to the core needs of different detection scenarios. First, combining the first, second, and third value distributions, significance analysis is performed on the gas concentration gradient and particulate matter concentration gradient data in the second value distribution. By calculating the deviation amplitude and trend stability of the two types of gradients from the environmental benchmark, quantitative significance indicators are generated. The higher the indicator value, the more obvious the detected gas or particulate matter signal is and the more likely it is to be related to the spread of fire source. Subsequently, the generated significance index is compared with a preset significance threshold: if the significance index is less than the preset threshold, it indicates that the current gas and particulate matter signals are weak and the diffusion characteristics of the ignition source are not obvious. In this case, priority should be given to ensuring data timeliness and high-temperature risk detection. Therefore, the weights of the first and third value distributions are set higher than the weight of the second value distribution, such as the first and third value distributions each accounting for 40%, and the second value distribution accounting for 20%. If the significance index is greater than or equal to the preset threshold, it indicates that significant gas or particulate matter signals have been detected and the ignition source diffusion path characteristics are clear. In this case, it is necessary to guide the sniffing drone to trace the source along the concentration gradient. Therefore, the weights are dynamically adjusted to reduce the weight of the first value distribution. With the third value distribution The weight of the second value distribution is increased. The weights are allocated as follows: the second value distribution accounts for 60%, the first value distribution and the third value distribution each account for 20%, ensuring that detection resources are tilted towards the concentration gradient direction that can accurately locate the fire source, thereby improving the efficiency of fire source tracking.

[0050] In one possible implementation, step S334 further includes: Identify the relative positions of the multiple grid cells and the suspected fire source area.

[0051] Based on the relative positions of the real-time wind direction and speed data, a wind direction distribution centered on the suspected fire source area is established. Grid cells located upwind are assigned a punitive weight for value penalty, while grid cells located downwind are assigned a reward weight for value reward.

[0052] Specifically, the core geographic coordinates of the suspected fire source area are determined, such as the latitude and longitude of the suspected fire source center point determined by reconnaissance drones through infrared thermal imaging and visible light image analysis. Simultaneously, the coordinate information of all grid units already divided in the area to be monitored is retrieved, i.e., the coordinates of the four corner vertices or the center coordinates of each grid unit. Then, using a spatial coordinate calculation algorithm, the coordinates of each grid unit are compared with the core coordinates of the suspected fire source area. First, the straight-line distance between each grid unit and the suspected fire source area is determined; for example, grid A is 200 meters away from the suspected area, and grid B is 50 meters away. Then, combined with geographic orientation determination rules, such as judging east-west and north-south directions based on latitude and longitude differences, the specific orientation of each grid unit relative to the suspected fire source area is determined, such as northeast, southwest, or due south. Finally, the distance and orientation information are integrated to form a complete relative position description of each grid unit, providing a spatial basis for subsequently adjusting the grid detection value weights based on wind direction and speed.

[0053] Centered on the core coordinates of the suspected fire source area, real-time wind direction and speed data are integrated to construct a wind direction distribution model covering the entire monitored area. This model clarifies the wind flow direction and diffusion range, thereby determining the wind direction attribute of each grid unit relative to the suspected fire source area: if the wind blows from a certain grid unit towards the suspected fire source area, that grid unit is the upwind area; if the wind blows from the suspected fire source area towards a certain grid unit, that grid unit is the downwind area. Subsequently, the detection value of grid cells was weighted based on wind direction: for upwind grid cells, the probability of sampling effective fire-related data was low because wind flow would hinder the diffusion of smoke and characteristic gases from the fire into the area; therefore, a penalty weight was assigned, such as multiplying the original grid detection value by a coefficient of 0.5-0.7, thus reducing their detection priority. For downwind grid cells, the pushing effect of the wind would make smoke and characteristic gases more likely to accumulate in the area, resulting in higher sampling efficiency and data validity; therefore, a reward weight was assigned, such as multiplying the original grid detection value by a coefficient of 1.3-1.6, thus increasing their detection priority. This weighting adjustment prioritizes guiding sniffing drones to downwind areas for sampling, significantly improving the efficiency of obtaining key fire information and providing more reliable data support for subsequent precise fire source location.

[0054] In one possible implementation, step S400 further includes: Step S410: Discretize the flight path of the sniffing drone within the preset future planning time into ordered path points to obtain multiple sets of particle position vectors.

[0055] Step S420: Based on the fitness function, calculate the fitness of the multiple sets of particle position vectors, update the particle position vectors, and obtain the optimal detection path with the highest fitness.

[0056] Specifically, by combining the endurance of the sniffing drone, the grid distribution characteristics of the area to be monitored, and the requirements of the detection mission, a preset future planning time is determined, such as 30 minutes or 1 hour, to ensure that the time range is within the effective operating time of the drone. Subsequently, for the flight range that the drone may cover within the planned time, a path discretization strategy is adopted. According to fixed spatial intervals, such as setting a path point every 40 meters or a time interval, such as recording a position point every 15 seconds, the originally continuous flight trajectory is decomposed into a series of spatial coordinate points containing latitude, longitude, and altitude information. These coordinate points strictly follow the flight sequence to form a discretized ordered path. To ensure the diversity of candidate schemes, the above discretization process is repeated to cover different flight directions and different high-value grid combinations, generating multiple sets of different ordered path point sequences. Finally, each set of ordered path point sequences is mathematically expressed in vector form, that is, multiple sets of "particle position vectors". Each particle position vector uniquely corresponds to a candidate flight path. The elements in the vector not only contain the coordinate data of the path points, but also implicitly contain the execution order of the path points, providing diverse initial candidate schemes for subsequent algorithm evaluation and selection of the optimal detection path.

[0057] A fitness function is constructed based on an information value map. For each group of particle position vectors, i.e., a candidate flight path, spatial coordinate matching is first used to determine all grid cells the path passes through on the information value map, including the grid cells directly containing the path points and the grid cells traversed by the path segments. Then, the detection value of each of these traversed grid cells is extracted. The quantitative scores obtained from the previous multi-dimensional evaluation are summed to obtain the "cumulative information value" of the candidate path. This cumulative value is the fitness value of the corresponding particle position vector. The higher the cumulative value, the stronger the potential of the path to obtain key fire-related information. Subsequently, based on the calculated fitness value, the particle position vectors are updated according to the rules of the particle swarm optimization algorithm: on the one hand, the historical best fitness value and corresponding path position of each group of particles are recorded; on the other hand, the global best fitness value and corresponding path position of all particles are calculated. Through a preset speed and position update formula, the coordinates and order of the path points in each particle position vector are adjusted, for example, fine-tuned towards the global best path direction, while retaining the advantage of covering high-value grids, generating a new round of candidate paths. Repeat the above fitness calculation-vector update iterative process until the fitness value no longer increases significantly or reaches the preset number of iterations. Finally, select the particle position vector with the highest fitness value. The ordered path point sequence corresponding to this vector is the optimal detection path of the sniffing drone. This can ensure that the drone maximizes the coverage of high information value areas within the planned time and efficiently obtains fire-related data.

[0058] In one possible implementation, step S400 further includes: The fitness function calculates the fitness value by traversing all grid cells on the information value map through which the particle's position vector passes.

[0059] Specifically, the detection value of each grid cell in the information value map, determined through prior assessment, is defined as a quantitative value score derived from multiple dimensions. This score directly reflects the potential of the grid cell to contain fire-related information. Then, for a candidate flight path corresponding to a given particle position vector, the coordinates of all ordered path points are extracted. Spatial matching is used to determine all grid cells that the path passes through on the information value map, including grids where path points directly fall and grids through which path segments pass. Next, the detection values ​​of these passed grid cells are summed to obtain the cumulative information value of the candidate path. A higher cumulative value indicates that the path can cover more high-value information areas, increasing the likelihood of obtaining key fire data such as characteristic gases and high-temperature signals. Finally, the fitness function directly uses this cumulative information value as the fitness value of the corresponding particle position vector, achieving an intuitive quantification of the candidate path's information acquisition capability. This provides a clear and comparable evaluation standard for subsequent path iteration optimization and optimal path selection.

[0060] Example 2, based on the same inventive concept as the UAV fire source intelligent identification method based on multimodal perception in the previous examples, such as... Figure 2 As shown, this application provides an intelligent fire source identification system for unmanned aerial vehicles (UAVs) based on multimodal perception. The system and method embodiments in this application are based on the same inventive concept. The system includes: The fire source suspected area positioning module 10 is used to acquire visible light and infrared images of the area to be monitored through a reconnaissance drone, and to perform initial positioning of the fire source suspected area.

[0061] The initial path planning module 20 is used to plan an initial path for the sniffing drone to approach the suspected fire source area based on the initial positioning results and real-time wind direction and speed data.

[0062] The value map construction module 30 is used to control the sniffing drone to fly along the initial path and use the onboard multimodal sensors to collect environmental perception data. Combined with the shared perception information synchronously received from the reconnaissance drone, it constructs a dynamic information value map in real time.

[0063] The location information output module 40 is used to optimize the detection path based on the dynamic information value map and with the goal of maximizing information collection efficiency under flight constraints, by using a particle swarm optimization algorithm to control the sniffing drone to fly along the optimal detection path and continue to perform environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, the module outputs fire source confirmation information and location information.

[0064] Furthermore, the system is also used to implement the following functions: The multimodal sensors include gas sensors and particulate matter sensors, and the sniffing drone carries a positioning device.

[0065] Furthermore, the system is also used to implement the following functions: The area to be monitored is gridded to construct multiple grid cells; the environmental sensing data carries detection location information; based on the detection location information, the shared sensing information, and the environmental sensing data, a further detection value analysis is performed on the multiple grid cells to generate multiple grid detection values; the multiple grid detection values ​​are marked to the multiple grid cells to establish the information value map.

[0066] Furthermore, the system is also used to implement the following functions: Based on the detection location information, the existing detection features of the sniffing drone for the multiple grid cells are identified; based on the environmental perception data, the gas concentration gradient and particulate matter concentration gradient of the multiple grid cells are identified; the temperature probability value in the infrared thermal image corresponding to the multiple grid cells is extracted from the shared perception information; the continued detection value of the multiple grid cells is evaluated by combining the existing detection features, the gas concentration gradient, the particulate matter concentration gradient, and the temperature probability value, thereby generating the detection value of the multiple grid cells.

[0067] Furthermore, the system is also used to implement the following functions: Based on the existing detection features, the value distribution of the multiple grid cells is performed to establish a first value distribution, wherein the initial value is proportional to the time during which any grid area has not been detected; based on the gas concentration gradient and the particulate matter concentration gradient, the degree of change in gas concentration and particulate matter concentration in each grid area is analyzed, and the value distribution is performed according to the degree of change to establish a second value distribution; based on the temperature probability value, a third value distribution is established: the first value distribution, the second value distribution, and the third value distribution are weighted to generate the detection value of the multiple grid cells.

[0068] Furthermore, the system is also used to implement the following functions: Based on the first value distribution, the second value distribution, and the third value distribution, a significance analysis of gas and particulate matter signals is performed to generate a significance index. If the significance index is less than a preset significance threshold, the weights of the first value distribution and the third value distribution are greater than the weight of the second value distribution. If the significance index is greater than or equal to the preset significance threshold, the weights of the first value distribution and the third value distribution are less than the weight of the second value distribution.

[0069] Furthermore, the system is also used to implement the following functions: Identify the relative positions of the multiple grid cells and the suspected fire source area; based on the relative positions of the real-time wind direction and speed data, establish a wind direction distribution centered on the suspected fire source area; assign a punitive weight to grid cells in the upwind direction for value penalty, and assign a reward weight to grid cells in the downwind direction for value reward.

[0070] Furthermore, the system is also used to implement the following functions: The flight path of the sniffing drone within the preset future planning time is discretized into ordered path points to obtain multiple sets of particle position vectors; based on the fitness function, the fitness of the multiple sets of particle position vectors is calculated and the particle position vectors are updated to obtain the optimal detection path with the highest fitness.

[0071] Furthermore, the system is also used to implement the following functions: The fitness function calculates the fitness value by traversing all grid cells on the information value map through which the particle's position vector passes.

[0072] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0073] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0074] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and variations fall within the scope of this application and its equivalents, this application intends to include such modifications and variations.

Claims

1. A method for intelligent identification of fire sources from unmanned aerial vehicles (UAVs) based on multimodal perception, characterized in that, include: Visible light and infrared images of the area to be monitored were obtained by reconnaissance drones, and the suspected fire source area was initially located. Based on the initial location results and real-time wind direction and speed data, an initial path is planned for the sniffing drone to approach the suspected fire source area. The sniffing drone is controlled to fly along the initial path and collects environmental perception data using its onboard multimodal sensors. Combined with the shared perception information synchronously received from the reconnaissance drone, a dynamic information value map is constructed in real time. Based on the dynamic information value map, with the goal of maximizing information collection efficiency under flight constraints, the detection path is optimized using a particle swarm optimization algorithm. The sniffing drone is controlled to fly along the optimal detection path and continue to perform environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, the fire source confirmation information and location information are output.

2. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 1, characterized in that, The multimodal sensors include gas sensors and particulate matter sensors, and the sniffing drone carries a positioning device.

3. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 1, characterized in that, The sniffing drone is controlled to fly along the initial path, and environmental perception data is collected using its onboard multimodal sensors. Combined with shared perception information synchronously received from the reconnaissance drone, a dynamic information value map is constructed in real time, including: The area to be monitored is gridded to construct multiple grid cells; The environmental sensing data carries detection location information. Based on the detection location information, the shared sensing information, and the environmental sensing data, a continued detection value analysis is performed on the multiple grid cells to generate multiple grid detection values. The value of the multiple grid detections is marked to the multiple grid cells to establish the information value map.

4. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 3, characterized in that, Based on the detection location information, the shared sensing information, and the environmental sensing data, a continued detection value analysis is performed on the multiple grid cells to generate multiple grid detection values, including: Based on the detection location information, identify the existing detection features of the sniffing drone for the multiple grid cells; Based on the environmental sensing data, the gas concentration gradient and particulate matter concentration gradient of the multiple grid cells are identified; Extract the temperature probability value from the infrared thermal image corresponding to the multiple grid cells from the shared sensing information; The existing detection features, the gas concentration gradient, the particulate matter concentration gradient, and the temperature probability value are combined to evaluate the continued detection value of the multiple grid cells, thereby generating the detection value of the multiple grid cells.

5. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 4, characterized in that, By combining the existing detection features, the gas concentration gradient, the particulate matter concentration gradient, and the temperature probability value, the continued detection value of the multiple grid cells is evaluated, generating the detection value of the multiple grid cells, including: Based on the existing detection features, the value distribution of the multiple grid cells is performed to establish a first value distribution, wherein the initial value is proportional to the time during which any grid region has not been detected. Based on the gas concentration gradient and the particulate matter concentration gradient, the degree of change in gas concentration and particulate matter concentration in each grid region is analyzed, and a value distribution is established according to the degree of change to create a second value distribution. A third value distribution is established based on the aforementioned temperature probability value: The first value distribution, the second value distribution, and the third value distribution are weighted to generate the plurality of grid detection values.

6. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 5, characterized in that, The first value distribution, the second value distribution, and the third value distribution are weighted to generate the plurality of grid detection values, including: Based on the first value distribution, the second value distribution, and the third value distribution, a significance analysis of the gas and particulate matter signals is performed to generate significant indices. If the significance index is less than a preset significance threshold, the weights of the first value distribution and the third value distribution are greater than the weight of the second value distribution. If the significance index is greater than or equal to a preset significance threshold, the weights of the first value distribution and the third value distribution are less than the weight of the second value distribution.

7. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 5, characterized in that, After generating the multiple grid detection values, the method further includes: Identify the relative positions of the multiple grid cells and the suspected fire source area; Based on the relative positions of the real-time wind direction and speed data, a wind direction distribution centered on the suspected fire source area is established. Grid cells located upwind are assigned a punitive weight for value penalty, while grid cells located downwind are assigned a reward weight for value reward.

8. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 1, characterized in that, Based on the dynamic information value map, and with the goal of maximizing information acquisition efficiency under flight constraints, the detection path is optimized using a particle swarm optimization algorithm, including: The flight path of the sniffing drone within the preset future planning time is discretized into ordered path points to obtain multiple sets of particle position vectors. Based on the fitness function, the fitness of the multiple sets of particle position vectors is calculated and the particle position vectors are updated to obtain the optimal detection path with the highest fitness.

9. The intelligent identification method for UAV fire sources based on multimodal perception as described in claim 8, characterized in that, The fitness function calculates the fitness value by traversing all grid cells on the information value map through which the particle's position vector passes.

10. A UAV fire source intelligent identification system based on multimodal perception, characterized in that, The system is used to implement the UAV fire source intelligent identification method based on multimodal perception as described in any one of claims 1-9, the system comprising: The fire source suspected area positioning module is used to acquire visible light and infrared images of the area to be monitored through reconnaissance drones, and to perform initial positioning of the suspected fire source area; The initial path planning module is used to plan an initial path for the sniffing drone to approach the suspected fire source area based on the initial positioning results and real-time wind direction and speed data. The value map construction module is used to control the sniffing drone to fly along the initial path and use the onboard multimodal sensors to collect environmental perception data. Combined with the shared perception information synchronously received from the reconnaissance drone, a dynamic information value map is constructed in real time. The location information output module is used to optimize the detection path based on the dynamic information value map, with the goal of maximizing information collection efficiency under flight constraints, by using a particle swarm optimization algorithm, and control the sniffing drone to fly along the optimal detection path to continue environmental perception. When the obtained environmental perception data meets the preset fire source confirmation conditions, the module outputs fire source confirmation information and location information.

Images