Intelligent low-altitude unmanned aerial vehicle cluster air-ground cooperative detection fire extinguishing system and method

Abstract

The invention discloses an intelligent low-altitude unmanned aerial vehicle cluster air-ground cooperative detection fire extinguishing system and method, a patrol plan is made by determining the patrol time interval of a patrol area of an unmanned aerial vehicle and the density of patrol routes, the system can dynamically adapt to the change of seasons and human activities, and the system and method are more accurate and more efficient. Fire missing detection can be effectively avoided; identifying a fire source by adopting a multi-source data fusion technology, determining whether the fire source exists in the forest region through the infrared image and the actual image of the forest region, and determining the intensity and the position of the fire source according to the smoke concentration when determining that the fire source exists in the forest region. The problem that the accuracy of infrared thermal imaging is affected and the detection precision is low due to the temperature difference of different vegetation, the change of the sunlight irradiation angle, water vapor in the atmosphere and the like can be solved. The fire trend prediction module and the fire extinguishing scheme generation module are adopted to automatically realize fire extinguishing of the forest area, the prediction result is efficient and accurate, and missing of the optimal fire extinguishing opportunity can be avoided.

Description

Technical Field

[0001] The present invention relates to the field of unmanned aerial vehicle technology, and in particular to an intelligent low-altitude unmanned aerial vehicle cluster air-to-ground collaborative fire detection and extinguishing system and method. Background Art

[0002] Forest fire is a very destructive natural disaster. Traditional forest fire fighting methods mainly rely on ground firefighters and equipment, but in some areas with complex terrain and inconvenient transportation, fire fighting work faces huge challenges. With the continuous advancement of science and technology, drone technology has developed rapidly. Drones have the advantages of flexibility and efficiency and have been widely used in many fields. In the field of forest fire fighting, the emergence of drones provides new ideas and methods to solve the problems of traditional fire fighting methods. Drones can ignore terrain obstacles, quickly reach the fire source area, and use the fire extinguishing bombs they carry to extinguish small fire sources in time, thereby effectively preventing the expansion of fires.

[0003] However, the existing drone-based fire-fighting systems still have the following shortcomings: 1. Existing drones detect fire sources by carrying infrared thermal imaging equipment to detect temperature, but the forest environment is complex, and the temperature differences between different vegetation, changes in sunlight angles, and water vapor in the atmosphere will affect the accuracy of infrared thermal imaging, resulting in low detection accuracy; 2. Existing drones detect fires by patrolling, which usually adopts fixed patrol routes and times. Due to the lack of dynamic planning, fires are prone to missed detection during periods of frequent fires; 3. Existing drone fire-fighting systems lack the function of evaluating fires and generating fire-fighting plans based on evaluation results. After receiving a fire warning, humans are required to analyze the data detected by the drone to determine the fire situation and formulate a fire-fighting plan. Since this process is time-consuming, it is easy to miss the best time to extinguish the fire. Summary of the invention

[0004] The present invention discloses an intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system and method to overcome the above-mentioned technical problems.

[0005] In order to achieve the above object, the technical solution of the present invention is:

[0006] An intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system, comprising: a patrol planning module, a data collection module, a fire source identification module, a fire trend prediction module, a fire extinguishing plan generation module and a fire extinguishing plan execution module;

[0007] The data collection module is used to obtain infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data and drone location data in real time through drones;

[0008] The patrol planning module is used to determine the patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculate the comprehensive risk value of the patrol area, so as to determine the time interval for patrolling the patrol area of ​​the drone and the density of the patrol route based on the comprehensive risk value; and patrol the forest area based on the time interval and the density of the patrol route;

[0009] The fire source identification module is used to determine whether there is a fire source in the forest area according to the infrared image of the forest area and the actual image of the forest area, and when it is determined that there is a fire source in the forest area, determine the intensity and location of the fire source according to the smoke concentration;

[0010] The fire trend prediction module is used to obtain the prediction results of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location; including the probability of fire occurrence, direction and speed of fire spread;

[0011] The fire extinguishing scheme generation module is used to obtain multiple fire extinguishing schemes when it is determined that there is a fire source in the forest area according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data and drone location data, fire source intensity and location, fire extinguishing resources, and the prediction results of the fire trend, and then determine the final fire extinguishing scheme based on the fire extinguishing effect, resource consumption and time cost;

[0012] The fire extinguishing scheme execution module is used to extinguish the fire in the forest area according to the final fire extinguishing scheme.

[0013] Further, the patrol planning module includes a patrol area division unit, a regional risk assessment unit and a patrol plan formulation unit;

[0014] The patrol plan formulation unit includes a patrol time allocation subunit and a patrol route planning subunit;

[0015] The patrol area division unit is used to determine the patrol area of ​​the drone based on the topographic data, vegetation data and human activity data of the forest area;

[0016] The regional risk assessment unit is used to obtain a comprehensive risk value of the patrol area of ​​the drone;

[0017] The patrol time allocation subunit is used to determine the time interval for patrolling the patrol area of ​​the drone according to the comprehensive risk value of the patrol area of ​​the drone;

[0018] The patrol route planning subunit is used to determine the density of the patrol route according to the comprehensive risk value of the patrol area of ​​the drone and the patrol time interval.

[0019] Furthermore, the method used to determine the patrol area of ​​the drone is as follows:

[0020] S1: Preprocessing of topographic data, vegetation data and human activity data in forest areas;

[0021] S2: obtaining topographic and geomorphic features, vegetation type features and human activity features according to the pre-processed topographic and geomorphic data, vegetation data and human activity data of the forest area, so as to obtain a feature vector of the patrol area;

[0022] The feature vector of the patrol area is obtained as follows:

[0023] F i =(T i ,V i ,H i )

[0024] in,

[0025] T i =T i1 ×α iT +T i2 ×β iT +T i3 ×γ iT

[0026] V i =V i1 ×α iV +V i2 ×β iV +V i3 ×γ iV

[0027] H i =H i1 ×α iH +H i2 ×β iH +H i3 ×γ iH

[0028] Where: T i represents the topographic features of the i-th patrol area; V i represents the vegetation type characteristics of the i-th patrol area; H i represents the human activity characteristics of the i-th patrol area; T i1 represents the altitude of the ith patrol area; T i2 is the slope of the ith patrol area, T i3is the slope direction of the i-th patrol area; α iT represents the weight coefficient of the altitude of the i-th patrol area; β iT represents the weight coefficient of the slope of the i-th patrol area; γ iT represents the weight coefficient of the slope aspect of the i-th patrol area, and α iT +β iT +γ iT =1; V i1 、V i2 、V i3 Respectively represent the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area; α iV , β iV , γ iV They represent the weight coefficients of the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area, respectively, and α iV +β iV +γ iV =1; H i1 , H i2 , H i3 They represent the population density influencing factor, tourist attraction influencing factor, and traffic route influencing factor of the i-th patrol area respectively; α iH , β iH , γ iH They represent the weight coefficients of the population density influencing factor, the tourist attractions influencing factor, and the traffic route influencing factor of the ith patrol area, respectively, and α iH +β iH +γ iH =1;

[0029] S3: Randomly determine the number of clusters K used to represent the total number of patrol areas, and select K initial cluster centers C1, C2, ..., C K ;

[0030] S4: Obtain the distance between the feature vector of the patrol area and the initial cluster center, so as to assign the i-th patrol area to the category to which the nearest cluster center belongs, obtain a new cluster, and then obtain the cluster center of the new cluster;

[0031] S5: Based on the cluster center of the new cluster, S4 is repeatedly executed until the termination condition is met; the final cluster is obtained to determine the patrol area of ​​the drone.

[0032] Furthermore, the formula used to obtain the comprehensive risk value of the patrol area of ​​the drone is as follows:

[0033]

[0034] Where: R represents the comprehensive risk value of the patrol area of ​​the drone; S is the seasonal and climate change index, S = α1×Q1+α2×Q2+α3×Q3+α4×Q4, Q1 is the temperature change index, Q2 is the humidity change index, Q3 is the precipitation change index, Q4 is the wind direction change index, α1, α2, α3, α4 are the weight coefficients of the temperature change index, humidity change index, precipitation change index, and wind direction change index, respectively, and α1+α2+α3+α4=1; T r is the topographic risk index, T r =T×α T , T is the topographic features, α T is the weight coefficient of topographic features; V r is the vegetation type risk index, V r =V×α V , V is the vegetation type characteristic, α V is the weight coefficient of vegetation type characteristics; H r is the risk index of human activities, H r =H×α H , H is the human activity characteristic, α H is the weight coefficient of human activity characteristics; ω S , are the weight coefficients of seasonal and climate change indicators, topographic and geomorphic characteristics, vegetation type risk indicators and human activity risk indicators, and

[0035] Further, the data collection module includes: an infrared image data acquisition unit, an actual image data acquisition unit, a smoke data acquisition unit, a position data acquisition unit, a meteorological data acquisition unit and a regional data acquisition unit;

[0036] The infrared image data acquisition unit is used to acquire a plurality of infrared images of the forest area;

[0037] The actual image data acquisition unit is used to obtain a plurality of actual images of the forest area;

[0038] Wherein, the plurality of infrared images are infrared images of different locations in the forest area; the plurality of actual images are actual images of different locations in the forest area;

[0039] The smoke data acquisition unit is used to obtain the smoke concentration in the air of the forest area;

[0040] The meteorological data acquisition unit is used to acquire meteorological data of the forest area, including ambient temperature, ambient humidity, wind direction, and wind speed;

[0041] The position data acquisition unit is used to obtain the position data of the drone;

[0042] The regional data acquisition unit is used to acquire topographic data, vegetation data, climate data and human activity data of the forest area.

[0043] Furthermore, the fire source identification module includes an infrared image processing unit, an actual image processing unit, and a data fusion unit;

[0044] The infrared image processing unit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area;

[0045] The actual image processing unit is used to determine whether there is open fire and smoke in the forest area according to the actual image and smoke density of the forest area;

[0046] The data fusion unit is used to determine the intensity and location of the fire source in the forest area according to whether there is open fire and smoke in the forest area when it is determined that there is a fire source in the forest area.

[0047] Furthermore, the fire source identification module also includes an algorithm adjustment unit;

[0048] The algorithm adjustment unit is used to adjust the algorithm parameters of the actual image processing unit according to meteorological data.

[0049] Furthermore, the infrared image processing unit includes a multi-frame image fusion subunit, a background subtraction subunit and a classifier subunit;

[0050] The multi-frame image fusion subunit is used to splice the infrared images of different positions in the forest area to obtain the spliced ​​infrared image of the forest area;

[0051] The background subtraction subunit is used to perform background subtraction processing on the spliced ​​infrared image of the forest area to obtain the infrared image of the forest area after the background is subtracted;

[0052] The classifier subunit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area after subtracting the background;

[0053] The actual image processing unit includes an open flame recognition subunit and a smoke recognition subunit;

[0054] The open fire identification subunit is used to determine whether there is an open fire in the forest area;

[0055] The smoke identification subunit is used to determine whether smoke exists in the forest area according to the smoke concentration.

[0056] Furthermore, the fire extinguishing scheme generation module includes an information integration unit, a scheme generation unit and an evaluation and optimization unit;

[0057] The information integration unit is used to obtain integrated fire extinguishing data based on the prediction results of fire trends, fire extinguishing resources, and water source distribution when it is determined that there is a fire source in the forest area; the fire extinguishing resources include the number of drones, the number of fire extinguishing bombs, and the capacity of fire extinguishing agents;

[0058] The solution generation unit is used to obtain multiple fire extinguishing solutions based on the integrated fire extinguishing data;

[0059] The evaluation and optimization unit is used to determine the final fire extinguishing plan based on the fire extinguishing effect, resource consumption and time cost.

[0060] A fire extinguishing method of an intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system comprises the following steps:

[0061] S101: Acquire infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data and drone location data;

[0062] S102: Determine the patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculate the comprehensive risk value of the patrol area, so as to determine the time interval for patrolling the patrol area of ​​the drone and the density of the patrol route based on the comprehensive risk value; and patrol the forest area based on the time interval and the density of the patrol route;

[0063] S103: determining whether there is a fire source in the forest area according to the real-time infrared image of the forest area and the real-time actual image of the forest area, and when it is determined that there is a fire source in the forest area, determining the intensity and location of the fire source according to the smoke density;

[0064] S104: Obtaining a prediction result of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location;

[0065] S105: Obtain multiple fire extinguishing plans based on ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data, drone location data, fire source intensity and location, fire extinguishing resources, and fire trend prediction results, and then determine the final fire extinguishing plan based on fire extinguishing effect, resource consumption, and time cost;

[0066] S106: Extinguish the fire in the forest area according to the final fire extinguishing plan.

[0067] Beneficial effects: The present invention provides an intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system and method, wherein the patrol planning module divides the patrol area based on topography, vegetation type and human activity, and determines the time interval for patrolling the patrol area of ​​the UAV and the density of the patrol route based on the comprehensive risk value, so as to formulate a patrol plan, which can dynamically adapt to changes in seasons and human activities, is more accurate, and can effectively avoid missed fires; at the same time, the present invention adopts multi-source data fusion technology to identify fire sources, and determines whether there is a fire source in the forest area through infrared images and actual images of the forest area, and when it is determined that there is a fire source in the forest area, determines the fire source intensity and location according to the smoke concentration, which can solve the problem of low detection accuracy caused by temperature differences of different vegetation, changes in sunlight angles and water vapor in the atmosphere, etc.; at the same time, the present invention adopts a fire trend prediction module and a fire extinguishing plan generation module to intelligently determine the final fire extinguishing plan to extinguish the forest area, which can avoid missing the best time to extinguish the fire. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0069] Figure 1 It is a structural block diagram of the fire extinguishing system of the present invention;

[0070] Figure 2 It is a structural block diagram of a data collection module in an embodiment of the present invention;

[0071] Figure 3 is a structural block diagram of a fire source identification module in an embodiment of the present invention;

[0072] Figure 4 It is a structural block diagram of a patrol planning module in an embodiment of the present invention;

[0073] Figure 5 A structural block diagram of a fire extinguishing scheme generation module in an embodiment of the present invention;

[0074] Figure 6 is a flow chart of a genetic algorithm of an evaluation and optimization unit in an embodiment of the present invention;

[0075] Figure 7 A flow chart of a K-Means clustering algorithm for a patrol area division unit in an embodiment of the present invention;

[0076] Figure 8It is a flow chart of the application method in an embodiment of the present invention. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0078] This embodiment introduces an intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system. Figure 1 As shown, it includes a data collection module, a data storage module, a fire source identification module, a patrol planning module, a fire trend prediction module, a fire extinguishing plan generation module and a fire extinguishing plan execution module;

[0079] The data collection module is used to obtain infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data and drone location data when using drones to patrol forest areas;

[0080] Preferably, if Figure 2 As shown, the data collection module includes: an infrared image data acquisition unit, an actual image data acquisition unit, a smoke data acquisition unit, a position data acquisition unit, a meteorological data acquisition unit and a regional data acquisition unit;

[0081] The infrared image data acquisition unit is used to acquire a plurality of infrared images of the forest area;

[0082] The actual image data acquisition unit is used to obtain a plurality of actual images of the forest area;

[0083] Wherein, the plurality of infrared images are infrared images of different locations in the forest area; the plurality of actual images are actual images of different locations in the forest area;

[0084] The smoke data acquisition unit is used to obtain the smoke concentration in the air of the forest area;

[0085] The meteorological data acquisition unit is used to acquire meteorological data of the forest area, including ambient temperature, ambient humidity, wind direction, and wind speed;

[0086] The position data acquisition unit is used to obtain the position data of the drone;

[0087] The regional data acquisition unit is used to acquire topographic data, vegetation data, climate data and human activity data of the forest area.

[0088] Specifically, the infrared image data acquisition unit of this embodiment uses an infrared thermal imaging camera to collect infrared images of forest areas; the actual image data acquisition unit uses a visible light camera to collect actual images of forest areas; the smoke data acquisition unit uses a smoke sensor to detect the smoke concentration in the air of the forest area; the position data acquisition unit is used to report the position data to the position data acquisition unit through a 5G communication link based on the GPS positioning module integrated in the drone, thereby obtaining the collected drone position data; the meteorological data acquisition unit is used to obtain temperature, humidity, wind direction and wind speed data. In one embodiment of the present invention, the detection drone is equipped with a meteorological detector, and through The meteorological data is reported to the meteorological data acquisition unit through the 5G communication link; in another embodiment, multiple micro-meteorological stations are deployed in the controlled forest area to cover the area, and the meteorological stations collect data and report it to the meteorological data acquisition unit; the regional data acquisition unit is used to obtain topographic data, vegetation data, climate data and human activity data in the forest area. For example, topographic data can be obtained through the national geographic information system or satellite remote sensing, vegetation data can be obtained through satellite remote sensing, climate data can be obtained from the meteorological information center, and human activity data is where there are roads or settlements. This can be obtained from local governments or actual exploration. All modules used are conventional modules in the field. This embodiment only uses them to implement the functions of this embodiment, and is not described in detail here.

[0089] The patrol planning module is used to determine the patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculate the comprehensive risk value of the patrol area, so as to determine the time interval for patrolling the patrol area of ​​the drone and the density of the patrol route based on the comprehensive risk value; and patrol the forest area based on the time interval and the density of the patrol route;

[0090] Preferably, if Figure 4 As shown, the patrol planning module includes a patrol area division unit, a regional risk assessment unit and a patrol plan formulation unit;

[0091] The patrol plan formulation unit includes a patrol time allocation subunit and a patrol route planning subunit;

[0092] The patrol area division unit is used to determine the patrol area of ​​the drone based on the topographic data, vegetation data and human activity data of the forest area;

[0093] Specifically, the patrol area division unit of this embodiment adopts the K-Means clustering algorithm to determine the patrol area of ​​the drone based on the topographic data, vegetation data and human activity data of the forest area, such as Figure 7 shown.

[0094] Preferably, the method used to determine the patrol area of ​​the drone is as follows:

[0095] S1. Preprocess the topographic data, vegetation data and human activity data of forest areas;

[0096] Specifically, in this embodiment, the topographic data, vegetation data and human activity data of the forest area are preprocessed, and conventional data preprocessing methods such as data cleaning, normalization and standardization are used to improve the data quality so that it has a unified data format and can be used by the algorithm.

[0097] S2: obtaining topographic and geomorphic features, vegetation type features and human activity features according to the pre-processed topographic and geomorphic data, vegetation data and human activity data of the forest area, so as to obtain a feature vector of the patrol area;

[0098] The feature vector of the patrol area is obtained as follows:

[0099] F i =(T i ,V i ,H i )

[0100] in,

[0101] T i =T i1 ×α iT +T i2 ×β iT +T i3 ×γ iT

[0102] V i =V i1 ×α iV +V i2 ×β iV +V i3 ×γ iV

[0103] H i =H i1 ×α iH +H i2 ×β iH +H i3 ×γ iH

[0104] Where: Ti represents the topographic features of the i-th patrol area; V i represents the vegetation type characteristics of the i-th patrol area; H i represents the human activity characteristics of the i-th patrol area; T i1 represents the altitude of the ith patrol area; T i2 is the slope of the ith patrol area, T i3 is the slope direction of the i-th patrol area; α iT represents the weight coefficient of the altitude of the i-th patrol area; β iT represents the weight coefficient of the slope of the i-th patrol area; γ iT represents the weight coefficient of the slope aspect of the i-th patrol area, and α iT +β iT +γ iT =1; V i1 、V i2 、V i3 Respectively represent the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area; α iV , β iV , γ iV They represent the weight coefficients of the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area, respectively, and α iV +β iV +γ iV =1; H i1 , H i2 , H i3 They represent the population density influencing factor, tourist attraction influencing factor, and traffic route influencing factor of the i-th patrol area respectively; α iH , β iH , γ iH They represent the weight coefficients of the population density influencing factor, the tourist attractions influencing factor, and the traffic route influencing factor of the ith patrol area, respectively, and α iH +β iH +γ iH =1;

[0105] S3: Randomly determine the number of clusters K used to represent the total number of patrol areas, and select K initial cluster centers C1, C2, ..., C K ;

[0106] Specifically, the determined number of clusters is the number of patrol areas finally divided, and the purpose is to integrate regional units with similar characteristics (conditions) into one area.

[0107] S4: Obtain the distance between the feature vector of the patrol area and the initial cluster center, so as to assign the i-th patrol area to the category to which the nearest cluster center belongs, obtain a new cluster, and then obtain the cluster center of the new cluster;

[0108] Specifically, for each patrol area, calculate its feature vector F i =(T i ,V i ,H i ) to each cluster center d(F i ,C k ) is a conventional technique in the field and will not be described in detail here. Specifically, the method for obtaining a new cluster center is to obtain the average value of the feature vectors of all patrol areas in the new cluster.

[0109] S5: Based on the cluster center of the new cluster, S4 is repeatedly executed until the termination condition is met; the final cluster is obtained to determine the patrol area of ​​the drone.

[0110] Specifically, the termination condition in this embodiment is that the position of the new cluster center does not change, or a preset number of iterations is reached.

[0111] Specifically, the cluster centers can represent the typical feature points of each patrol area. In the subsequent patrol planning, these centers are used as the identification of each area to facilitate the distinction and management of different areas.

[0112] The regional risk assessment unit is used to obtain a comprehensive risk value of the patrol area of ​​the drone;

[0113] Preferably, the formula used to obtain the comprehensive risk value of the patrol area of ​​the drone is as follows:

[0114]

[0115] Where: R represents the comprehensive risk value of the patrol area of ​​the drone; S is the seasonal and climate change index, S = α1×Q1+α2×Q2+α3×Q3+α4×Q4, Q1 is the temperature change index, Q2 is the humidity change index, Q3 is the precipitation change index, Q4 is the wind direction change index, α1, α2, α3, α4 are the weight coefficients of the temperature change index, humidity change index, precipitation change index, and wind direction change index, respectively, and α1+α2+α3+α4=1; T r is the topographic risk index, T r =T×α T , T is the topographic features, α T is the weight coefficient of topographic features; V r is the vegetation type risk index, V r =V×αV , V is the vegetation type characteristic, α V is the weight coefficient of vegetation type characteristics; H r is the risk index of human activities, H r =H×α H , H is the human activity characteristic, α H is the weight coefficient of human activity characteristics; ω S , are the weight coefficients of seasonal and climate change indicators, topographic and geomorphic characteristics, vegetation type risk indicators and human activity risk indicators, and

[0116] Specifically, the weight coefficient of each indicator is obtained by technical personnel in the field based on the evaluation of the fire impact status of each indicator, and will not be described in detail here.

[0117] The patrol time allocation subunit is used to determine the time interval for patrolling the patrol area of ​​the drone according to the comprehensive risk value of the patrol area of ​​the drone;

[0118] The patrol route planning subunit is used to determine the density of the patrol route according to the comprehensive risk value of the patrol area of ​​the drone and the patrol time interval;

[0119] Specifically, high-risk areas require more frequent patrols to ensure that fire hazards can be discovered in time and countermeasures can be taken. Therefore, the higher the comprehensive risk value of the area, the more patrol time should be allocated. In this embodiment, a corresponding relationship table between the comprehensive risk value and the patrol time is set. Based on the comprehensive risk value, the patrol area is divided into high-risk areas, medium-risk areas and local priority areas. The division method is a conventional technology in the field and is not described in detail here. Then, the high-risk area is patrolled once an hour, the medium-risk area is patrolled once every two hours, and the low-risk area is patrolled once every three hours.

[0120] Specifically, the method of determining the density of the patrol route according to the comprehensive risk value of the patrol area of ​​the drone and the patrol time interval in this embodiment is a prior art and will not be described in detail here.

[0121] The fire source identification module is used to determine whether there is a fire source in the forest area according to the infrared image of the forest area and the actual image of the forest area, and when it is determined that there is a fire source in the forest area, determine the intensity and location of the fire source according to the smoke density; and then adjust the algorithm parameters of the fire source identification module according to the meteorological data;

[0122] Preferably, if Figure 3 As shown, the fire source identification module includes an infrared image processing unit, an actual image processing unit, a data fusion unit and an algorithm adjustment unit;

[0123] The infrared image processing unit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area;

[0124] The actual image processing unit is used to determine whether there is open fire and smoke in the forest area according to the actual image and smoke density of the forest area;

[0125] The data fusion unit is used to determine the intensity and location of the fire source in the forest area according to whether there is open fire and smoke in the forest area when it is determined that there is a fire source in the forest area;

[0126] Specifically, since searching for open flames and smoke in actual images may not be accurate, it may not be recognized or the accuracy is low due to vegetation obstruction, weather factors, etc.; therefore, this embodiment determines the fire source through infrared images and actual images to ensure accurate identification of the fire source.

[0127] Preferably, the fire source identification module further includes an algorithm adjustment unit;

[0128] The algorithm adjustment unit is used to adjust the algorithm parameters of the actual image processing unit according to meteorological data.

[0129] Specifically, the actual image processing unit of this embodiment uses an algorithm capable of self-learning to determine whether there is open fire and smoke in the forest area. During the calculation process, the algorithm parameters need to be continuously adjusted.

[0130] Preferably, the infrared image processing unit includes a multi-frame image fusion subunit, a background subtraction subunit and a classifier subunit;

[0131] The actual image processing unit includes an open flame recognition subunit and a smoke recognition subunit;

[0132] The multi-frame image fusion subunit is used to splice the infrared images of different positions in the forest area to obtain the spliced ​​infrared image of the forest area;

[0133] The background subtraction subunit is used to perform background subtraction processing on the spliced ​​infrared image of the forest area to obtain the infrared image of the forest area after the background is subtracted;

[0134] The classifier subunit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area after subtracting the background; specifically, the method used by the classifier subunit to distinguish between fire sources and non-fire sources is a prior art in the field and is not described in detail here.

[0135] The open fire identification subunit is used to determine whether an open fire exists in a forest area by using an image recognition algorithm;

[0136] The smoke recognition subunit is used to determine whether there is smoke in the forest area according to the smoke concentration using an image recognition algorithm;

[0137] Specifically, the fire source recognition module of this embodiment uses machine learning algorithms, such as neural networks, to train a large number of fire source images annotated with meteorological data. The trained model can automatically learn and adjust the parameters of the fire source recognition algorithm according to the input meteorological data to achieve adaptive parameter optimization. The purpose is to remove the interference caused by meteorological factors and improve the robustness of the algorithm. This solves the problem that the existing fixed parameter image recognition algorithm has low recognition accuracy of images collected in different environments.

[0138] Specifically, in this embodiment, whether there is a fire source in the forest area is determined based on the infrared image of the forest area, whether there is an open flame and smoke in the forest area is determined based on the actual image of the forest area, and whether there is a fire source in the forest area is determined based on the results output by the infrared image processing unit and the actual image processing unit. The methods used are all conventional technologies and are not described in detail here.

[0139] Specifically, the open flame identification subunit and the smoke identification subunit are both utilizations of existing technologies and will not be described in detail here.

[0140] The fire trend prediction module is used to obtain the prediction results of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location; including the probability of fire occurrence, direction and speed of fire spread;

[0141] Specifically, the fire trend prediction module in this embodiment uses a neural network to predict fire trends. The input features of the neural network include wind direction, wind speed, temperature, humidity, fire source location, fire source intensity, slope, slope direction and vegetation combustion coefficient. The output is the direction and speed of fire spread. The specific method is the existing technology in the field and is not described in detail here.

[0142] The fire extinguishing scheme generation module is used to obtain multiple fire extinguishing schemes when it is determined that there is a fire source in a forest area, based on the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data and drone position data, fire source intensity and fire source location, and fire extinguishing resources (number of drones, number of fire extinguishing bombs, capacity of fire extinguishing agent), and the prediction results of fire trends, and then determine the final fire extinguishing scheme based on the fire extinguishing effect, resource consumption and time cost.

[0143] Specifically, the drone position can be used to determine the location of the fire source, and the trajectory formed by the drone position can be regarded as a patrol path, which can be used to analyze the coverage of the patrol area and subsequent dynamic patrol path planning.

[0144] Preferably, if Figure 5 As shown, the fire extinguishing scheme generation module includes an information integration unit, a scheme generation unit and an evaluation and optimization unit;

[0145] The information integration unit is used to obtain integrated fire extinguishing data based on the prediction results of fire trends, fire extinguishing resources, and water source distribution when it is determined that there is a fire source in the forest area; the fire extinguishing resources include the number of drones, the number of fire extinguishing bombs, and the capacity of fire extinguishing agents;

[0146] The solution generation unit is used to obtain multiple fire extinguishing solutions based on the integrated fire extinguishing data;

[0147] The evaluation and optimization unit is used to determine the final fire extinguishing plan based on the fire extinguishing effect, resource consumption and time cost.

[0148] Specifically, the prediction results of the fire trend in this embodiment include the probability of fire occurrence, the direction and speed of fire spread. Combined with the topography, vegetation type, water source distribution and other information of the forest area, the available fire-fighting resources are considered, such as the number of drones, the number of fire-fighting bombs, the capacity of fire-fighting agents, etc. Based on the integrated information, a variety of possible fire-fighting schemes are generated. For example, in the case of a small fire, a small number of drones are dispatched to carry fire-fighting bombs for precise delivery; in the case of a large fire, multiple drones may be required to work together, and at the same time, ground fire-fighting forces are combined to extinguish the fire. Among them, the methods used to obtain multiple fire-fighting schemes and determine the final fire-fighting scheme are conventional technologies for those skilled in the art and are not described in detail here.

[0149] The fire extinguishing scheme execution module is used to extinguish the fire in the forest area according to the final fire extinguishing scheme. Specifically, it controls the drone cluster to extinguish the fire and notifies the ground fire extinguishing personnel to rush to the fire scene.

[0150] Specifically, the fire extinguishing system of this embodiment also includes a data storage module, which is respectively connected to the data collection module, the fire source identification module, the patrol planning module, the fire trend prediction module, the fire extinguishing plan generation module and the fire extinguishing plan execution module to store process data, which can facilitate technical personnel in the field to analyze and study the fire conditions in forest areas.

[0151] This embodiment also discloses a fire extinguishing method of an intelligent low-altitude UAV cluster air-to-ground collaborative fire extinguishing system, including the following steps: Figure 8 As shown:

[0152] S101: Determine the patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculate the comprehensive risk value of the patrol area, so as to determine the time interval for patrolling the patrol area of ​​the drone and the density of the patrol route based on the comprehensive risk value; and patrol the forest area based on the time interval and the density of the patrol route;

[0153] S102: Acquire infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data, and drone location data;

[0154] S103: determining whether there is a fire source in the forest area according to the infrared image of the forest area and the actual image of the forest area, and when it is determined that there is a fire source in the forest area, determining the intensity and location of the fire source according to the smoke density;

[0155] S104: Obtaining a prediction result of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location;

[0156] S105: Obtain multiple fire extinguishing plans based on ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data, drone location data, fire source intensity and location, fire extinguishing resources, and fire trend prediction results, and then determine the final fire extinguishing plan based on fire extinguishing effect, resource consumption, and time cost;

[0157] Specifically, the final fire extinguishing scheme in this embodiment is obtained by using a genetic algorithm based on the evaluation optimization unit, and the fitness function of the genetic algorithm is: F = ω1×E1+ω2×E2+ω3×E3, E1 is the fire extinguishing effect index, E2 is the resource consumption index, E3 is the time cost index, ω1, ω2, ω3 are the weight coefficients of each index, and ω1+ω2+ω3=1, such as Figure 6 shown.

[0158] S106: Extinguish the fire in the forest area according to the final fire extinguishing plan.

[0159] Specifically, in this embodiment, when patrolling a forest area, the drone uses a data collection module to collect infrared image data, actual image data, and smoke data during the patrol process, and hands them over to the fire source identification module for fire source identification; after the fire source identification module identifies the fire source, the fire trend prediction module predicts the fire trend based on the fire source data (fire source size and range), meteorological data, and geographic data; and then generates a fire extinguishing plan based on the predicted results, and the optimal one is handed over to the fire extinguishing plan execution module for execution.

[0160] Beneficial effects: The present invention provides an intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system and method, wherein the patrol planning module adopts a clustering algorithm to divide the patrol area based on topography, vegetation type and human activity, and determines the time interval for patrolling the patrol area of ​​the UAV and the density of the patrol route based on the comprehensive risk value, so as to formulate a patrol plan, which can dynamically adapt to changes in seasons and human activities, is more accurate, and can effectively avoid missed fires; the fire trend prediction module adopts a neural network to predict fire trends, and the fire extinguishing plan generation module adopts a genetic algorithm to obtain the most suitable fire extinguishing plan based on the prediction results, which has the advantages of high efficiency and accuracy, and can avoid missing the best fire extinguishing opportunity. At the same time, the present invention adopts multi-source data fusion technology to identify the fire source, and determines whether there is a fire source in the forest area through the infrared image and the actual image of the forest area. When it is determined that there is a fire source in the forest area, the fire source intensity and the fire source location are determined according to the smoke concentration. This can solve the problem that the accuracy of infrared thermal imaging is affected by the temperature difference of different vegetation, the change of sunlight angle and the water vapor in the atmosphere, resulting in low detection accuracy. At the same time, the present invention adopts a fire trend prediction module to use a neural network to predict the fire trend, and uses a fire extinguishing plan generation module to use a genetic algorithm to determine the final fire extinguishing plan to extinguish the forest area. The prediction result is efficient and accurate, and can avoid missing the best time to extinguish the fire.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system, characterized in that: include: Patrol planning module, data collection module, fire source identification module, fire trend prediction module, fire extinguishing plan generation module and fire extinguishing plan execution module; The data collection module is used to obtain infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data and drone location data in real time through drones; The patrol planning module is used to determine the patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculate the comprehensive risk value of the patrol area, so as to determine the time interval for patrolling the patrol area of ​​the drone and the density of the patrol route based on the comprehensive risk value; patrolling forest areas at a density based on the stated time intervals and patrol routes; The fire source identification module is used to determine whether there is a fire source in the forest area according to the infrared image of the forest area and the actual image of the forest area, and when it is determined that there is a fire source in the forest area, determine the intensity and location of the fire source according to the smoke concentration; The fire trend prediction module is used to obtain the prediction result of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location; Including the probability of fire occurrence, direction and speed of fire spread; The fire extinguishing scheme generation module is used to obtain multiple fire extinguishing schemes when it is determined that there is a fire source in the forest area according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data and drone location data, fire source intensity and location, fire extinguishing resources, and the prediction results of the fire trend, and then determine the final fire extinguishing scheme based on the fire extinguishing effect, resource consumption and time cost; The fire extinguishing scheme execution module is used to extinguish the fire in the forest area according to the final fire extinguishing scheme.

2. According to claim 1, the intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system is characterized in that: The patrol planning module includes a patrol area division unit, a regional risk assessment unit and a patrol plan formulation unit; The patrol plan formulation unit includes a patrol time allocation subunit and a patrol route planning subunit; The patrol area division unit is used to determine the patrol area of ​​the drone based on the topographic data, vegetation data and human activity data of the forest area; The regional risk assessment unit is used to obtain a comprehensive risk value of the patrol area of ​​the drone; The patrol time allocation subunit is used to determine the time interval for patrolling the patrol area of ​​the drone according to the comprehensive risk value of the patrol area of ​​the drone; The patrol route planning subunit is used to determine the density of the patrol route according to the comprehensive risk value of the patrol area of ​​the drone and the patrol time interval.

3. According to claim 2, the intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system is characterized in that: The method used to determine the patrol area of ​​the drone is as follows: S1: Preprocessing of topographic data, vegetation data and human activity data in forest areas; S2: obtaining topographic and geomorphic features, vegetation type features and human activity features according to the pre-processed topographic and geomorphic data, vegetation data and human activity data of the forest area, so as to obtain a feature vector of the patrol area; The feature vector of the patrol area is obtained as follows: F i =(T i ,V i ,H i ) in, T i =T i1 ×α iT +T i2 ×β iT +T i3 ×γ iT V i =V i1 ×α iV +V i2 ×β iV +V i3 ×γ iV H i =H i1 ×α iH +H i2 ×β iH +H i3 ×γ iH Where: T i represents the topographic features of the i-th patrol area; V i represents the vegetation type characteristics of the i-th patrol area; H i represents the human activity characteristics of the i-th patrol area; T i1 represents the altitude of the ith patrol area; T i2 is the slope of the ith patrol area, T i3 is the slope direction of the i-th patrol area; α iT represents the weight coefficient of the altitude of the i-th patrol area; β iT represents the weight coefficient of the slope of the i-th patrol area; γ iT represents the weight coefficient of the slope aspect of the i-th patrol area, and α iT +β iT +γ iT =1; V i1 、V i2 、V i3 Respectively represent the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area; α iV , β iV , γ iV They represent the weight coefficients of the coniferous forest characteristics, broad-leaved forest characteristics, and mixed forest characteristics of the i-th patrol area, respectively, and α iV +β iV +γ iV =1; H i1 , H i2 , H i3 They represent the population density influencing factor, tourist attraction influencing factor, and traffic route influencing factor of the i-th patrol area respectively; α iH , β iH , γ iH They represent the weight coefficients of the population density influencing factor, the tourist attractions influencing factor, and the traffic route influencing factor of the ith patrol area, respectively, and α iH +β iH +γ iH =1; S3: Randomly determine the number of clusters K used to represent the total number of patrol areas, and select K initial cluster centers C1, C2, ..., C K ; S4: Obtain the distance between the feature vector of the patrol area and the initial cluster center, so as to assign the i-th patrol area to the category to which the nearest cluster center belongs, obtain a new cluster, and then obtain the cluster center of the new cluster; S5: Based on the cluster center of the new cluster, S4 is repeatedly executed until the termination condition is met; the final cluster is obtained to determine the patrol area of ​​the drone.

4. According to claim 2, the intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system is characterized in that: The formula used to obtain the comprehensive risk value of the patrol area of ​​the drone is as follows: Where: R represents the comprehensive risk value of the patrol area of ​​the drone; S is the seasonal and climate change index, S = α1×Q1+α2×Q2+α3×Q3+α4×Q4, Q1 is the temperature change index, Q2 is the humidity change index, Q3 is the precipitation change index, Q4 is the wind direction change index, α1, α2, α3, α4 are the weight coefficients of the temperature change index, humidity change index, precipitation change index, and wind direction change index, respectively, and α1+α2+α3+α4=1; T r is the topographic risk index, T r =T×α T , T is the topographic features, α T is the weight coefficient of topographic features; V r is the vegetation type risk index, V r =V×α V , V is the vegetation type characteristic, α V is the weight coefficient of vegetation type characteristics; H r is the risk index of human activities, H r =H×α H , H is the human activity characteristic, α H is the weight coefficient of human activity characteristics; ω S ,ω Tr ,ω Vr ,ω Hr are the weight coefficients of seasonal and climate change indicators, topographic and geomorphic characteristics, vegetation type risk indicators and human activity risk indicators, and there is ω S +ω Tr +ω Vr +ω Hr =1.

5. According to claim 1, the intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system is characterized in that: The data collection module includes: an infrared image data acquisition unit, an actual image data acquisition unit, a smoke data acquisition unit, a position data acquisition unit, a meteorological data acquisition unit and a regional data acquisition unit; The infrared image data acquisition unit is used to acquire a plurality of infrared images of the forest area; The actual image data acquisition unit is used to obtain a plurality of actual images of the forest area; Wherein, the plurality of infrared images are infrared images of different locations in the forest area; the plurality of actual images are actual images of different locations in the forest area; The smoke data acquisition unit is used to obtain the smoke concentration in the air of the forest area; The meteorological data acquisition unit is used to acquire meteorological data of the forest area, including ambient temperature, ambient humidity, wind direction, and wind speed; The position data acquisition unit is used to obtain the position data of the drone; The regional data acquisition unit is used to acquire topographic data, vegetation data, climate data and human activity data of the forest area.

6. According to claim 1, the intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system is characterized in that: The fire source identification module includes an infrared image processing unit, an actual image processing unit, and a data fusion unit; The infrared image processing unit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area; The actual image processing unit is used to determine whether there is open fire and smoke in the forest area according to the actual image and smoke density of the forest area; The data fusion unit is used to determine the intensity and location of the fire source in the forest area according to whether there is open fire and smoke in the forest area when it is determined that there is a fire source in the forest area.

7. The intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system according to claim 1 is characterized in that: The fire source identification module also includes an algorithm adjustment unit; The algorithm adjustment unit is used to adjust the algorithm parameters of the actual image processing unit according to meteorological data.

8. The intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system according to claim 6 is characterized in that: The infrared image processing unit includes a multi-frame image fusion subunit, a background subtraction subunit and a classifier subunit; The multi-frame image fusion subunit is used to splice the infrared images of different positions in the forest area to obtain the spliced ​​infrared image of the forest area; The background subtraction subunit is used to perform background subtraction processing on the spliced ​​infrared image of the forest area to obtain the infrared image of the forest area after the background is subtracted; The classifier subunit is used to determine whether there is a fire source in the forest area based on the infrared image of the forest area after subtracting the background; The actual image processing unit includes an open flame recognition subunit and a smoke recognition subunit; The open fire identification subunit is used to determine whether there is an open fire in the forest area; The smoke identification subunit is used to determine whether smoke exists in the forest area according to the smoke concentration.

9. The intelligent low-altitude UAV cluster air-to-ground collaborative fire detection and extinguishing system according to claim 1 is characterized in that: The fire extinguishing scheme generation module includes an information integration unit, a scheme generation unit and an evaluation and optimization unit; The information integration unit is used to obtain integrated fire extinguishing data based on the prediction results of fire trends, fire extinguishing resources, and water source distribution when it is determined that there is a fire source in the forest area; the fire extinguishing resources include the number of drones, the number of fire extinguishing bombs, and the capacity of fire extinguishing agents; The solution generation unit is used to obtain multiple fire extinguishing solutions based on the integrated fire extinguishing data; The evaluation and optimization unit is used to determine the final fire extinguishing plan based on the fire extinguishing effect, resource consumption and time cost.

10. A fire extinguishing method of an intelligent low-altitude UAV cluster air-to-ground collaborative fire extinguishing system according to any one of claims 1 to 9, characterized in that: The steps include: S101: Acquire infrared images of forest areas, actual images, smoke concentration in the air, ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, meteorological data, human activity data and drone location data; S102: determining a patrol area of ​​the drone according to the topographic data, vegetation data and human activity data of the forest area, and calculating a comprehensive risk value of the patrol area, so as to determine a time interval for patrolling the patrol area of ​​the drone and a density of patrol routes based on the comprehensive risk value; patrolling forest areas at a density based on the stated time intervals and patrol routes; S103: determining whether there is a fire source in the forest area according to the real-time infrared image of the forest area and the real-time actual image of the forest area, and when it is determined that there is a fire source in the forest area, determining the intensity and location of the fire source according to the smoke density; S104: Obtaining a prediction result of the fire trend according to the ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, fire source intensity and fire source location; S105: Obtain multiple fire extinguishing plans based on ambient temperature, ambient humidity, wind direction, wind speed, topographic data, vegetation data, human activity data, drone location data, fire source intensity and location, fire extinguishing resources, and fire trend prediction results, and then determine the final fire extinguishing plan based on fire extinguishing effect, resource consumption, and time cost; S106: Extinguish the fire in the forest area according to the final fire extinguishing plan.

Images