Fire extinguishing method of fire-fighting drone based on visual detection and fire-fighting drone system

Abstract

The present invention discloses a fire extinguishing method of a fire-fighting drone based on visual detection and a fire-fighting drone system. The present invention is based on the detection of smoke temperature and surrounding environmental temperature by a thermal induction sensor module, quickly discovers surrounding fire areas through a visual algorithm, eliminates the interference of smoke and high temperature of the surrounding environment, and establishes a map of the surrounding fire temperature, regionalizes it, and grades the fire severity. The fire extinguishing area is preferentially selected based on the classification of the fire severity, which not only improves the ground center personnel's more intuitive understanding of the severity of the fire, but also improves the intelligence of the fire-fighting drone's autonomous fire extinguishing, improves the fire extinguishing efficiency, and solves the problem that existing fire-fighting drones are unclear about the fire situation on site and difficult to distinguish the fire severity due to factors such as smoke and environment.

Description

Technical Field

[0001] The present invention relates to the technical field of firefighting drones, and more particularly to a firefighting drone system and a firefighting drone method based on visual detection. Background Art

[0002] Drones are already widely used in firefighting. Water sprayers are installed on drones, and ground operators remotely control the direction and location of the water jets. However, due to operators' limited understanding of the fire situation, it is difficult to extinguish fires quickly and effectively.

[0003] The autonomous firefighting drone system and method disclosed in CN201910503542.3 includes a ground station and a firefighting drone. The ground station and the firefighting drone transmit and share control commands and fire image information via a ground-to-air digital communication link. The ground station is used to send mission instructions to the firefighting drone, monitor the drone's status in real time, and further understand the fire situation based on the fire image information and flight attitude information transmitted back by the drone. The firefighting drone is used to collect fire image information, extinguish fires, and dynamically plan firefighting missions and flight paths in real time, enabling autonomous trajectory planning and obstacle avoidance. This drone can autonomously respond to fires without unified command from a command center. While the firefighting drone disclosed in this patent can improve operators' understanding of the fire situation on site, in the presence of thick smoke, it is difficult for operators to effectively judge the fire's development with the naked eye. Furthermore, the drone has difficulty determining the firefighting point, making autonomous firefighting impossible.

[0004] In CN202020803208.8, a fire investigation drone uses a thermal imager outside the casing. The thermal imager cover is provided with a transparent dust cover, which is fixedly connected to the pod shell. The dust cover is connected to a cleaning component for wiping smoke from the dust cover, thereby reducing the impact of smoke on the investigation results. Although the use of thermal imagers can better observe the fire situation in a dense smoke environment, the drone or operator cannot have a specific control over the overall fire situation in the surrounding area. For example, due to interference from factors such as the temperature of high-temperature smoke particles and the ambient temperature, it is difficult to grasp the specific fire situation in the overall area and quickly judge the severity of the fire, resulting in a reduction in fire extinguishing efficiency. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a fire extinguishing method for fire-fighting drones based on visual detection, in order to address the shortcomings of existing fire-fighting drones in that they are unable to clearly see the fire situation on site and find it difficult to distinguish the extent of the fire due to factors such as smoke, dust, and the environment.

[0006] Another technical problem that the present invention needs to solve is to provide a fire-fighting drone system based on a fire-fighting drone fire-fighting method of visual detection.

[0007] The purpose of the present invention is achieved through the following technical solutions:

[0008] A fire extinguishing method using a firefighting drone based on visual detection, comprising the following steps:

[0009] S1. The firefighting drone uses a high-definition camera to locate the source of thick smoke, then uses a thermal sensor to record the temperature and labels the temperature image as T1.

[0010] S2. Control the firefighting drone to take off above a smoke-free area of ​​the fire, observe and demarcate the fire area, determine the coordinates of the fire center, and use a thermal sensor to record the smoke temperature. Mark the temperature image as T2.

[0011] S3. After continuing to use the high-definition camera to locate the dense smoke at the center of the fire, the aircraft scans a 360-degree temperature distribution image, records the temperature image, and creates a 3D map for locating the fire area. The temperature image is labeled T3.

[0012] S4. Grayscale the temperature images of T1, T2, and T3 to obtain three sets of threshold values ​​T1', T2', and T3' at different temperatures on the grayscale image. Subtract the threshold value T2' from the threshold value T3' to obtain the threshold temperature T.

[0013] S5. Using the threshold temperature T as the standard threshold for detecting the actual fire temperature, the temperature distribution area in the 3D grayscale image is obtained, and the difference D is compared with the threshold T1'. The range of the difference D is set as the fire judgment threshold Z. The fire severity priority is established based on the area occupied by the fire judgment threshold Z image;

[0014] S6. Based on the priority of the established fire severity map, control the firefighting drone to extinguish fires in areas with higher priority.

[0015] Furthermore, T1 uses an average value as the temperature value.

[0016] Furthermore, in step S5, the difference D within 5% of the threshold value T1' is used as the fire determination threshold.

[0017] Furthermore, in step S6, a high-definition camera and a thermal sensor are used to determine whether the fire at the current location is extinguished. If the fire at the current location is extinguished, the fire is extinguished in the next priority fire area.

[0018] Furthermore, the fire extinguishing judgment criteria are that the camera cannot identify the open fire source and the average value of the temperature sensor detecting the threshold of the area is lower than the set threshold.

[0019] Furthermore, the regional threshold is a smoke temperature threshold.

[0020] Furthermore, in step S3, the fire severity map is transmitted to the ground central station via the ground-to-air data transmission system, and the operator can adjust the fire extinguishing priority according to the specific fire situation.

[0021] A firefighting drone system includes a firefighting drone and a ground center. The firefighting drone and the ground center establish data communication through a data transmission module. The firefighting drone includes a flight control task processor, a visual detection module and a firefighting delivery module. The flight control task processor is used to control its own flight, receive flight instructions from the ground center, and send and process collected fire information. The flight control task processor includes a waypoint planning module, a fire assessment module and a fire information transmission module. The waypoint planning module is used for flight control and route planning of the firefighting drone. The fire assessment module is connected to the visual detection module for analyzing the severity of the fire. The fire information transmission module is connected to the firefighting delivery module for the firefighting drone's firefighting delivery module to deliver the firefighting information within the fire area.

[0022] Furthermore, the visual detection module includes a thermal sensor, a high-definition camera and a pan-tilt platform, and the high-definition camera is installed on the pan-tilt platform to collect 360° fire information.

[0023] Furthermore, the thermal sensing sensor includes one or more of an infrared sensing camera and a thermistor sensor.

[0024] Furthermore, the fire delivery module includes one or more of a moored water hose and a fire bag.

[0025] Compared with the prior art, the beneficial effects are:

[0026] The present invention is based on the detection of smoke temperature and the temperature of the surrounding environment of the fire by the thermal induction sensor module. It quickly discovers the surrounding fire area through a visual algorithm, eliminates the interference of smoke and the high temperature of the surrounding environment, and establishes a map of the surrounding fire temperature. It regionalizes it and grades the fire severity. The fire extinguishing area is prioritized by the classification of the fire severity. This not only improves the ground center personnel's more intuitive understanding of the severity of the fire, but also improves the intelligence of the fire-fighting drone's autonomous fire extinguishing and improves the fire extinguishing efficiency. At the same time, the operator can also adjust the priority of potential high-risk areas according to the actual specific situation to avoid the expansion of the fire, thereby further improving the fire extinguishing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a diagram of the firefighting drone system;

[0028] Figure 2This is a flowchart for firefighting drones;

[0029] Figure 3 This is the algorithm diagram for fire severity. DETAILED DESCRIPTION

[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0031] It should be noted that if there are directional indications (such as up, down, left, right, front, and back) in the embodiments of the present invention, the directional indications are only used to explain the relative positional relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indications will also change accordingly. If there are descriptions of "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features.

[0032] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "provided with," "connected," etc., should be understood in a broad sense. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; it may refer to a mechanical connection or an electrical connection; it may refer to a direct connection or an indirect connection through an intermediate medium; it may refer to internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0033] In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is mutually contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0034] Example 1

[0035] This embodiment provides a firefighting drone system, including a firefighting drone and a ground center, wherein the firefighting drone and the ground center are connected via ground-to-air data communication.

[0036] like Figure 1The firefighting UAV includes a flight control task processor, a visual detection module and a firefighting delivery module. The flight control task processor is used to control its own flight, receive flight instructions from the ground center, and send and process the collected fire information.

[0037] The flight control task processor includes a waypoint planning module, a fire assessment module and a fire information transmission module. The waypoint planning module is used for flight control and route planning of the fire-fighting drone. The fire assessment module is connected to the visual detection module for analyzing the severity of the fire. The fire information transmission module and the fire delivery module are used for the fire delivery module of the fire-fighting drone to deliver firefighting equipment within the fire area.

[0038] The visual detection module includes a thermal sensor, a high-definition camera and a three-axis gimbal. The thermal sensor is used to collect fire temperature information, the high-definition camera is used to collect flames and surrounding environment information, and the three-axis gimbal is used to steer the high-definition camera.

[0039] The fire-fighting delivery module includes a moored fire-fighting water hose and a direction control module. The direction control module controls the water spraying direction of the moored fire-fighting water hose according to the fire priority of the fire assessment module and the determination of the target by a high-definition camera.

[0040] Example 2

[0041] like Figure 2-3 This embodiment provides a fire extinguishing method for a firefighting drone based on visual detection based on embodiment 1, and the steps include:

[0042] S1. The firefighting drone uses a high-definition camera to locate the source of thick smoke, then uses a thermal sensor to record the temperature and labels the temperature image as T1.

[0043] S2. Control the firefighting drone to take off above a smoke-free area of ​​the fire, observe and demarcate the fire area, determine the coordinates of the fire center, and use a thermal sensor to record the smoke temperature. Mark the temperature image as T2.

[0044] S3. After continuing to use the high-definition camera to locate the dense smoke at the center of the fire, the aircraft scans a 360-degree temperature distribution image, records the temperature image, and creates a 3D map for locating the fire area. The temperature image is labeled T3.

[0045] S4. Grayscale the temperature images of T1, T2, and T3 to obtain three sets of threshold values ​​T1', T2', and T3' at different temperatures on the grayscale image. Subtract the threshold value T2' from the threshold value T3' to obtain the threshold temperature T.

[0046] S5. Using the threshold temperature T as the standard threshold for detecting the actual fire temperature, the temperature distribution area in the 3D grayscale image is obtained. This temperature distribution area is compared with the threshold value T1' to obtain the difference D. The threshold value where the difference D falls within 5% of T1' is set as the fire judgment threshold Z. The fire severity priority is established based on the area occupied by the fire judgment threshold Z image.

[0047] S3. Based on the priority of the established fire severity map, the firefighting drone will fly to the higher priority area to extinguish the fire, and use high-definition cameras and thermal sensors to monitor the fire situation in the area. If the camera cannot identify the open flame source and the temperature sensor detects that the average smoke temperature threshold in the area is lower than the set threshold, it indicates that the fire is extinguished. The firefighting drone will then fly to the next priority area to extinguish the fire.

[0048] Example 3

[0049] This embodiment provides a fire extinguishing method for a firefighting drone based on visual detection based on embodiment 1, and the steps include:

[0050] S1. The firefighting drone uses a high-definition camera to locate the source of thick smoke, then uses a thermal sensor to record the temperature and labels the temperature image as T1.

[0051] S2. Control the firefighting drone to take off above a smoke-free area of ​​the fire, observe and demarcate the fire area, determine the coordinates of the fire center, and use a thermal sensor to record the smoke temperature. Mark the temperature image as T2.

[0052] S3. After continuing to use the high-definition camera to locate the dense smoke at the center of the fire, the aircraft scans a 360-degree temperature distribution image, records the temperature image, and creates a 3D map for locating the fire area. The temperature image is labeled T3.

[0053] S4. Grayscale the temperature images of T1, T2, and T3 to obtain three sets of threshold values ​​T1', T2', and T3' at different temperatures on the grayscale image. Subtract the threshold value T2' from the threshold value T3' to obtain the threshold temperature T.

[0054] S5. Using the threshold temperature T as the standard threshold for detecting the actual fire temperature, the temperature distribution area in the 3D grayscale image is obtained. This temperature distribution area is compared with the threshold value T1' to obtain the difference D. The threshold range of the difference D within 5% of T1' is set as the fire judgment threshold Z. The fire severity priority is established based on the area occupied by the fire judgment threshold Z image, and the image is sent to the ground center for confirmation.

[0055] S3. The ground center adjusts and confirms the priority of the fire area based on the specific environment and fire situation, and sends it to the firefighting drone. Based on the priority of the updated fire severity map, the firefighting drone flies to the higher-priority area to extinguish the fire, and uses high-definition cameras and thermal sensors to monitor the fire situation in the area. If the camera cannot identify the open flame source and the temperature sensor detects that the average smoke temperature threshold in the area is lower than the set threshold, it indicates that the fire is extinguished, and the firefighting drone flies to the next priority area to extinguish the fire.

[0056] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Those skilled in the art will appreciate that other variations or modifications can be made based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A fire extinguishing method using a fire-fighting drone based on visual detection, characterized in that the steps include: S1. The firefighting drone uses a high-definition camera to locate the source of thick smoke, then uses a thermal sensor to record the temperature and labels the temperature image as T1. S2. Control the firefighting drone to take off above a smoke-free area of ​​the fire, observe and demarcate the fire area, determine the coordinates of the fire center, and use a thermal sensor to record the smoke temperature. Mark the temperature image as T2. S3. After continuing to use the high-definition camera to locate the dense smoke at the center of the fire, the aircraft scans a 360-degree temperature distribution image, records the temperature image, and creates a 3D map for locating the fire area. The temperature image is labeled T3. S4. Grayscale the temperature images of T1, T2, and T3 to obtain three sets of threshold values ​​T1', T2', and T3' at different temperatures on the grayscale image. Subtract the threshold value T2' from the threshold value T3' to obtain the threshold temperature T. S5. Using the threshold temperature T as the standard threshold for detecting the actual fire temperature, the temperature distribution area in the 3D grayscale image is obtained, and the difference D is compared with the threshold T1'. The range of the difference D is set as the fire judgment threshold Z. The fire severity priority is established based on the area occupied by the fire judgment threshold Z image; S6. Based on the priority of the established fire severity map, control the firefighting drone to extinguish fires in areas with higher priority.

2. The fire extinguishing method of the fire-fighting drone based on visual detection according to claim 1 is characterized in that: In step S1 , the threshold T1 ′ uses the average value of T1 as the temperature value.

3. The fire extinguishing method of the fire-fighting drone based on visual detection according to claim 1 is characterized in that: In step S5, the difference D within 5% of the threshold value T1' is used as the fire determination threshold.

4. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 1 is characterized in that: In step S6, a high-definition camera and a thermal sensor are used to determine whether the fire at the current location is extinguished. If the fire at the current location is extinguished, the fire is extinguished in the next priority fire area.

5. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 4 is characterized in that: The fire extinguishing judgment criteria are that the camera cannot identify the open fire source and the average value of the temperature sensor detecting the threshold T3' in the area is lower than the set threshold.

6. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 1 is characterized in that: In step S5, the fire severity map is transmitted to the ground central station via the ground-to-air data transmission system, and the operator can adjust the fire extinguishing priority according to the specific fire situation.

7. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 1, characterized in that: The fire-fighting drone system of the fire-fighting drone extinguishing method based on visual detection includes a fire-fighting drone and a ground center. The fire-fighting drone and the ground center establish data communication through a data transmission module. The fire-fighting drone includes a flight control task processor, a visual detection module and a fire-fighting delivery module. The flight control task processor is used to control its own flight, receive flight instructions from the ground center, and send and process the collected fire information. The flight control task processor includes a waypoint planning module, a fire assessment module and a fire information transmission module. The waypoint planning module is used for flight control and route planning of the fire-fighting drone. The fire assessment module is connected to the visual detection module for analyzing the severity of the fire. The fire information transmission module is connected to the fire-fighting delivery module for the fire-fighting drone to deliver the fire-fighting delivery module to the fire area.

8. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 7 is characterized in that: The visual detection module includes a thermal sensor, a high-definition camera and a pan-tilt platform, and the high-definition camera is installed on the pan-tilt platform to collect 360° fire information.

9. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 8, characterized in that: The thermal sensing sensor includes one or more of an infrared sensing camera and a thermistor sensor.

10. The fire extinguishing method of a fire-fighting drone based on visual detection according to claim 7, characterized in that: The fire delivery module includes one or more types of moored water hoses and fire bags.

Images