Unmanned aerial vehicle control method based on emergency events in chemical industry park

Abstract

The invention relates to the technical field of unmanned aerial vehicles, in particular to an unmanned aerial vehicle control method based on emergency events in a chemical industry park, and the method comprises the steps: generating a three-dimensional temperature field image through the cooperation of a thermal imaging camera, a close-range camera and a three-dimensional laser camera of a fire-fighting unmanned aerial vehicle; reconstructing flame in the three-dimensional temperature field image through a flame convolutional neural network model to generate a flame three-dimensional slice; determining a fire extinguishing stage according to the flame three-dimensional slice; according to the fire extinguishing stage, the fire extinguishing agent type, the foam multiple and the throwing tracking position of the fire extinguishing agent foam echelon superposition throwing strategy executed by the fire-fighting unmanned aerial vehicle are determined; and control parameters, communication laser wavelength and energy transfer laser wavelength of a laser communication energy transfer system of the fire-fighting unmanned aerial vehicle are determined according to the launching tracking position and the flame three-dimensional slice. The fire-fighting unmanned aerial vehicle can automatically execute fire extinguishing agent foam echelon superposition throwing, it is guaranteed that the fire-fighting unmanned aerial vehicle can rapidly reduce the fire scene temperature of a chemical industry park, and fire expansion and reburning are prevented.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a control method of unmanned aerial vehicles based on emergency events in a chemical park. Background Art

[0002] The chemical process flow in the chemical industrial park is complex, with many reaction units, various raw materials, strict process control and high technology intensity. Fire accidents in chemical plants in chemical industrial parks are the focus of prevention. At the same time, due to the internal supply closed loop formed by industries represented by fine chemical parks, upstream and downstream industries are prone to chain reactions of accidents. Once a fire occurs, the fire will quickly spread in all directions along the container pipeline, forming a large area of ​​combustion. The toxic gases and smoke commonly seen in chemical fires are highly toxic and pose a serious threat to human health, which may cause poisoning, suffocation and even death. In addition, the release of toxic gases and smoke will also pollute the surrounding environment, affecting the ecological balance and human health.

[0003] Therefore, chemical parks prefer to use firefighting drones for remote firefighting to avoid threats to human health from toxic gases and smoke.

[0004] In the process of firefighting in chemical parks, the tactic of "three superpositions and three stages" of staggered release of fire extinguishing foam is usually adopted. It promotes the foam coverage layer by releasing low, medium and high multiple foams and fire extinguishing agents in stages, forming a "low-multiple main fire extinguishing, medium-multiple pushing low-multiple, high-multiple strong supplement" of foam and fire extinguishing agents. Superimposed coverage, forming a three-dimensional "full foam" firefighting force, which can improve firefighting efficiency and adapt to different types of fire scenes in chemical parks.

[0005] However, how to enable firefighting drones to automatically perform the step-by-step and overlapping release of fire extinguishing foam according to the fire situation to adapt to the fire extinguishing method in chemical parks is a technical problem that needs to be solved. Summary of the invention

[0006] To this end, the present invention provides a UAV control method based on emergency events in a chemical park. The process of the fire-fighting UAV executing the strategy of cascaded superimposed delivery of fire-extinguishing foam is determined through a three-dimensional temperature field image, so that the fire-fighting UAV can automatically execute the cascaded superimposed delivery of fire-extinguishing foam according to the accurate fire situation, ensuring that the fire-fighting UAV can quickly reduce the fire scene temperature in the chemical park and prevent the fire from expanding and reigniting.

[0007] To achieve the above purpose, the present invention proposes a UAV control method based on emergency events in a chemical park, comprising:

[0008] The firefighting drone's thermal imaging camera, close-range camera and 3D laser camera work together to generate a 3D temperature field image that reflects the fire situation in the chemical park;

[0009] Reconstructing the flame in the three-dimensional temperature field image through a flame convolutional neural network model to generate a three-dimensional flame slice;

[0010] determining a fire extinguishing stage based on the three-dimensional slice of the flame;

[0011] According to the fire extinguishing stage, determine the type of fire extinguishing agent, foam multiple and delivery tracking position of the fire-fighting drone to execute the fire extinguishing agent foam cascade superposition delivery strategy;

[0012] According to the delivery tracking position and the three-dimensional slice of the flame, the control parameters, communication laser wavelength and energy transmission laser wavelength of the laser communication energy transmission system of the firefighting drone are determined.

[0013] Furthermore, the steps of generating a three-dimensional temperature field image reflecting the fire situation in the chemical park by using the thermal imaging camera, the close-range camera and the three-dimensional laser camera of the firefighting drone include:

[0014] Matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field;

[0015] Controlling the three-dimensional laser camera to generate a three-dimensional image of the chemical park at the acquisition point;

[0016] The characteristic points of the temperature field are matched with the characteristic points of the three-dimensional image to generate the three-dimensional temperature field image.

[0017] Furthermore, the step of matching the acquisition points of the temperature field with the feature points of the three-dimensional image to generate the temperature field image includes:

[0018] Extracting the temperature building entity boundary of the temperature field and the image building entity boundary of the chemical park in the three-dimensional image through a convolutional neural network model;

[0019] According to the temperature building entity boundary and the image building entity boundary, the temperature field and the three-dimensional image are matched to generate the three-dimensional temperature field image.

[0020] Furthermore, the step of matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field includes:

[0021] The acquisition points of the thermal imaging camera and the acquisition points of the close-range camera are matched through an image registration and fusion model to generate the temperature field.

[0022] In the above scheme, the images collected by the thermal imaging camera, close-range camera and three-dimensional laser camera are fused through a convolutional neural network model and an image registration and fusion model to generate an accurate and visual three-dimensional temperature field image that can reflect the fire height and fire temperature.

[0023] Furthermore, the step of determining the fire extinguishing stage according to the three-dimensional flame slices includes:

[0024] Determining the fire combustion stage according to the flame core temperature value in the flame three-dimensional slice and the flame spread speed in a plurality of the flame three-dimensional slices;

[0025] According to the flame morphology in the current three-dimensional slice of the flame, it is determined whether the flame is in a passivation state; and according to the fire burning stage and whether the flame is in a passivation state, the fire extinguishing stage is determined.

[0026] In the above scheme, accurate identification of the fire extinguishing stage is achieved based on various flame / fire information reflected in the three-dimensional temperature field image.

[0027] Furthermore, a pipeline for conveying fire extinguishing agent is provided on the fire-fighting drone, and the pipeline conveys fire extinguishing agent of different types and different foam multiples through a wireless control valve, and the wireless control valve communicates with the fire-fighting drone;

[0028] The steps of determining the type of fire extinguishing agent and the foam multiple for the fire-fighting drone to implement the fire extinguishing agent foam stacking strategy according to the fire extinguishing stage include:

[0029] If the fire extinguishing stage is in the initial stage, the firefighting drone controls the wireless control valve to release low-multiple foam fire extinguishing agent and / or gas fire extinguishing agent;

[0030] If the fire extinguishing stage is in the middle stage, the firefighting drone controls the wireless control valve to release the medium-multiple foam fire extinguishing agent and / or special fire extinguishing agent;

[0031] If the fire extinguishing stage is in the later stage, the firefighting drone controls the wireless control valve to release a high-multiplicity foam fire extinguishing agent.

[0032] Furthermore, there are at least two firefighting drones that implement the fire extinguishing agent foam cascade superposition delivery strategy;

[0033] The steps of determining the release tracking position of the fire-fighting drone to implement the fire-extinguishing foam cascade superposition release strategy according to the fire extinguishing stage include:

[0034] If the fire extinguishing stage is in the early stage, the release and tracking positions of the multiple firefighting drones are controlled to be within the disaster area above the flames;

[0035] If the fire extinguishing stage is in the middle stage, the release and tracking positions of the plurality of firefighting drones are controlled to be located above the buildings within the disaster area above the flames and within the set range around the flames;

[0036] If the fire extinguishing stage is in the later stage, the release and tracking positions of the multiple firefighting drones are controlled to be directly above the flames.

[0037] Furthermore, the affected range and the set range are calculated and determined based on the flame height and area in the three-dimensional slice of the flame.

[0038] Furthermore, when the release and tracking positions of the multiple firefighting drones are located within the disaster area, the different positions of the firefighting drones above the flames encircle the flames;

[0039] When the release and tracking positions of the multiple firefighting drones are within the set range, the multiple points located above the building form an array coverage of the building.

[0040] In the above scheme, the fire-fighting drone can quickly cool and isolate the flames in the early stage of the fire, expand the fire-fighting range in the middle stage of the fire, and expand the fire-fighting range in the late stage of the fire to enhance the fire-fighting effect and effectively prevent re-ignition.

[0041] Further, the step of determining the control parameters and laser parameters of the laser communication energy transmission system of the firefighting drone according to the delivery tracking position and the three-dimensional temperature field image includes:

[0042] The firefighting drone performs rough tracking according to the difference between the delivery tracking position and the spot position by setting the proportional parameter, integral parameter and differential parameter of the PID control algorithm of the laser communication energy transmission system, wherein the proportional parameter, the integral parameter and the differential parameter are determined according to the flame height and area in the three-dimensional temperature field image;

[0043] The firefighting drone sets the laser wavelength of the laser communication energy transmission system according to the distance between the delivery tracking position and the laser emitting end.

[0044] In the above scheme, the fire situation is reflected according to the three-dimensional temperature field image, ensuring that the atmospheric interference caused by the fire has little impact on the laser communication energy transmission system. The fire-fighting drone performs dynamic PID adjustment with different sensitivities to adapt to the fire situation.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] 1. The images collected by the thermal imaging camera, close-range camera and 3D laser camera are fused through the convolutional neural network model and image registration fusion model to generate an accurate and visual 3D temperature field image that can reflect the fire height and fire temperature.

[0047] 2. It realizes accurate identification of the fire extinguishing stage based on various flame / fire information reflected in the three-dimensional temperature field image.

[0048] 3. Firefighting drones can quickly cool and isolate flames in the early stages of a fire, expand the scope of firefighting in the middle stages of a fire, and expand the scope of firefighting in the late stages of a fire to enhance the firefighting effect and effectively prevent re-ignition.

[0049] 4. Ensure that the atmospheric interference caused by the fire has little impact on the laser communication energy transmission system. The firefighting drone performs dynamic PID adjustment with different sensitivities to adapt to the fire situation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A schematic diagram of the general flow of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention;

[0051] Figure 2 Detailed flowchart of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention;

[0052] Figure 3 It is a schematic diagram of a three-dimensional temperature field image generation process of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention;

[0053] Figure 4 A schematic diagram of three-dimensional slices generated from a three-dimensional temperature field image of a method for controlling a drone based on an emergency event in a chemical park according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0055] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0056] It should be noted that, in the description of the present invention, terms such as "up", "down", "left", "right", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the drawings. This is merely for the convenience of description and does not indicate or imply that the device or element must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it cannot be understood as a limitation on the present invention.

[0057] In addition, it should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0058] like Figures 1 to 4 As shown, the present invention provides a UAV control method based on emergency events in a chemical park, which determines the process of the fire-fighting UAV executing the strategy of cascaded superimposed delivery of fire-extinguishing foam through a three-dimensional temperature field image, so that the fire-fighting UAV can automatically execute the cascaded superimposed delivery of fire-extinguishing foam according to the accurate fire situation, and ensures that the fire-fighting UAV can quickly reduce the fire scene temperature in the chemical park to prevent the fire from expanding and reigniting.

[0059] in, Figure 1 A schematic diagram of the general flow of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention; Figure 2 Detailed flowchart of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention; Figure 3 It is a schematic diagram of a three-dimensional temperature field image generation process of a method for controlling a UAV based on an emergency event in a chemical park according to an embodiment of the present invention; Figure 4 A schematic diagram of three-dimensional slices generated from a three-dimensional temperature field image of a method for controlling a drone based on an emergency event in a chemical park according to an embodiment of the present invention.

[0060] like Figures 1 to 4 As shown, this embodiment proposes a UAV control method based on chemical park emergency events, including: generating a three-dimensional temperature field image reflecting the fire situation in the chemical park through the thermal imaging camera, close-range camera and three-dimensional laser camera of the fire-fighting UAV; determining the fire extinguishing stage according to the three-dimensional temperature field image; according to the fire extinguishing stage, determining the type of fire extinguishing agent, foam multiple and delivery tracking position of the fire-fighting UAV to implement the fire-fighting agent foam tiered superposition delivery strategy; according to the delivery tracking position and the three-dimensional temperature field image, determining the control parameters and laser beam parameters of the laser communication energy transmission system of the fire-fighting UAV.

[0061] It is understandable that the firefighting drone can determine the danger level of the fire situation in the chemical park based on the three-dimensional temperature field image, and if the flame height exceeds 3 meters, an emergency alarm signal is generated.

[0062] Furthermore, the step of generating a three-dimensional temperature field image reflecting the fire situation in the chemical park by cooperating with the thermal imaging camera, close-range camera and three-dimensional laser camera of the fire-fighting drone includes: matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field; controlling the three-dimensional laser camera to generate a three-dimensional image of the chemical park at the acquisition points; matching the feature points of the temperature field with the feature points of the three-dimensional image to generate the three-dimensional temperature field image.

[0063] The temperature field described in this embodiment specifically refers to a two-dimensional image or a three-dimensional image that reflects the temperature values ​​at different points. In the figure, low temperature is blue and high temperature is red. The temperature value is reflected by the depth of red, that is, the temperature field reflects the distribution state of temperature in space.

[0064] Furthermore, the step of matching the acquisition points of the temperature field with the feature points of the three-dimensional image to generate the temperature field image includes: extracting the temperature building entity boundary of the temperature field and the image building entity boundary of the chemical park in the three-dimensional image through a convolutional neural network model; matching the temperature field and the three-dimensional image according to the temperature building entity boundary and the image building entity boundary to generate the three-dimensional temperature field image. The temperature building entity boundary and the image building entity boundary appear as a regular closed figure of a building in the three-dimensional image.

[0065] Specifically, the convolutional neural network model is implemented based on the convolutional neural network of YOLOv8 and Kitti dataset. YOLOv8 is the next generation algorithm model developed by Ultra Lyticics after the YOLOv5 algorithm, and currently supports image classification, object detection and instance segmentation tasks. The convolutional neural network model includes: a new backbone network, a new Ancher-Free detection head and a new loss function. The YOLOv8 convolutional neural network model is used to realize the detection of the building entity boundary and the temperature building entity boundary of the chemical park image based on the YOLOV8 model and the Kitti dataset, and the Pyside6 library is used to build an interface system on the computer communicating with the firefighting drone, complete the development of the detection page, and visualize its three-dimensional temperature field image. By adjusting the detection confidence threshold and the IOU threshold, the accuracy of the detection is made more appropriate.

[0066] The Kitti dataset used in this embodiment marks the temperature / image building entity boundaries of different types of buildings in the chemical park, and the different types of buildings in the chemical park include production plants, gas supply stations, chemical warehouses, environmental protection facilities, office facilities and living facilities, and the dataset contains a total of 8,000 pictures. The categories in this dataset have a large number of rotations and different lighting conditions and different fire conditions, which are helpful for training a more robust detection model. The Kitti detection and recognition dataset of this experiment contains 7,500 pictures in the training set and 500 pictures in the verification set. Since the YOLOv8 algorithm has restrictions on the size of the input picture, all pictures need to be adjusted to the same size. In order to reduce the distortion of the picture as much as possible without affecting the detection accuracy, all pictures are adjusted to a size of 640x640 and the original aspect ratio is maintained. In addition, in order to enhance the generalization ability and robustness of the model, data enhancement techniques are used, including random rotation, scaling, cropping and color conversion, etc., to expand the data set and reduce the risk of overfitting.

[0067] Furthermore, the step of matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field includes: matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera through an image registration and fusion model to generate the temperature field.

[0068] It is understandable that due to the different focal lengths of the infrared light of the thermal imaging camera and the visible light of the close-range camera, the image sizes of spatial objects in the two images are different. At the same time, there is a deviation in the optical center of the infrared and visible light hardware systems. Even if the image is scaled according to the focal length, the image size of the same object in different images cannot be the same.

[0069] Therefore, the method adopted is to take the leftmost, rightmost, topmost and bottommost ends of the building as collection points, calculate the pixel difference between the collection points in the image collected by the thermal imaging camera and the image collected by the close-range camera, obtain the image scaling ratio based on the collected images, and unify the sizes of the spatial objects in the two images.

[0070] In the above scheme, the images collected by the thermal imaging camera, close-range camera and three-dimensional laser camera are fused through a convolutional neural network model and an image registration and fusion model to generate an accurate and visual three-dimensional temperature field image that can reflect the fire height and fire temperature.

[0071] Furthermore, the step of determining the fire combustion stage according to the three-dimensional temperature field image includes: reconstructing the flame in the three-dimensional temperature field image through a flame convolutional neural network model to generate a three-dimensional flame slice, and determining the fire combustion stage according to the flame core temperature value in the three-dimensional flame slice and the flame spread speed in multiple three-dimensional flame slices; determining whether the flame is in a passivation state according to the flame morphology in the current three-dimensional flame slice; and determining the fire extinguishing stage according to the fire combustion stage and whether the flame is in a passivation state.

[0072] Specifically, the flame temperature field is reconstructed through the flame convolutional neural network model. The flame convolutional neural network model sets two flame data sets, namely unimodal flame and multimodal flame. The unimodal flame data set is used to determine the network structure and to verify the reconstruction efficiency and reconstruction accuracy of the flame convolutional neural network model (CNN) reconstruction algorithm. Multimodal flame has non-axisymmetric characteristics, so a multimodal flame data set is constructed to verify the ability of the convolutional neural network model to reconstruct the non-axisymmetric flame temperature field.

[0073] In the process of constructing the single-peak flame data set, the temperature field distribution formula used is as follows:

[0074]

[0075] Wherein, z and r are the axial and radial coordinates of the temperature point, Z and R are the axial and radial coordinates of the temperature point of the flame in the three-dimensional temperature field image, respectively. f and v are adjustable coefficients, and different temperature fields can be obtained. The value range of f is set to 0.8-1.2 with an interval of 0.01; the value range of v is 0.8-1.1 with an interval of 0.01. The grid of the three-dimensional temperature field image is divided into 10x5x10, which represent the axial, radial and circumferential directions respectively.

[0076] A total of 5,000 different sets of images were obtained through the temperature field distribution formula, of which 4,500 sets were used as training sets and 500 sets were used as test sets to train and test the flame convolutional neural network composed of input layer, hidden layer and output layer, where the hidden layer includes convolution layer, pooling layer and fully connected layer. The flame convolutional neural network model fitted by the test is used to recognize new three-dimensional temperature field images and generate flame three-dimensional slices.

[0077] If the flame convolutional neural network model identifies the flame in the three-dimensional temperature field image as more than three groups of multi-peak flames, or a multi-peak flame with a length greater than 3 meters, or a multi-peak flame with a height greater than 2.5 meters, then it is judged that the flame is not in a passivation state; otherwise, it is judged that the flame is in a passivation state.

[0078] The flame three-dimensional slice is as follows Figure 4As shown, the flame core temperature is determined by comparing the highest temperatures of multiple slices in the three-dimensional temperature field image. The flame core temperature is the average temperature of the slice with the highest average temperature and its two adjacent slices.

[0079] The flame spread speed is the flame height increase in 0.5 seconds multiplied by 0.8, plus the flame extension length in 0.5 seconds multiplied by 0.5. The flame spread speed is usually greater than 0.1 m / s.

[0080] If the flame core temperature is lower than 500°C and the flame spread speed is lower than 0.2 m / s, the fire combustion stage is judged to be the initial stage. If the flame core temperature is higher than 500°C and lower than 900°C, or the flame spread speed is greater than 0.2 m / s, the fire combustion stage is judged to be the development stage. If the flame core temperature is higher than 500°C and lower than 900°C, and the flame spread speed is greater than 0.2 m / s, the fire combustion stage is judged to be the fierce stage.

[0081] If the flame is in a passivation state and the fire burning stage is the development stage / violent stage, then the fire extinguishing stage is determined to be the early stage; if the flame is not in a passivation state, or the fire burning stage is the development stage, then the fire extinguishing stage is determined to be the middle stage; if the flame is not in a passivation state, and the fire burning stage is the early stage, then the fire extinguishing stage is determined to be the late stage.

[0082] In the above scheme, accurate identification of the fire extinguishing stage is achieved based on various flame / fire information reflected in the three-dimensional temperature field image.

[0083] Furthermore, a pipeline for conveying fire extinguishing agent is arranged on the fire-fighting drone, and the pipeline conveys fire extinguishing agent of different types and different foam multiples through a wireless control valve, and the wireless control valve communicates with the fire-fighting drone; the step of determining the type and foam multiple of fire extinguishing agent for the fire-fighting drone to execute the fire-fighting agent foam cascade superposition delivery strategy according to the fire extinguishing stage includes: if the fire extinguishing stage is in the early stage, the fire-fighting drone controls the wireless control valve to deliver low-multiple foam fire extinguishing agent and / or gas fire extinguishing agent; if the fire extinguishing stage is in the middle stage, the fire-fighting drone controls the wireless control valve to deliver medium-multiple foam fire extinguishing agent and / or special fire extinguishing agent; if the fire extinguishing stage is in the late stage, the fire-fighting drone controls the wireless control valve to deliver high-multiple foam fire extinguishing agent.

[0084] It is understandable that the fire extinguishing agents commonly used in chemical park fires include dry powder fire extinguishers, gas fire extinguishers, foam fire extinguishers and special fire extinguishers. Dry powder fire extinguishers are suitable for extinguishing initial fires of solid materials and live equipment; gas fire extinguishers have the advantages of good fire extinguishing effect and non-conductivity; foam fire extinguishers are suitable for extinguishing solid materials and liquid fires; special fire extinguishing agents such as Lith-X can reduce the fire source temperature and passivate the flames of lithium-ion batteries in specific scenarios. In actual fire fighting and rescue work, gas fire extinguishing agents can be used first to quickly reduce the fire source temperature, followed by dry powder fire extinguishing agents, which react chemically with lithium-ion batteries to further reduce the fire source temperature and passivate the flames, and finally dry powder fire extinguishers can be used for additional fire fighting to ensure the safety of personnel and equipment.

[0085] Furthermore, there are at least two firefighting drones that execute the fire extinguishing foam staggered superimposed delivery strategy; the step of determining the delivery tracking position of the firefighting drones that execute the fire extinguishing foam staggered superimposed delivery strategy according to the fire extinguishing stage includes: if the fire extinguishing stage is in the early stage, the delivery tracking positions of the multiple firefighting drones are controlled to be located within the disaster-stricken area above the flame; if the fire extinguishing stage is in the middle stage, the delivery tracking positions of the multiple firefighting drones are controlled to be located above the buildings within the disaster-stricken area above the flame and within a set range around the flame; if the fire extinguishing stage is in the late stage, the delivery tracking positions of the multiple firefighting drones are controlled to be located directly above the flame.

[0086] Furthermore, the affected range and the set range are calculated and determined based on the flame height and area in the three-dimensional slice of the flame.

[0087] Specifically, the distance from the lowest point to the highest point of the flame in the three-dimensional flame slice reconstructed from the three-dimensional temperature field image is determined as the flame height, and the average area of ​​the flame in the slice with the highest average temperature and its four adjacent slices is determined as the flame area.

[0088] The affected range is determined by adding the first height correction value to the circumference of the flame area of ​​3 meters. The first height correction value is calculated by multiplying the flame height by the first flame coefficient. The first flame coefficient is determined according to the fire extinguishing stage, which is 0.5 in the initial stage and 0.3 in the middle stage. That is, the more serious the fire situation is, the farther the fire-fighting drone is from the center of the flame to ensure its safety.

[0089] The set range is determined by adding a second height correction value to the circumference of the flame area of ​​1 meter. The second height correction value is calculated by multiplying the flame height by 0.2 to ensure that the set range covered by the firefighting drone is safe and dense.

[0090] Furthermore, when the release and tracking positions of multiple firefighting drones are within the disaster-stricken area, the different positions above the flames form an enclosure for the flames; when the release and tracking positions of multiple firefighting drones are within the set range, the multiple points above the building form an array coverage of the building.

[0091] Specifically, multiple firefighting drones can form irregularly connected enclosures around the flames from different directions, such as east, west, south, and north. Multiple firefighting drones can form a tic-tac-toe array covering the building.

[0092] In the above scheme, the fire-fighting drone can quickly cool and isolate the flames in the early stage of the fire, expand the fire-fighting range in the middle stage of the fire, and expand the fire-fighting range in the late stage of the fire to enhance the fire-fighting effect and effectively prevent re-ignition.

[0093] Furthermore, the step of determining the control parameters and laser parameters of the laser communication energy transmission system of the fire-fighting drone according to the delivery tracking position and the three-dimensional temperature field image includes: the fire-fighting drone performs coarse tracking by setting the proportional parameters, integral parameters and differential parameters of the PID control algorithm of the laser communication energy transmission system according to the difference between the delivery tracking position and the spot position, wherein the proportional parameters, the integral parameters and the differential parameters are determined according to the flame height and area in the three-dimensional temperature field image; the fire-fighting drone sets the laser wavelength of the laser communication energy transmission system according to the distance between the delivery tracking position and the laser transmitting end.

[0094] Specifically, the spot detector of the firefighting drone sends the detected spot image signal to the spot processing unit, and obtains the actual spot position after the corresponding image filtering and positioning algorithm processing. The delivery tracking position is the target position, and the difference between the spot position and the target position is taken as the error. The controller makes control decisions and outputs control signals through the PID control algorithm according to the feedback error. The control signal is transmitted to the driver to drive the motor, thereby achieving the rough tracking effect of the firefighting drone.

[0095] If the flame height in the three-dimensional flame slice reconstructed from the three-dimensional temperature field image exceeds 5 meters, the proportional gain is set greater than the integral gain and the differential gain for faster adjustment, and the communication wavelength of the laser communication energy transmission system is set to 1550nm and the power supply wavelength is set to 850nm.

[0096] If the flame height is lower than 5 meters, or the flame area is lower than 6 square meters, the integral gain is set greater than the proportional gain and the differential gain to perform medium-speed and relatively stable adjustment, and the communication wavelength of the laser communication energy transmission system is set to 1310nm and the power supply wavelength is set to 650nm.

[0097] If the flame height is lower than 5 meters and the flame area is lower than 6 square meters, the proportional gain is set to 0, and only the integral gain and differential gain are performed to achieve slow and smooth adjustment. The communication wavelength of the laser communication energy transmission system is set to 1310nm and the power supply wavelength is set to 650nm.

[0098] It is understandable that since flames in the atmosphere produce different amounts of smoke, the atmosphere has different absorption effects on lasers. When the amount of smoke is large, the laser wavelength is adjusted to reduce the impact of the smoke on the laser communication energy transmission system.

[0099] In the above scheme, the fire situation is reflected according to the three-dimensional temperature field image, ensuring that the atmospheric interference caused by the fire has little impact on the laser communication energy transmission system. The fire-fighting drone performs dynamic PID adjustment with different sensitivities to adapt to the fire situation.

[0100] It can be understood that in this embodiment, the images collected by the thermal imaging camera, the close-range camera and the three-dimensional laser camera are fused through the convolutional neural network model and the image registration fusion model to generate an accurate and visual three-dimensional temperature field image that can reflect the fire height and fire temperature. According to the various flame / fire information reflected in the three-dimensional temperature field image, the accurate identification of the fire extinguishing stage is realized. The fire-fighting drone can quickly cool and isolate the flame in the early stage of the fire, expand the fire-fighting drone in the middle stage of the fire, and expand the fire-fighting drone in the late stage of the fire to enhance the fire-fighting effect and effectively prevent the re-ignition. To ensure that the atmospheric interference generated by the fire has little effect on the laser communication energy transmission system, the fire-fighting drone performs dynamic PID adjustment with different sensitivities to adapt to the fire situation.

[0101] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

[0102] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for controlling unmanned aerial vehicles based on emergency events in a chemical park, characterized in that: include: The firefighting drone's thermal imaging camera, close-range camera and 3D laser camera work together to generate a 3D temperature field image that reflects the fire situation in the chemical park; Reconstructing the flame in the three-dimensional temperature field image through a flame convolutional neural network model to generate a three-dimensional flame slice; determining a fire extinguishing stage based on the three-dimensional slice of the flame; According to the fire extinguishing stage, determine the type of fire extinguishing agent, foam multiple and delivery tracking position of the fire-fighting drone to execute the fire extinguishing agent foam cascade superposition delivery strategy; According to the delivery tracking position and the three-dimensional slice of the flame, the control parameters, communication laser wavelength and energy transmission laser wavelength of the laser communication energy transmission system of the firefighting drone are determined.

2. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 1, characterized in that: The steps of generating a three-dimensional temperature field image reflecting the fire situation in the chemical park by using the thermal imaging camera, close-range camera and three-dimensional laser camera of the fire-fighting drone include: Matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field; Controlling the three-dimensional laser camera to generate a three-dimensional image of the chemical park at the acquisition point; The characteristic points of the temperature field are matched with the characteristic points of the three-dimensional image to generate the three-dimensional temperature field image.

3. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 2, characterized in that: The steps of matching the acquisition points of the temperature field with the feature points of the three-dimensional image to generate the temperature field image include: Extracting the temperature building entity boundary of the temperature field and the image building entity boundary of the chemical park in the three-dimensional image through a convolutional neural network model; According to the temperature building entity boundary and the image building entity boundary, the temperature field and the three-dimensional image are matched to generate the three-dimensional temperature field image.

4. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 2, characterized in that: The steps of matching the acquisition points of the thermal imaging camera with the acquisition points of the close-range camera to generate a temperature field include: The acquisition points of the thermal imaging camera and the acquisition points of the close-range camera are matched through an image registration and fusion model to generate the temperature field.

5. The method for controlling a UAV based on an emergency event in a chemical park according to any one of claims 1 to 4, characterized in that: The steps to determine the fire extinguishing stage based on the 3D slices of the flames include: Determining the fire combustion stage according to the flame core temperature value in the flame three-dimensional slice and the flame spread speed in a plurality of the flame three-dimensional slices; Determining whether the flame is in a passivation state according to the flame morphology in the current three-dimensional slice of the flame; The fire extinguishing stage is determined according to the fire burning stage and whether the flame is in a passivation state.

6. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 5, characterized in that: The firefighting drone is provided with a pipeline for conveying fire extinguishing agent, and the pipeline conveys fire extinguishing agents of different types and different foam multiples through a wireless control valve, and the wireless control valve communicates with the firefighting drone; The steps of determining the type of fire extinguishing agent and the foam multiple for the fire-fighting drone to implement the fire extinguishing agent foam stacking strategy according to the fire extinguishing stage include: If the fire extinguishing stage is in the initial stage, the firefighting drone controls the wireless control valve to release low-multiple foam fire extinguishing agent and / or gas fire extinguishing agent; If the fire extinguishing stage is in the middle stage, the firefighting drone controls the wireless control valve to release the medium-multiple foam fire extinguishing agent and / or special fire extinguishing agent; If the fire extinguishing stage is in the later stage, the firefighting drone controls the wireless control valve to release a high-multiplicity foam fire extinguishing agent.

7. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 6, characterized in that: There are at least two firefighting drones that implement the fire extinguishing agent foam cascade superposition delivery strategy; The steps of determining the release tracking position of the fire-fighting drone to implement the fire-extinguishing foam cascade superposition release strategy according to the fire extinguishing stage include: If the fire extinguishing stage is in the early stage, the release and tracking positions of the multiple firefighting drones are controlled to be within the disaster area above the flames; If the fire extinguishing stage is in the middle stage, the release and tracking positions of the plurality of firefighting drones are controlled to be located above the buildings within the disaster area above the flames and within the set range around the flames; If the fire extinguishing stage is in the later stage, the release and tracking positions of the multiple firefighting drones are controlled to be directly above the flames.

8. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 7, characterized in that: The affected range and the set range are determined by calculation based on the flame height and area in the three-dimensional slice of the flame.

9. The method for controlling unmanned aerial vehicles based on emergency events in a chemical park according to claim 7, characterized in that: When the release and tracking positions of the multiple firefighting drones are within the disaster area, the different positions of the firefighting drones above the flames form an enclosure around the flames; When the release and tracking positions of the multiple firefighting drones are within the set range, the multiple points located above the building form an array coverage of the building.

10. The method for controlling a UAV based on an emergency event in a chemical park according to any one of claims 1 to 4, characterized in that: The steps of determining the control parameters, communication laser wavelength and energy transmission laser wavelength of the laser communication energy transmission system of the firefighting UAV according to the delivery tracking position and the flame three-dimensional slice include: The proportional parameters, integral parameters and differential parameters of the PID control algorithm used by the laser communication energy transmission system to perform coarse tracking of the delivery tracking position, as well as the communication laser wavelength and the energy transmission laser wavelength are determined according to the flame height and area in the three-dimensional flame slice.

Images