A spraying unmanned aerial vehicle for firefighting and a spraying control optimization method and system thereof

Abstract

The application provides a spraying unmanned plane for fire fighting and a spraying control optimization method and system thereof, and the method comprises the following steps: collecting image data and real-time geographical position in the process of unmanned plane cruising to obtain a capture frame matrix of each frame of image; tracing an image pixel matrix of the real-time position of the unmanned plane according to the capture frame matrix; estimating the fire-fighting water volume of the monitored area of the unmanned plane according to the image pixel matrix of the real-time position of the unmanned plane obtained by tracing, and controlling the spraying head of the unmanned plane to spray water with the corresponding spraying water volume for fire fighting. The application can accurately determine the position of the forest fire source and multiple fire points through image collection optimization, and then guide the unmanned plane to accurately determine the required spraying water volume and reach the positions of multiple fire sources in the shortest and fastest planned path, so as to ensure fire extinguishing while avoiding water resource waste, and there is no risk of collision and falling caused by being ignited according to the planned optimal travel path.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of unmanned aerial vehicle fire fighting, and particularly relates to a spraying unmanned aerial vehicle for fire fighting and a spraying control optimization method and system thereof. BACKGROUND

[0002] Forests, as an important natural resource, are vital to our survival. Unfortunately, tens of thousands of hectares of forests are destroyed by fires every year in China and all over the world, and the resource loss caused by tree felling and forest fires or even natural disasters is incalculable. In recent years, unmanned aerial vehicles have become intelligent systems with multiple functions from simple aircraft, and have been widely used in military and civilian fields.

[0003] Unmanned aerial vehicles are small, portable, stable in flight, high-precision positioning, and flexible and diverse in gimbal bearing, and can carry image functions of remote sensing, infrared, visible light, multispectral systems, etc., and the remote sensing images carried by them are not inferior to existing satellite remote sensing image systems. They can provide timely and accurate feedback information and can seamlessly monitor forest areas according to the pre-designed flight route, height, etc. requirements. Secondly, unmanned aerial vehicles have high inclusiveness, small weight and volume, and are easy to take off and land in complex forest environments. In forest surveys, the working environment is generally in mountainous areas, and it is difficult to find suitable takeoff and landing sites for fixed-wing aircraft that need to slide and slide. In the case of not meeting the conditions, takeoff can cause great damage to the aircraft, while multi-rotor and single-rotor aircraft can take off vertically and do not require fixed takeoff sites. They have strong weather interference resistance, light weight and simple maintenance.

[0004] For the inspection unmanned aerial vehicle, the forest image information returned by the unmanned aerial vehicle is complex, and the manual observation has a high missed detection rate. Therefore, it is a key point and difficulty to enable the unmanned aerial vehicle to quickly and accurately identify flames in a complex environment for realizing real-time autonomous monitoring of the inspection unmanned aerial vehicle. For the fire extinguishing unmanned aerial vehicle, the unmanned aerial vehicle flies fast, and manual control of the unmanned aerial vehicle is difficult to operate. The forest environment is complex, and it is difficult to obtain complete map information. Therefore, it is a key to enable the unmanned aerial vehicle to quickly determine the source position of the forest fire in the case of incomplete environmental information, accurately determine the flame occurrence position from the image collected by the unmanned aerial vehicle, and accurately provide the spraying water amount required by the unmanned aerial vehicle for fire extinguishing and plan an optimal travel path without collision and being caused to fall by a fire. SUMMARY

[0005] The present application aims at the above-mentioned defects, and provides a spraying unmanned aerial vehicle for fire fighting and a spraying control optimization method and system thereof. The present application can accurately determine the positions of forest fire sources and multiple fire points through image acquisition optimization, and then guide the unmanned aerial vehicle to accurately determine the required spraying water volume and reach the positions of multiple fire sources in the shortest and fastest planned path, so as to ensure fire extinguishing and avoid waste of water resources, and the optimal planned travel path can avoid collision and falling caused by being ignited.

[0006] The present application provides the following technical scheme: a spraying control optimization method of a spraying unmanned aerial vehicle for fire fighting, comprising the following steps:

[0007] S101, collecting image data and real-time geographical position in the unmanned aerial vehicle cruising process, and analyzing the capture frame matrix of each frame of image according to the length L and width W of the monitored area of the unmanned aerial vehicle;

[0008] S102, tracing the image pixel matrix of the real-time position of the unmanned aerial vehicle according to the capture frame matrix;

[0009] S103, estimating the fire fighting spraying water volume of the monitored area of the unmanned aerial vehicle with the length L and width W according to the image pixel matrix of the real-time position of the unmanned aerial vehicle obtained by tracing in the S102 step, and controlling the unmanned aerial vehicle spraying head to spray water with the corresponding spraying water volume for fire fighting and extinguishing.

[0010] Further, each capture frame matrix is composed of the blue visible light spectrum and the green visible light spectrum and the near-infrared spectrum intensity NIR t calculated GNDVI t spectrum intensity channel;

[0011] G t is the collected real-time green visible light spectrum intensity, B t is the collected real-time blue visible light spectrum intensity; SGNDVI t is the green visible light spectrum intensity G t removed from the near-infrared spectrum intensity NIR t at t moment. t calculated SGNDVI t spectrum intensity.

[0012] Further, the capture frame matrix F(x t ,y t ) obtained in the step S101 is as follows:

[0013]

[0014] wherein, x t= 0,1,...,W-1,y t = 0,1,...,L-1;x t is the x-axis coordinate of the projection plane of the ground where the UAV is located in real time by the GPRS positioning module. t is the y-axis coordinate of the projection plane of the ground where the UAV is located in real time by the GPRS positioning module.

[0015] Further, the calculation formula of the image pixel matrix P(x t ,y t ) of the real-time location of the UAV in the S103 step is as follows:

[0016]

[0017] wherein, is the average coefficient of each image pixel processing frame.

[0018] Further, the calculation steps of the average coefficient of each image pixel processing frame are as follows:

[0019] S201, real-time spectral acquisition and analysis calculation, output normalized difference vegetation index NDVI t at t time;

[0020] S202, according to the normalized difference vegetation index NDVI t acquired and calculated in the S201 step, calculate the near-infrared spectral intensity NIR t in the image data corresponding to the capture frame matrix;

[0021] S203, according to the calculated near-infrared spectral intensity NIR t , calculate the spectral intensity GNDVI t calculated from the near-infrared spectral intensity NIR t by removing the green visible light spectral intensity at t time;

[0022] S204, according to the calculated spectral intensity GNDVI t , calculate the average coefficient

[0023]

[0024] Further, the formula for calculating the near-infrared spectral intensity NIR t in the image data corresponding to the capture frame matrix in the S202 step is as follows:

[0025]

[0026] wherein, Rt The red visible light spectrum intensity collected in real time.

[0027] Further, the S203 step calculates the spectrum intensity GNDVI t The formula is as follows:

[0028]

[0029] Wherein, G t The green visible light spectrum intensity collected in real time.

[0030] The application also provides a spraying unmanned plane for fire fighting using the method.

[0031] The application also provides a spraying unmanned plane control optimization system for fire fighting using the method, comprising an image acquisition module, a GreenSeeker spectrum sensor, a GPRS positioning module and a central analysis control module.

[0032] The image acquisition module is used to acquire the red visible light spectrum intensity, the blue visible light spectrum intensity and the green visible light spectrum intensity of real-time images in the process of unmanned plane cruising.

[0033] The GreenSeeker spectrum sensor is used to acquire and analyze and calculate in real time in the process of unmanned plane cruising, and output the normalized difference vegetation index NDVI t at time t.

[0034] The GPRS positioning module is used to position the x-axis coordinate and the y-axis coordinate of the projection plane of the ground where the unmanned plane is located in real time.

[0035] The central analysis control module is used to analyze the capture frame matrix F(x t ,y t ) of each frame of image according to the length L and the width W of the monitored area of the unmanned plane, trace the image pixel matrix of the real-time location of the unmanned plane, estimate the fire-fighting water volume of the monitored area of the unmanned plane with the length L and the width W, and control the unmanned plane spraying head to spray water with the corresponding fire-fighting water volume for fire fighting.

[0036] Further, the image acquisition module is a binocular camera or an infrared camera.

[0037] The application has the following beneficial effects:

[0038] 1. The spraying control optimization method for the spraying unmanned plane for fire fighting can acquire the blue visible light spectrum and the green visible light spectrum of each frame of image of the capture frame of the real-time location of the unmanned plane through the image acquisition module, and calculate the normalized difference vegetation index NDVI tThe calculated GNDVI obtained by removing the green visible light spectral intensity t The captured frame matrix model of spectral intensity, the GNDVI calculated by removing the green visible light spectral intensity t Spectral intensity, the captured frame matrix model provided by the application can accurately locate the position of the forest fire occurrence, and calculate the GNDVI t The near-infrared spectrum NIR of spectral intensity t The normalized difference vegetation index NDVI at t time is output by real-time collection and analysis calculation of the GreenSeeker spectral sensor during the unmanned aerial vehicle cruising t The calculated GNDVI improves the accuracy of the calculation.

[0039] 2, The method provided by the application based on the visual forest fire monitoring system replaces the traditional manual monitoring and alarm method, avoids the manual remote control unmanned aerial vehicle inspection operation and the high difficulty and poor effect of naked eye detection, the phenomenon of untimely alarm and false alarm, and the waste of fire extinguishing resources.

[0040] 3, The method and system provided by the application and the unmanned aerial vehicle can accurately calculate the required water spraying amount for the fire public opinion range calculated by image acquisition, effectively improve the accuracy of fire public opinion discovery after planning the fire extinguishing route of the unmanned aerial vehicle, and avoid the phenomenon of unmanned aerial vehicle crash before reaching the fire fighting position after being ignited by the fire flame during the flight.

[0041] 4, The method for detecting the position of the fire source provided by the application, the target detection is carried out through the image acquisition module, the object category in the single frame image is recognized and the position information is obtained, the position of the target is marked out by a rectangular frame with length L and width W, the neural network structure is designed by applying the target detection algorithm of deep learning, the network model which can recognize the object category and position information is obtained by a large number of sample training, the network model completely replaces the complex steps such as pretreatment, feature extraction and classification recognition of traditional algorithm, avoids the low calculation accuracy of traditional target detection algorithm which extracts the information features of image by artificial and designs the classifier to compare the features to obtain the object category and feature information, and the phenomenon of false alarm easily occurs. BRIEF DESCRIPTION OF DRAWINGS

[0042] In the following, the application will be described in more detail based on embodiments and with reference to the accompanying drawings. In which:

[0043] Figure 1 The spraying control optimization method flowchart of the spraying unmanned aerial vehicle provided by the application for fire fighting;

[0044] Figure 2 The average coefficient method flow chart for calculating each image pixel processing frame in the control optimization method provided by the present application is shown in the figure;

[0045] Figure 3 The spray control optimization system structure schematic diagram of the spray unmanned aerial vehicle for fire fighting provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0047] Embodiment 1

[0048] As shown in the figure, the flow chart of the spray control optimization method of the spray unmanned aerial vehicle for fire fighting provided by the present embodiment, the method provided by the present embodiment includes the following steps: Figure 1

[0049] S101, collect image data and real-time geographic position in the unmanned aerial vehicle cruising process, and analyze the capture frame matrix of each frame image according to the length L and width W of the region monitored by the unmanned aerial vehicle;

[0050] S102, according to the capture frame matrix, trace the image pixel matrix of the real-time position of the unmanned aerial vehicle;

[0051] S103, according to the image pixel matrix of the real-time position of the unmanned aerial vehicle obtained by tracing in the step S102, estimate the fire-fighting water volume of the region monitored by the unmanned aerial vehicle with the length L and width W, and control the unmanned aerial vehicle spray head to spray water with the corresponding spray water volume for fire fighting.

[0052] Embodiment 2

[0053] As shown in the figure, the flow chart of the spray control optimization method of the spray unmanned aerial vehicle for fire fighting provided by the present embodiment, the method provided by the present embodiment includes the following steps: Figure 1

[0054] ​​S101, collect image data and real-time geographic position in the process of unmanned aerial vehicle cruising, and transmit to the central analysis control module, and the central analysis control module obtains the capture frame matrix of each frame of image according to the length L and width W of the region monitored by the unmanned aerial vehicle; each capture frame matrix is represented by a matrix with the maximum width W and length L and the origin at the upper left corner of the Cartesian plane, and the spectral intensity channel group is composed of the near-infrared spectral intensity NIR corresponding to the blue visible light spectrum and the green visible light spectrum and the green visible light spectrum intensity removed from the near-infrared spectral intensity NIR t The calculated GNDVI t Spectral intensity channel composition

[0055] Wherein, G t is the real-time green visible light spectrum intensity collected by the image acquisition module, B t is the real-time blue visible light spectrum intensity collected by the image acquisition module; GNDVI t is the GNDVI calculated at time t from the near-infrared spectral intensity NIR t minus the green visible light spectrum intensity G t ; GNDVI t is the spectral intensity

[0056] According to the x-axis coordinate x t and y-axis coordinate y t of the projection plane of the ground where the unmanned aerial vehicle is located positioned by the GPRS real-time positioning module, the capture frame matrix F(x t ,y t ) obtained in step S101 is as follows:

[0057]

[0058] Wherein, x t =0,1,…,W―1,y t =0,1,…,L―1

[0059] S102, the central analysis control module traces the image pixel matrix P(x t ,y t ) of the real-time position of the unmanned aerial vehicle according to the capture frame matrix F(x t ,y t )

[0060] S103, the central analysis control module estimates the fire sprinkling water volume of the region monitored by the unmanned aerial vehicle with length L and width W according to the image pixel matrix P(x t ,y t ) of the real-time position of the unmanned aerial vehicle traced in step S102, and controls the unmanned aerial vehicle spray head to spray water with corresponding sprinkling water volume for fire extinguishing;

[0061] The calculation formula of the image pixel matrix P(x t ,y t ) of the real-time location of the traceability unmanned aerial vehicle is as follows:

[0062]

[0063] wherein, is the average coefficient of each image pixel processing frame.

[0064] As shown in Figure 2 , the calculation steps of the average coefficient of each image pixel processing frame are as follows:

[0065] S201, the GreenSeeker spectral sensor performs real-time spectral acquisition and analysis calculation, and outputs the normalized difference vegetation index NDVI t at time t and transmits it to the central analysis control module;

[0066] S202, the GreenSeeker spectral sensor performs real-time spectral acquisition and analysis calculation, and outputs the normalized difference vegetation index NDVI t at time t, and the formula is as follows:

[0067]

[0068] R t is the real-time red visible light spectral intensity collected by the image acquisition module.

[0069] Therefore, the central analysis control module can reversely calculate the near-infrared spectral intensity NIR t in the image data corresponding to the capture frame matrix according to the NDVI t collected by the GreenSeeker spectral sensor:

[0070]

[0071] S203, according to the calculated near-infrared spectral intensity NIR t , the spectral intensity GNDVI t calculated by removing the green visible light spectral intensity from the near-infrared spectral intensity NIR t at time t is calculated:

[0072]

[0073] S204, according to the calculated spectral intensity GNDVI t , the average coefficient of each image pixel processing frame is calculated:

[0074]

[0075] The present application also provides a spraying unmanned aerial vehicle for fire fighting using the method provided in Embodiment 1 or Embodiment 2.

[0076] Embodiment 3

[0077] As shown in Figure 3 , the present embodiment provides a spraying unmanned aerial vehicle control optimization system for fire fighting using the method provided in Embodiment 1 or Embodiment 2, which comprises an image acquisition module, a GreenSeeker spectral sensor, a GPRS positioning module, and a central analysis control module.

[0078] The image acquisition module is used to acquire the red visible light spectral intensity, the blue visible light spectral intensity, and the green visible light spectral intensity of the real-time image during the unmanned aerial vehicle cruising.

[0079] The GreenSeeker spectral sensor is used to acquire and analyze and calculate in real time during the unmanned aerial vehicle cruising, and output the normalized difference vegetation index NDVI at time t. t ;

[0080] The GPRS positioning module is used to position the x-axis coordinate and the y-axis coordinate of the projection plane of the ground where the unmanned aerial vehicle is located in real time.

[0081] The central analysis control module is used to analyze the capture frame matrix F(x t ,y t ) of each frame of image according to the length L and the width W of the monitored area of the unmanned aerial vehicle, trace the image pixel matrix of the real-time location of the unmanned aerial vehicle, and estimate the fire-fighting water volume of the monitored area of the unmanned aerial vehicle with the length L and the width W, so as to control the unmanned aerial vehicle to spray out water with the corresponding fire-fighting water volume for fire fighting.

[0082] In a preferred embodiment provided by the present application, the image acquisition module is a binocular camera

[0083] In another preferred embodiment provided by the present application, the image acquisition module is an infrared camera.

[0084] The binocular camera or the infrared camera is arranged on the unmanned aerial vehicle, and the real-time image acquisition of the fire-fighting monitored area is carried out by the unmanned aerial vehicle carrying the binocular camera or the infrared camera.

[0085] In a preferred embodiment provided by the present application, the system further comprises an LED display module, which is used to output the data acquired by the image acquisition module, the GreenSeeker spectral sensor, and the GPRS positioning module in real time.

[0086] While the present application has been described with reference to the preferred embodiments, it is to be understood that various modifications can change the scope of the present application to which they are not intended to deviate. In particular, the technical features mentioned in the various embodiments can be combined in any manner, provided that there is no structural conflict. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A spray control optimization method for a spraying drone for firefighting, characterized by, The method comprises the following steps: S101, collecting image data and real-time geographical position in the process of unmanned aerial vehicle cruising, and analyzing a capture frame matrix of each frame of image according to the length L and the width W of the region monitored by the unmanned aerial vehicle; S102, according to the capture frame matrix, tracing the image pixel matrix of the real-time position of the unmanned aerial vehicle; S103, according to the image pixel matrix of the real-time position of the unmanned aerial vehicle traced in the step S102, estimating the fire-fighting water quantity of the region monitored by the unmanned aerial vehicle with the length L and the width W, and controlling the unmanned aerial vehicle to spray water with the corresponding fire-fighting water quantity from the spraying head for fire-fighting; wherein the capture frame matrix obtained in step S101 As follows: ； wherein , ; The image pixel matrix of the real-time location of the tracing unmanned aerial vehicle in the S103 step The calculation formula is as follows: ； wherein, process the average coefficient of the frame for each image pixel.

2. The method of claim 1, wherein, each said capture frame matrix is composed of spectral intensities corresponding to a blue visible light spectrum and a green visible light spectrum and a near infrared spectrum intensity de- greened from the green visible light spectrum intensity calculated spectral intensity channel composition; ; is the collected real-time green visible light spectrum intensity, is the collected real-time blue visible light spectrum intensity; is the near-infrared spectrum intensity at time t is the green visible light spectrum intensity removed from the near-infrared spectrum intensity is the calculated spectrum intensity.

3. The method of claim 2, wherein, the average coefficient of each image pixel processing frame The calculation steps are as follows: S201, real-time spectrum acquisition and analysis calculation, output the normalized difference vegetation index at t time ; S202、According to the S201 step, the normalized difference vegetation index collected and calculated , calculating the near-infrared spectral intensity in the image data corresponding to the captured frame matrix ; S203、According to the calculated near-infrared spectrum intensity , the spectrum intensity calculated by removing the green visible light spectrum intensity from the near-infrared spectrum intensity at time t ;​ S204、according to the calculated spectral intensity , calculate the average coefficient of each image pixel processing frame : 。 4. The method of claim 3, wherein, The S202 step calculates the near-infrared spectral intensity in the image data corresponding to the captured frame matrix The formula is as follows: wherein, is the collected real-time red visible light spectral intensity.

5. The method of claim 4, wherein, The S203 step calculates the spectral intensity The formula is as follows: wherein is the collected real-time green visible light spectral intensity.

6. The spraying unmanned aerial vehicle for fire-fighting using the method according to claim 1.

7. The spray drone control optimization system for firefighting using the method as claimed in claim 1, wherein, The device comprises an image collection module, a GreenSeeker spectrum sensor, a GPRS positioning module, and a central analysis control module. The image collection module is used to collect the red visible light spectrum intensity, the blue visible light spectrum intensity, and the green visible light spectrum intensity of real-time images in the process of unmanned aerial vehicle cruising. The GreenSeeker spectral sensor is used to collect and analyze in real time during the UAV cruise process, and output the normalized difference vegetation index at time t ; The GPRS positioning module is used to position the x-axis coordinate and the y-axis coordinate of the projection plane of the ground where the unmanned aerial vehicle is located in real time. The central analysis control module is configured to analyze a capture frame matrix of each frame of image according to a length L and a width W of a region monitored by the unmanned aerial vehicle trace an image pixel matrix of a real-time location of the unmanned aerial vehicle, estimate a fire-fighting water volume of the region monitored by the unmanned aerial vehicle with the length L and the width W, and control the unmanned aerial vehicle to spray water with a corresponding water volume from the spraying head for fire-fighting.

8. The system of claim 7, wherein, The image collection module is a binocular camera or an infrared camera.

Images