A method and system for detecting forest fire risk areas using drones

Abstract

The present invention relates to the field of image processing and artificial intelligence technology, and discloses a method and system for detecting forest fire risk blocks by unmanned aerial vehicles. The method includes: constructing and training a target detection model based on block classification, and using the trained target detection model based on block classification to realize forest fire risk block detection by unmanned aerial vehicles; the target detection model based on block classification uses a block mapper to map different blocks in the unmanned aerial vehicle image to block features; a multi-stage sampling network is used to sample block features at multiple scales, and a dimensionality reduction mapping layer is used to align the scales to obtain multi-scale block features; the multi-scale block features are fused through the Hadamard product, and the fused multi-scale block features are mapped to the block category probability using a block classifier. The present invention realizes the positioning of risk blocks by regional classification, avoids the huge amount of calculation caused by precise positioning in the prior art, and prolongs the available time of the unmanned aerial vehicle.

Description

Technical Field

[0001] The present invention relates to the field of image processing and artificial intelligence technology, and in particular to a method and system for detecting forest fire risk areas using an unmanned aerial vehicle. Background Art

[0002] Forest fires threaten people's lives and property. The complex terrain of forests makes it difficult to quickly locate and identify fire risk areas. Currently, target detection algorithms in drones are mostly used to give a detection frame for the precise location of the target in the input image. However, existing target detection algorithms require huge calculations to provide accurate location information, and the limited power of drones cannot support large amounts of calculations. Summary of the invention

[0003] The purpose of the present invention is to solve the problem of low detection efficiency in the prior art.

[0004] The technical solution adopted by the present invention to solve the technical problem is: to provide a method for detecting forest fire risk blocks by using a drone, comprising the following steps:

[0005] Obtain drone image data with block category labels as target detection dataset;

[0006] Build a target detection model based on block classification;

[0007] The cross entropy loss function is used to calculate the loss of block category probability and block category label, and the target detection dataset is used to train the target detection model based on block classification;

[0008] Use the trained block classification-based target detection model to detect forest fire risk blocks using drones;

[0009] The block classification-based target detection model uses a block mapper to map different blocks in drone images into block features; uses a multi-stage sampling network to sample block features at multiple scales, and uses a dimensionality reduction mapping layer to align scales to obtain multi-scale block features; then fuses the multi-scale block features through the Hadamard product, and finally uses a block classifier to map the fused multi-scale block features to block category probabilities.

[0010] Preferably, the step of obtaining drone image data with block category labels as a target detection data set includes:

[0011] Resize all drone images to a uniform height of H and width of W;

[0012] Data preprocessing of drone images using random horizontal flipping.

[0013] Preferably, the block mapper receives the drone image and maps different blocks in the drone image into block features, expressed as:

[0014] ;

[0015] in, is the drone image data, B is the number of samples in each batch, H and W are the height and width of the drone image respectively; Conv represents the convolution layer operation, and Reshape represents the spatial transformation operation; Represents block characteristics, is the number of blocks, h and w represent the height and width of the blocks in the drone image, respectively.

[0016] Preferably, the multi-stage sampling network includes K stages of the network Net k , k∈[0, K-1] represents the stage of the network, and the network at all stages will The input features of size are transformed into The feature output of size; when k=0, the network Net 0 Taking the output of the block mapper As input, output block features ; When k>0, the network Net k Network k-1 Output As input, output block features .

[0017] Preferably, the scale alignment is performed using the dimension reduction mapping layer, and multiple block features output by the multi-stage sampling network are processed using the same fully connected layer to align the block features. Dimensionality reduction into features , expressed as:

[0018] ;

[0019] Among them, FC represents the fully connected layer.

[0020] Preferably, the multi-scale block features are fused through the Hadamard product, which is expressed as:

[0021] ;

[0022] in, Represents the block features after feature fusion, Represents the Hadamard product operation.

[0023] Preferably, the block features after feature fusion are mapped to block category probabilities by using a block classifier, which is implemented through a fully connected layer and a probability mapping function, and is expressed as:

[0024] ;

[0025] Among them, FC represents the fully connected layer, which converts the number of channels of the input feature from D to M, where M is equal to the number of categories; Softmax represents the probability mapping function, which maps the features output by the fully connected layer to the category probability .

[0026] Preferably, the cross entropy loss function is used to calculate the loss of block category probability and block category label, which is expressed as:

[0027] ;

[0028] Among them, CE represents the cross entropy loss function, which is used to calculate the loss value of block category probability and block category label; GT b,n,m represents the label of the nth block in the bth batch of data on the mth category; p b,n,m It represents the probability that the nth block in the bth batch of data belongs to the mth category; L is the block category loss.

[0029] Preferably, the method of using the trained block classification-based target detection model to realize the detection of forest fire risk blocks by drones is specifically as follows:

[0030] The drone image data is input into the trained block classification-based target detection model to obtain the category probability. The category with the largest probability value is taken as the prediction result. The prediction result includes the category prediction of different blocks in the drone image.

[0031] The present invention also provides a UAV forest fire risk block detection system, comprising:

[0032] The dataset acquisition module obtains drone image data with block category labels as the target detection dataset.

[0033] Model building module, building a target detection model based on block classification;

[0034] The model training module uses the cross entropy loss function to calculate the block category probability and the loss of the block category label, and uses the target detection dataset to train the target detection model based on block classification;

[0035] The target detection module uses the trained block classification-based target detection model to detect forest fire risk blocks using drones;

[0036] The block classification-based target detection model uses a block mapper to map different blocks in drone images into block features; uses a multi-stage sampling network to sample block features at multiple scales, and uses a dimensionality reduction mapping layer to align scales to obtain multi-scale block features; then fuses the multi-scale block features through the Hadamard product, and finally uses a block classifier to map the fused multi-scale block features to block category probabilities.

[0037] The present invention has the following beneficial effects: the present invention learns features of different scales through a multi-stage sampling network, and locates the position of the target in different blocks of the image through block classification, so that the drone can achieve target positioning with a smaller amount of calculation in a forest fire scene where the precise position of the target is not required, thereby extending the available time of the drone and improving the detection efficiency of the drone.

[0038] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments, but the present invention is not limited to the embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A method step diagram of an embodiment of the present invention;

[0040] Figure 2 is a model structure diagram of an embodiment of the present invention;

[0041] Figure 3 4 is a system structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0042] See also Figure 1 FIG. 1 is a diagram showing steps of a method according to an embodiment of the present invention, comprising the following steps:

[0043] S101, obtaining drone image data with block category labels as a target detection dataset;

[0044] S102, constructing a target detection model based on block classification;

[0045] S103, using a cross entropy loss function to calculate the block category probability and the loss of the block category label, and using the target detection data set to train a target detection model based on block classification;

[0046] S104, using the trained block classification-based target detection model to implement UAV forest fire risk block detection;

[0047] The block classification-based target detection model can be found in Figure 2As shown in the figure, different blocks in the drone image are mapped into block features using a block mapper; the block features are sampled at various scales using a multi-stage sampling network, and the scales are aligned using a dimensionality reduction mapping layer to obtain multi-scale block features; the multi-scale block features are then fused through the Hadamard product, and finally the block classifier is used to map the fused multi-scale block features to block category probabilities.

[0048] Specifically, the method of obtaining drone image data with block category labels as a target detection data set includes: uniformly adjusting the sizes of all drone images to a height of H and a width of W; and then performing data preprocessing on the drone images using random horizontal flipping.

[0049] Specifically, the block classification-based target detection model uses a stage sampling block classification method to achieve target detection, including the following steps:

[0050] S201, using a block mapper to map different blocks in the drone image into block features, as shown in formula (1):

[0051] (1)

[0052] in, is the drone image data, B is the number of samples in each batch, H and W represent the height and width of the drone image respectively; the block mapper consists of the convolution layer Conv and the spatial transformation operation Reshape; the convolution kernel and step size of the convolution layer Conv are consistent with the block size, that is, the height is h, the width is w, the number of input channels is 3, the number of output channels is C, and the size of its output feature is ; The spatial transformation operation Reshape flattens the features output by the convolution layer Conv in the height and width dimensions, and swaps them with the channel dimensions to output block features , is the number of blocks.

[0053] S202, using a multi-stage sampling network to sample block features at multiple scales, as shown in formula (2):

[0054] (2)

[0055] Among them, the multi-stage sampling network contains 4 stages, Net 0 For the network at stage 0, Net k For the network at stage k, they all have input shape The features of , and the output shape is Characteristics of the network at stage 0 0 Takes the output f of the block mapper as input and outputs the block features of stage 0 ; In the subsequent stages of the network, the network Net of stage k k The network Net at stage k-1 k-1 Output As input, it outputs the block features of stage k . This embodiment selects the ViT-Base model as the backbone network of the multi-stage sampling network, and evenly places the twelve-layer Transformer blocks of ViT-Base in four stages to construct a multi-stage sampling network, namely: stage 0 uses the first three layers of Transformer blocks, stage 1 uses the next three layers of Transformer blocks, stage 2 uses the next three layers of Transformer blocks, and stage 3 uses the last three layers of Transformer blocks.

[0056] S203, using the dimension reduction mapping layer to align the scales of block features of multiple scales, and using the Hadamard product to fuse the scale-aligned block features, as shown in formula (3):

[0057] (3)

[0058] Among them, FC is a fully connected layer, which is used for dimensionality reduction mapping. Its input channel number is C and its output channel number is ; The block features of the four stages are processed using the same dimensionality reduction mapping FC to align them to the same scale; after dimensionality reduction mapping, the block features of the kth stage Reduced to ; It is the block feature after feature fusion, which is obtained by performing Hadamard product operation on the block features of the four stages.

[0059] S204, using a block classifier to map the block features after feature fusion to the block category probability, and using a cross entropy loss function to calculate the loss of the block category probability and the block category label, as shown in formula (4):

[0060] (4)

[0061] Among them, the block classifier consists of a fully connected layer FC and a probability mapping function Softmax; the number of input channels of the fully connected layer FC is D, and the number of output channels is M; M is equal to the number of categories; the probability mapping function Softmax maps the output features of the fully connected layer FC to category probabilities; is the category probability of each block; CE is the cross entropy loss function, which is used to calculate the loss value of the block category probability and the block category label; GT b,n,m represents the label of the nth block in the mth category in the bth batch of data; p b,n,mIt represents the probability that the nth block belongs to the mth category in the bth batch of data; L is the block category loss.

[0062] Specifically, the number of categories can be set according to actual identification requirements, for example, set to 2, including two categories: 1. Risk-free blocks, 2. Risk blocks with fire sources; or set to 4, including 4 categories: 1. Safe blocks with water sources; 2. Low-risk blocks without water sources; 3. Medium-risk blocks with accumulation of flammable materials; 4. High-risk blocks with fire sources.

[0063] Specifically, S103 is as follows: inputting the drone image data into a trained target detection model based on block classification to obtain the category probability, and taking the category with the largest probability value as the prediction result, which is manifested as each block of the image being divided into different risk categories, that is, it can be determined that a certain block of the image belongs to a fire risk area. Figure 2 As shown, the number of categories is 2, and the prediction result is the 01 grid in the figure. When the grid value is 0, it means category 0 (such as no risk), and when the grid value is 1, it means category 1 (such as risk).

[0064] join Figure 3 FIG. 1 is a system structure diagram of an embodiment of the present invention, including:

[0065] The data set acquisition module 301 acquires drone image data with block category labels as a target detection data set.

[0066] A model building module 302 builds a target detection model based on block classification;

[0067] The model training module 303 uses a cross entropy loss function to calculate the block category probability and the loss of the block category label, and uses the target detection data set to train a target detection model based on block classification;

[0068] The target detection module 304 uses the trained block classification-based target detection model to implement UAV forest fire risk block detection.

[0069] In this embodiment, the inference stage is: use the target detection model obtained in the training stage to infer the drone image, use the block category probability output by the block classifier as the classification basis, and select the category with the highest probability in the block as the category into which the block is classified.

[0070] The key of the present invention is to learn features of different scales through a multi-stage sampling network, and locate the position of the target on different blocks of the image by block classification. Specifically, the drone image is divided into N blocks, and N features are generated accordingly, and then N features are fused; each of the N fused features has K scales. This method can reduce the computational complexity of the detection method, extend the available time of the drone, and thus improve the detection efficiency of the drone. Therefore, the present invention can be widely used in target detection applications carried by drones.

[0071] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for detecting forest fire risk areas using an unmanned aerial vehicle, characterized in that: The following steps are involved: Obtain drone image data with block category labels as target detection dataset; Build a target detection model based on block classification; The cross entropy loss function is used to calculate the loss of block category probability and block category label, and the target detection dataset is used to train the target detection model based on block classification; Use the trained block classification-based target detection model to detect forest fire risk blocks using drones; The block classification-based target detection model uses a block mapper to map different blocks in the drone image into block features; A multi-stage sampling network is used to sample block features at multiple scales, and the scales are aligned using a dimensionality reduction mapping layer to obtain multi-scale block features. The multi-scale block features are then fused through the Hadamard product, and finally the block classifier is used to map the fused multi-scale block features to the block category probability.

2. The method for detecting forest fire risk areas by using a drone according to claim 1, characterized in that: The method of obtaining drone image data with block category labels as a target detection dataset includes: Resize all drone images to a uniform height of H and width of W; Data preprocessing of drone images using random horizontal flipping.

3. The method for detecting forest fire risk areas by using a drone according to claim 1, characterized in that: The block mapper receives the drone image and maps different blocks in the drone image into block features, which are expressed as: f = Reshape(Conv(img)); in, is the drone image data, B is the number of samples in each batch, H and W are the height and width of the drone image respectively; Conv represents the convolution layer operation, and Reshape represents the spatial transformation operation; Represents block characteristics, is the number of blocks, h and w represent the height and width of the blocks in the drone image, respectively.

4. The method for detecting forest fire risk areas by using a drone according to claim 3, characterized in that: The multi-stage sampling network contains K stages of the network Net k , k∈[0, K-1] represents the stage of the network, and the network at all stages will The input features of size are transformed into The feature output of size; k =0, the network Net 0 Take the output f of the block mapper as input and output the block feature g 0 ; k >0, network Net k Network k-1 The output of g k-1 As input, output block features 5. The method for detecting forest fire risk areas by using a drone according to claim 4, characterized in that: The dimension reduction mapping layer is used to align the scales, and the multiple block features output by the multi-stage sampling network are processed using the same fully connected layer to align the block features. Dimensionality reduction into multi-scale block features It is expressed as: h k =FC(g k ),k=0,1,2,...,K-1; Among them, FC represents the fully connected layer.

6. The method for detecting forest fire risk areas by using a drone according to claim 5, characterized in that: The fusion of multi-scale block features through Hadamard product is expressed as: in, represents the block feature after feature fusion, and Π represents the Hadamard product operation.

7. The method for detecting forest fire risk areas by using a drone according to claim 6, characterized in that: The block classifier is used to map the block features after feature fusion to the block category probability, which is realized through the fully connected layer and the probability mapping function, and is expressed as: p = Softmax(FC(y)); Among them, FC represents the fully connected layer, which converts the number of channels of the input feature from D to M, where M is equal to the number of categories; Softmax represents the probability mapping function, which maps the features output by the fully connected layer to the category probability p.

8. The method for detecting forest fire risk areas by using a drone according to claim 7, characterized in that: The cross entropy loss function is used to calculate the loss of block category probability and block category label, which is expressed as: Among them, CE represents the cross entropy loss function, which is used to calculate the loss value of block category probability and block category label; GT b,n,m represents the label of the nth block in the bth batch of data on the mth category; p b,n,m It represents the probability that the nth block in the bth batch of data belongs to the mth category; L is the block category loss.

9. The method for detecting forest fire risk areas by using a drone according to claim 1, characterized in that: The trained block classification-based target detection model is used to realize the detection of forest fire risk blocks by drones, specifically: The drone image data is input into the trained block classification-based target detection model to obtain the category probability. The category with the largest probability value is taken as the prediction result. The prediction result includes the category prediction of different blocks in the drone image.

10. A UAV forest fire risk block detection system, characterized in that: include: The data set acquisition module acquires drone image data with block category labels as target detection data set; Model building module, building a target detection model based on block classification; The model training module uses the cross entropy loss function to calculate the block category probability and the loss of the block category label, and uses the target detection dataset to train the target detection model based on block classification; The target detection module uses the trained block classification-based target detection model to detect forest fire risk blocks using drones; The block classification-based target detection model uses a block mapper to map different blocks in the drone image into block features; A multi-stage sampling network is used to sample block features at multiple scales, and the scales are aligned using a dimensionality reduction mapping layer to obtain multi-scale block features. The multi-scale block features are then fused through the Hadamard product, and finally the block classifier is used to map the fused multi-scale block features to the block category probability.

Images