Forest fire prevention unmanned aerial vehicle early warning system and method based on laser point cloud recognition

Abstract

The invention relates to the technical field of fire early warning, and discloses a forest fire prevention unmanned aerial vehicle early warning method based on laser point cloud recognition, and the method comprises the steps: obtaining initial point cloud image data obtained by scanning a forest region through an unmanned aerial vehicle; performing image denoising processing on the initial point cloud image data to obtain target point cloud image data; performing dynamic target identification on the target point cloud image data to extract a dynamic feature image; performing flame and smoke feature extraction according to the dynamic feature image, and generating a fire feature vector; and inputting the fire feature vector into a preset KELM model for fire identification, transmitting an identification result to a background, performing alarm processing when the fire is identified, and starting acousto-optic early warning at the same time. According to the invention, the point cloud image is acquired through the laser radar carried by the unmanned aerial vehicle, deep processing is carried out, flame and smoke characteristics are acquired, and fire identification is carried out by using the preset KELM model, so that the detection efficiency of the forest fire is improved, and rapid detection and early warning of the fire are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of fire warning, and particularly relates to a forest fire prevention UAV warning system and method based on laser point cloud recognition. Background Technique

[0002] Forest fires are one of the primary natural disasters that destroy forests. Forest fires occur globally every year. Such fires usually last for a long time and are difficult to extinguish, causing serious harm to forestry production. In 2021, 616 forest fires occurred in China, covering an area of more than 4000 hm 2 . If forest fires can be detected and contained in the impending or initial stage, the least amount of resources can be used to prevent large forest fires and avoid potential hazards in the later stage. Due to limitations in technical levels, in the early days, humans usually used traditional methods to detect forest fires. For example, the method used in China in the early days was to conduct manual patrols on the ground to find out if a fire had occurred. In addition, there are some other traditional forest fire detection methods, such as lookout tower monitoring, remote sensing technology, and aerial patrols, etc. However, these traditional detection methods each have different defects. Manual patrols and lookout tower detections are inefficient and have high labor costs. Satellite remote sensing can only provide a complete image of the earth once every 1-2 days and cannot achieve rapid real-time detection. Summary of the Invention

[0003] The present invention provides a forest fire prevention UAV warning system and method based on laser point cloud recognition. By using a lidar carried by the UAV to obtain a point cloud image and perform in-depth processing, the characteristics of flames and smoke are obtained, and a preset KELM model is used for fire recognition, improving the detection efficiency of forest fires and realizing rapid detection and warning of fires.

[0004] The present invention provides a forest fire prevention UAV warning method based on laser point cloud recognition, including:

[0005] Obtaining initial point cloud image data obtained by the UAV scanning a forest area; wherein, the UAV flies on a preset monitoring route and is equipped with a lidar for scanning;

[0006] Performing image denoising processing on the initial point cloud image data to obtain target point cloud image data;

[0007] Performing dynamic target recognition on the target point cloud image data to extract the dynamic feature image in the target point cloud image data;

[0008] Extracting the characteristics of flames and smoke according to the dynamic feature image, and generating a fire feature vector by using the characteristics of flames and smoke;

[0009] Input the fire feature vector into a preset KELM model for fire recognition, and transmit the recognition result to the background. When a fire is recognized, alarm processing is performed, and at the same time, an audible and visual warning is activated.

[0010] Further, the step of performing image denoising processing on the initial point cloud image data to obtain the target point cloud image data includes:

[0011] Randomly select a point a to be denoised from the initial point cloud image data i , and calculate the distance between point a i and other points in the initial point cloud image data. The calculation formula is:

[0012]

[0013] where d ij represents the distance between point a i and other point a j (x i , y i , z i ) represents the three-dimensional coordinates of point a i , (x j , y j , z j ) represents the three-dimensional coordinates of point a j ;

[0014] Establish a neighbor set of point a ij , denoted as A(a i ), and establish the unit normal vector between point a i and a i in A(a i ) j

[0015] Calculate the inner product B of the distance vector between point a i and point a j and the normal vector from point a i to , that is:

[0016] B = [a i - a j , b i

[0017] Calculate the Gaussian kernel functions of the bilateral filtering function in the spatial domain and in the frequency domain, denoted as f 1 (||d ij ||), f 2 (B). The calculation formula is:

[0018] ​

[0019] Among them, c represents point a i and point a j The distance to a i The influence factor on a represents point a i and point a j The distance of is in The projection on a i The influence factor;

[0020] According to f 1 (||d ij ||), f 2 (B) Calculate the bilateral filtering weight factor, and the calculation formula is:

[0021]

[0022] Among them, D represents the bilateral filtering weight factor;

[0023] Substitute D and into the formula Get the denoised point cloud data;

[0024] Repeat the above process, traverse all the point cloud data, complete the denoising of all the point cloud data, and obtain the target point cloud image data.

[0025] Furthermore, the step of performing dynamic target recognition on the target point cloud image data to extract the dynamic feature image in the target point cloud image data includes:

[0026] Use the Euclidean clustering method to segment the background point cloud and the dynamic target point cloud in the target point cloud image data;

[0027] Use the Freeman chain code vector to detect the edge features of the dynamic target to obtain the dynamic feature image.

[0028] Furthermore, the step of using the Euclidean clustering method to segment the background point cloud and the dynamic target point cloud in the target point cloud image data includes:

[0029] Search for the neighboring points of each point cloud in the target point cloud image data and compare the Euclidean distances between each point cloud and its neighboring points;

[0030] Normalize the minimum value and perform Euclidean iteration with other classes. When the Euclidean distance between any two classes is less than the set threshold, the segmentation ends;

[0031] Let the set of the target point cloud image data be S=(1, 2, 3,..., k), the total number of image background point clouds be N, the total number of dynamic target point clouds be n, and calculate the average value of the whole image point cloud as W1 = ∑(n × N) / N, where U is the total proportion of the background point cloud in the image a = N 1 / N, where U is the total proportion of the dynamic target point cloud in the image b = N 1 / n, calculate the between-class variance to complete the segmentation of the background and the dynamic target. The segmentation algorithm is V = U + U b .

[0032] Furthermore, the step of detecting the edge features of the dynamic target using the Freeman chain code vector to obtain the dynamic feature image includes:

[0033] Search for non-zero pixel points of the dynamic target in the target point cloud image data, and record the coordinates of the first non-zero pixel point found as (x 0 , y 0 );

[0034] Take the first non-zero pixel point found as a new starting point for searching, and search for pixel points in the neighborhood in a clockwise direction from "0" to "8", and at the same time determine whether the pixel point is 0; if so, record the pixel point coordinates as (x 1 , y 1 ), otherwise continue to search for the next non-zero pixel point and repeat this operation;

[0035] The coordinates of the new search center point are (x 1 , y 1 ), repeat the above two steps in the direction of 8→7, and record the coordinates of the next adjacent pixel point found as (x 2 , y 2 );

[0036] Repeat the above steps until the point cloud coordinates found coincide with the initial point cloud coordinates (x 0 , y 0 ), the edge feature detection ends, realizing the recognition of the dynamic target, and obtaining the dynamic feature image.

[0037] Furthermore, the step of extracting the flame and smoke features based on the dynamic feature image and generating the fire feature vector using the flame and smoke features includes:

[0038] Project the dynamic feature image onto a two-dimensional plane using the z coordinate to generate a depth map or a grayscale map, and quantize the projected depth values into the grayscale range, and normalize the quantized grayscale values to obtain the two-dimensional dynamic grayscale map of the dynamic feature image;

[0039] Extract the flame and smoke features from the two-dimensional dynamic grayscale image, including shape features and texture features; among them, the shape features include circularity and the number of sharp corners, and the texture features include energy, entropy, inverse difference moment, inertia moment, and correlation moment:

[0040] The calculation formula for circularity is:

[0041]

[0042] where S is the area of the dynamic feature image, and L is the contour perimeter of the dynamic feature image;

[0043] The calculation formula for energy is:

[0044]

[0045] The calculation formula for the entropy value is:

[0046]

[0047] The calculation formula for the inverse difference moment is:

[0048]

[0049] The calculation formula for the inertia moment is:

[0050]

[0051] The calculation formula for the correlation moment is:

[0052]

[0053] where P is the pixel value of (i,j), G is the order of the gray-level co-occurrence matrix, and δ,θ respectively represent the distance and direction of P;

[0054] Generate a fire feature vector according to the shape features and texture features, and its formula is:

[0055] F = [C, N, M E , M ENT , M IDM , M IM , M COR , S E , S ENT , S IDM , S IM , S COR

[0056] where F is the fire feature vector, C is circularity, N is the number of sharp corners, M ENT , M IDM , M IM , M COR ​The average differences of energy, entropy, inverse difference moment, inertia moment, and correlation moment are S respectively. E , S ENT , S IDM , S IM , S COR The standard deviations of energy, entropy, inverse difference moment, inertia moment, and correlation moment are respectively.

[0057] Furthermore, in the step of inputting the fire feature vector into a preset KELM model for fire recognition, transmitting the recognition result to the background, and performing alarm processing and turning on the sound and light warning when a fire is recognized, the preset KELM model includes:

[0058] The output result of the extreme learning machine is:

[0059]

[0060] The Gaussian radial basis kernel function is used as the kernel function of the kernel extreme learning machine, and its expression is:

[0061]

[0062] where T is the target output corresponding to the input sample, and δ is the bandwidth of the Gaussian radial basis kernel function;

[0063] Taking the fire feature vector as the input of the KELM model, normalizing different eigenvalue of the fire to [-1, 1], and its formula is:

[0064]

[0065] where x i is the i-th (i = 1, 2,..., 12) dimensional feature quantity in the fire feature vector, x max and x min are the maximum and minimum values of this feature quantity respectively. Taking the recognition rate of KELM as the fitness function of the PSO algorithm, as the preset KELM model, its steps are:

[0066] Performing particle initialization or updating the particle swarm, and using the test data for classification processing; where the test data is extracted from the training dataset of the normalized fire feature vector;

[0067] Calculating the classification accuracy rate of KELM and judging whether the termination condition is satisfied;

[0068] If not, then return to perform particle initialization or update the particle swarm; if so, obtain the optimal parameters C, δ, and use the optimal parameters C, δ to establish the preset KELM model.

[0069] The present invention also provides a forest fire prevention UAV warning system based on laser point cloud recognition, including:

[0070] An acquisition module, configured to acquire initial point cloud image data obtained by the UAV scanning a forest area; wherein, the UAV flies on a preset monitoring route and is equipped with a lidar for scanning;

[0071] A processing module, configured to perform image denoising processing on the initial point cloud image data to obtain target point cloud image data;

[0072] An extraction module, configured to perform dynamic target recognition on the target point cloud image data to extract a dynamic feature image from the target point cloud image data;

[0073] A generation module, configured to extract flame and smoke features according to the dynamic feature image, and generate a fire feature vector by using the flame and smoke features;

[0074] A warning module, configured to input the fire feature vector into a preset KELM model for fire recognition, transmit the recognition result to the background, perform alarm processing when a fire is recognized, and simultaneously turn on an audible and visual alarm.

[0075] The present invention also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0076] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0077] The beneficial effects of the present invention are as follows:

[0078] The present invention acquires initial point cloud image data obtained by the UAV scanning a forest area; performs image denoising processing on the initial point cloud image data to obtain target point cloud image data; performs dynamic target recognition on the target point cloud image data to extract a dynamic feature image; extracts flame and smoke features according to the dynamic feature image, and generates a fire feature vector, and finally inputs the fire feature vector into a preset KELM model for fire recognition, performs alarm processing when a fire is recognized, and simultaneously turns on an audible and visual alarm, improves the detection efficiency of forest fires, realizes rapid detection and early warning of fires, and saves labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention.

[0080] Figure 2 It is a schematic structural diagram of the device according to an embodiment of the present invention.

[0081] Figure 3 Schematic diagram of the internal structure of a computer device according to an embodiment of the present invention.

[0082] The realization of the object of the present invention, functional features and advantages will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0083] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0084] As Figure 1 shown, the present invention provides a forest fire prevention UAV warning method based on laser point cloud recognition, including:

[0085] S1. Obtain the initial point cloud image data scanned by the UAV for the forest area; wherein, the UAV flies on a preset monitoring route and is equipped with a lidar for scanning;

[0086] S2. Perform image denoising processing on the initial point cloud image data to obtain target point cloud image data.

[0087] For the initial point cloud image data captured by the lidar, point cloud data denoising processing is required to improve the data quality. The noise point cloud is an objective interference data caused by the measurement instrument itself and other external environments, etc., which seriously affects the quality of the point cloud data and must be removed. The selected method is the bilateral filtering algorithm. The point cloud data denoising process, that is, step S2 specifically includes:

[0088] S201. Randomly select a point a to be denoised from the initial point cloud image data i , and calculate the distance between point a i and other points in the initial point cloud image data. The calculation formula is:

[0089]

[0090] where d ij represents the distance between point a i and other point a j , (x i , y i , z i ) represents the three-dimensional coordinates of point a i , (x j , y j , z j ) represents the three-dimensional coordinates of point a j ;

[0091] S202. Establish point a according to d ij ​i The set of neighbors, denoted as A(a i ), and establish the unit normal vector i between point a i and A(a j )

[0092] S203. Calculate the inner product B of the distance vector between point a i and point a j and the normal vector from point a i to , that is:

[0093] B = [a i - a j , b i

[0094] S204. Calculate the Gaussian kernel functions of the bilateral filtering function in the spatial domain and in the frequency domain, denoted as f 1 (||d ij ||), f 2 (B), and the calculation formulas are:

[0095]

[0096] where c represents the influence factor of the distance between point a i and point a j on a i , represents the influence factor of the projection of the distance between point a i and point a j on on a i ;

[0097] S205. Calculate the bilateral filtering weight factor according to f 1 (||d ij ||), f 2 (B), and the calculation formula is:

[0098]

[0099] where D represents the bilateral filtering weight factor;

[0100] S206. Substitute D and into the formula to obtain the denoised point cloud data;

[0101] S207. Repeat the above process, traverse all the point cloud data, complete the denoising of all the point cloud data, and obtain the target point cloud image data.

[0102] ​S3. Perform dynamic target recognition on the target point cloud image data to extract the dynamic feature image in the target point cloud image data.

[0103] Step S3 specifically includes:

[0104] S301. Use the Euclidean clustering method to segment the background point cloud and the dynamic target point cloud in the target point cloud image data.

[0105] Use the Euclidean clustering method to segment the background point cloud and the dynamic target point cloud in the target point cloud image data, divide the point cloud into smaller parts, and establish an effective topological relationship between the scattered point clouds. The Euclidean clustering method uses the distance information between point clouds as the basis for judging segmentation and does not require the color information of points. Therefore, introducing the Euclidean clustering method can improve the efficiency of target recognition. The specific process is as follows:

[0106] Search for the neighboring points of each point cloud in the target point cloud image data and compare the Euclidean distance between each point cloud and its neighboring points;

[0107] Normalize the minimum value and perform Euclidean iteration with other classes. When the Euclidean distance between any two classes is less than the set threshold, the segmentation ends;

[0108] Let the set of the target point cloud image data be S = (1, 2, 3,..., k), the total number of image background point clouds be N, the total number of dynamic target point clouds be n, and calculate the average value of the whole image point cloud as W 1 = ∑(n × N) / N, the total proportion of the image occupied by the background point cloud is U a = N 1 / N, the total proportion of the image occupied by the dynamic target point cloud is U b = N 1 / n, calculate the between-class variance, complete the segmentation of the background and the dynamic target, and the segmentation algorithm is V = U + U b .

[0109] S302. Use the Freeman chain code vector to detect the edge features of the dynamic target to obtain the dynamic feature image. Specifically, it includes:

[0110] S3021. Search for the pixel points in the target point cloud image data where the dynamic target is not zero, and record the coordinates of the first non-zero pixel point found as (x 0 , y 0 );

[0111] S3022. Use the first non-zero pixel point found as a new starting point for searching, and search for the pixel points in the neighborhood in a clockwise direction from "0" to "8", and at the same time judge whether the pixel point is 0; if so, then record the coordinates of the pixel point as (x 1 , y1 ), if not, continue to find the next non - zero pixel point and repeat this operation;

[0112] S3023. The coordinates of the new search center point are (x 1 , y 1 ). Repeat the previous two steps in the direction of 8→7, and record the coordinates of the next adjacent pixel point found as (x 2 , y 2 );

[0113] S3024. Repeat the above steps until the point cloud coordinates found coincide with the initial point cloud coordinates (x 0 , y 0 ). The edge feature detection ends, realizing dynamic target recognition and obtaining a dynamic feature image.

[0114] S4. Extract flame and smoke features according to the dynamic feature image, and generate a fire feature vector using the flame and smoke features.

[0115] Step S4 specifically includes:

[0116] S401. Project the dynamic feature image onto a two - dimensional plane using the z - coordinate to generate a depth map or a grayscale map, and quantize the projected depth value into the grayscale range (0 to 255). Normalize the quantized grayscale value to ensure it is between 0 and 255, obtaining a two - dimensional dynamic grayscale map of the dynamic feature image;

[0117] S402. Extract flame and smoke features from the two - dimensional dynamic grayscale map, including shape features and texture features; among them, the shape features include circularity and the number of sharp corners, and the texture features include energy, entropy, inverse difference moment, inertia moment, and correlation moment. The textures of flames and smoke are significantly different from other interfering objects. The gray - level co - occurrence matrix GLCM considers second - order statistics and studies pixel pairs with a certain relationship, which not only reflects the distribution characteristics of brightness but also the position distribution characteristics between pixels.

[0118] 1) Circularity refers to the degree of similarity between the contour of the target area and a circle, which can be used to measure the complexity of the object contour. The calculation formula for circularity is:

[0119]

[0120] where S is the area of the dynamic feature image and L is the contour perimeter of the dynamic feature image;

[0121] 2) The number of sharp corners. Since there is an edge jitter phenomenon during the combustion of the flame, many small sharp corners will be generated. Therefore, the number of sharp corners is used as a feature.

[0122] 3) Energy reflects the degree of uniformity of the grayscale distribution of the image. The calculation formula for energy is:

[0123]

[0124] 4) Entropy reflects the degree of disorder of the image. The calculation formula for the entropy value is:

[0125]

[0126] 5) The inverse difference moment reflects the degree of local change of the image texture. The calculation formula for the inverse difference moment is:

[0127]

[0128] 6) The moment of inertia describes the spatial difference of the grayscale distribution of the image. The calculation formula for the moment of inertia is:

[0129]

[0130] 7) The correlation moment reflects the linear correlation between pixel grayscale values. The calculation formula for the correlation moment is:

[0131]

[0132] Among them, P is the pixel value of (i, j), G is the order of the gray-level co-occurrence matrix, and δ and θ respectively represent the distance and direction of P;

[0133] S403. Generate a fire feature vector according to the shape feature and the texture feature. The formula is:

[0134] F = [C, N, M E , M ENT , M IDM , M IM , M COR , S E , S ENT , S IDM , S IM , S COR

[0135] Among them, F is the fire feature vector, C is the circularity, N is the number of sharp corners, and M ENT , M IDM , M IM , M COR are respectively the average differences of energy, entropy, inverse difference moment, moment of inertia, and correlation moment, and S E , S ENT , S IDM , S IM , S COR ​They are the standard deviations of energy, entropy, inverse difference moment, moment of inertia, and correlation moment respectively.

[0136] S5. Input the fire feature vector into a preset KELM model for fire recognition, and transmit the recognition result to the background. When a fire is recognized, alarm processing is performed, and at the same time, acoustic and optical warnings are activated.

[0137] The extreme learning machine (ELM) is a fast and efficient single-hidden-layer feedforward neural network. Compared with the traditional single-hidden-layer feedforward neural network, ELM has a faster learning speed and generalization ability.

[0138] The mathematical model of the extreme learning machine is expressed as:

[0139] Y = Hβ

[0140] Among them, H is the output matrix of the hidden layer, β is the output weight matrix, and Y is the output matrix of the extreme learning machine. Its learning purpose is to minimize the training error and the hidden layer weight norm, and its objective function is:

[0141]

[0142] Among them, T is the target output corresponding to the input sample. According to the KKT theory, the output result of the extreme learning machine is:

[0143]

[0144] The Gaussian radial basis kernel function is used as the kernel function of the kernel extreme learning machine, and its expression is:

[0145]

[0146] Among them, T is the target output corresponding to the input sample, and δ is the bandwidth of the Gaussian radial basis kernel function;

[0147] Take the fire feature vector as the input of the KELM model, and normalize different eigenvalue of the fire to [-1, 1]. Its formula is:

[0148]

[0149] Among them, x i is the i-th (i = 1, 2,..., 12) dimensional feature quantity in the fire feature vector, x max and x min are the maximum and minimum values of this feature quantity respectively. Take the recognition rate of KELM as the fitness function of the PSO algorithm. As a preset KELM model, its steps are:

[0150] Initialize particles or update the particle swarm, and perform classification processing using test data; wherein, the test data is extracted from the training dataset of the normalized fire feature vectors;

[0151] Calculate the classification accuracy rate of KELM, and determine whether the termination condition is satisfied;

[0152] If not, return to initialize particles or update the particle swarm; if so, obtain the optimal parameters C and δ, and use the optimal parameters C and δ to establish a preset KELM model.

[0153] The present invention obtains the initial point cloud image data scanned by the drone for the forest area; performs image denoising processing on the initial point cloud image data to obtain the target point cloud image data; performs dynamic target recognition on the target point cloud image data to extract the dynamic feature image; extracts the fire and smoke features according to the dynamic feature image, and generates the fire feature vector, and finally inputs the fire feature vector into the preset KELM model for fire recognition. When a fire is recognized, an alarm is processed, and at the same time, the sound and light warning is turned on, which improves the detection efficiency of forest fires, realizes the rapid detection and warning of fires, and saves labor costs.

[0154] The present invention also provides a forest fire prevention drone warning system based on laser point cloud recognition, including:

[0155] An acquisition module 1, configured to acquire the initial point cloud image data scanned by the drone for the forest area; wherein, the drone flies on a preset monitoring route and is equipped with a lidar for scanning;

[0156] A processing module 2, configured to perform image denoising processing on the initial point cloud image data to obtain the target point cloud image data;

[0157] An extraction module 3, configured to perform dynamic target recognition on the target point cloud image data to extract the dynamic feature image in the target point cloud image data;

[0158] A generation module 4, configured to extract the fire and smoke features according to the dynamic feature image, and generate a fire feature vector by using the fire and smoke features;

[0159] An early warning module 5, configured to input the fire feature vector into a preset KELM model for fire recognition, and transmit the recognition result to the background. When a fire is recognized, an alarm is processed, and at the same time, the sound and light warning is turned on.

[0160] In one embodiment, the processing module 2 includes:

[0161] A selection unit, configured to randomly select a point a to be denoised from the initial point cloud image data i , and calculate the point ai The distance between it and other points in the initial point cloud image data, the calculation formula is:

[0162]

[0163] Where d ij represents the distance between point a i and other points a j , (x i , y i , z i ) represents the three-dimensional coordinates of point a i , (x j , y j , z j ) represents the three-dimensional coordinates of point a j ;

[0164] A unit is established to establish the neighbor set of point a ij , denoted as A(a i ), and establish the unit normal vector between point a i and a i in A(a i ) j

[0165] The first calculation unit is used to calculate the inner product B of the distance vector between point a i and point a j and the normal vector from point a i to , that is:

[0166] B = [a i - a j , b i

[0167] The second calculation unit is used to calculate the Gaussian kernel functions of the bilateral filtering function in the spatial domain and in the frequency domain, denoted as f 1 (||d ij ||), f 2 (B), and the calculation formula is:

[0168]

[0169] Where c represents the influence factor of the distance between point a i and point a j on a i , represents the influence factor of the projection of the distance between point a i and point a j on a on a i ;

[0170] A third calculation unit, configured to calculate a bilateral filtering weight factor according to f 1 (||d ij ||), f 2 (B), and the calculation formula is:

[0171]

[0172] where D represents the bilateral filtering weight factor;

[0173] A substitution unit, configured to substitute D and into the formula to obtain the denoised point cloud data;

[0174] A traversal unit, configured to repeat the above process, traverse all the point cloud data, complete the denoising of all the point cloud data, and obtain the target point cloud image data.

[0175] In one embodiment, the extraction module 3 includes:

[0176] A segmentation unit, configured to segment the background point cloud and the dynamic target point cloud in the target point cloud image data by using the Euclidean clustering method;

[0177] A detection unit, configured to detect the edge features of the dynamic target by using the Freeman chain code vector to obtain a dynamic feature image.

[0178] In one embodiment, the segmentation unit includes:

[0179] configured to search for neighboring points of each point cloud in the target point cloud image data and compare the Euclidean distance between each point cloud and its neighboring points;

[0180] normalize the minimum value and perform Euclidean iteration with other classes, and when the Euclidean distance between any two classes is less than a set threshold, the segmentation ends;

[0181] Let the set of the target point cloud image data be S=(1, 2, 3,..., k), the total number of image background point clouds be N, and the total number of dynamic target point clouds be n. Calculate the average value of the whole image point cloud as W 1 =∑(n×N) / N, the total proportion of the background point cloud in the image is U a =N 1 / N, the total proportion of the dynamic target point cloud in the image is U b =N 1 / n, calculate the between-class variance, complete the segmentation of the background and the dynamic target, and the segmentation algorithm is V = U + U b .

[0182] In one embodiment, the detection unit includes:

[0183] Search sub-unit, which is used to search for pixel points with non-zero dynamic targets in the target point cloud image data, and record the coordinates of the first non-zero pixel point found as (x 0 , y 0 );

[0184] Search sub-unit, which is used to take the first non-zero pixel point found as a new starting point for searching, and search for pixel points in the neighborhood in a clockwise direction from "0" to "8", and at the same time judge whether the pixel point is 0; if so, then record the pixel point coordinates as (x 1 , y 1 ), if not, then continue to find the next non-zero pixel point and repeat this operation;

[0185] Repeat sub-unit, which is used for the new search center point coordinates to be (x 1 , y 1 ), repeat the previous two steps in the direction of 8→7, and record the coordinates of the next adjacent pixel point found as (x 2 , y 2 );

[0186] Stop sub-unit, which is used to repeat the above steps until the searched point cloud coordinates coincide with the initial point cloud coordinates (x 0 , y 0 ), the edge feature detection ends, the dynamic target recognition is realized, and the dynamic feature image is obtained.

[0187] In one embodiment, the generation module 4 includes:

[0188] Projection unit, which is used to project the dynamic feature image onto a two-dimensional plane using the z coordinate to generate a depth map or a grayscale map, quantize the projected depth value into the grayscale range, and normalize the quantized grayscale value to obtain the two-dimensional dynamic grayscale map of the dynamic feature image;

[0189] Feature extraction unit, which is used to extract flame and smoke features according to the two-dimensional dynamic grayscale map, including shape features and texture features; among them, the shape features include circularity and the number of sharp corners, and the texture features include energy, entropy, inverse difference moment, inertia moment, relationship moment:

[0190] The calculation formula for circularity is:

[0191]

[0192] Among them, S is the area of the dynamic feature image, and L is the contour perimeter of the dynamic feature image;

[0193] The calculation formula for energy is:

[0194]

[0195] The calculation formula for the entropy value is as follows:

[0196]

[0197] The calculation formula for the inverse difference moment is as follows:

[0198]

[0199] The calculation formula for the moment of inertia is as follows:

[0200]

[0201] The calculation formula for the correlation moment is as follows:

[0202]

[0203] Where P is the pixel value of (i, j), G is the order of the gray-level co-occurrence matrix, and δ and θ respectively represent the distance and direction of P;

[0204] A vector generation unit, configured to generate a fire feature vector according to the shape feature and the texture feature, and its formula is:

[0205] F = [C, N, M E , M ENT , M IDM , M IM , M COR , S E , S ENT , S IDM , S IM , S COR

[0206] Where F is the fire feature vector, C is the circularity, N is the number of sharp corners, M ENT , M IDM , M IM , M COR are respectively the average differences of energy, entropy, inverse difference moment, moment of inertia, and correlation moment, and S E , S ENT , S IDM , S IM , S COR are respectively the standard deviations of energy, entropy, inverse difference moment, moment of inertia, and correlation moment.

[0207] In one embodiment, in the warning module 5, the preset KELM model includes:

[0208] The output result of the extreme learning machine is:

[0209]

[0210] ​The Gaussian radial kernel function is used as the kernel function of the kernel extreme learning machine, and its expression is:

[0211]

[0212] where T is the target output corresponding to the input sample, and δ is the bandwidth of the Gaussian radial kernel function;

[0213] Taking the fire feature vector as the input of the KELM model, different eigenvalue of the fire is normalized to [-1, 1], and its formula is:

[0214]

[0215] where x i is the i-th (i = 1, 2,..., 12) dimensional feature quantity in the fire feature vector, x max and x min are the maximum value and the minimum value of this feature quantity respectively. Taking the recognition rate of KELM as the fitness function of the PSO algorithm, as a preset KELM model, its steps are:

[0216] Perform particle initialization or update the particle swarm, and use the test data for classification processing; wherein, the test data is extracted from the training data set of the normalized fire feature vector;

[0217] Calculate the classification accuracy rate of KELM, and determine whether the termination condition is met;

[0218] If not, return to perform particle initialization or update the particle swarm; if so, obtain the optimal parameters C, δ, and use the optimal parameters C, δ to establish the preset KELM model.

[0219] Each of the above modules, units, and subunits is used to correspondingly execute each step in the above forest fire prevention UAV warning method based on laser point cloud recognition, and its specific implementation manner refers to the above method embodiment and will not be elaborated here.

[0220] As Figure 3 shown, the present invention also provides a computer device, which can be a server, and its internal structure can be as Figure 3As shown in the figure. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store all the data required for the process of the forest fire prevention UAV warning method based on laser point cloud recognition. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it implements the forest fire prevention UAV warning method based on laser point cloud recognition.

[0221] Those skilled in the art can understand that Figure 3 the structure shown in the figure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.

[0222] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements any one of the above forest fire prevention UAV warning methods based on laser point cloud recognition.

[0223] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium provided in the present application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or an external cache. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0224] It should be noted that in this text, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, apparatus, article or method including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent in such a process, apparatus, article or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, apparatus, article or method including such an element.

[0225] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A forest fire prevention drone early warning method based on laser point cloud recognition, characterized in that: include: Acquire initial point cloud image data obtained by scanning a forest area with a drone; wherein the drone flies on a preset monitoring route and carries a laser radar for scanning; Performing image denoising processing on the initial point cloud image data to obtain target point cloud image data; Performing dynamic target recognition on the target point cloud image data to extract dynamic feature images from the target point cloud image data; Extracting flame and smoke features according to the dynamic feature image, and generating a fire feature vector using the flame and smoke features; The fire feature vector is input into a preset KELM model for fire identification, and the identification result is transmitted to the background. When a fire is identified, an alarm is processed and an audible and visual warning is turned on at the same time.

2. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 1 is characterized in that: The step of performing image denoising processing on the initial point cloud image data to obtain target point cloud image data comprises: A point a to be denoised is randomly selected from the initial point cloud image data. i , and calculate point a i The distance between the initial point cloud image data and other points is calculated as follows: Among them, d ij Represents point a i With other points a j The distance between i ,y i ,z i ) represents point a i The three-dimensional coordinates (x j ,y j ,z j ) represents point a j The three-dimensional coordinates of According to d ij Establish point a i The neighbor set of i ), and establish point a i with A(a i ) j The unit normal vector between Calculate point a i With point a j The distance vector between point a i arrive The inner product B of the normal vector is: B=[a i -a j ,b i Calculate the Gaussian kernel function of the bilateral filter function in the spatial domain and in the frequency domain, denoted as f1(||d ij ||), f2(B), the calculation formula is: Where c represents point a i With point a j The distance to a i The impact factor, Indicates point a i With point a j The distance is The projection on a i Impact factor; According to f1(||d ij ||), f2(B) calculates the bilateral filtering weight factor, and the calculation formula is: Where D represents the bilateral filtering weight factor; D and Substitute into the formula Get the denoised point cloud data; Repeat the above process, traverse all point cloud data, complete all point cloud data denoising, and obtain the target point cloud image data.

3. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 1 is characterized in that: The step of performing dynamic target recognition on the target point cloud image data to extract dynamic feature images from the target point cloud image data comprises: Using the Euclidean clustering method to segment the background point cloud and the dynamic target point cloud in the target point cloud image data; The Freeman chain code vector is used to detect the edge features of dynamic targets and obtain a dynamic feature image.

4. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 3 is characterized in that: The step of segmenting the background point cloud and the dynamic target point cloud in the target point cloud image data by using the Euclidean clustering method comprises: Searching for neighboring points of each point cloud in the target point cloud image data, and comparing the Euclidean distance between each point cloud and the neighboring points; Normalize the minimum value and perform Euclidean iteration with other classes. When the Euclidean distance between any two classes is less than the set threshold, the segmentation ends. Assume that the set of target point cloud image data is S = (1, 2, 3, ..., k), the total number of image background point clouds is N, the total number of dynamic target point clouds is n, the average value of the full image point cloud is calculated as W1 = ∑ (n × N) / N, and the total proportion of the background point cloud in the image is U a =N1 / N, the proportion of the dynamic target point cloud in the total image is U b =N1 / n, calculate the inter-class variance, complete the background and dynamic target segmentation, the segmentation algorithm is V=U+U b .

5. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 3 is characterized in that: The step of using the Freeman chain code vector to detect the edge features of the dynamic target to obtain the dynamic feature image includes: Searching for non-zero pixel points of the dynamic target in the target point cloud image data, and marking the coordinates of the first non-zero pixel point found as (x0, y0); The first non-zero pixel is used as the new starting point for searching. The neighboring pixels are searched clockwise from "0" to "8". At the same time, it is determined whether the pixel is 0. If so, the coordinates of the pixel are marked as (x1, y1). If not, continue to search for the next non-zero pixel and repeat this operation. The new search center point coordinates are (x1, y1). Repeat the first two steps in the direction of 8→7 and mark the coordinates of the next adjacent pixel point found as (x2, y2). Repeat the above steps until the searched point cloud coordinates coincide with the initial point cloud coordinates (x0, y0), the edge feature detection is completed, the dynamic target recognition is realized, and the dynamic feature image is obtained.

6. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 1 is characterized in that: The step of extracting flame and smoke features according to the dynamic feature image and generating a fire feature vector using the flame and smoke features comprises: Projecting the dynamic feature image onto a two-dimensional plane using a z coordinate to generate a depth map or a grayscale map, quantizing the projected depth value within a grayscale range, and normalizing the quantized grayscale value to obtain a two-dimensional dynamic grayscale map of the dynamic feature image; According to the two-dimensional dynamic grayscale image, the flame and smoke features are extracted, including shape features and texture features; wherein the shape features include roundness and the number of sharp corners, and the texture features include energy, entropy, inverse moment, moment of inertia, and relationship moment: The calculation formula for circularity is: Among them, S is the area of ​​the dynamic feature image, and L is the contour perimeter of the dynamic feature image; The energy calculation formula is: The calculation formula of entropy value is: The calculation formula of the inverse moment is: The formula for calculating the moment of inertia is: The calculation formula of the correlation moment is: Where P is the pixel value of (i, j), G is the order of the gray-level co-occurrence matrix, δ and θ represent the distance and direction of P respectively; A fire feature vector is generated according to the shape feature and texture feature, and its formula is: F=[C,N,M E ,M ENT ,M IDM ,M IM ,M COR ,S E ,S ENT ,S IDM ,S IM ,S COR ] Among them, F is the fire characteristic vector, C is the circularity, N is the number of sharp corners, M is ENT ,M IDM ,M IM ,M COR are the average differences of energy, entropy, inverse moment, moment of inertia, and correlation moment, respectively. E ,S ENT ,S IDM ,S IM ,S COR They are the standard deviations of energy, entropy, inverse moment, moment of inertia, and correlation moment, respectively.

7. The forest fire prevention drone early warning method based on laser point cloud recognition according to claim 6 is characterized in that: In the step of inputting the fire feature vector into a preset KELM model for fire identification, transmitting the identification result to the background, and performing alarm processing when a fire is identified, and simultaneously starting the sound and light warning, the preset KELM model includes: The output of the extreme learning machine is: The Gaussian radial kernel function is used as the kernel function of the kernel extreme learning machine, and its expression is: Where T is the target output corresponding to the input sample, and δ is the bandwidth of the Gaussian radial kernel function; The fire feature vector is used as the input of the KELM model, and the different feature values ​​of the fire are normalized to [-1,1]. The formula is: Among them, x i is the i-th (i=1,2,…,12)-th dimension feature value in the fire feature vector, x max With x min are the maximum and minimum values ​​of the feature value respectively, and the recognition rate of KELM is used as the fitness function of the PSO algorithm. As the preset KELM model, the steps are as follows: Initializing particles or updating particle groups, and using test data for classification processing; wherein the test data is extracted from a training data set of normalized fire feature vectors; Calculate the classification accuracy of KELM and determine whether the termination condition is met; If not, it returns to initialize particles or update the particle swarm; if so, it obtains the optimal parameters C,δ, and uses the optimal parameters C,δ to establish the preset KELM model.

8. A forest fire prevention drone early warning system based on laser point cloud recognition, characterized in that: include: An acquisition module is used to acquire initial point cloud image data obtained by a drone scanning a forest area; wherein the drone flies on a preset monitoring route and carries a laser radar for scanning; A processing module, used for performing image denoising processing on the initial point cloud image data to obtain target point cloud image data; An extraction module, used for performing dynamic target recognition on the target point cloud image data to extract dynamic feature images from the target point cloud image data; A generation module, used for extracting flame and smoke features according to the dynamic feature image, and generating a fire feature vector using the flame and smoke features; The early warning module is used to input the fire feature vector into a preset KELM model for fire identification, and transmit the identification result to the background. When a fire is identified, an alarm is processed and an audible and visual early warning is turned on.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images