Intelligent unmanned aerial vehicle fire early warning system and path optimization method thereof

Abstract

The invention discloses an intelligent unmanned aerial vehicle fire early warning system and a path optimization method thereof. The system comprises an unmanned aerial vehicle platform carrying a multi-modal sensor, a multi-modal data fusion analysis module based on deep learning and a dynamic path planning module. The method comprises the following steps: processing visual data by adopting a Gaussian background hybrid modeling method, and extracting flame and smoke fire features; and an abnormal temperature rise area in the monitoring area is analyzed, and a patrol path is optimized by adopting a spider bee algorithm based on historical fire behavior data and a real-time environment risk level in combination with a dynamic feature analysis result obtained by carrying out background modeling on a visible light image through a Gaussian mixture model by an image recognition unit. The system can rapidly arrive at a fire scene or a high-risk area, capture flame image features and temperature tiny changes, reduce false alarms and missing alarms, automatically plan an optimal patrol path according to fire safety requirements and historical fire data, and ensure key monitoring of the high-risk area.

Description

Technical Field

[0001] The present invention relates to the cross - field of unmanned aerial vehicle systems and intelligent path planning technologies, and particularly to an intelligent unmanned aerial vehicle fire warning system and its path optimization method. Background Art

[0002] The stability and recovery ability of the power system are key indicators to measure the modernization level of a country's infrastructure. The power outage scenario generation and system recovery modeling system under extreme events has emerged as an important technical means to enhance the resilience of the power grid and ensure energy security.

[0003] With the continuous growth of China's power demand, the complexity and vulnerability of the power grid are also increasing. Traditional power grid management methods often rely on manual experience and post - event response, making it difficult to effectively cope with sudden extreme events. At the same time, the power grid recovery process lacks scientific decision - making support, resulting in long power outage times and low recovery efficiency.

[0004] Extreme weather such as strong storms, extreme heat, and freezing poses a serious threat to the stable operation of the power system. Once the power grid is damaged, it will directly lead to large - scale power outages, having a major impact on social and economic activities and people's lives. Therefore, how to improve the resilience and recovery ability of the power grid under extreme events has become an urgent problem in the current power industry.

[0005] From a technical perspective, traditional power grid management and recovery methods are no longer able to meet the requirements of modern complex power grids. On the one hand, traditional methods lack the ability to predict and simulate extreme events and cannot accurately assess the vulnerability of the power grid; on the other hand, the recovery process lacks scientific decision - making support and often relies on empirical judgment, resulting in low efficiency. Summary of the Invention

[0006] Object of the Invention: The object of the present invention is to provide an intelligent unmanned aerial vehicle fire warning system and its path optimization method.

[0007] Technical Solution: The intelligent unmanned aerial vehicle fire warning system described in the present invention includes an unmanned aerial vehicle platform equipped with multi - modal sensors, a multi - modal data fusion and analysis module based on deep learning, and a dynamic path planning module; the multi - modal data fusion and analysis module based on deep learning outputs a fire risk assessment result by integrating visual data, thermal imaging data, and temperature sensor data, and the multi - modal sensors include a high - definition camera, a thermal imager, and a temperature sensor.

[0008] Furthermore, the multimodal data fusion analysis module includes an image recognition unit and a temperature analysis unit. The temperature analysis unit analyzes the abnormal temperature rise areas in the monitoring area based on the infrared image data collected by the thermal imaging sensor and the point cloud temperature data collected by the temperature sensor. At the same time, in combination with the dynamic feature analysis results obtained by the image recognition unit through Gaussian mixture model for background modeling of visible light images, it performs spatio-temporal correlation matching with the abnormal temperature rise areas to generate a fire warning signal.

[0009] Furthermore, the image recognition unit uses the Gaussian background mixture modeling method to process visual data and extract flame and smoke features.

[0010] The intelligent UAV fire warning path optimization method of the present invention includes the following steps:

[0011] (1) Use the Gaussian background mixture modeling method to process visual data and extract fire features such as flames and smoke;

[0012] (2) Analyze the abnormal temperature rise areas in the monitoring area. At the same time, in combination with the dynamic feature analysis results obtained by the image recognition unit through Gaussian mixture model for background modeling of visible light images, based on historical fire data and real-time environmental risk levels, use the spider wasp algorithm to optimize the patrol path.

[0013] Furthermore, step (1) includes:

[0014] Using the Gaussian mixture background modeling method, for the video images collected on site, the pixel value distribution of each pixel point of the image usually follows a multi-modal Gaussian distribution. The Gaussian distribution is also called the normal distribution. If it is defined that the random variable X follows the normal distribution, it can be denoted as X~N(μ, σ 2 ), and its probability density function is as follows, where μ represents the mean of the random variable x, and P(X t ) represents the variance of the random variable x:

[0015]

[0016] Assume that the observation value set of a certain pixel point in the video image is X = {X1, X2, X3... X t}, where X t represents the pixel value of this pixel point at time t; X t represents a gray value in a grayscale image, and usually represents a vector in an RGB image. The probability density function P(X t ) is as follows:

[0017]

[0018] Among them, K represents the number of Gaussian distributions, and the usually set value is 3 to 5; ω i，t represents the weight of the i-th Gaussian distribution at time t; represents the probability density function of the i-th Gaussian distribution at time t; u i，t represents the mean of the i-th Gaussian distribution at time t; represents the covariance matrix of the i-th Gaussian distribution at time t; n represents the dimension of the pixel value X t If the color components of each pixel are independent of each other and have the same variance, the covariance is as follows:

[0019]

[0020] Among them, represents the variance, and I represents the identity matrix.

[0021] Furthermore, the Gaussian background mixture modeling method includes:

[0022] (1.1) Background model initialization, initializing a Gaussian mixture model for each pixel;

[0023] (1.2) Traverse all pixel values of the new image, and match the new pixel value X t with its corresponding K Gaussian distributions. The matching principle is as follows:

[0024] |X t -u i，t-1 |≤2.5, σ i，t-1 ;

[0025] (1.3) If the pixel value X t matches the i-th Gaussian distribution, then this pixel is a background pixel, otherwise it is a foreground pixel;

[0026] (1.4) If the pixel is a background pixel, update the parameters of its corresponding Gaussian distribution;

[0027] (1.5) If no Gaussian distribution in step (1.2) matches the new pixel value X t , replace the Gaussian distribution with the lowest weight, and set its mean to the current pixel value X t , and set the standard deviation and weight to the initial values;

[0028] (1.6) After updating the Gaussian mixture model, sort each Gaussian function in descending order according to the value of a i . The mathematical expression of a is as follows:

[0029] a i =ω i，t ÷σ i，t

[0030] (1.7) Select the top B models as the background, and its mathematical expression is as follows:

[0031]

[0032] Among them, T is defined as a preset threshold, which reflects the proportion of the weight of the background model in the K independent Gaussian distribution components.

[0033] Furthermore, the calculation principle of step (1.4) is expressed as:

[0034]

[0035] Among them, θ represents the learning rate of the model, and ρ represents the learning rate of the parameters

[0036] Furthermore, step (2) includes:

[0037] (2.1) For each path P, calculate the fitness value, and the objective function is as follows:

[0038]

[0039] Among them, Fitness(P) is the fitness value of path P, and the larger the value, the better the path performance. L(P) is the total length of path P, (x i , y i ) is the coordinate of the i-th node in path P, C(P) is the proportion of the high-risk area covered by path P, R(i) is the fire risk value of node i in path P, j∈Map is the sum of the fire risk values of all nodes in the entire map, and β is the weight coefficient used to balance the path length and risk coverage;

[0040] (2.2) Initialize the population and parameters. The population matrix of the spider wasp algorithm is as follows:

[0041]

[0042] Among them, X is the spider wasp population; X i is the i-th spider wasp; x ij is the value of the j-th problem variable proposed by the i-th spider wasp; N is the total number of spider wasp population members; m is the number of decision variables;

[0043] (2.3) Select the search space, locate the most promising area, and the position update formula is as follows:

[0044]

[0045] Among them, is the latest position of the spider wasp, i.e., the optimal UAV patrol coordinates (x i , y i ); is the position of the i-th spider wasp; are the positions of the a-th and b-th spider wasps respectively. a and b are two randomly selected indices from the population to determine the search direction; rn is a random number conforming to the normal distribution; r1, r2, r3, and r4 are random numbers uniformly distributed between [0, 1]; is the position of the c-th spider wasp, where c is a randomly selected index from the population; l is a random number uniformly distributed between [-2, 1]; H is the upper limit of the objective function; L is the lower limit of the objective function.

[0046] (2.4) Search and track the prey. The spider wasp pursues the prey by flying, and the position update formula is as follows:

[0047]

[0048] where t is the current iteration number; t max is the maximum iteration number; r5 and r6 are random numbers uniformly distributed between [0, 1]; vc is a random number normally distributed between [-k, k],

[0049] (2.5) Capture the prey,

[0050]

[0051] where X * is the current best solution, i.e., the optimal UAV patrol coordinates (x i , y i ); γ is a number generated during the flight of the spider wasp; U is a binary number, assigned according to the following formula:

[0052] (2.6) Algorithm improvement. Improve the position formula in step (2.3) by introducing a spiral search strategy to improve the global search performance. The improved formula is as follows:

[0053]

[0054] where, is the position of the spider wasp at the next iteration moment, the optimal UAV patrol coordinates (x i , y i ); is the current best position, i.e., the historical optimal solution; is the current position of the i-th spider wasp; β is the spiral shape control parameter, and its value is dynamically adjusted with the iteration number; t is the current iteration number; tmax is the maximum number of iterations.

[0055] Advantages: Compared with the prior art, the present invention has the following remarkable advantages: The present invention has core advantages such as fast response, high-precision monitoring, intelligent analysis, scientific path planning, strong environmental adaptability, and real-time alarm. It can quickly reach the fire scene or high-risk areas, capture the characteristics of flame images and tiny temperature changes, reduce false alarms and missed alarms through deep learning algorithms, and automatically plan the optimal patrol path according to fire safety requirements and historical fire data to ensure key monitoring of high-risk areas. It is applicable to various scenarios such as urban fire protection, forest fire prevention, and safety monitoring in industrial parks, can significantly improve the intelligent level of fire safety management, reduce the incidence of fire accidents, and has broad application prospects and social value. Brief Description of the Drawings

[0056] Figure 1 is the flowchart of the path optimization method of the present invention;

[0057] Figure 2 is the flowchart of the spider wasp algorithm. Detailed Embodiments

[0058] The technical solutions of the present invention will be further described below with reference to the accompanying drawings.

[0059] The intelligent UAV fire warning system described in the present invention includes a UAV platform equipped with multi-modal sensors (high-definition camera, thermal imager, temperature sensor), a multi-modal data fusion analysis module based on deep learning, and a dynamic path planning module.

[0060] The multi-modal data fusion analysis module based on deep learning outputs a fire risk assessment by integrating visual data, thermal imaging data, and temperature sensor data.

[0061] As Figure 1 shown, the specific steps of the intelligent UAV fire warning path optimization method described in the present invention are as follows:

[0062] (1) The image recognition unit processes the visual data using the Gaussian mixture background modeling method and extracts fire characteristics such as flames and smoke.

[0063] Using the Gaussian mixture background modeling method, for the video images collected on-site, the pixel value distribution of each pixel point of the image usually follows a multi-modal Gaussian distribution. The Gaussian distribution is also called the normal distribution. If the random variable X is defined to follow the normal distribution, it can be denoted as X ∼ N(μ, σ 2 ), and its probability density function is as follows, where μ represents the mean of the random variable x, and P(X t ) represents the variance of the random variable x.

[0064]

[0065] Assume that the set of observed values of a certain pixel point in the video image is \(X = \{X_1, X_2, X_3 \cdots X\}\), where \(X\) represents the pixel value of this pixel point at time \(t\). \(X\) represents a grayscale value in a grayscale image, and usually represents a vector in an RGB image. The probability density function \(P(X)\) is as follows: t} t represents the pixel value of this pixel point at time \(t\). t In a grayscale image, it represents a grayscale value, while in an RGB image, it usually represents a vector. The probability density function \(P(X)\) is as follows: t ) is as follows:

[0066]

[0067] where \(K\) represents the number of Gaussian distributions, and the usually set value is 3 - 5; \(\omega\) represents the weight of the \(i\)-th Gaussian distribution at time \(t\); i，t represents the weight of the \(i\)-th Gaussian distribution at time \(t\); represents the probability density function of the \(i\)-th Gaussian distribution at time \(t\); \(\mu\) represents the mean of the \(i\)-th Gaussian distribution at time \(t\); i，t represents the mean of the \(i\)-th Gaussian distribution at time \(t\); represents the covariance matrix of the \(i\)-th Gaussian distribution at time \(t\); \(n\) represents the dimension of the pixel value \(X\). If the color components of each pixel point are independent of each other and have the same variance, then the covariance is as follows: t is as follows:

[0068]

[0069] where represents the variance, and \(I\) represents the identity matrix.

[0070] The specific implementation steps of the image recognition unit using the Gaussian background mixture modeling method in step (1) are as follows:

[0071] (1.1) Background model initialization, initialize a Gaussian mixture model for each pixel point, and the parameters (mean, variance) of the Gaussian distribution can be set as initial values based on the first few frames of the video or randomly selected.

[0072] (1.2) Traverse all pixel values of the new image, and match the new pixel value \(X\) with its corresponding \(K\) Gaussian distributions. The matching principle is as follows: t is matched with the \(K\) Gaussian distributions corresponding to it. The matching principle is as follows:

[0073] \(\vert X\) t - \(\mu\) i，t-1 \vert\leq2.5\sigma\) (4) i，t-1 (4)

[0074] (1.3) If the pixel value \(X\) t is matched with the \(i\)-th Gaussian distribution, then this pixel point is a background pixel point, otherwise it is a foreground pixel point.

[0075] (1.4) If the pixel is a background pixel, update the parameters of its corresponding Gaussian distribution, and its calculation principle is as follows:

[0076]

[0077] Among them, θ represents the learning rate of the model, and ρ represents the learning rate of the parameters.

[0078] (1.5) If there is no Gaussian distribution matching the new pixel value X in the above step (1.2), replace the Gaussian distribution with the lowest weight, and set its mean to the current pixel value X t , and set the standard deviation and weight to the initial values. t

[0079] (1.6) After updating the Gaussian mixture model, sort each Gaussian function in descending order according to the value of a i , and the mathematical expression of a is as follows:

[0080] a i = ω i，t ÷ σ i，t (6)

[0081] (1.7) Select the first B models as the background, and its mathematical expression is as follows:

[0082]

[0083] Among them, T is defined as a preset threshold, which reflects the weight proportion of the background model in the K independent Gaussian distribution components. When the T value is large, the background model is comprehensive, but the calculation takes a long time. When T is small, the background model is one-sided, so the value is between 0.6 and 0.9.

[0084] (2) Based on the infrared image data collected by the thermal imaging sensor and the point cloud temperature data collected by the temperature sensor, analyze the abnormal temperature rise area in the monitoring area. At the same time, combine the dynamic feature analysis results obtained by the image recognition unit through the Gaussian mixture model for background modeling of the visible light image, and perform spatio-temporal correlation matching with the abnormal temperature rise area. When the preset fire warning conditions are met, generate a fire warning signal.

[0085] The dynamic path planning module optimizes the patrol path using the spider wasp algorithm based on historical fire data and real-time environmental risk levels. The specific steps are as follows:

[0086] (2.1) For each path P, calculate the fitness value, and the objective function is as follows:

[0087]

[0088]

[0089] Among them, Fitness(P) is the fitness value of path P, and the larger the value, the better the path performance. L(P) is the total length of path P, (x i , y i ) is the coordinate of the i-th node in path P, C(P) is the proportion of high-risk areas covered by path P, R(i) is the fire risk value of node i in path P, j∈Map is the sum of fire risk values of all nodes in the entire map, and β is the weight coefficient used to balance path length and risk coverage.

[0090] (2.2) Initialize the population and parameters. The population matrix of the spider wasp algorithm is as follows:

[0091]

[0092] Among them, X is the spider wasp population; X i is the i-th spider wasp; x ij is the value of the j-th problem variable proposed by the i-th spider wasp; N is the total number of spider wasp population members; m is the number of decision variables.

[0093] (2.3) Select the search space and locate the most promising area. The position update formula is as follows:

[0094]

[0095] Among them, is the latest position of the spider wasp, that is, the optimal UAV patrol coordinate (x i , y i ); is the position of the i-th spider wasp; are the positions of the a-th and b-th spider wasps respectively. a and b are two randomly selected indicators from the population to determine the search direction; rn is a random number conforming to the normal distribution; r1, r2, r3, and r4 are random numbers uniformly distributed between [0, 1]; is the position of the c-th spider wasp, where c is a randomly selected indicator from the population; l is a random number uniformly distributed between [-2, 1]; H is the upper limit of the objective function; L is the lower limit of the objective function.

[0096] (2.4) Search for and track the prey. The spider wasp hunts the prey by flying. The position update formula is as follows:

[0097]

[0098] In the formula, t is the current iteration number; t maxis the maximum number of iterations; r5 and r6 are random numbers uniformly distributed between [0, 1]; vc is a random number normally distributed between [-k, k].

[0099] (2.5) Capture the prey.

[0100]

[0101] In the formula, X * is the current best solution, that is, the optimal UAV patrol coordinates (x i , y i ); γ is a number generated during the flight of the spider wasp; U is a binary number, assigned according to the following formula:

[0102] (2.6) Algorithm improvement. Improve the position formula in step (2.3) by introducing a spiral search strategy to improve the global search performance. The improved formula is as follows:

[0103]

[0104] Among them, is the position of the spider wasp at the next iteration moment, the optimal UAV patrol coordinates (x i , y i ); is the current best position (i.e., the historical optimal solution); is the current position of the i-th spider wasp; β is the spiral shape control parameter, and its value is dynamically adjusted with the number of iterations; t is the current number of iterations; t max is the maximum number of iterations.

[0105] This strategy introduces a combination of exponential function and cosine function, enabling the spider wasp to approach the optimal solution along a spiral trajectory during the search process, enhancing the algorithm's ability to jump out of local optima, and thus improving the global search performance.

Claims

1. An intelligent UAV fire warning system, characterized in that, It includes a drone platform equipped with multi-modal sensors, a multi-modal data fusion analysis module based on deep learning, and a dynamic path planning module; the multi-modal data fusion analysis module based on deep learning outputs a fire risk assessment result by integrating visual data, thermal imaging data, and temperature sensor data, and the multi-modal sensors include a high-definition camera, a thermal imager, and a temperature sensor.

2. The intelligent UAV fire warning system according to claim 1, wherein, The multi-modal data fusion analysis module includes an image recognition unit and a temperature analysis unit. The temperature analysis unit analyzes the abnormal temperature rise area in the monitoring area based on the infrared image data collected by the thermal imaging sensor and the point cloud temperature data collected by the temperature sensor. At the same time, in combination with the dynamic feature analysis result obtained by the image recognition unit through Gaussian mixture model for background modeling of visible light images, it performs spatio-temporal correlation matching with the abnormal temperature rise area to generate a fire warning signal.

3. The intelligent UAV fire warning system according to claim 2, wherein The image recognition unit processes the visual data using the Gaussian background mixture modeling method and extracts flame and smoke features.

4. An intelligent UAV fire warning path optimization method, characterized in that It includes the following steps: (1) Process the visual data using the Gaussian background mixture modeling method and extract fire features such as flame and smoke; (2) Analyze the abnormal temperature rise area in the monitoring area. At the same time, in combination with the dynamic feature analysis result obtained by the image recognition unit through Gaussian mixture model for background modeling of visible light images, optimize the patrol path using the spider wasp algorithm based on historical fire data and real-time environmental risk levels.

5. The intelligent UAV fire warning path optimization method according to claim 4, wherein The step (1) includes: Using the Gaussian mixture background modeling method, for the video images collected on site, the pixel value distribution of each pixel in the image usually follows a multi-modal Gaussian distribution. The Gaussian distribution is also known as the normal distribution. If the random variable X is defined to follow the normal distribution, it can be denoted as X ~ N(μ, σ 2 ), and its probability density function is as follows. Here, μ represents the mean of the random variable x, and P(X t ) represents the variance of the random variable x: Suppose the set of observed values of a certain pixel point in the video image is \(X = \{X_1, X_2, X_3 \cdots X\}\), where \(X\) t represents the pixel value of this pixel point at time \(t\); \(X\) t represents the pixel value of this pixel point at time \(t\); \(X\) t represents a gray value in a grayscale image, and usually represents a vector in an RGB image. The probability density function \(P(X)\) t is as follows: Among them, K represents the number of Gaussian distributions, and the usually set value is 3 to 5; ω i，t represents the weight of the i-th Gaussian distribution at time t; represents the probability density function of the i-th Gaussian distribution at time t; u i，t represents the mean of the i-th Gaussian distribution at time t; represents the covariance matrix of the i-th Gaussian distribution at time t; n represents the dimension of the pixel value X t When the color components of each pixel point are independent of each other and have the same variance, the covariance is as follows: Among them, represents variance, and I represents the identity matrix.

6. The intelligent UAV fire warning path optimization method according to claim 4, characterized in that The Gaussian background mixture modeling method includes: (1.1) Background model initialization, initialize a Gaussian mixture model for each pixel point; (1.2) Traverse all pixel values of the new image and set the new pixel value X t Match it with its corresponding K Gaussian distributions. The matching principle is as follows: |X t -u i，t-1 |≤2.5, σ i，t-1 ; (1.3) If the pixel value X t matches the i-th Gaussian distribution, then this pixel point is a background pixel point; otherwise, it is a foreground pixel point. (1.4) If the pixel point is a background pixel point, update the parameters of its corresponding Gaussian distribution; (1.5) If there is no Gaussian distribution in step (1.2) that matches the new pixel value X t , then replace the Gaussian distribution with the lowest weight, and set its mean to the current pixel value X t , and set the standard deviation and weight to the initial values; After updating the Gaussian mixture model, sort each Gaussian function in descending order according to the value of a i The mathematical expression of a is as follows: a i = ω i，t ÷ σ i，t ; (1.7) Select the first B models as the background, and its mathematical expression is as follows: Among them, T is defined as a preset threshold, which reflects the proportion of the weight of the background model in the K independent Gaussian distribution components.

7. The intelligent UAV fire warning path optimization method according to claim 6, wherein The calculation principle of the step (1.4) is expressed as: Among them, α represents the learning rate of the model, and ρ represents the learning rate of the parameters.

8. The intelligent UAV fire warning path optimization method according to claim 4, wherein The step (2) includes: (2.1) For each path P, calculate the fitness value, and the objective function is as follows: Among them, Fitness(P) is the fitness value of path P, and the larger the value, the better the path performance. L(P) is the total length of path P, (x i , y i ) is the coordinate of the i-th node in path P, C(P) is the proportion of high-risk areas covered by path P, R(i) is the fire risk value of node i in path P, j∈Map is the sum of the fire risk values of all nodes in the entire map, and β is the weight coefficient used to balance the path length and risk coverage; (2.2) Initialize the population and parameters, and the population matrix of the spider wasp algorithm is as follows: Among them, X is the spider wasp population; X i is the i-th spider wasp; x ij is the value of the j-th problem variable proposed by the i-th spider wasp; N is the total number of spider wasp population members; m is the number of decision variables; (2.3) Select the search space, locate the most promising area, and the position update formula is as follows: Among them, is the latest position of the spider wasp, that is, the optimal UAV patrol coordinates (x i , y i ); is the position of the i-th spider wasp; are the positions of the a-th and b-th spider wasps respectively. a and b are two randomly selected indicators from the population to determine the search direction; rn is a random number conforming to the normal distribution; r1, r2, r3, and r4 are random numbers uniformly distributed between [0, 1]; is the position of the c-th spider wasp, where c is a randomly selected indicator from the population; l is a random number uniformly distributed between [-2, 1]; H is the upper limit of the objective function; L is the lower limit of the objective function; (2.4) Search for and track the prey. The spider wasp pursues the prey by flying, and the position update formula is as follows: where t is the current iteration number; t max is the maximum iteration number; r5 and r6 are random numbers uniformly distributed between [0, 1]; vc is a random number normally distributed between [-k, k], (2.5) Capture the prey. Among them, X * is the current best solution, that is, the optimal drone patrol coordinates (x i , y i ); P is a number generated during the flight of the spider wasp; U is a binary number, which is assigned according to the following formula: (2.6) Algorithm improvement, improve the position formula in step (2.3), and improve the global search performance by introducing a spiral search strategy. The improved formula is as follows: Among them, is the position of the spider wasp at the next iteration moment, and the optimal UAV patrol coordinates (x i , y i ); is the current best position, i.e., the historical optimal solution; is the current position of the i-th spider wasp; β is the spiral shape control parameter, and its value is dynamically adjusted with the number of iterations; t is the current iteration number; t max is the maximum number of iterations.

Images