Fire spreading path dynamic prediction method and system based on artificial intelligence

Abstract

The invention belongs to the technical field of artificial intelligence, and discloses a fire spreading path dynamic prediction method and system based on artificial intelligence. The method comprises the following steps: acquiring image data, sensor data, meteorological data, fire spreading rate and burnt area of a fire scene; performing feature extraction on the image data to obtain image features; performing feature extraction on the sensor data to obtain environment features; performing feature extraction on the meteorological data to obtain meteorological features; calculating correlation weights among the different modal features, and performing weighted fusion on the different modal features based on the correlation weights to obtain aligned feature vectors; dynamically adjusting a model prediction time step length based on a reinforcement learning mechanism to obtain a dynamic time step length; performing fire hazard path dynamic prediction based on a path prediction model according to the dynamic time step length; according to the method, the change of the fire spreading path can be carefully captured, so that the prediction accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically, to a method and system for dynamically predicting a fire spread path based on artificial intelligence. Background Art

[0002] Traditional fire spread prediction methods are mainly based on physical models and empirical formulas. For example, theoretical models based on heat conduction, heat convection and heat radiation equations, as well as empirical rules summarized according to different fuel types and environmental conditions. However, these methods have many limitations. On the one hand, physical models often require a lot of simplified assumptions about fire scenes, such as assuming that fuel is evenly distributed and environmental conditions remain unchanged. Actual fire scenes are complex and changeable, and these assumptions are difficult to hold. On the other hand, the scope of application of empirical formulas is limited. The fuel characteristics, meteorological conditions and building structures in different regions vary greatly, making it difficult for empirical formulas to accurately predict the fire spread path under various circumstances.

[0003] The Chinese patent application with publication number CN118195110A discloses a real-time fire evacuation path planning method and system based on deep learning. In response to receiving a fire command, the fire scene information is determined, and the fire scene information at least includes the road planning in the fire scene; the monitoring data collected in real time by the monitoring equipment of the fire scene is obtained; the monitoring data is mapped in low-dimensional space to obtain multiple first monitoring data feature matrices; based on the first influence feature recognition configuration, the influence feature recognition is performed on the multiple first monitoring data feature matrices to determine the influence monitoring data basis corresponding to the multiple first monitoring data feature matrices; the influence monitoring data is input into the target fire spread trend inference model obtained in advance based on deep learning training to obtain the target spread trend information, and the target spread trend information includes the time information of the fire spreading to each road in the fire scene; based on the road planning and the target spread trend information, the target evacuation path is generated and sent. The invention effectively ensures that the personnel in the fire scene can be evacuated quickly and reliably, and effectively guarantees the evacuation efficiency when a fire occurs.

[0004] Although the above method can meet most scenarios, research and practical application of the above method and existing technology have found that the above method and existing technology have at least the following defects:

[0005] When fusing multimodal data such as image data, sensor data, and meteorological information, the feature space and representation methods of different modal data vary greatly; the spread speed and characteristics of fire in the early and late stages of development vary greatly, and a fixed model prediction time step is difficult to meet the prediction needs of different stages.

[0006] In view of this, the present invention proposes a fire spread path dynamic prediction method and system based on artificial intelligence to solve the above problems. Summary of the invention

[0007] In order to overcome the above-mentioned defects of the prior art and to achieve the above-mentioned purpose, the present invention provides the following technical solution: a method for dynamic prediction of fire spread path based on artificial intelligence, comprising the following steps:

[0008] Collect image data, sensor data, meteorological data, fire spread rate and burned area at the fire scene;

[0009] Performing feature extraction on the image data based on the first extraction model to obtain image features;

[0010] Extracting features from the sensor data based on the second extraction model to obtain environmental features;

[0011] Extracting features from the meteorological data based on the third extraction model to obtain meteorological features;

[0012] Calculate the correlation weights between different modal features, and perform weighted fusion of different modal features based on the correlation weights to obtain aligned feature vectors; different modal features include image features, environmental features, and meteorological features;

[0013] The aligned feature vector is used as the input of the reinforcement learning mechanism, and the prediction time step of the path prediction model is dynamically adjusted based on the reinforcement learning mechanism to obtain a dynamic time step;

[0014] Dynamic prediction of fire path is performed based on the path prediction model according to the dynamic time step.

[0015] Furthermore, the method for obtaining the dynamic time step includes:

[0016] Define an agent, which is used to select and execute actions in the action space;

[0017] Define the state space S, which includes the fire spread rate, the area of ​​the burned area, the temperature and the meteorological data, and obtain the state vector s corresponding to the state space S;

[0018] Define the action space A, according to the preset time step value range [Δa min ,Δa max ], where Δa max is the maximum time step; Δa min is the minimum time step value; the preset time step value range is discretized into N values, and the action space A corresponding to the action containing N time step values ​​is obtained. 1 , a 2 , …, a N}, the nth action in the action space A, that is, the value a of the nth time step n Satisfy Δamin ≤a n ≤Δa max , n=1,…,N;

[0019] Design the reward function r based on the error between the predicted 3D coordinate change of action a and the actual 3D coordinate change;

[0020] Initialize the reinforcement learning mechanism: select a deep Q network to simulate the Q-value function Q(s, a), which represents the expected cumulative reward obtained by performing action a under state vector s;

[0021] During the training process, the agent interacts with the state vector and presets the experience playback buffer D to record the four-tuple (s t , a t , r t ,s t+1 ), where s t is the state vector corresponding to time step t, a t is the action taken at time step t, r t is the reward obtained at time step t, s t+1 is the state vector corresponding to the next time step;

[0022] A set of four-tuples (s) are randomly drawn from the experience replay buffer D j , a j , r j ,s j+1 ), where j = 1, ..., M, is the number of quaternions corresponding to different time steps. j , a j , r j ,s j+1 ) Update Q network parameters;

[0023] Step H: The agent selects actions using the ε-greedy strategy;

[0024] Step I: After training is completed, the action with the largest Q value is selected as the prediction time step, and the path prediction model is evaluated based on the evaluation index according to the preset index threshold: when the evaluation index reaches the preset index threshold, the corresponding preset time step is obtained.

[0025] Further, according to the four-tuple (s j , a j , r j ,s j+1 ) The method for updating the Q network parameters includes:

[0026] Combine the reward obtained at time step j and the reward at s j+1 The maximum value of the Q value of all possible actions a' under the deep Q network is used to calculate the target network Q value yj ;

[0027] Update the main Q network parameters by minimizing the loss function; the main Q network is used to calculate the current state vector s j Next select action a j Q value;

[0028] The Adam optimizer is used to iteratively optimize the loss function to obtain updated parameters.

[0029] Furthermore, the method of selecting an action using the ε-greedy strategy includes:

[0030] Use the selection probability ε to randomly select an action, and use the selection probability 1-ε to select the action with the maximum Q value calculated by the current Q network.

[0031] Furthermore, the method for obtaining the aligned feature vector includes:

[0032] Calculate the similarity matrix between different modal features:

[0033] The similarity matrix between the image features and the environmental features, the similarity matrix between the image features and the meteorological features, and the similarity matrix between the environmental features and the meteorological features are calculated respectively through the conversion function;

[0034] The features of different modalities are weightedly fused according to the similarity matrix to obtain the aligned feature vector.

[0035] Furthermore, the method for obtaining the burned area includes:

[0036] Obtain bird's-eye view images of the fire area using drones equipped with cameras;

[0037] The bird's-eye view image is subjected to denoising and enhancement processing to obtain an original bird's-eye view image, and then subjected to grayscale processing to obtain a grayscale bird's-eye view image;

[0038] The grayscale bird's-eye view image is segmented according to a preset segmentation threshold, and pixels in the grayscale bird's-eye view image whose pixel values ​​are greater than the preset segmentation threshold are marked as burned areas, and pixels whose pixel values ​​are less than the preset segmentation threshold are marked as unburned areas; the original bird's-eye view image is converted from the RGB color interval to the HSV color space, and the HSV color space is divided into I, E and H intervals according to hue, saturation and brightness, respectively, and the intervals are numbered accordingly, and the interval numbers to which the hue, saturation and brightness of each pixel belong are counted, and the interval numbers corresponding to the hue, saturation and brightness of the pixel are spliced ​​as color features;

[0039] The gray-level co-occurrence matrix of the gray-level bird's-eye view image is statistically calculated, and the eigenvalues ​​of the gray-level co-occurrence matrix are calculated in the neighborhood of each pixel. The eigenvalues ​​include contrast, correlation, energy, and entropy. The eigenvalues ​​are concatenated to obtain texture features.

[0040] Select K seed points located in the determined burned area in the grayscale bird's-eye view image, and record the grayscale value, texture feature and color feature of each seed point in the grayscale bird's-eye view image;

[0041] Define growth rules: for pixels adjacent to the seed point, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed point and the seed point respectively through Euclidean distance; obtain comprehensive similarity by weighted average of texture feature similarity and color feature similarity; compare the comprehensive similarity with a preset comprehensive similarity threshold, and when the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area;

[0042] Starting from the seed point, adjacent pixels that meet the conditions are added to the burned area according to the growth rule until no new pixels meet the growth conditions;

[0043] The burned area is calculated based on the product of the image resolution and the number of pixels marked as burned after segmentation.

[0044] Furthermore, the method for obtaining the color feature includes:

[0045] Divide hue, saturation and brightness into I, E and H intervals respectively, and number the intervals accordingly;

[0046] Convert the original bird's-eye view image from RGB color space to HSV color space;

[0047] Methods for converting the original bird's-eye view image from RGB color space to HSV color space include:

[0048] The brightness is calculated by the maximum component value of the original bird's-eye view image in the R channel, the G channel, and the B channel;

[0049] The saturation is calculated by the difference between the maximum component value of the original bird's-eye view image in the R channel, the G channel and the B channel and the minimum component value of the original bird's-eye view image in the R channel, the G channel and the B channel;

[0050] Differentiate and calculate hues based on lightness and saturation scores;

[0051] For each pixel in the HSV image, obtain its hue, saturation and brightness values; determine the interval number to which the hue, saturation and brightness values ​​of each pixel belong, and concatenate the corresponding hue interval number, saturation interval number and brightness interval number as a color feature.

[0052] Furthermore, the method for obtaining the texture feature includes:

[0053] Count the frequency of pixel pairs with grayscale values ​​X and Y in the grayscale bird's-eye view image in a neighborhood of size C×C, where C is the size of the neighborhood, and use the frequency as the element T(X,Y) of the grayscale co-occurrence matrix T of size H×H; H is the number of rows and columns of the grayscale co-occurrence matrix T;

[0054] Select a pixel as the center point, obtain the corresponding neighborhood of size C×C, and traverse the pixel pairs in the corresponding neighborhood of size C×C according to the preset spatial position relationship;

[0055] For each pair of adjacent pixels, obtain the corresponding grayscale values ​​X and Y, and add 1 to the corresponding element T(X, Y) in the grayscale co-occurrence matrix T;

[0056] Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that meet the spatial position relationship in the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X, Y);

[0057] Based on the gray-level co-occurrence matrix P(X, Y), the contrast, correlation, energy and entropy are calculated respectively;

[0058] Contrast, correlation, energy and entropy are concatenated as texture features.

[0059] Furthermore, a method for dynamically predicting a fire path based on the path prediction model includes:

[0060] Initialize the path prediction model parameters, substitute the aligned feature vector and the initial dynamic time step into the path prediction model, predict the three-dimensional coordinate change of the fire spread boundary, and continuously collect new image data, new sensor data, and new meteorological data from the fire scene to update the aligned feature vectors. Obtain a new aligned feature vector, and combine it with the dynamic time step updated by the reinforcement learning mechanism to predict the next round of fire spread path through the path prediction model; repeat the above process of updating the aligned feature vector and the dynamic time step, and continuously iterate and predict the fire spread path.

[0061] Furthermore, the method for obtaining the fire spread rate includes:

[0062] Preselect two monitored points, the positions of the two monitored points are x 1 and x 2 , measure the temperature of the monitored point through infrared detection equipment, and compare the temperature of the monitored point with the preset temperature threshold. When the temperature of the monitored point reaches the preset temperature threshold, it means that the monitored point is in a burning state. The position of the monitored point is counted as x 1 and x2 The time τ when the temperature of the two monitored points reaches the preset temperature threshold 1 and τ 2 The fire spread rate V is obtained based on the ratio of the monitoring point position change value to the time change value of the combustion state.

[0063] Furthermore, the training method of the first extraction model includes:

[0064] Collecting a group A of image training data in advance, where the image training data includes image data and image features corresponding to the image data;

[0065] Each group of image training data is used as the input of the first extraction model. The first extraction model uses the image features corresponding to each group of image data as output, and uses the actual image features corresponding to each group of image data as prediction targets; minimizing the sum of prediction errors of all image features is used as the training target; the first extraction model is trained until the sum of prediction errors reaches convergence, and the training is stopped; the first extraction model is a deep neural network model; the loss function value of the first extraction model is the mean square error.

[0066] The fire spread path dynamic prediction system based on artificial intelligence is used to implement the fire spread path dynamic prediction method based on artificial intelligence, including:

[0067] Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate and burned area of ​​the fire scene;

[0068] The first processing module: extracts features from the image data based on the first extraction model to obtain image features;

[0069] The second extraction module: extracts features from the sensor data based on the second extraction model to obtain environmental features;

[0070] The third extraction module: extracts features from the meteorological data based on the third extraction model to obtain meteorological features;

[0071] Feature alignment module: used to calculate the correlation weights between different modal features, and perform weighted fusion of different modal features based on the correlation weights to obtain the aligned feature vector; different modal features include image features, environmental features, and meteorological features;

[0072] Step adjustment module: used to use the aligned feature vector as the input of the reinforcement learning mechanism, dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism, and obtain the dynamic time step;

[0073] Path prediction module: Dynamic prediction of fire path is performed based on the path prediction model according to the dynamic time step.

[0074] The technical effects and advantages of the fire spread path dynamic prediction method and system based on artificial intelligence of the present invention are as follows:

[0075] The present invention maps data of different modalities, such as image data, sensor data, and meteorological data, into a unified feature space through feature fusion technology, so that the model can process data from these different sources simultaneously; it helps to explore the intrinsic correlation between data of different modalities and utilize the complementarity of data of each modality to more comprehensively understand the fire scene; by corresponding to different development stages of the fire, the prediction time step can be dynamically adjusted, and the changes in the fire spread path can be more finely captured, thereby improving the accuracy of the prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a flow chart of a method for dynamic prediction of fire spread path based on artificial intelligence of the present invention;

[0077] Figure 2 It is a schematic flow chart of the method for obtaining the area of ​​the burned region of the present invention;

[0078] Figure 3 It is a schematic flow chart of a method for obtaining a dynamic time step of the present invention;

[0079] Figure 4 It is a block diagram of the fire spread path dynamic prediction system based on artificial intelligence of the present invention;

[0080] Figure 5 This is a schematic diagram of the interface of the fire spread path dynamic prediction system based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0081] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] Example 1

[0083] See also Figure 1 As shown, the fire spread path dynamic prediction method based on artificial intelligence described in this embodiment includes the following steps:

[0084] Collect image data, sensor data, meteorological data, fire spread rate and burned area of ​​the fire scene; the image data of the fire scene is collected based on drones equipped with high-definition cameras, including image data of the fire scene from different angles. The image data can clearly show the shape, size and position of the flame. Through image analysis, the advancement direction and speed of the flame front can be directly observed, providing a basis for predicting the next direction of the fire.

[0085] Sensor data is collected based on temperature sensors and smoke sensors. Sensor data includes the temperature and smoke concentration at the fire scene. Collecting temperature data can accurately depict the heat distribution at the fire scene, predict the heat propagation path in the building structure, and then predict the fire spread path. Changes in smoke concentration can reflect the air flow. By setting up smoke sensors at multiple locations and combining information on wind direction and ventilation systems, the impact of air flow patterns on fire spread can be inferred.

[0086] Meteorological data is collected through wind speed sensors, wind direction sensors and humidity sensors, including wind speed, wind direction and humidity at the fire scene; wind provides additional oxygen supply for the combustion process and quickly transfers the heat generated by combustion to unburned areas, accelerating the combustion reaction; smoke will also spread with the wind direction; understanding wind speed and wind direction can predict the propagation path of smoke, which is crucial for assessing the impact of fire on the surrounding environment; environmental humidity affects the humidity and drying speed of the burning material, as well as the evaporative cooling effect.

[0087] Methods for obtaining fire spread rate include:

[0088] Preselect two monitored points, the positions of the two monitored points are x 1 and x 2 , measure the temperature of the monitored point through infrared detection equipment, and compare the temperature of the monitored point with the preset temperature threshold. When the temperature of the monitored point reaches the preset temperature threshold, it means that the monitored point is in a burning state. The position of the monitored point is counted as x 1 and x 2 The time τ when the two monitored points change from the non-combustion state to the combustion state 1 and τ 2 , that is, the position of the monitored point is x 1 and x 2 The time when the temperature of the two monitored points reaches the preset temperature threshold is based on the speed formula Obtain the fire spread rate V. Fire will spread between different types of materials and structures. Different materials have different combustion characteristics, resulting in different fire spread rates. By dynamically adjusting the time step according to the spread rate, the simulation step can be reasonably updated when the fire spreads from one material to another (such as from a wooden structure to a steel structure building) to accurately reflect the spread of fire in different media. By accurately obtaining the spread rate, the position of the flame front at different time points can be updated in real time. For example, on a two-dimensional plane, if the position and spread rate of the flame front at a certain moment are known, the position where the flame will reach at the next moment can be predicted, thereby accurately depicting the dynamic spread path of the flame in space.

[0089] Reference Figure 2 , the methods for obtaining the burned area include:

[0090] Obtain bird's-eye views of the fire area using drones equipped with high-precision cameras;

[0091] The bird's-eye view image is subjected to denoising and enhancement processing to obtain an original bird's-eye view image, and then subjected to grayscale processing to obtain a grayscale bird's-eye view image;

[0092] The grayscale bird's-eye view image is segmented according to a preset segmentation threshold, and pixels in the grayscale bird's-eye view image whose pixel values ​​are greater than the preset segmentation threshold are marked as burned areas, and pixels whose pixel values ​​are less than the preset segmentation threshold are marked as unburned areas; the original bird's-eye view image is converted from the RGB color interval to the HSV color space, and the HSV color space is divided into I, E and H intervals according to hue, saturation and brightness, respectively, and the intervals are numbered accordingly, and the interval numbers to which the hue, saturation and brightness of each pixel belong are counted, and the interval numbers corresponding to the hue, saturation and brightness of the pixel are spliced ​​as color features;

[0093] Methods for obtaining color features include:

[0094] Divide hue, saturation and brightness into I, E and H intervals respectively, and number the intervals accordingly;

[0095] Convert the original bird's-eye view image from RGB color space to HSV color space;

[0096] Methods for converting the original bird's-eye view image from RGB color space to HSV color space include:

[0097] The lightness VM is calculated as follows:

[0098] VM = max(R, G, B);

[0099] Among them, R, G, and B are the component values of the R channel, G channel, and B channel in the RGB color space of the original aerial image; max(R, G, B) is the maximum value among the three component values of R, G, and B;

[0100] The saturation is calculated as follows:

[0101] Calculate the saturation variable δ = max(R, G, B) - min(R, G, B); where min(R, G, B) is the minimum value among the three component values of R, G, and B;

[0102] When VM = 0, the saturation VB = 0;

[0103] When VM ≠ 0, the saturation

[0104] The hue is calculated as follows:

[0105] When δ = 0 or VB = 0, the hue VS = 0;

[0106] When R = max(R, G, B) and G ≥ B, the hue

[0107] When R = max(R, G, B) and G < B, the hue

[0108] When G = max(R, G, B), the hue

[0109] When B = max(R, G, B), the hue

[0110] For each pixel in the HSV image, obtain its hue, saturation, and value; determine the interval number to which the values of the hue, saturation, and value of each pixel belong, and splice the corresponding hue interval number, saturation interval number, and value interval number as the color feature.

[0111] Statistically analyze the gray-level co-occurrence matrix of the gray-scale aerial image, and calculate the eigenvalues of the gray-level co-occurrence matrix within the neighborhood of each pixel. The eigenvalues include contrast, correlation, energy, and entropy, and splice the eigenvalues to obtain the texture feature;

[0112] The method for obtaining the texture feature includes:

[0113] Statistically analyze the frequency of occurrence of pixel pairs with gray-level values of X and Y in the neighborhood of size C × C in the gray-scale aerial image. C is the neighborhood size, and use the frequency as the element T(X, Y) of the gray-level co-occurrence matrix T of size H × H; H is the number of rows and columns of the gray-level co-occurrence matrix T;

[0114] Select a pixel as the center point, obtain the corresponding neighborhood of size C×C, and traverse the pixel pairs in the corresponding neighborhood of size C×C according to the preset spatial position relationship (such as horizontal adjacent, vertical adjacent, diagonal adjacent, etc.);

[0115] For each pair of adjacent pixels, obtain the corresponding grayscale values ​​X and Y, and add 1 to the corresponding element T(X, Y) in the grayscale co-occurrence matrix T; for example, if the grayscale value of the current pixel is 10 and the grayscale value of the adjacent pixel on the right is 15, then add 1 to T(10, 15), and repeat this process until all pixel pairs that meet the spatial position relationship in the neighborhood are traversed.

[0116] Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that meet the spatial position relationship in the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X, Y); the gray-level co-occurrence matrix P(X, Y) is calculated as follows:

[0117]

[0118] Among them, P(X,Y) is the gray level co-occurrence matrix, which represents the probability of occurrence of pixel pair (X,Y).

[0119] Calculate the contrast based on the gray-level co-occurrence matrix P(X,Y):

[0120]

[0121] Among them, Contrast is contrast;

[0122] Calculate the correlation based on the gray-level co-occurrence matrix P(X,Y):

[0123]

[0124] Among them, Correlation is the correlation; μ X is the mean of the grayscale value X, that is, for all possible grayscale values ​​X, the probability of occurrence P(X, Y) of the corresponding pixels is weighted and summed to obtain the average level of the grayscale value X in the selected neighborhood; μ Y is the mean of the grayscale value Y, that is, for all possible grayscale values ​​Y, the probability of occurrence P(X, Y) of the corresponding pixels is weighted and summed to obtain the average level of the grayscale value Y in the selected neighborhood; σ X is the standard deviation of the gray value X, that is, the degree of dispersion of the gray value X in the selected neighborhood; σ Y is the standard deviation of the gray value Y, that is, the degree of dispersion of the gray value Y in the selected neighborhood;

[0125] Calculate energy based on the gray-level co-occurrence matrix P(X,Y):

[0126]

[0127] Among them, Energy is energy;

[0128] Calculate entropy based on the gray-level co-occurrence matrix P(X,Y):

[0129]

[0130] Among them, Entropy is entropy.

[0131] Contrast, correlation, energy and entropy are concatenated as texture features.

[0132] Select K seed points located in the determined burned area in the grayscale bird's-eye view image, and record the grayscale value, texture feature and color feature of each seed point in the grayscale bird's-eye view image;

[0133] Define growth rules: for pixels adjacent to the seed point, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed point and the seed point respectively through Euclidean distance; obtain the comprehensive similarity by weighted average of the texture feature similarity and the color feature similarity; compare the comprehensive similarity with the preset comprehensive similarity threshold, and when the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area.

[0134] Starting from the seed point, adjacent pixels that meet the conditions are added to the burned area according to the growth rule until no new pixels meet the growth conditions.

[0135] According to the image resolution FB, the number of pixels W marked as burned areas after segmentation is counted, and the area of ​​the burned area G is calculated based on the formula G = W × FB. In the early stage of a fire, the area of ​​the burned area grows relatively slowly; as the fire develops, the area of ​​the burned area may expand rapidly; by monitoring the rate of change of the area of ​​the burned area, we can understand what stage of development the fire is in; this change in the development of the fire can be used as a basis for dynamically adjusting the time step. In the slow development stage of the fire, the time step can be appropriately increased to reduce the amount of calculation; in the rapid spread stage, reducing the time step can more finely capture the rapid change process of the fire and ensure the accuracy of the simulation; and the boundary position of the burned area intuitively shows the frontier of the fire spread; by continuously monitoring the changes in the area of ​​the burned area, the advancement speed of the flame in different directions can be determined, thereby predicting the direction of the fire spread.

[0136] Performing feature extraction on the image data based on the first extraction model to obtain image features;

[0137] The training method of the first extraction model includes:

[0138] A group A of image training data is collected in advance, where the image training data includes image data and image features corresponding to the image data.

[0139] Each group of image training data is used as the input of the first extraction model. The first extraction model uses the image features corresponding to each group of image data as output, and the actual image features corresponding to each group of image data as prediction targets; minimizing the sum of prediction errors of all image features is used as the training target; the first extraction model is trained until the sum of prediction errors reaches convergence and the training is stopped; the first extraction model is a deep neural network model.

[0140] The loss function value of the first extraction model is the mean square error.

[0141] Prediction error formula: a is the ath group of image data; A is the number of groups of image data; is the image feature corresponding to the a-th group of image data; z a is the actual image feature corresponding to the a-th group of image data.

[0142] Based on the second extraction model, feature extraction is performed on the sensor data to obtain environmental features; the training method of the second extraction model can refer to the training method of the first extraction model.

[0143] Based on the third extraction model, feature extraction is performed on the meteorological data to obtain meteorological features; the training method of the third extraction model can refer to the training method of the first extraction model.

[0144] The correlation weights between different modal features are calculated, and based on the correlation weights, the different modal features are weightedly fused to obtain the aligned feature vectors; the different modal features include image features, environmental features, and meteorological features.

[0145] Methods for obtaining aligned feature vectors include:

[0146] Calculate the similarity matrix between different modal features:

[0147] Calculate the similarity matrix A between image features and environment features IS :

[0148]

[0149] Calculate the similarity matrix A between image features and meteorological features IM :

[0150]

[0151] Calculate the similarity matrix A between environmental features and meteorological features IM :

[0152]

[0153] Among them, f I is the image feature; f S is the environmental characteristic; M is the meteorological feature; Softmax is the conversion function used to convert the similarity value into probability distribution;

[0154] The features of each modality are weighted and fused according to the similarity matrix to obtain the aligned feature vector:

[0155] F=α 1 A IS f I +α 2 A IM f I +β 1 A IM f M +β 2 A SM f M +γ 1 A IS f s +γ 2 A SM f S ;

[0156] Among them, α 1 , α 2 , β 1 , β 2 , γ 1 and γ 2 are adjustment parameters; F is the aligned feature vector.

[0157] The aligned feature vector is used as the input of the reinforcement learning mechanism, and the prediction time step of the path prediction model is dynamically adjusted based on the reinforcement learning mechanism to obtain a dynamic time step;

[0158] Reference Figure 3 , methods for obtaining dynamic time steps include:

[0159] Define an agent, which is used to select and execute actions;

[0160] Define a state space S, which includes the fire spread rate, the burned area, and the aligned feature vectors, and obtain a state vector s corresponding to the state space S;

[0161] Define the action space A, according to the preset time step value range [Δa min , Δa max ], where Δa max is the maximum time step; Δamin is the minimum time step value; the preset time step value range is discretized into N values, and the action space A corresponding to the action containing N time step values ​​is obtained. 1 , a 2 , …, a N}, the nth action in the action space A, that is, the value a of the nth time step n Satisfy Δa min ≤a n ≤Δa max , n=1,…,N;

[0162] Design reward function r(s, a): r(s, a) = -β·∈; where s is the state vector; a is the selected action, i.e., the time step; β is the scaling factor used to adjust the reward amplitude; ∈ is the error between the predicted three-dimensional coordinate change based on action a and the actual three-dimensional coordinate change;

[0163] Initialize the reinforcement learning mechanism and select a deep Q network to simulate the Q-value function Q(s, a). The Q-value function Q(s, a) represents the expected cumulative reward obtained by performing action a under the state vector s.

[0164] During the training process, the agent interacts with the state vector and presets the experience playback buffer D to record the four-tuple (s t , a t , r t ,s t+1 ), where s t is the state vector corresponding to time step t, a t is the action taken at time step t, r t is the reward obtained at time step t, s t+1 is the state vector corresponding to the next time step.

[0165] A set of four-tuples (s) are randomly drawn from the experience replay buffer D j , a j , r j ,s j+1 ), where j = 1, ..., M, is the number of quadruplets corresponding to different time steps, s j is the state vector corresponding to time step j, a j is the action taken at time step j, r j is the reward obtained at time step j, s j+1 is the state vector corresponding to the next time step; according to the quaternion (s j , a j , r j ,s j+1 ) Update Q network parameters;

[0166] According to the four-tuple (s j , a j , r j ,s j+1 ) The method for updating the Q network parameters includes:

[0167] Calculate the target network Q value y of the deep Q network j :

[0168]

[0169] Among them, γ is the discount factor, which is used to weigh the importance of future rewards; Q(s j+1 , a'; Θ - ) is used with parameter Θ - The target network is calculated in s j+1 The Q value of all possible actions a', the parameter Θ of the target network - The target network is used to predict the next state vector s after executing action a', which is obtained by periodically assigning values ​​from the main Q network. j+1 The maximum Q value under the condition, the main Q network is used to calculate the current state vector s j Next select action a j Q value; Indicates taking Q(s j+1 , a'; Θ - )’s maximum value;

[0170] Update the main Q network parameters Θ by minimizing the loss function; loss function The definition is as follows:

[0171]

[0172] Among them, Q(s j , a j ; Θ) is the main Q network calculated with parameter Θ at s j Next action a j Q value;

[0173] The Adam optimizer is used to optimize the loss function Perform iterative optimization to obtain the updated parameter Θ′. The update formula of the parameter Θ is:

[0174]

[0175] Among them, λ is the learning rate; is the loss function with respect to the parameter Θ t The gradient of ; Adam is the Adam optimization function, and the Adam optimizer adaptively adjusts the learning rate based on the gradient information.

[0176] The agent uses the ε-greedy strategy to select actions, that is, it uses the selection probability ε to randomly select an action, and uses the selection probability 1-ε to select the action that the current Q network thinks has the largest Q value, as follows:

[0177]

[0178] Among them, a t The action selected for time step t; s t is the state vector corresponding to time step t; Q(s t ,a; Θ) is the main Q network calculated with parameter Θ at s t The Q value of the next action a; Indicates taking Q(s t ,a;Θ) maximum value;

[0179] After the training is completed, the action with the largest Q value is selected as the prediction time step, and the path prediction model is evaluated based on the evaluation index according to the preset index threshold: when the evaluation index reaches the preset index threshold, the corresponding preset time step is obtained and can be applied to the dynamic prediction process of the fire path; the evaluation index includes the closeness of the prediction time step to the true optimal time step and the accuracy of the fire spread prediction based on the prediction time step.

[0180] Perform dynamic prediction of fire paths based on path prediction models according to dynamic time steps;

[0181] The methods for dynamic prediction of fire paths based on path prediction models include:

[0182] Initialize the path prediction model parameters, substitute the aligned feature vector and the initial dynamic time step into the path prediction model, predict the three-dimensional coordinate change of the fire spread boundary, and the training method of the path prediction model can refer to the training method of the first extraction model; continue to collect new image data, new sensor data and new meteorological data from the fire scene to update the aligned feature vectors, obtain new aligned feature vectors, and combine the dynamic time step updated by the reinforcement learning mechanism to predict the next round of fire spread path through the path prediction model; repeat the above process of updating the aligned feature vectors and dynamic time steps, and continuously iterate and predict the fire spread path.

[0183] Example 2

[0184] See also Figure 4-Figure 5 As shown, the fire spread path dynamic prediction system based on artificial intelligence in this embodiment includes:

[0185] Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate and burned area of ​​the fire scene;

[0186] The first processing module: extracts features from the image data based on the first extraction model to obtain image features;

[0187] The second extraction module: extracts features from the sensor data based on the second extraction model to obtain environmental features;

[0188] The third extraction module: extracts features from the meteorological data based on the third extraction model to obtain meteorological features;

[0189] Feature alignment module: used to calculate the correlation weights between different modal features, and perform weighted fusion of different modal features based on the correlation weights to obtain the aligned feature vector; different modal features include image features, environmental features, and meteorological features;

[0190] Step adjustment module: used to use the aligned feature vector as the input of the reinforcement learning mechanism, dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism, and obtain the dynamic time step;

[0191] Path prediction module: Dynamic prediction of fire path is performed based on the path prediction model according to the dynamic time step.

[0192] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

[0193] Finally: The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A fire spread path dynamic prediction method based on artificial intelligence, characterized in that: The steps include: Collect image data, sensor data, meteorological data, fire spread rate and burned area at the fire scene: Performing feature extraction on the image data based on the first extraction model to obtain image features; Extracting features from the sensor data based on the second extraction model to obtain environmental features; Extracting features from the meteorological data based on the third extraction model to obtain meteorological features; Calculate the correlation weights between different modal features, and perform weighted fusion of different modal features based on the correlation weights to obtain aligned feature vectors; different modal features include image features, environmental features, and meteorological features; The aligned feature vector is used as the input of the reinforcement learning mechanism, and the prediction time step of the path prediction model is dynamically adjusted based on the reinforcement learning mechanism to obtain a dynamic time step; Dynamic prediction of fire path is performed based on the path prediction model according to the dynamic time step.

2. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1 is characterized in that: The method of obtaining the dynamic time step includes: Define an agent, which is used to select and execute actions in the action space; Define the state space S, which includes the fire spread rate, the area of ​​the burned area, the temperature and the meteorological data, and obtain the state vector s corresponding to the state space S; Define the action space A, according to the preset time step value range [Δa min , Δa max ], where Δa max is the maximum time step; Δa min The preset time step value range is discretized into N values, and the action space A corresponding to the action containing n time step values ​​is obtained. N }, the nth action in the action space A, that is, the value a of the nth time step n Satisfy Δa min ≤a n ≤Δa max , n=1,…,N; Design the reward function r based on the error between the predicted 3D coordinate change of action a and the actual 3D coordinate change; Initialize the reinforcement learning mechanism and select a deep Q network to simulate the Q-value function Q(s, a). The Q-value function Q(s, a) represents the expected cumulative reward obtained by performing action a under the state vector s. During the training process, the agent interacts with the state vector and presets the experience playback buffer D to record the four-tuple (s t , a t , r t ,s t+1 ), where s t is the state vector corresponding to time step t, a t is the action taken at time step t, r t is the reward obtained at time step t, s t+1 is the state vector corresponding to the next time step; A set of four-tuples (s) are randomly drawn from the experience replay buffer D j , a j , r j ,s j+1 ), where j = 1, ..., M, is the number of quaternions corresponding to different time steps. j , a j , r j ,s j+1 ) Update Q network parameters; The agent uses an ε-greedy strategy to select actions; After the training is completed, the action with the largest Q value is selected as the prediction time step, and the path prediction model is evaluated based on the evaluation index according to the preset index threshold: when the evaluation index reaches the preset index threshold, the corresponding preset time step is obtained.

3. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 2 is characterized in that: According to the four-tuple (s j , a j , r j ,s j+1 ) The method for updating the Q network parameters includes: Combine the reward obtained at time step j and the reward at s j+1 The maximum value of the Q value of all possible actions a' under the deep Q network is used to calculate the target network Q value y j ; Update the main Q network parameters by minimizing the loss function; the main Q network is used to calculate the current state vector s j Next select action a j Q value; The Adam optimizer is used to iteratively optimize the loss function to obtain updated parameters.

4. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 2 is characterized in that: The method of selecting an action using the ε-greedy strategy includes: Use the selection probability ε to randomly select an action, and use the selection probability 1-ε to select the action with the maximum Q value calculated by the current Q network.

5. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1 is characterized in that: The method for obtaining the aligned feature vector includes: Calculate the similarity matrix between different modal features: The similarity matrix between the image features and the environmental features, the similarity matrix between the image features and the meteorological features, and the similarity matrix between the environmental features and the meteorological features are calculated respectively through the conversion function; The features of different modalities are weightedly fused according to the similarity matrix to obtain the aligned feature vector.

6. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1, characterized in that: The method for obtaining the area of ​​the burned region includes: Obtain bird's-eye view images of the fire area using drones equipped with cameras; The bird's-eye view image is subjected to denoising and enhancement processing to obtain an original bird's-eye view image, and then subjected to grayscale processing to obtain a grayscale bird's-eye view image; The grayscale bird's-eye view image is segmented according to a preset segmentation threshold, and pixels in the grayscale bird's-eye view image whose pixel values ​​are greater than the preset segmentation threshold are marked as burned areas, and pixels whose pixel values ​​are less than the preset segmentation threshold are marked as unburned areas; the original bird's-eye view image is converted from the RGB color interval to the HSV color space, and the HSV color space is divided into I, E and H intervals according to hue, saturation and brightness, respectively, and the intervals are numbered accordingly, and the interval numbers to which the hue, saturation and brightness of each pixel belong are counted, and the interval numbers corresponding to the hue, saturation and brightness of the pixel are spliced ​​as color features; The gray-level co-occurrence matrix of the gray-level bird's-eye view image is statistically calculated, and the eigenvalues ​​of the gray-level co-occurrence matrix are calculated in the neighborhood of each pixel. The eigenvalues ​​include contrast, correlation, energy, and entropy. The eigenvalues ​​are concatenated to obtain texture features. Select K seed points located in the determined burned area in the grayscale bird's-eye view image, and record the grayscale value, texture feature and color feature of each seed point in the grayscale bird's-eye view image; Define growth rules: for pixels adjacent to the seed point, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed point and the seed point respectively through Euclidean distance; obtain comprehensive similarity by weighted average of texture feature similarity and color feature similarity; compare the comprehensive similarity with a preset comprehensive similarity threshold, and when the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area; Starting from the seed point, adjacent pixels that meet the conditions are added to the burned area according to the growth rule until no new pixels meet the growth conditions; The burned area is calculated based on the product of the image resolution and the number of pixels marked as burned after segmentation.

7. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 6 is characterized in that: The method for obtaining the color feature includes: Divide hue, saturation and brightness into I, E and H intervals respectively, and number the intervals accordingly; Convert the original bird's-eye view image from RGB color space to HSV color space; Methods for converting the original bird's-eye view image from RGB color space to HSV color space include: The brightness is calculated by the maximum component value of the original bird's-eye view image in the R channel, the G channel, and the B channel; The saturation is calculated by the difference between the maximum component value of the original bird's-eye view image in the R channel, the G channel and the B channel and the minimum component value of the original bird's-eye view image in the R channel, the G channel and the B channel; Differentiate and calculate hues based on lightness and saturation scores; For each pixel in the HSV image, obtain its hue, saturation and brightness values; determine the interval number to which the hue, saturation and brightness values ​​of each pixel belong, and concatenate the corresponding hue interval number, saturation interval number and brightness interval number as a color feature.

8. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 6, characterized in that: The method for obtaining the texture feature includes: Count the frequency of pixel pairs with grayscale values ​​X and Y in the grayscale bird's-eye view image in a neighborhood of size C×C, where C is the size of the neighborhood, and use the frequency as the element T(X, Y) of the grayscale co-occurrence matrix T of size H×H; H is the number of rows and columns of the grayscale co-occurrence matrix T; Select a pixel as the center point, obtain the corresponding neighborhood of size X×X, and traverse the pixel pairs in the corresponding neighborhood of size C×C according to the preset spatial position relationship; For each pair of adjacent pixels, obtain the corresponding grayscale values ​​X and Y, and add 1 to the corresponding element T(X, Y) in the grayscale co-occurrence matrix T; Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that meet the spatial position relationship in the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X, Y); Based on the gray-level co-occurrence matrix P(X, Y), the contrast, correlation, energy and entropy are calculated respectively; Contrast, correlation, energy and entropy are concatenated as texture features.

9. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1, characterized in that: The method for dynamically predicting a fire path based on the path prediction model includes: Initialize the path prediction model parameters, substitute the aligned feature vector and the initial dynamic time step into the path prediction model, predict the three-dimensional coordinate change of the fire spread boundary, and continuously collect new image data, new sensor data, and new meteorological data from the fire scene to update the aligned feature vectors. Obtain a new aligned feature vector, and combine it with the dynamic time step updated by the reinforcement learning mechanism to predict the next round of fire spread path through the path prediction model; repeat the above process of updating the aligned feature vector and the dynamic time step, and continuously iterate and predict the fire spread path.

10. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1, characterized in that: The method for obtaining the fire spread rate includes: Preselect two monitored points, the positions of the two monitored points are x1 and x2 respectively, measure the temperature of the monitored points by infrared detection equipment, compare the temperature of the monitored points with a preset temperature threshold, when the temperature of the monitored points reaches the preset temperature threshold, it indicates that the monitored points are in a burning state, count the time τ1 and τ2 when the temperature of the two monitored points with the positions of the monitored points x1 and x2 reaches the preset temperature threshold, and obtain the fire spread rate V based on the ratio of the change value of the monitoring point position to the time change value of the burning state.

11. The method for dynamic prediction of fire spread path based on artificial intelligence according to claim 1, characterized in that: The training method of the first extraction model includes: Collecting a group A of image training data in advance, where the image training data includes image data and image features corresponding to the image data; Each group of image training data is used as the input of the first extraction model. The first extraction model uses the image features corresponding to each group of image data as output, and uses the actual image features corresponding to each group of image data as prediction targets; minimizing the sum of prediction errors of all image features is used as the training target; the first extraction model is trained until the sum of prediction errors reaches convergence, and the training is stopped; the first extraction model is a deep neural network model; the loss function value of the first extraction model is the mean square error.

12. A fire spread path dynamic prediction system based on artificial intelligence, used to implement the fire spread path dynamic prediction method based on artificial intelligence as claimed in any one of claims 1 to 11, characterized in that: include: Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate and burned area of ​​the fire scene; The first processing module: extracts features from the image data based on the first extraction model to obtain image features; The second extraction module: extracts features from the sensor data based on the second extraction model to obtain environmental features; The third extraction module: extracts features from the meteorological data based on the third extraction model to obtain meteorological features; Feature alignment module: used to calculate the correlation weights between different modal features, and perform weighted fusion of different modal features based on the correlation weights to obtain the aligned feature vector; different modal features include image features, environmental features, and meteorological features; Step adjustment module: used to use the aligned feature vector as the input of the reinforcement learning mechanism, dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism, and obtain the dynamic time step; Path prediction module: Dynamic prediction of fire path is performed based on the path prediction model according to the dynamic time step.

Images