Dynamic Prediction Method and System for Fire Spread Path Based on Artificial Intelligence

Abstract

The present invention belongs to the field of artificial intelligence technology, and discloses a method and system for dynamically predicting the fire spread path based on artificial intelligence. The method includes: collecting image data, sensor data, meteorological data, fire spread rate and the area of the burned area at the fire scene; extracting features from the image data to obtain image features; extracting features from the sensor data to obtain environmental features; extracting features from the meteorological data to obtain meteorological features; calculating the correlation weights between different modal features, and performing weighted fusion on different modal features based on the correlation weights to obtain an aligned feature vector; dynamically adjusting the model prediction time step based on a reinforcement learning mechanism to obtain a dynamic time step; and performing dynamic prediction of the fire path based on the path prediction model according to the dynamic time step. The present invention can finely capture the changes in the fire spread path, thereby improving the prediction accuracy.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology. More specifically, the present invention relates to a dynamic prediction method and system for fire spread paths based on artificial intelligence. Background Art

[0002] Traditional fire spread prediction methods mainly rely on physical models and empirical formulas. For example, theoretical models based on heat conduction, heat convection, and heat radiation equations, as well as empirical rules summarized according to different fuel types and environmental conditions. However, these methods have many limitations. On the one hand, physical models often require a large number of simplifying assumptions about the fire scene, such as assuming uniform fuel distribution and constant environmental conditions. In actual fire scenes, which are complex and variable, these assumptions are difficult to hold. On the other hand, the applicable range of empirical formulas is limited. There are significant differences in fuel characteristics, meteorological conditions, and building structures in different regions, making it difficult for empirical formulas to accurately predict fire spread paths in various situations.

[0003] Chinese patent application with publication number CN118195110A discloses a method and system for real-time fire evacuation path planning based on deep learning. In response to receiving a fire start instruction, fire scene information is determined, and the fire scene information at least includes road planning within the fire scene; monitoring data collected in real time by monitoring devices at the fire scene is obtained; the monitoring data is subjected to low-dimensional space mapping to obtain a plurality of first monitoring data feature matrices; based on a first influence feature recognition configuration, influence feature recognition is performed on the plurality of first monitoring data feature matrices to determine the basis of influence monitoring data corresponding to the plurality of first monitoring data feature matrices; the influence monitoring data is input into a target fire spread trend inference model pre-trained based on deep learning to obtain target spread trend information, and the target spread trend information includes the time information when the fire spreads to each road in the fire scene; based on the road planning and the target spread trend information, a target evacuation path is generated and sent. This invention effectively ensures that people in the fire scene can evacuate quickly and reliably, effectively guaranteeing the evacuation efficiency in case of a fire.

[0004] Although the above method can meet most scenarios, through research and practical application of the above method and existing technologies, it is found that the above method and existing technologies have at least the following partial defects:

[0005] When fusing multi-modal data such as image data, sensor data, and meteorological information, the feature spaces and representation methods of different modal data vary greatly; the spread speeds and characteristics in the initial and later stages of fire development are very different, and a fixed model prediction time step is difficult to meet the prediction requirements in different stages.

[0006] In view of this, the present invention proposes a dynamic prediction method and system for fire spread paths based on artificial intelligence to solve the above problems. Summary of the Invention

[0007] To overcome the above-mentioned defects of the prior art and to achieve the above object, the present invention provides the following technical solution: A dynamic prediction method for fire spread paths based on artificial intelligence, comprising the following steps:

[0008] Collect image data, sensor data, meteorological data, fire spread rate, and the area of the burned area at the fire scene;

[0009] Extract features from the image data based on a first extraction model to obtain image features;

[0010] Extract features from the sensor data based on a second extraction model to obtain environmental features;

[0011] Extract features from the meteorological data based on a third extraction model to obtain meteorological features;

[0012] Calculate the correlation weights between different modal features, and perform weighted fusion on different modal features based on the correlation weights to obtain an aligned feature vector; different modal features include image features, environmental features, and meteorological features;

[0013] Use the aligned feature vector as the input of a reinforcement learning mechanism, and dynamically adjust the prediction time step of a path prediction model based on the reinforcement learning mechanism to obtain a dynamic time step;

[0014] Perform dynamic prediction of the fire path based on the path prediction model according to the dynamic time step.

[0015] Further, the method for obtaining the dynamic time step includes:

[0016] Define an agent, which is used to select and execute actions in the action space;

[0017] Define a state space S, where the state space S includes the fire spread rate, the area of the burned area, temperature, and meteorological data, and obtain a state vector s corresponding to the state space S;

[0018] Define an action space A. According to the preset time step value range [Δa min , Δa max , where Δa max is the maximum value of the time step; Δa min is the minimum value of the time step; discretize the preset time step value range into N values to obtain an action space A = {a1, a2,..., a N} corresponding to N time step values. The nth action in the action space A, that is, the value a n of the nth time step, satisfies Δa min ≤ a n≤Δa max , n = 1, …, N;

[0019] Design a reward function r based on the error between the predicted three - dimensional coordinate change and the true three - dimensional coordinate change obtained from action a;

[0020] Initialize the reinforcement learning mechanism: Select a deep Q - network to simulate the Q - value function Q(s, a), where the Q - value function Q(s, a) represents the expected cumulative reward obtained by performing action a under the state vector s;

[0021] During the training process, the agent interacts with the state vector. Preset an experience replay buffer D to record the quadruple (s t , a t , r t , s t+1 ) corresponding to time step t, where s t is the state vector corresponding to time step t, a t is the action taken at time step t, r t is the reward obtained at time step t, and s t+1 is the state vector corresponding to the next time step;

[0022] Randomly sample a set of quadruples (s j , a j , r j , s j+1 ) from the experience replay buffer D, where j = 1, …, M is the number of quadruples corresponding to different time steps. Update the Q - network parameters according to the quadruple (s j , a j , r j , s j+1 );

[0023] Step H: The agent selects an action using an ε - greedy policy;

[0024] Step I: After training is completed, select the action with the largest Q - value as the predicted time step length, and evaluate the path prediction model based on the evaluation metric according to the preset metric threshold: When the evaluation metric reaches the preset metric threshold, obtain the corresponding preset time step length.

[0025] Furthermore, the method for updating the Q - network parameters according to the quadruple (s j , a j , r j , s j+1 ) includes:

[0026] Calculate the target network Q - value y of the deep Q - network by combining the reward obtained at time step j and the maximum value of the Q - values of all possible actions a' under s j+1 ; j ;

[0027] Update the parameters of the main Q-network by minimizing the loss function; the main Q-network is used to calculate the current state vector s j Select the action a j of the Q value;

[0028] Use the Adam optimizer to iteratively optimize the loss function to obtain updated parameters.

[0029] Furthermore, the method for selecting an action using the ε-greedy strategy includes:

[0030] Randomly select an action with the selection probability ε, and select the action with the maximum Q value calculated by the current Q-network with the selection probability 1 - ε.

[0031] Furthermore, the method for obtaining the aligned feature vector includes:

[0032] Calculate the similarity matrix between different modality features:

[0033] Calculate the similarity matrix between image features and environmental features, the similarity matrix between image features and meteorological features, and the similarity matrix between environmental features and meteorological features respectively through a transformation function;

[0034] Weightedly fuse different modality features according to the similarity matrix to obtain the aligned feature vector.

[0035] Furthermore, the method for obtaining the area of the burned area includes:

[0036] Obtain an aerial view image of the fire area through a drone equipped with a camera;

[0037] Denoise and enhance the aerial view image to obtain the original aerial view image, and then perform grayscale processing to obtain the grayscale aerial view image;

[0038] Segment the grayscale aerial view image according to a preset segmentation threshold, mark the pixels with pixel values greater than the preset segmentation threshold in the grayscale aerial view image as the burned area, and mark the pixels with pixel values less than the preset segmentation threshold as the unburned area; convert the original aerial view image from the RGB color range to the HSV color space, divide the HSV color space into I, E, and H intervals according to hue, saturation, and lightness respectively, and number the intervals correspondingly, count the interval numbers to which the hue, saturation, and lightness of each pixel belong, and splice the interval numbers corresponding to the hue, saturation, and lightness of the pixel as the color feature;

[0039] Statistical gray-level co-occurrence matrix of the grayscale aerial view image, and calculate the eigenvalues of the gray-level co-occurrence matrix in the neighborhood of each pixel. The eigenvalues include contrast, correlation, energy, and entropy, and splice the eigenvalues to obtain the texture feature;

[0040] Select K seed points located within the determined burned area in the grayscale aerial image, and record the grayscale value, texture feature, and color feature of each seed point in the grayscale aerial image.

[0041] Define the growth rule: For the pixels adjacent to the seed points, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed points and the seed points respectively through the Euclidean distance; obtain the comprehensive similarity by weighted averaging the texture feature similarity and color feature similarity; compare the comprehensive similarity with the preset comprehensive similarity threshold. When the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area.

[0042] Starting from the seed points, add the eligible adjacent pixels to the burned area according to the growth rule until no new pixels meet the growth conditions.

[0043] Calculate the area of the burned area according to the product of the image resolution and the number of pixels marked as the burned area after segmentation.

[0044] Furthermore, the method for obtaining the color feature includes:

[0045] Divide the hue, saturation, and lightness into I, E, and H intervals respectively, and number the intervals correspondingly.

[0046] Convert the original aerial image from the RGB color space to the HSV color space.

[0047] The method for converting the original aerial image from the RGB color space to the HSV color space includes:

[0048] Calculate the lightness through the maximum component value of the original aerial image in the R channel, G channel, and B channel.

[0049] Calculate the saturation through the difference between the maximum component value of the original aerial image in the R channel, G channel, and B channel and the minimum component value of the original aerial image in the R channel, G channel, and B channel.

[0050] Distinguish and calculate the hue according to the scores of the lightness and saturation.

[0051] For each pixel in the HSV image, obtain the values of its hue, saturation, and lightness; determine the interval numbers to which the values of the hue, saturation, and lightness of each pixel belong, and splice the corresponding hue interval number, saturation interval number, and lightness interval number as the color feature.

[0052] Furthermore, the method for obtaining the texture feature includes:

[0053] Statistically calculate the frequency of pixel pairs with gray values X and Y respectively in the grayscale overhead image within a neighborhood of size C×C, where C is the neighborhood size, and use the frequency as the element T(X, Y) of the gray-level co-occurrence matrix T of size H×H; H is the number of rows and columns of the gray-level co-occurrence matrix T;

[0054] Select a pixel as the center point, obtain the corresponding neighborhood of size C×C, and traverse the pixel pairs within the corresponding neighborhood of size C×C according to the preset spatial position relationship;

[0055] For each pair of adjacent pixels, obtain the corresponding gray values X and Y, and increment the corresponding element T(X, Y) in the gray-level co-occurrence matrix T by 1;

[0056] Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that meet the spatial position relationship within the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X, Y);

[0057] Calculate the contrast, correlation, energy, and entropy based on the gray-level co-occurrence matrix P(X, Y) respectively;

[0058] Concatenate the contrast, correlation, energy, and entropy as the texture features.

[0059] Furthermore, the method for dynamically predicting the fire path based on the path prediction model includes:

[0060] Initialize the path prediction model parameters, substitute the aligned feature vector and the initial dynamic time step into the path prediction model to predict the three-dimensional coordinate change of the fire spread boundary, continuously collect new image data, new sensor data, and new meteorological data at the fire scene to update the aligned feature vector, obtain a new aligned feature vector, combine the dynamically updated time step by the reinforcement learning mechanism, and perform the next round of fire spread path prediction through the path prediction model; repeat the above process of updating the aligned feature vector and the dynamic time step, and continuously iterate to predict the fire spread path.

[0061] Furthermore, the method for obtaining the fire spread rate includes:

[0062] Pre-select two monitored points, the positions of the two monitored points are x1 and x2 respectively, measure the temperature of the monitored points through an infrared detection device, compare the temperature of the monitored points with a preset temperature threshold, when the temperature of the monitored point reaches the preset temperature threshold, it means that the monitored point is in a burning state, statistically calculate the times τ1 and τ2 when the temperatures of the two monitored points at positions x1 and x2 reach the preset temperature threshold, and obtain the fire spread rate V based on the ratio of the change in the monitored point position to the change in time of the burning state.

[0063] Furthermore, the training method of the first extraction model includes:

[0064] Pre-collect the image training data of Group A, where the image training data includes image data and the corresponding image features of the image data;

[0065] Take each group of image training data as the input of the first extraction model. The first extraction model takes the image features corresponding to each group of image data as the output and the actual image features corresponding to each group of image data as the prediction target. Take minimizing the sum of the prediction errors of all image features as the training target. Train the first extraction model until the sum of the prediction errors converges and then stop training. The first extraction model is a deep neural network model. The loss function value of the first extraction model is the mean square error.

[0066] An artificial intelligence-based dynamic fire spread path prediction system for implementing the artificial intelligence-based dynamic fire spread path prediction method, including:

[0067] Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate, and the area of the burned area at the fire scene;

[0068] First processing module: Based on the first extraction model, extract features from the image data to obtain image features;

[0069] Second extraction module: Based on the second extraction model, extract features from the sensor data to obtain environmental features;

[0070] Third extraction module: Based on the third extraction model, extract features from the meteorological data to obtain meteorological features;

[0071] Feature alignment module: used to calculate the correlation weights between different modality features, and perform weighted fusion on different modality features based on the correlation weights to obtain the aligned feature vector. Different modality features include image features, environmental features, and meteorological features;

[0072] Step adjustment module: used to take the aligned feature vector as the input of the reinforcement learning mechanism, and dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism to obtain the dynamic time step;

[0073] Path prediction module: Based on the dynamic time step, perform dynamic fire path prediction according to the path prediction model.

[0074] The technical effects and advantages of the artificial intelligence-based dynamic fire spread path prediction method and system of the present invention:

[0075] The present invention maps data of different modalities, such as image data, sensor data, and meteorological data, to a unified feature space through a feature fusion technology, enabling the model to process these data from different sources simultaneously; it helps to explore the internal correlations between different modalities of data and utilize the complementarity of each modality of data, so as to understand the fire scene more comprehensively; by corresponding to different development stages of the fire, the prediction time step can be dynamically adjusted, and the changes in the fire spread path can be captured more precisely, thereby improving the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] Figure 1 It is a schematic flow chart of the method for dynamically predicting the fire spread path based on artificial intelligence according to the present invention;

[0077] Figure 2 It is a schematic flow chart of the method for obtaining the area of the burned area according to the present invention;

[0078] Figure 3 It is a schematic flow chart of the method for obtaining the dynamic time step according to the present invention;

[0079] Figure 4 It is a block diagram of the system for dynamically predicting the fire spread path based on artificial intelligence according to the present invention;

[0080] Figure 5 It is a schematic diagram of the interface of the system for dynamically predicting the fire spread path based on artificial intelligence according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0081] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0082] Embodiment 1

[0083] Please refer to Figure 1 As shown, the method for dynamically predicting the fire spread path based on artificial intelligence in this embodiment includes the following steps:

[0084] Collect image data, sensor data, meteorological data, fire spread rate, and the area of the burned area at the fire scene; the image data at the fire scene is collected based on a drone equipped with a high-definition camera, including image data of the fire scene from different angles, and the image data can clearly show the shape, size, and position of the flame. By analyzing the image, the advancing direction and speed of the flame front can be directly observed, providing a basis for predicting the next direction of the fire.

[0085] The sensor data is collected based on temperature sensors and smoke sensors. The sensor data includes the temperature at the fire scene and the smoke concentration. Collecting temperature data can accurately depict the heat distribution at the fire scene, predict the propagation path of heat in the building structure, and further predict the fire spread path. The change in smoke concentration can reflect the situation of air flow. By setting smoke sensors at multiple locations and combining information on wind direction and ventilation systems, the impact of air flow patterns on fire spread can be inferred.

[0086] The meteorological data is collected through wind speed sensors, wind direction sensors, and humidity sensors, and includes the wind speed, wind direction, and humidity at the fire scene. Wind provides additional oxygen supply for the combustion process and quickly transfers the heat generated by combustion to the unburned area, accelerating the combustion reaction. Smoke also spreads along the wind direction. Understanding the wind speed and wind direction can predict the propagation path of smoke, which is crucial for evaluating the impact of the fire on the surrounding environment. Environmental humidity affects the humidity and drying speed of combustible materials, as well as the evaporative cooling effect.

[0087] The methods for obtaining the fire spread rate include:

[0088] Pre-select two monitored points, with the positions of the two monitored points being x1 and x2 respectively. Measure the temperature of the monitored points through infrared detection equipment, and compare the temperature of the monitored points with a preset temperature threshold. When the temperature of the monitored point reaches the preset temperature threshold, it indicates that the monitored point is in a burning state. Statistically record the times τ1 and τ2 when the two monitored points at positions x1 and x2 change from a non-burning state to a burning state exactly, that is, the times when the temperatures of the two monitored points at positions x1 and x2 reach the preset temperature threshold. Based on the velocity formula Obtain the fire spread rate V. Fires can spread between different types of materials and structures. Different materials have different combustion characteristics, resulting in different fire spread rates. By dynamically adjusting the time step according to the spread rate, when the fire spreads from one material to another (such as from a wooden structure to a steel structure building), the simulation step can be reasonably updated to accurately reflect the propagation of the fire in different media. By accurately obtaining the spread rate, the position of the flame front at different time points can be updated in real time. For example, in a two-dimensional plane, if the position and spread rate of the flame front at a certain moment are known, the position where the flame will reach at the next moment can be predicted, and thus the dynamic spread path of the flame in space can be accurately depicted.

[0089] Refer to Figure 2 The methods for obtaining the area of the burned area include:

[0090] Obtain an aerial view image of the fire area through a drone equipped with a high-precision camera;

[0091] Denoise and enhance the aerial image to obtain the original aerial image, and then perform grayscale processing to obtain the grayscale aerial image;

[0092] Segment the grayscale aerial image according to a preset segmentation threshold, mark the pixels with pixel values greater than the preset segmentation threshold in the grayscale aerial image as the burned area, and mark the pixels with pixel values less than the preset segmentation threshold as the unburned area; Convert the original aerial image from the RGB color range to the HSV color space, divide the HSV color space into I, E, and H intervals according to hue, saturation, and lightness respectively, and number the intervals correspondingly. Count the interval numbers to which the hue, saturation, and lightness of each pixel belong, and splice the interval numbers corresponding to the hue, saturation, and lightness to which the pixel belongs as the color feature;

[0093] The method for obtaining the color feature includes:

[0094] Divide the hue, saturation, and lightness into I, E, and H intervals respectively, and number the intervals correspondingly;

[0095] Convert the original aerial image from the RGB color space to the HSV color space;

[0096] The method for converting the original aerial image from the RGB color space to the HSV color space includes:

[0097] Calculate the lightness VM as follows:

[0098] VM = max(R, G, B);

[0099] Wherein, R, G, and B are the component values of the R channel, G channel, and B channel in the RGB color space of the original aerial image respectively; max(R, G, B) is the maximum value among the three component values of R, G, and B;

[0100] Calculate the saturation as follows:

[0101] Calculate the saturation variable δ = max(R, G, B) - min(R, G, B); where min(R, G, B) is the minimum value among the three component values of R, G, and B;

[0102] When VM = 0, the saturation VB = 0;

[0103] When VM ≠ 0, the saturation

[0104] Calculate the hue as follows:

[0105] When δ = 0 or VB = 0, the hue VS = 0;

[0106] When R = max(R, G, B) and G ≥ B, the hue

[0107] When R = max(R, G, B) and G < B, the hue

[0108] When G = max(R, G, B), the hue

[0109] When B = max(R, G, B), the hue

[0110] For each pixel in the HSV image, obtain the values of its hue, saturation, and lightness; determine the interval numbers to which the values of the hue, saturation, and lightness of each pixel belong, and splice the corresponding hue interval number, saturation interval number, and lightness interval number as the color feature.

[0111] Statistical gray-level co-occurrence matrix of the gray-scale aerial image, and calculate the eigenvalues of the gray-level co-occurrence matrix in the neighborhood of each pixel. The eigenvalues include contrast, correlation, energy, and entropy, and splice the eigenvalues to obtain the texture feature;

[0112] The methods for obtaining the texture feature include:

[0113] Statistical frequency of pixel pairs with gray values X and Y in the neighborhood of size C×C in the gray-scale aerial image, where C is the neighborhood size, and use the frequency as the element T(X, Y) of the gray-level co-occurrence matrix T of size H×H; H is the number of rows and columns of the gray-level co-occurrence matrix T;

[0114] Select a pixel as the center point, obtain the corresponding neighborhood of size C×C, and traverse the pixel pairs in the corresponding neighborhood of size C×C according to the preset spatial position relationship (such as horizontally adjacent, vertically adjacent, diagonally adjacent, etc.);

[0115] For each pair of adjacent pixels, obtain the corresponding gray values X and Y, and increment the corresponding element T(X, Y) in the gray-level co-occurrence matrix T by 1; for example, if the gray value of the current pixel is 10 and the gray value of its adjacent pixel on the right is 15, then increment T(10, 15) by 1, and repeat this process until all pixel pairs that meet the spatial position relationship in the neighborhood are traversed.

[0116] Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that meet the spatial position relationship in the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X, Y); the calculation of the gray-level co-occurrence matrix P(X, Y) is as follows:

[0117]

[0118] Among them, P(X,Y) is the gray-level co-occurrence matrix, representing the probability of the pixel pair (X,Y) appearing.

[0119] Calculate the contrast based on the gray-level co-occurrence matrix P(X,Y):

[0120]

[0121] Among them, Contrast is the contrast;

[0122] Calculate the correlation based on the gray-level co-occurrence matrix P(X,Y):

[0123]

[0124] Among them, Correlation is the correlation; μ X is the mean value of the gray value X, that is, for all possible gray values X, weighted summation is performed according to the corresponding pixel pair occurrence probability P(X,Y) to obtain the average level of the gray value X in the selected neighborhood; μ Y is the mean value of the gray value Y, that is, for all possible gray values Y, weighted summation is performed according to the corresponding pixel pair occurrence probability P(X,Y) to obtain the average level of the gray value Y in the selected neighborhood; σ X is the standard deviation of the gray value X, that is, the degree of dispersion of the gray value X in the selected neighborhood; σ Y is the standard deviation of the gray value Y, that is, the degree of dispersion of the gray value Y in the selected neighborhood;

[0125] Calculate the energy based on the gray-level co-occurrence matrix P(X,Y):

[0126]

[0127] Among them, Energy is the energy;

[0128] Calculate the entropy based on the gray-level co-occurrence matrix P(X,Y):

[0129]

[0130] Among them, Entropy is the entropy.

[0131] Concatenate the contrast, correlation, energy, and entropy as texture features.

[0132] Select K seed points located in the determined burned area in the gray-scale overview image, and record the gray value, texture feature, and color feature of each seed point in the gray-scale overview image;

[0133] Define the growth rule: For the pixels adjacent to the seed point, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed point and the seed point respectively by Euclidean distance; obtain the comprehensive similarity by weighted averaging the texture feature similarity and color feature similarity; compare the comprehensive similarity with the preset comprehensive similarity threshold. When the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area.

[0134] Starting from the seed point, add the eligible adjacent pixels to the burned area according to the growth rule until no new pixels meet the growth conditions.

[0135] According to the image resolution FB, count the number of pixels W marked as the burned area after segmentation, and calculate the burned area G based on the formula G = W × FB. In the initial stage of the fire, the growth of the burned area is relatively slow; as the fire develops, the burned area may expand rapidly; by monitoring the change rate of the burned area, we can understand what stage the fire is in; this change in fire development can be used as a basis for dynamically adjusting the time step. In the slow development stage of the fire, the time step can be appropriately increased to reduce the computational amount; while in the rapid spread stage, reducing the time step can more precisely capture the rapid change process of the fire and ensure the accuracy of the simulation; moreover, the boundary position of the burned area intuitively shows the front of the fire spread; by continuously monitoring the change of the burned area, we can determine the advancing speed of the flame in different directions, so as to predict the direction of fire spread.

[0136] Extract features from the image data based on the first extraction model to obtain image features;

[0137] The training method of the first extraction model includes:

[0138] Pre-collect a set of image training data A, where the image training data includes image data and the corresponding image features of the image data.

[0139] Take each set of image training data as the input of the first extraction model. The first extraction model takes the image features corresponding to each set of image data as the output and the actual image features corresponding to each set of image data as the prediction target; take minimizing the sum of the prediction errors of all image features as the training target; train the first extraction model until the sum of the prediction errors converges and then stop training; the first extraction model is a deep neural network model.

[0140] The loss function value of the first extraction model is the mean square error.

[0141] Prediction error formula: a is the a-th set of image data; A is the number of sets of image data; is the image feature corresponding to the a-th group of image data; z a is the actual image feature corresponding to the a-th group of image data.

[0142] Feature extraction is performed on the sensor data based on the second extraction model to obtain environmental features; the training method of the second extraction model can refer to the training method of the first extraction model.

[0143] Feature extraction is performed on the meteorological data based on the third extraction model to obtain meteorological features; the training method of the third extraction model can refer to the training method of the first extraction model.

[0144] Calculate the correlation weights between different modality features, and perform weighted fusion on different modality features based on the correlation weights to obtain the aligned feature vector; different modality features include image features, environmental features, and meteorological features.

[0145] The method for obtaining the aligned feature vector includes:

[0146] Calculate the similarity matrix between different modality features:

[0147] Calculate the similarity matrix A between the image feature and the environmental feature IS :

[0148]

[0149] Calculate the similarity matrix A between the image feature and the meteorological feature IM :

[0150]

[0151] Calculate the similarity matrix A between the environmental feature and the meteorological feature IM :

[0152]

[0153] where f I is the image feature; f S is the environmental feature; f M is the meteorological feature; Softmax is a conversion function used to convert the similarity value into a probability distribution;

[0154] Perform weighted fusion on each modality feature according to the similarity matrix to obtain the aligned feature vector:

[0155] F = α1A IS f I + α2A IM f I + β1A IM f M + β2A SM fM +γ1A IS f s +γ2A SM f S ;

[0156] Wherein, α1, α2, β1, β2, γ1 and γ2 are all adjustment parameters; F is the aligned feature vector.

[0157] Taking the aligned feature vector as the input of the reinforcement learning mechanism, dynamically adjusting the prediction time step of the path prediction model based on the reinforcement learning mechanism to obtain a dynamic time step;

[0158] Referring to Figure 3 , the method for obtaining the dynamic time step includes:

[0159] Defining an agent, which is used to select and execute actions;

[0160] Defining a state space S, where the state space S includes the fire spread rate, the area of the burned area, and the aligned feature vector, and obtaining the state vector s corresponding to the state space S;

[0161] Defining an action space A, according to the preset time step value range [Δa min , Δa max , wherein, Δa max is the maximum time step; Δa min is the minimum time step; discretizing the preset time step value range into N values, obtaining an action space A = {a1, a2,..., a N} corresponding to N time step values, and the nth action in the action space A, that is, the value a n of the nth time step satisfies Δa min ≤ a n ≤ Δa max , n = 1,..., N;

[0162] Designing a reward function r(s, a): r(s, a) = -β·∈; wherein, s is the state vector; a is the selected action, that is, the time step; β is a scaling coefficient used to adjust the reward amplitude; ∈ is the error between the predicted three-dimensional coordinate change amount based on the action a and the true three-dimensional coordinate change amount;

[0163] Initializing the reinforcement learning mechanism, selecting a deep Q-network to simulate the Q-value function Q(s, a), and the Q-value function Q(s, a) represents the expected cumulative reward obtained by executing the action a under the state vector s;

[0164] During the training process, the agent interacts with the state vector, and a preset experience replay buffer D is used to record the quadruple (s t , at , r t , s t+1 ), where s t is the state vector corresponding to time step t, a t is the action taken at time step t, r t is the reward obtained at time step t, s t+1 is the state vector corresponding to the next time step.

[0165] Randomly sample a set of quadruples (s j , a j , r j , s j+1 ) from the experience replay buffer D, where j = 1, …, M, is the number of quadruples corresponding to different time steps, s j is the state vector corresponding to time step j, a j is the action taken at time step j, r j is the reward obtained at time step j, s j+1 is the state vector corresponding to the next time step; update the Q-network parameters according to the quadruple (s j , a j , r j , s j+1 );

[0166] The method for updating the Q-network parameters according to the quadruple (s j , a j , r j , s j+1 ) includes:

[0167] Calculate the target network Q-value y j :

[0168]

[0169] where γ is the discount factor, used to weigh the importance of future rewards; Q(s j+1 , a'; Θ - ) is the Q-value of all possible actions a' calculated by the target network with parameters Θ - under s j+1 , the parameters Θ - of the target network are obtained by periodically assigning values from the main Q-network, the target network is used to predict the maximum Q-value under the next state vector s j+1 after executing the action a', and the main Q-network is used to calculate the Q-value of the action a j selected under the current state vector s j ; denotes taking the maximum value of Q(s j+1 , a'; Θ - );

[0170] Update the parameters Θ of the main Q-network by minimizing the loss function; the loss function is defined as follows:

[0171]

[0172] where Q(s j , a j ; Θ) is the Q-value of action a j under state s j calculated by the main Q-network with parameters Θ;

[0173] Use the Adam optimizer to iteratively optimize the loss function to obtain the updated parameters Θ′, and the update formula for parameters Θ is:

[0174]

[0175] where λ is the learning rate; is the gradient of the loss function with respect to parameters Θ t ; Adam is the Adam optimization function, and the Adam optimizer adaptively adjusts the learning rate based on the gradient information.

[0176] The agent selects actions using the ε-greedy policy, that is, randomly selects an action with selection probability ε, and selects the action with the largest Q-value considered by the current Q-network with selection probability 1 - ε. Specifically, as follows:

[0177]

[0178] where a t is the action selected at time step t; s t is the state vector corresponding to time step t; Q(s t , a; Θ) is the Q-value of action a t under state s calculated by the main Q-network with parameters Θ; denotes taking the maximum value of Q(s t , a; Θ);

[0179] After training is completed, select the action with the largest Q-value as the predicted time step length, and evaluate the path prediction model based on the evaluation metrics according to the preset metric thresholds: when the evaluation metrics reach the preset metric thresholds, obtain the corresponding preset time step length, and it can be applied to the dynamic prediction process of the fire path; the evaluation metrics include the closeness of the predicted time step length to the true optimal time step length and the accuracy of the fire spread prediction based on the predicted time step length.

[0180] Perform dynamic prediction of the fire path based on the path prediction model according to the dynamic time step length;

[0181] The method for dynamically predicting the fire path based on the path prediction model includes:

[0182] Initialize the parameters of the path prediction model, substitute the aligned feature vectors and the initial dynamic time step into the path prediction model to predict the three-dimensional coordinate change of the fire spread boundary. The training method of the path prediction model can refer to the training method of the first extraction model; continuously collect new image data, new sensor data and new meteorological data at the fire scene to update the aligned feature vectors, obtain new aligned feature vectors, combine the dynamically updated time step by the reinforcement learning mechanism, and perform the next round of fire spread path prediction through the path prediction model; repeat the above process of updating the aligned feature vectors and the dynamic time step, and continuously iterate to predict the fire spread path.

[0183] Embodiment 2

[0184] Please refer to Figures 4 - 5 As shown, the dynamic fire spread path prediction system based on artificial intelligence in this embodiment includes:

[0185] Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate and area of the burned area at the fire scene;

[0186] First processing module: perform feature extraction on the image data based on the first extraction model to obtain image features;

[0187] Second extraction module: perform feature extraction on the sensor data based on the second extraction model to obtain environmental features;

[0188] Third extraction module: perform feature extraction on the meteorological data based on the third extraction model to obtain meteorological features;

[0189] Feature alignment module: used to calculate the correlation weights between different modality features, and perform weighted fusion on different modality features based on the correlation weights to obtain aligned feature vectors; different modality features include image features, environmental features and meteorological features;

[0190] Step adjustment module: used to take the aligned feature vectors as the input of the reinforcement learning mechanism, and dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism to obtain the dynamic time step;

[0191] Path prediction module: perform dynamic fire path prediction based on the path prediction model according to the dynamic time step.

[0192] As described above, it is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the said claims.

[0193] Finally: The above description is only the preferred embodiment of the present invention and is not used to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A dynamic prediction method for fire spread paths based on artificial intelligence, characterized in that, It includes the following steps: Collect image data, sensor data, meteorological data, fire spread rate, and burned area of the fire scene; Extract features from the image data based on the first extraction model to obtain image features; Extract features from the sensor data based on the second extraction model to obtain environmental features; Extract features from the meteorological data based on the third extraction model to obtain meteorological features; Calculate the correlation weights between different modal features, and perform weighted fusion on different modal features based on the correlation weights to obtain an aligned feature vector; different modal features include image features, environmental features, and meteorological features; Use the aligned feature vector as the input of the reinforcement learning mechanism, and dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism to obtain a dynamic time step: Define an agent, which is used to select and execute actions in the action space; Define the state space S, where the state space S includes the fire spread rate, burned area, temperature, and meteorological data, and obtain the state vector s corresponding to the state space S; Define the action space A, which takes values according to the preset time step range [Δa min , Δa max , where Δa max is the maximum value of the time step; Δa min is the minimum value of the time step. Discretize the preset time step range into N values to obtain the action space A = {a1, a2, …, a N} corresponding to the actions with N time step values; Design a reward function r based on the error between the predicted three-dimensional coordinate change amount and the true three-dimensional coordinate change amount for the action a; Initialize the reinforcement learning mechanism and select a deep Q-network to simulate the Q-value function Q(s,a); During the training process, the agent interacts with the state vector, and a preset experience replay buffer D is used to record the quadruple (s t , a t , r t , s t+1 ) corresponding to the time step t, where s t is the state vector corresponding to the time step t, a t is the action taken at the time step t, r t is the reward obtained at the time step t, and s t+1 is the state vector corresponding to the next time step; Randomly sample a set of quadruples (s j , a j , r j , s j+1 ) from the experience replay buffer D, where j = 1, …, M is the number of quadruples corresponding to different time steps. Update the Q-network parameters according to the quadruples (s j , a j , r j , s j+1 ). The agent selects actions using an ε-greedy strategy; After training is completed, select the action with the largest Q value as the prediction time step, and evaluate the path prediction model based on the evaluation index according to the preset index threshold: when the evaluation index reaches the preset index threshold, obtain the corresponding preset time step; Perform dynamic prediction of the fire path based on the path prediction model according to the dynamic time step.

2. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, wherein The nth action in the action space A, i.e., the value a at the nth time step n satisfies Δa min ≤a n ≤Δa max , n = 1, …, N; The Q-value function Q(s,a) represents the expected cumulative reward obtained by performing the action a under the state vector s.

3. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 2, characterized in that, The method for updating the Q-network parameters according to the quadruple (s j , a j , r j , s j+1 ) includes: Calculate the target network Q-value y of the deep Q-network by combining the reward obtained at time step j and the maximum value of the Q-values of all possible actions a' under s j+1 j ;​ Update the parameters of the main Q-network by minimizing the loss function; the main Q-network is used to calculate the Q-value of the current state vector s j to select the action a j ; Use the Adam optimizer to iteratively optimize the loss function to obtain updated parameters.

4. The method for dynamically predicting the fire spread path based on artificial intelligence according to claim 2, wherein, The method of using the ε-greedy strategy to select actions includes: Randomly select an action with the selection probability ε, and select the action with the largest Q value calculated by the current Q-network with the selection probability 1-ε.

5. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, characterized in that The method of obtaining the aligned feature vector includes: Calculate the similarity matrix between different modal features: Calculate the similarity matrix between the image features and the environmental features, the similarity matrix between the image features and the meteorological features, and the similarity matrix between the environmental features and the meteorological features respectively through the transformation function; Perform weighted fusion on different modal features according to the similarity matrix to obtain an aligned feature vector.

6. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, characterized in that The method of obtaining the burned area includes: Obtain an aerial view image of the fire area through a drone equipped with a camera; Denoise and enhance the aerial view image to obtain the original aerial view image, and then perform grayscale processing to obtain the grayscale aerial view image; Segment the grayscale aerial image according to a preset segmentation threshold, mark the pixels with pixel values greater than the preset segmentation threshold in the grayscale aerial image as the burned area, and mark the pixels with pixel values less than the preset segmentation threshold as the unburned area; convert the original aerial image from the RGB color range to the HSV color space, divide the HSV color space into I, E, and H intervals according to hue, saturation, and lightness respectively, and number the intervals correspondingly, count the interval numbers to which the hue, saturation, and lightness of each pixel belong, and splice the interval numbers corresponding to the hue, saturation, and lightness to which the pixel belongs as the color feature; Statistically analyze the gray-level co-occurrence matrix of the grayscale aerial image, and calculate the eigenvalues of the gray-level co-occurrence matrix within the neighborhood of each pixel. The eigenvalues include contrast, correlation, energy, and entropy, and splice the eigenvalues to obtain the texture feature; Select K seed points located within the determined burned area in the grayscale aerial image, and record the gray value, texture feature, and color feature of each seed point in the grayscale aerial image; Define the growth rule: for the pixels adjacent to the seed points, calculate the texture feature similarity and color feature similarity between the pixels adjacent to the seed points and the seed points respectively through the Euclidean distance; obtain the comprehensive similarity by weighted averaging the texture feature similarity and the color feature similarity; compare the comprehensive similarity with the preset comprehensive similarity threshold, and when the comprehensive similarity is greater than the preset comprehensive similarity threshold, add the pixel to the burned area; otherwise, add the pixel to the unburned area; Starting from the seed points, add the eligible adjacent pixels to the burned area according to the growth rule until no new pixels meet the growth conditions; Calculate the burned area according to the product of the image resolution and the number of pixels marked as the burned area after segmentation.

7. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 6, characterized in that The method for obtaining the color feature includes: Divide the hue, saturation, and lightness into I, E, and H intervals respectively, and number the intervals correspondingly; Convert the original aerial image from the RGB color space to the HSV color space; The method for converting the original aerial image from the RGB color space to the HSV color space includes: Calculate the lightness through the maximum component values of the original aerial image in the R channel, G channel, and B channel; Calculate the saturation through the difference between the maximum component values of the original aerial image in the R channel, G channel, and B channel and the minimum component values of the original aerial image in the R channel, G channel, and B channel; Differentiate and calculate the hue according to the scores of the lightness and saturation; For each pixel in the HSV image, obtain its hue, saturation, and lightness values; judge the interval numbers to which the hue, saturation, and lightness values of each pixel belong, and splice the corresponding hue interval number, saturation interval number, and lightness interval number as the color feature.

8. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 6, characterized in that The method for obtaining the texture feature includes: Statistically analyze the frequency of occurrence of pixel pairs with gray values of X and Y respectively in the neighborhood with a size of C×C in the grayscale aerial image. C is the neighborhood size, and use the frequency as the element T(X,Y) of the gray-level co-occurrence matrix T with a size of H×H; H is the number of rows and columns of the gray-level co-occurrence matrix T; Select a pixel as the center point, obtain the corresponding neighborhood with a size of C×C, and traverse the pixel pairs within the corresponding neighborhood with a size of C×C according to the preset spatial position relationship; For each pair of adjacent pixels, obtain the corresponding gray values X and Y, and increment the corresponding element T(X,Y) in the gray-level co-occurrence matrix T by 1; Divide all elements of the gray-level co-occurrence matrix T by the total number of pixel pairs that conform to the spatial position relationship within the neighborhood to obtain the normalized gray-level co-occurrence matrix P(X,Y); Calculate the contrast, correlation, energy, and entropy based on the gray-level co-occurrence matrix P(X,Y) respectively; Concatenate the contrast, correlation, energy, and entropy as the texture features.

9. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, characterized in that The method for dynamically predicting the fire path based on the path prediction model includes: Initialize the path prediction model parameters, substitute the aligned feature vector and the initial dynamic time step into the path prediction model to predict the three-dimensional coordinate change of the fire spread boundary. Continuously collect new image data, new sensor data, and new meteorological data at the fire scene to update the aligned feature vector, obtain a new aligned feature vector, combine the dynamically updated time step by the reinforcement learning mechanism, and perform the next round of fire spread path prediction through the path prediction model; repeat the above process of updating the aligned feature vector and the dynamic time step, and continuously iterate to predict the fire spread path.

10. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, characterized in that, The method for obtaining the fire spread rate includes: Pre-select two monitored points, the positions of the two monitored points are x1 and x2 respectively. Measure the temperature of the monitored points through an infrared detection device, and compare the temperature of the monitored points with a preset temperature threshold. When the temperature of the monitored point reaches the preset temperature threshold, it means that the monitored point is in a burning state. Statistically record the times τ1 and τ2 when the temperatures of the two monitored points with positions x1 and x2 reach the preset temperature threshold, and obtain the fire spread rate V based on the ratio of the change in the position of the monitored point to the change in time of the burning state.

11. The dynamic prediction method for fire spread path based on artificial intelligence according to claim 1, characterized in that The training method of the first extraction model includes: Pre-collect A groups of image training data, and the image training data includes image data and the corresponding image features of the image data; Use each group of image training data as the input of the first extraction model. The first extraction model outputs the corresponding image features of each group of image data, and uses the actual image features corresponding to each group of image data as the prediction target; use minimizing the sum of the prediction errors of all image features as the training target; train the first extraction model until the sum of the prediction errors converges and then stop training; the first extraction model is a deep neural network model; the loss function value of the first extraction model is the mean square error.

12. An artificial intelligence-based dynamic fire spread path prediction system for implementing the artificial intelligence-based dynamic fire spread path prediction method according to any one of claims 1-11, characterized in that, It includes: Data acquisition module: used to collect image data, sensor data, meteorological data, fire spread rate, and the area of the burned area at the fire scene; First processing module: extract features from the image data based on the first extraction model to obtain image features; Second extraction module: extract features from the sensor data based on the second extraction model to obtain environmental features; Third extraction module: extract features from the meteorological data based on the third extraction model to obtain meteorological features; Feature alignment module: used to calculate the correlation weights between different modality features, and perform weighted fusion on different modality features based on the correlation weights to obtain the aligned feature vectors; different modality features include image features, environmental features, and meteorological features; Step adjustment module: used to take the aligned feature vectors as the input of the reinforcement learning mechanism, dynamically adjust the prediction time step of the path prediction model based on the reinforcement learning mechanism, and obtain the dynamic time step; Path prediction module: perform dynamic prediction of the fire path based on the dynamic time step using the path prediction model.

Images