Long-term fire heat release rate prediction method based on fire scene image recognition

Abstract

The invention relates to the technical field of fire science and computer vision, in particular to a long-term fire heat release rate prediction method based on fire scene image recognition. The method comprises the following steps: selecting a plurality of groups of fire video data from an NIST fire calorimetric database, preprocessing the fire video data, and extracting key image frames from image frames corresponding to the selected fire video data by adopting a dense frame extraction mode, the HRR prediction method comprises the following steps: extracting a plurality of key image frames, marking the heat release rate of the corresponding key image frames, establishing a deep learning model based on Bi-LSTM and Attention mechanisms, inputting the plurality of extracted key image frames into the deep learning model, extracting spatial-temporal characteristics by the deep learning model, and generating an HRR prediction value corresponding to a future fire image. According to the method, the time sequence features and the spatial features of the fire scene image sequence are extracted, and the attention mechanism and the backbone architecture of the bidirectional long-short-term memory network are combined, so that the high-precision prediction of the long-time history and the long-time-history heat release rate of the fire can be realized.

Description

Technical Field

[0001] This invention relates to the fields of fire science and computer vision technology, and more specifically, to a method for predicting long-term fire heat release rates based on fire scene image recognition. Background Technology

[0002] With the continuous development of society, the frequency of fire accidents is increasing year by year, especially in high-risk environments such as industrial and urban buildings. Fires not only pose a serious threat to human life and property, but may also lead to environmental pollution and ecological damage. The hazard of a fire is closely related to the heat release rate (HRR), which is an important parameter for assessing fire severity, predicting fire spread, and designing fire emergency response strategies. Therefore, accurately predicting the heat release rate of a fire is of great significance for fire prevention and control, emergency response, and post-disaster assessment.

[0003] The fire development process goes through several key stages: Incipient stage, the HRR gradually increases from the ignition point; Fire Growth stage, during which the fire spreads rapidly in the enclosed space, and the HRR rises sharply; Fully Developed stage, the HRR reaches its peak and remains high, marking the strongest stage of the fire; Decay stage, the HRR gradually decreases, and the fire weakens until it is extinguished.

[0004] In many fire tests or real-world fire scenarios, closed-circuit television (CCTV) cameras and mobile device cameras are commonly used to acquire fire videos, record changes in flames and smoke, and measure relevant fire parameters. The fire image frames extracted from these videos contain information about the behavior and characteristics of the fire, such as the size, color, brightness, and oscillation frequency of flames and smoke, as well as their changes over time. In-depth analysis of fire scene images can reveal crucial information about the fire's development.

[0005] In recent years, with the rapid advancements in computer vision and image recognition technologies, especially the widespread application of deep learning models, fire parameter prediction based on video images has gradually become a promising research direction. Fire scene images contain rich information about flames, smoke, and thermal radiation, which is of great value for accurately determining the stage, intensity, and spread trend of a fire. Summary of the Invention

[0006] The purpose of this invention is to provide a method for predicting long-term fire heat release rate based on fire scene image recognition, so as to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, a method for predicting long-term fire heat release rates based on fire scene image recognition is provided, comprising the following steps:

[0008] S1. Select multiple sets of fire video data from the NIST fire calorimetry database and preprocess the fire video data.

[0009] S2. The key image frames corresponding to the selected fire video data are extracted by dense frame extraction, and the heat release rate is labeled on the corresponding key image frames to form HRR labels. Multiple pairs of fire scene images and HRR labels are obtained, and the obtained data are used to build a training dataset.

[0010] S3. A deep learning model is built based on Bi-LSTM and Attention mechanism to capture the temporal dependence of key image frames and the relationship between HRR labels;

[0011] S4. Input the extracted key image frames into the deep learning model. The deep learning model extracts spatiotemporal features and generates HRR prediction values ​​corresponding to future fire images.

[0012] S5. The deep learning model is trained in multiple rounds, and its generalization ability is tested through various fire scenarios to verify its high prediction accuracy and stability in high-brightness scenarios and complex combustible material conditions. After verification, the model is trained and optimized.

[0013] As a further improvement to this technical solution, the step of preprocessing the fire video data in S1 is as follows:

[0014] ① Standardize video format;

[0015] ② Decompose the video frames and detect and remove abnormal frames;

[0016] ③ Preprocess the image frames of the fire scene;

[0017] ④ Enhancement and classification of fire scene image frames;

[0018] ⑤ Data storage and management.

[0019] As a further improvement to this technical solution, the step of forming the HRR tag in S2 is as follows:

[0020] ① Synchronous acquisition: Select fire video data containing HRR curves from the NIST database;

[0021] ② Timestamp alignment: The video frames and HRR data are calibrated to use seconds as the unit of timestamp.

[0022] ③ Label generation: Automatically assign the corresponding HRR value as a regression label to each image frame;

[0023] ④ Data format storage: Write the images and labels to a CSV file or a JSON file.

[0024] As a further improvement to this technical solution, the step of establishing the deep learning model in S3 is as follows:

[0025] ① Model architecture design:

[0026] An Att-BiLSTM model based on a bidirectional long short-term memory network and an attention mechanism is used to capture the temporal dependence and complex nonlinear features in fire scene image sequences.

[0027] ② Input and Output Design:

[0028] The processed sequence of fire scene images is input, and the image frames are converted into multidimensional tensors and then linearly transformed to fit the input layer of the model. In addition, HRR labels are used as supervision signals to guide the model to learn the mapping relationship between image sequences and HRR values.

[0029] Output data: The model output is the predicted HRR value for a future time period;

[0030] ③ Design of the key layer:

[0031] The key layer consists of an upper path and a lower path. The upper path takes the HRR label as input and guides the network to learn the key features of HRR during training. The lower path is responsible for processing the pre-processed fire scene image sequence. The outputs of the upper and lower paths are then stitched together in the Connect layer.

[0032] ④ Activation functions and regularization strategies.

[0033] As a further improvement to this technical solution, the step of concatenating the upper and lower paths at the Connect layer after output is as follows:

[0034] (1) HRR label information of the upper path: heat release rate label information of the upper path output;

[0035] (2) Image sequence features of the next path: The output features are usually a multi-dimensional vector;

[0036] (3) Concatenation operation: Concatenate the HRR label information with the image sequence features along the feature dimension;

[0037] (4) Generate multidimensional feature vectors: The spliced ​​feature vectors will contain image spatial features, temporal features and heat release rate label information, forming a new feature vector.

[0038] As a further improvement to this technical solution, the timing steps for capturing key image frames in S3 are as follows:

[0039] The bidirectional long short-term memory network is a network structure composed of forward LSTM and backward LSTM;

[0040] The steps of forward LSTM capturing the forward temporal dependent features of image frames:

[0041] ① Input Construction: Arrange the fire scene image sequence in chronological order as a three-dimensional tensor input;

[0042] ② Feature extraction: Each image frame can first be processed by a network such as CNN to extract spatial features and convert them into a two-dimensional feature vector sequence;

[0043] ③ Forward LSTM input: The feature vector sequence is input into the LSTM unit in chronological order. The LSTM receives the current input frame by frame and updates it by combining the hidden state of the previous frame.

[0044] ④ State update and memory transfer: The feedforward LSTM uses a gating mechanism to determine which historical information to retain;

[0045] ⑤ Output hidden state sequence: The output hidden state at each step serves as the "temporal context enhancement feature" of that frame, representing the semantic position and dependencies of the image in the forward temporal flow at that moment;

[0046] The steps of backward LSTM to capture inverse time-dependent features are as follows:

[0047] ① Input preparation: Sort the image sequence in reverse chronological order and construct a temporally reversed tensor;

[0048] ② Spatial feature extraction: Spatial features of each image frame can be extracted first through CNN to form a reversed feature sequence;

[0049] ③ Reverse LSTM input processing: Input the reversed features into the LSTM unit in sequence, and the LSTM calculates the hidden state step by step from time T to time 1;

[0050] ④ Gating mechanism for modeling future information: LSTM uses the hidden states of the current frame and future frames to extract the inverse time dependency information of fire information through forget gate, input gate, and output gate mechanisms;

[0051] ⑤ Output hidden state sequence: Reverse the output hidden state sequence back to its original time order so that it corresponds to the "future-aware features" at each original time point of the frame.

[0052] As a further improvement to this technical solution, the step in S5 of testing the generalization ability of the model through multiple fire scenarios is as follows:

[0053] ① Input preparation: The original image sequence collected at the fire scene is preprocessed; the processed image sequence is converted into a multidimensional tensor input model, and then input into the model batch by batch for processing. After each batch is input into the deep learning model, the output is calculated step by step.

[0054] ② Feature extraction: The fire scene image sequence is first processed by the Bi-LSTM layer. The forward LSTM captures the positive temporal dependency features of the image frames; the backward LSTM captures the negative temporal dependency features. The two are combined to form complete temporal context information; key features are weighted; the features output by Bi-LSTM are weighted through the Attention layer; the Attention mechanism calculates the importance of each image frame in the prediction task and assigns higher weights to highly relevant frames.

[0055] ③ Optimization process:

[0056] (1) Forward propagation: The sequence of fire scene images is used as input and passes through the Bi-LSTM layer, Attention layer and fully connected layer of the model in sequence to generate future transient or long-term HRR prediction values.

[0057] (2) Loss calculation and gradient backpropagation: Calculate the loss between the predicted value and the actual HRR value, and update the model weights through the backpropagation algorithm;

[0058] ④ Training result analysis: Plot the curves of training and validation loss as a function of rounds, and analyze the convergence of the model and the risk of overfitting.

[0059] As a further improvement to this technical solution, the step of weight allocation of the features output by the Bi-LSTM through the Attention layer is as follows:

[0060] (1) Input features: The temporal features output by Bi-LSTM are used as the input of Attention;

[0061] (2) Calculate attention score: evaluate the relevance of each image frame to the prediction task through a scoring function;

[0062] (3) Normalized score: The attention score is converted into probability weights using the Softmax function;

[0063] (4) Weighted summation: The features of each frame are weighted according to the attention weights to form a global feature representation.

[0064] As a further improvement to this technical solution, the Attention mechanism calculates the importance of each image frame in the prediction task as follows:

[0065] (1) Input feature preparation: Obtain the temporal feature sequence from the feature extraction module, where each element is the feature vector of the corresponding image frame;

[0066] (2) Calculate Attention weights: Use the Attention mechanism to calculate the importance of each frame in the prediction task, calculate the score by the similarity between the query and the key, and then normalize the score to obtain the weight of each frame.

[0067] (3) Weighted summation: The input features are weighted and summed using the obtained weights to obtain the weighted feature representation of each frame;

[0068] (4) Generate final prediction: Input the weighted features into the subsequent prediction module to generate the final prediction result.

[0069] As a further improvement to this technical solution, the training and optimization steps of the model are as follows:

[0070] ① Data splitting: During the dataset construction phase, the total dataset is divided into training set and validation set according to the proportion. The training set is used to update the model parameters, and the validation set is used to evaluate the model's performance on unseen data.

[0071] ② Definition of loss function: Mean squared error is one of the most commonly used loss functions in regression models, used to measure the difference between predicted and actual values.

[0072] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0073] 1. This method for predicting long-term fire heat release rate based on fire image recognition extracts the temporal and spatial features of fire image sequences and combines an attention mechanism and a bidirectional long short-term memory network backbone architecture to achieve high-precision prediction of long-term fire heat release rate. Using fire video image sequences as input, through multi-step data processing, network training and optimization, a time series prediction model for fire heat release rate has been successfully established. This model can not only provide real-time early warning support for fire rescue, but also provide an important reference for fire combustion dynamics research.

[0074] 2. In this method for predicting long-term fire heat release rate based on fire image recognition, a bidirectional long short-term memory network (Bi-LSTM) in deep learning is used in combination with an attention mechanism. This method can capture long-term dependencies in fire image sequences and accurately extract key features, thereby improving the prediction accuracy of fire heat release rate. Attached Figure Description

[0075] Figure 1 This is a flowchart illustrating the overall steps of the present invention;

[0076] Figure 2 This is a schematic diagram of the fire scene video image frame extraction process of the present invention;

[0077] Figure 3 This is a schematic diagram of the dataset and its preprocessing procedure of the present invention;

[0078] Figure 4 This is a diagram of the deep learning model architecture of the present invention;

[0079] Figure 5 A model evaluation diagram using the NIST database for the burning of cardboard boxes according to the present invention;

[0080] Figure 6 A model evaluation diagram using the NIST database during the combustion of the trash can according to the present invention;

[0081] Figure 7 A model evaluation diagram using the NIST database during the combustion of the plastic chair of the present invention;

[0082] Figure 8 The NIST database was used to evaluate the model during combustion of the box-type gas burner of the present invention.

[0083] Figure 9 A model evaluation diagram using the NIST database was generated during the combustion of the work vehicle with a laptop and printer according to the present invention.

[0084] Figure 10 The diagram shows the model evaluation using the NIST database for the combustion of propanol liquid according to the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0086] Example 1

[0087] Please see Figures 1-10 As shown, the purpose of this embodiment is to provide a method for predicting long-term fire heat release rate based on fire scene image recognition, including the following steps:

[0088] S1. Select multiple sets of fire video data from the NIST fire calorimetry database and preprocess the fire video data.

[0089] The NIST Fire Calorimetry Database utilizes a public database accumulated over a long period by the National Institute of Standards and Technology (NIST). This proposal selects 40 fire experiment videos from the NIST database. These fire experiments have peak heat release rates (PHRR) ranging from 10.5 kW to 4174 kW and total heat release ranging from 0 to 5120 MJ, covering a variety of possible scenarios from small to large fires, demonstrating broad applicability and representativeness. The combustion materials used in some of the combustion tests with different ignition sources are mostly common solid fuels found in daily life, such as wood, paper, and chemicals. Using common solid fuels in daily life can fully reflect the common combustion characteristics in actual fires.

[0090] The fire experiments ranged in duration from 4 minutes to 115 minutes, covering the entire process from ignition to complete combustion, ensuring the integrity and diversity of the experimental data.

[0091] Preprocessing of fire video data is a crucial step in the entire research process. Its goal is to extract high-quality feature information from the raw fire scene video, providing a reliable foundation for subsequent dataset construction and model training.

[0092] The steps for preprocessing fire scene video data are as follows:

[0093] ① Standardize video format;

[0094] Since the fire videos in the NIST fire calorimetry database may come from different experimental equipment and have inconsistent formats and resolutions, it is necessary to unify the video formats. The H.264 encoding format and standard resolution (1920×1080) are used for unified processing to ensure the consistency and compatibility of subsequent data processing.

[0095] ② Decompose the video frames and detect and remove abnormal frames;

[0096] Fire videos are decomposed into independent image frames. Abnormal frames in the video are detected by indicators such as image sharpness, average brightness, and noise level. Frames that are too dark, too bright, or blurry due to equipment shake are defined as abnormal frames. These abnormal frames will affect the model's judgment of the fire scene and therefore need to be removed.

[0097] To ensure the quality of the training data, we determine whether an image frame is "too dark" or "too bright" based on the histogram mean of the image brightness (the average grayscale value of the image). The specific thresholds are as follows:

[0098] Too dark (image is too black): The grayscale mean of the image is less than 30. In such images, the flame is not obvious or the background is almost completely black, and it cannot provide effective heat release characteristics.

[0099] The image is too bright (too white), with a grayscale average of 225 or higher. The image may be overexposed, have light source interference, or have severely lost information about the fire area.

[0100] For abnormally blurry frames (such as those caused by device jitter), the Laplacian variance algorithm is used to determine image sharpness: if the Laplacian variance is less than 100, it is considered a blurry frame. Such images lack edge information and will seriously interfere with the model's extraction of fire contours and dynamic features.

[0101] ③ The image frames of the fire scene are preprocessed to improve data quality and consistency of model training, including:

[0102] (1) Clean and denoise the image frames of the fire scene, and retain key features such as flames and smoke;

[0103] (2) Use image cropping to unify the cropped image area: Adjust the resolution of the fire scene image to reduce computational overhead and background noise. The fire1 dataset adjusts the image to 64×64 pixels to extract key information such as the temporal changes of flames and smoke distribution. This cropping strategy not only reduces computational cost, but also effectively reduces the interference of background noise on the model, thereby significantly improving the accuracy and robustness of the model prediction.

[0104] (3) Random horizontal flipping: The fire scene image is horizontally flipped with a probability of 0.5 to increase data diversity and enhance the model's ability to adapt to changes in the direction of the fire scene image.

[0105] Assume the input image matrix is ​​I, with dimensions H×W×c, where H is the image height, W is the width, c is the number of channels, and (i,j) is the image position. The horizontal flip operation can be represented as:

[0106] I flipped (i,j,c)=I(i,W-j+1,c)

[0107] Where I(i,j,c) represents the pixel value at position (i,j) in channel c of the image; I flipped (i,j,c) are the pixel values ​​at the corresponding positions in the flipped image;

[0108] For each image, the flipping condition I output It can be represented as:

[0109]

[0110] p is the probability of flipping, set to 0.5. This technique enables the model to learn the features of fire scene images at different angles, thereby reducing the model's sensitivity to image orientation and improving its adaptability to diverse fire scenarios.

[0111] (4) Pixel value normalization: Normalize the image pixel values ​​from the range of [0,255] to the range of [0,1] to ensure the stability of the input data distribution and improve the training speed and accuracy of the model;

[0112] Assume the image pixel value matrix is ​​Y, and its pixel value range is [Y]. min ,Y max The normalization operation can be achieved using the following formula:

[0113]

[0114] Where Y(i,j) represents the pixel value of the image at position (i,j); Y normalized (i,j) are the normalized pixel values; Y min =0,Y max =255, which can prevent the difference in pixel values ​​from being too large and reduce data bias. This normalization process can effectively prevent the model training from being unstable or overfitting due to the large difference in pixel values, while speeding up the convergence speed of the model.

[0115] Finally, the preprocessed fire images were further divided into a training set (80%) and a validation set (20%) for training and validation of the deep learning model. To ensure the broad applicability of the research results and avoid potential data bias, both sets of data included fire experimental images with different fuel types and heat release rates (HRR) ranges.

[0116] Preprocessing image frames through cleaning, cropping, normalization, and random flipping significantly improves data quality and consistency, providing a clear and representative data foundation for the model. It also enhances the model's robustness through data augmentation techniques, increasing its adaptability to different transformations, thereby reducing overfitting and improving generalization performance. Furthermore, preprocessing accelerates model training, avoids instability caused by uneven data distribution, and helps the model extract key features more efficiently, thus improving prediction accuracy.

[0117] ④ Enhancement and classification of fire scene image frames:

[0118] Flame and smoke features are enhanced by adjusting contrast and sharpening, making fire scene images clearer. Background noise that may obscure key features is processed using denoising algorithms to reduce the impact of background interference on the model. During classification, images are categorized according to the experimental scenario and type of combustible material, such as ordinary laboratory environment, extreme temperature environment, ventilated environment, and enclosed environment, as well as different combustible materials (such as wood, paper, chemicals, etc.).

[0119] ⑤ Data storage and management:

[0120] The processed image frames are grouped according to the experiment number, and video frames with the same scene and the same burning material are stored in a unified database; a hierarchical folder structure and naming rules are used to manage the data to ensure efficient retrieval and retrieval.

[0121] The experiments are numbered and grouped logically based on fire type, scenario, and experimental conditions, as follows:

[0122] Numbering basis:

[0123] Each experimental video is accompanied by an experimental number, such as: Exp01, Exp02...;

[0124] The number represents a specific experimental scenario, including information such as the type of fire source, the type of combustible material, and ventilation conditions;

[0125] Grouping principle:

[0126] Archived according to the experiment number, the processed image frames are placed in separate subfolders according to their respective experiment numbers, that is, the image frames in the Exp01 experiment are placed together;

[0127] At the same time, the processed image frames are constructed into two subsets to form high-resolution and low-resolution subsets;

[0128] High resolution (e.g., 50×108): used to capture local details, serving transient HRR prediction;

[0129] Low resolution (e.g., 64×64): used for modeling macroscopic time series features, serving long-term forecasting;

[0130] S2. The key image frames corresponding to the selected fire video data are extracted by dense frame extraction, and the heat release rate is labeled on the corresponding key image frames to form HRR labels. During labeling, the key image frames are paired with the corresponding heat release rates one by one to form image-HRR training samples with a unified structure. Each image frame is accurately aligned with the HRR label corresponding to that moment. Multiple pairs of fire scene images and HRR labels are obtained, and the obtained data are used to build a training dataset.

[0131] To ensure the integrity of the temporal relationship between frames, this scheme adopts a dense sampling strategy of 30 frames per second (FPS) to extract continuous image frames from the image frames. This can capture the dynamic changes in the fire process without missing important feature information, ensuring that the dataset has high temporal resolution and the integrity required for scientific research.

[0132] To achieve a one-to-one correspondence between images and heat release rate (HRR) data, the following method is used to generate HRR labels during the data construction process:

[0133] ① Synchronous acquisition: Select fire video data containing HRR curves from the NIST database to ensure that the video frames (30 frames per second) are consistent with the time points of the HRR experiment recording;

[0134] ② Timestamp alignment: Video frames and HRR data are calibrated with timestamps in seconds, ensuring that each image frame is precisely matched to the HRR value corresponding to its time.

[0135] ③ Label generation: Automatically assign the corresponding HRR value as a regression label to each image frame;

[0136] ④ Data format storage: Write the images and labels into a CSV or JSON file for use in training the model.

[0137] The heat release rate (HRR) data were derived from sensor measurements in the raw experimental data of the NIST Fire Calorimetry Database. These values ​​were obtained using oxygen consumption calorimetry. Fire image data and corresponding HRR data were acquired from the fire experimental database, covering various fire scenarios and fuel types, including solid, liquid, and gaseous fuels. Keyframes were extracted from fire videos at a frame rate of 30 FPS using dense frame extraction, and each image frame was labeled with its corresponding HRR value. This resulted in the construction of multiple sets of high-quality training datasets paired with HRR annotations.

[0138] This dataset contains 27,231 pairs of fire image-HRR pairs. Each pair consists of a high-resolution fire image (1920×1080 pixels) and its corresponding HRR value at a given time point. The database comprehensively covers the entire process of a fire, from its initial ignition stage and peak combustion to its complete burnout, providing rich information on fire scenes and combustion stages.

[0139] S3. A deep learning model is built based on Bi-LSTM and Attention mechanism to capture the temporal dependence of key image frames and the relationship between HRR labels;

[0140] Bi-LSTM (Bidirectional Long Short-Term Memory) is a network structure composed of a forward LSTM and a backward LSTM. This structure can simultaneously consider past and future information in sequence data, thus more comprehensively capturing the contextual relationships and long-distance dependencies of the data. It can not only utilize the forward information of the sequence (past frames) but also learn information from future frames through the backward LSTM, which helps to capture the temporal information of fire evolution from the entire sequence. Using Bi-LSTM to capture the temporal dependencies in key fire image frames, that is, capturing the temporal sequence of changes between key fire image frames, such as the temporal changes of flame expansion and smoke spread.

[0141] The steps of forward LSTM capturing the forward temporal dependent features of image frames:

[0142] ① Input Construction: Arrange the fire scene image sequence in chronological order as a three-dimensional tensor input (e.g., shape T×H×W×C, where T is the number of frames);

[0143] ② Feature extraction: Each image frame can first be processed by a network such as CNN to extract spatial features, which are then converted into a two-dimensional feature vector sequence with a shape of T×D (D is the feature dimension);

[0144] ③ Forward LSTM input: The feature vector sequence is input into the LSTM unit in chronological order. The LSTM receives the current input frame by frame and updates it by combining the hidden state of the previous frame.

[0145] ④ State update and memory transfer: The feedforward LSTM uses a gating mechanism (input gate, forget gate, output gate) to determine which historical information to retain in order to capture trend features that change over time, such as "flame growth".

[0146] ⑤ Output hidden state sequence: The output hidden state at each step serves as the "temporal context enhancement feature" of that frame, representing the semantic position and dependencies of the image in the forward time flow at that moment.

[0147] The steps of backward LSTM to capture inverse time-dependent features are as follows:

[0148] ① Input preparation: Reverse the image sequence in time (from the last frame to the first frame) and construct a time-reverse tensor: the original sequence shape is [x1,x2,...,x_T], and the input becomes [x_T,...,x2,x1];

[0149] ② Spatial feature extraction: Spatial features of each image frame can be extracted first through CNN to form a reversed feature sequence;

[0150] ③ Reverse LSTM input processing: Input the reversed features into the LSTM unit in sequence, and the LSTM calculates the hidden state step by step from time T to time 1;

[0151] ④ Gating mechanism for modeling future information: LSTM uses the hidden states of the current frame and future frames to extract inverse time-dependent information of fire information such as "smoke diffusion" and "flame stabilization" through forget gate, input gate and output gate mechanisms;

[0152] ⑤ Output hidden state sequence: Reverse the output hidden state sequence back to its original time order so that it corresponds to the "future-aware features" at each original time point of the frame.

[0153] LSTM is a special type of recurrent neural network (RNN). LSTM introduces a structure called a "memory unit" to address the vanishing and exploding gradient problems during long training sequences. Each LSTM unit consists of an input gate i. t A gate of oblivion t An output gate o t A candidate unit state A cell state c t and a hidden state h t Composition, such as Figure 4 As shown, i t This determines whether the current input information is written to c. t , and f t The decision was made by c. t Has the information in the text been forgotten? t The decision of whether or not information is output from memory cells is calculated as follows:

[0154]

[0155]

[0156] h t =o t ×tanh(c t )

[0157] Where σ is the sigmoid function. These are concatenation operators; + and × represent element-wise addition and multiplication, respectively. W (x) and b (x) Let X represent the weight matrix and bias vector of gate X, respectively.

[0158] The Attention mechanism extracts key features from critical fire image frames. It assigns different weights to each time step (critical image frame), ensuring that the model can focus on the most predictive image frames when predicting HRR. For example, the flame spread or smoke changes in the early stage of a fire will be given higher weights, and the model will focus on these critical moments.

[0159] Temporal dependencies and the source of features

[0160] The capture of temporal dependence and key image frame features is based on the visual features of the key fire image frames themselves and their corresponding heat release rate data;

[0161] Specifically: temporal dependence comes from the dynamic evolution between consecutive frames in an image sequence; for example, the expansion of a flame from one point to another, the flow of smoke, etc., are all temporal information that cannot be ignored in an image sequence; key features come from the changing features in fire images, such as changes in information such as flames, smoke, and temperature, which have a direct impact on HRR prediction.

[0162] By analyzing these time-dependent and key features, the model can more accurately predict the development trend of a fire, thus providing a scientific basis for fire prevention and control. For example, the model can predict the possible paths of fire spread based on changes in flame spread rate and smoke concentration, helping firefighters to develop more effective firefighting strategies.

[0163] The steps to build a deep learning model are as follows:

[0164] ① Model architecture design:

[0165] An Att-BiLSTM model based on a bidirectional long short-term memory network (Bi-LSTM) and an attention mechanism is used to capture the temporal dependence and complex nonlinear features in fire scene image sequences.

[0166] The Bi-LSTM module can simultaneously capture both positive and negative dependencies in time series, making it suitable for processing long-term dependency features in fire image sequences. Through this module, dynamic change information of the fire process can be extracted, such as flame growth and smoke diffusion.

[0167] Attention mechanism: An Attention module is added to the output of Bi-LSTM to assign weights to fire image frames at different time points, focusing on the frames that contribute the most to HRR prediction, enhancing the model's sensitivity to key temporal features, and further improving prediction accuracy.

[0168] ② Input and output design;

[0169] The input consists of a sequence of 45 consecutive images of the fire scene. The images are converted into a 1×45×64×64 multidimensional tensor and then linearly transformed to fit the input layer of the model. In addition, HRR labels are used as supervision signals to guide the model in learning the mapping relationship between the image sequence and HRR values.

[0170] Output data: The model output is the predicted HRR values ​​for the next 45 frames;

[0171] ③ Design of the key layer:

[0172] refer to Figure 4 The key layer consists of an upper path and a lower path. The upper path takes HRR labels as input to provide supervision signals to the model and guide the network to learn the key features of HRR during training. The upper path contains three hidden layers, each with 128, 256, and 256 units respectively. The input dimension is 1×45, and after linear transformation and Dropout layer processing, the output dimension is 1×240.

[0173] The next path is responsible for processing the preprocessed fire scene image sequence. The input dimension is 1×45×64×64. This path consists of 3 Bi-LSTM layers and 1 Attention layer. The dimension is further reduced to 1×16 through linear activation and Dropout layers.

[0174] After the upper and lower paths are output, they are concatenated in the Connect layer to form a 256-dimensional vector, which integrates image features and HRR signal information to enhance the model's ability to understand complex fire scenarios. The concatenated result is further reduced to 45 dimensions through a fully connected layer (FC layer) and a Dropout layer, which outputs the predicted future long-term HRR value.

[0175] The steps for concatenating the HRR label information of the upper path and the image sequence features of the lower path at the Connect layer are as follows:

[0176] (1) HRR label information of the upper path: The upper path outputs HRR (heat release rate) label information, which is usually a scalar or vector representing a certain time-series characteristic or target value of the fire. The shape can be [T] or [1] (depending on the task settings).

[0177] (2) Image sequence features of the next path: The next path extracts the spatial and temporal features of the image sequence through feature extraction modules such as convolutional neural network (CNN) or LSTM. The output features are usually a multi-dimensional vector with the shape

[256] (i.e., 256-dimensional feature vector).

[0178] (3) Concatenation operation: The HRR label information and the image sequence features are concatenated in the feature dimension. Usually, the concatenation operation is to merge the two vectors by aligning them in the dimension. If the HRR label information is a scalar (such as a value), it will be expanded to the same dimension as the image sequence features through the broadcast mechanism; if the HRR label information is a vector, its dimension will be directly aligned with the image sequence features.

[0179] (4) Generate a 256-dimensional feature vector: The concatenated feature vector will contain image spatial features, temporal features and heat release rate label information, forming a new feature vector with a final dimension of 256 (if the original features already meet the 256-dimensional requirement).

[0180] ④ Activation functions and regularization strategies: ReLU, Tanh and Softmax activation functions are used, which are suitable for different levels of data flow to ensure nonlinear expressiveness and stability; multiple Dropout layers (with a parameter of 0.05) are introduced into the network structure to prevent overfitting and enhance the model's generalization ability.

[0181] S4. Input the extracted key image frames into the deep learning model. The deep learning model extracts spatiotemporal features and generates HRR prediction values ​​corresponding to future fire images.

[0182] S5. Train the deep learning model in multiple rounds and test its generalization ability through various fire scenarios to verify its high prediction accuracy and stability in high-brightness scenarios and complex combustible conditions. After verification, train and optimize the deep learning model to ensure the reliability of practical applications.

[0183] To ensure high prediction accuracy and stability of the model in practical applications, we took the following steps to verify the capabilities of the fire scenario prediction model:

[0184] Multi-scenario simulation:

[0185] The robustness of the deep learning model was verified by simulating a variety of different fire scenarios, including high-brightness scenarios and complex background scenarios, which have different lighting conditions, smoke density and flame spread patterns.

[0186] The simulation scenarios include ordinary laboratory environments, extreme temperature environments, ventilated environments, and enclosed environments, ensuring the model's adaptability to various real fire environments;

[0187] Combustion condition validation: Validation was conducted for different combustibles (such as wood, paper, chemicals, etc.). These materials have significant differences in heat release rate (HRR) changes during fire. Therefore, the model needs to be able to adapt to different combustible characteristics. The prediction accuracy of the model was tested through actual fire test data of multiple combustibles to ensure its ability to respond to a variety of combustibles in practical applications.

[0188] High-precision HRR prediction: Acquire high-precision heat release rate data in a simulated environment and compare it with the model prediction results to verify the accuracy of the model. By comparing the error between the actual HRR and the predicted HRR, the high prediction accuracy of the deep learning model is ensured.

[0189] Long-term stability verification: In long-term fire scenario simulations, the model is tested to see if it can maintain stable prediction performance in different stages (such as the initial stage, expansion stage, and stabilization stage of a fire). By splitting the training dataset and the test dataset, the model is verified to maintain consistent prediction results in multiple runs.

[0190] In the prediction stage of HRR prediction values ​​corresponding to future fire images, the core task of the deep learning model is to generate future long-term heat release rate (HRR) prediction values ​​based on the input sequence of key image frames of the fire scene.

[0191] The steps for testing the generalization ability of a model through various fire scenarios include the following five key steps, as detailed below:

[0192] ① Input preparation: The original image sequence collected at the fire scene is preprocessed; the processed image sequence is converted into a multidimensional tensor input model; in order to optimize computational efficiency and hardware resource utilization, the data is grouped in a small batch (batch size) manner, so that the multi-frame fire scene image sequence is divided into multiple batches according to the set small batch (e.g., 8 images per batch), and input into the model batch by batch for parallel or sequential processing. After each batch is input into the deep learning model, the output is calculated step by step.

[0193] ② Feature Extraction: The fire scene image sequence is first processed by a Bi-LSTM layer. The forward LSTM captures the positive time-dependent features of the image frames, such as the flame growth trend; the backward LSTM captures the negative time-dependent features, such as the smoke dissipation pattern; the two are combined to form complete temporal context information; key features are weighted; the features output by Bi-LSTM are weighted through an Attention layer; the Attention mechanism calculates the importance of each image frame in the prediction task and assigns higher weights to highly relevant frames, thereby improving the expressive power of key features.

[0194] The steps for weighting the features output by Bi-LSTM through the Attention layer are as follows:

[0195] (1) Input features: The temporal features output by Bi-LSTM are used as the input of Attention;

[0196] (2) Calculate attention score: evaluate the relevance of each image frame to the prediction task using a scoring function (such as dot-product);

[0197] (3) Normalized score: The attention score is converted into probability weights using the Softmax function;

[0198] (4) Weighted summation: The features of each frame are weighted according to the attention weights to form a global feature representation.

[0199] The steps of the attention mechanism to calculate the importance of each image frame in the prediction task are as follows:

[0200] (1) Input feature preparation: Obtain the temporal feature sequence from the feature extraction module (such as Bi-LSTM), where each element is the feature vector of the corresponding image frame;

[0201] (2) Calculate Attention weights: Use the Attention mechanism to calculate the importance of each frame in the prediction task, calculate the score by the similarity between the query and the key, and then normalize the score to obtain the weight of each frame.

[0202] (3) Weighted summation: The input features are weighted and summed using the obtained weights to obtain the weighted feature representation of each frame;

[0203] (4) Generate final prediction: Input the weighted features into the subsequent prediction module (such as the fully connected layer) to generate the final prediction result.

[0204] The method for assigning higher weights: In the Attention mechanism, higher weights refer to those frames that have a greater impact on the final prediction task. This "higher" weight is usually a value obtained through Softmax normalization, representing the relative importance of each frame in the entire sequence. Generally speaking, the weight value of "higher" weight depends on the task requirements, but usually some values ​​are selected, such as the top 20% of frames with the highest weights, which will play a key role in the final prediction. The result of Softmax normalization is that the weight of each frame is in the range of [0,1], and the sum of all weights is 1. Generally speaking, a high weight value (e.g., greater than 0.5) indicates that the frame contributes more to the prediction task, while a low weight value (e.g., less than 0.1) indicates that the frame contributes less to the task.

[0205] ③ Optimization process:

[0206] (1) Forward propagation: The sequence of fire scene images is used as input and passes through the Bi-LSTM layer, Attention layer and fully connected layer of the model in sequence to generate future transient or long-term HRR prediction values.

[0207] (2) Loss calculation and gradient backpropagation: Calculate the loss between the predicted value and the actual HRR value, and update the model weights through the backpropagation algorithm;

[0208] ④ Training result analysis: Plot the curves of training and validation loss as a function of rounds, and analyze the convergence of the model and the risk of overfitting.

[0209] The generalization ability of the testing model is comprehensively examined through various evaluation metrics and testing strategies to assess the model's predictive power and applicability, specifically including the following aspects:

[0210] The coefficient of determination (R²) is an important evaluation metric for regression models, used to measure the degree to which the model explains the variance in the data. Its formula is:

[0211]

[0212] in, It is the average of the actual values. It is the residual sum of squares (the sum of squares of the differences between the predicted and actual values). R² is the sum of squares (the sum of squared differences between the actual values ​​and the mean). R² ranges from 0 to 1; a value closer to 1 indicates that the model better explains the variation in the data, while a value closer to 0 indicates that the model has no explanatory power. In some cases, R² can be negative, meaning that the model's predictive performance is worse than a simple mean prediction.

[0213] ① In high-brightness fire scenarios, brightness changes can affect the feature extraction and analysis capabilities of fire scene images. Therefore, this study selected three sets of high-brightness fire test cases, including burning scenarios of cardboard boxes, rubber trash cans, and plastic chairs, to verify the model's adaptability to high-brightness scenarios: comparing the actual HRR values ​​of high-brightness scenarios with the model's predicted values, and analyzing the coefficient of determination (R²). 2 ) and MSE values, the results show that R 2 The values ​​are all above 0.90, indicating that the model's prediction performance is stable under high brightness conditions; Fit verification: Scatter plots and line graphs of actual and predicted values ​​were plotted to ensure that the predicted trend is consistent with the actual trend, further illustrating the model's robustness to high brightness scenes.

[0214] ② Assessment of Complex Combustible Material Fire Scenarios: Real-world fires involve a wide variety of combustible materials, including solids, liquids, and gases, leading to significant differences in combustion characteristics. This study selected three complex combustible material fire cases from the NIST database for testing: a box-type gas burner, a work cart with a laptop and printer, and a propanol liquid fire. The predicted HRR values ​​for each scenario were compared with the actual values ​​to analyze the model's generalization ability under different combustion characteristics. The test results show that HRR... 2 The values ​​are all greater than 0.86, indicating that the model can adapt to fire scenarios with different fuel types. For rapid changes in the combustion process (such as flame bursts or smoke rise), the model can capture its dynamic characteristics, showing strong time-dependent modeling capabilities. Through the above detailed tests and evaluations, the reliability and applicability of the model in different fire scenarios are finally confirmed, providing comprehensive data support and technical assurance for practical applications.

[0215] To further validate the model's generalization ability, this study selected entirely new data outside the training set for testing. Specifically, multiple fire test cases with different heat release rate (HRR) ranges were selected from the NIST Fire Calorimetry Database. These cases were not included in the 20% validation set used during training, and therefore represented entirely new and unseen samples for the model. In this evaluation process, the model predicted the future long-term HRR of these unseen fire scenarios and related combustibles based on the knowledge learned during training. This method can comprehensively evaluate the model's performance on unknown data, reflecting its generalization ability and providing a scientific basis and reliable support for its application in practical fire prediction tasks.

[0216] ① Performance evaluation of models in high-brightness fire scenarios

[0217] Changes in brightness within a fire environment significantly impact the brightness and contrast of fire images, thereby affecting the model's performance in image feature extraction and analysis. Therefore, the brightness conditions of the fire environment are a crucial factor influencing the generalization ability of heat release rate (HRR) prediction models. To evaluate the model's predictive performance in high-brightness fire scenarios, this study selected three fire test cases with high brightness from the NIST fire calorimetry database. The burning items in the selected cases included cardboard boxes, rubber trash cans, and plastic chairs. The experimental results are as follows: Figures 5-7 As shown, even under high brightness conditions, the model can still accurately predict the heat release rate (HRR) of different combustibles, and the coefficient of determination (R²) of the test cases is also high. 2 The values ​​were all above 0.90, and the scatter plot and control plot showed a good fit, which fully proved the accuracy of the model prediction.

[0218] The comprehensive experimental results show that the proposed model has strong adaptability to changes in brightness in fire scenarios and can achieve prediction outputs that are highly consistent with the measured values ​​under high brightness conditions. This finding further verifies the model's generalization ability and provides reliable support for its practical fire prediction applications under different brightness environments, demonstrating its wide applicability in complex fire scenarios.

[0219] ② Performance verification of the model under complex combustible fire scenarios

[0220] The presence of complex combustibles significantly increases the difficulty of predicting the actual heat release rate (HRR) of a fire. These combustibles may include various forms such as solids, liquids, and gases, leading to significant differences in the color, shape, and other characteristics of flames and smoke. To evaluate the predictive reliability of the proposed model in fire scenarios involving complex combustibles, this study selected three typical fire scenarios from the NIST Fire Calorimetry Database for validation: a "box-type gas burner," a "work vehicle with a laptop and printer," and a "propanol liquid fire." These scenarios effectively simulate diverse combustible conditions in real fires, aiming to test the model's generalization ability in handling complex fire scenarios.

[0221] Experimental results are as follows Figures 8-10 As shown, the left column displays a scatter plot, and the right column displays the corresponding line graph, used to intuitively analyze the model's predictive performance in complex fire scenarios. By comparing the actual HRR of three fire scenarios with the prediction results of the deep learning model, it was found that despite the complex fuel load and fire propagation characteristics of the fire scenarios involved, the model can still reasonably predict the HRR. Experimental results show that the proposed model's coefficient of determination (R²) is high in all test scenarios. 2 All values ​​are above 0.86, which can accurately reflect the changing trend of HRR during combustion.

[0222] This result demonstrates that the designed deep learning model exhibits high adaptability and predictive ability in handling HRR prediction tasks in complex combustible fire scenarios, providing strong support for its application in diverse real-world fire scenarios.

[0223] Model training and optimization: Model training and optimization are core steps to ensure the performance of the prediction system. The aim is to enable the model to learn effective feature representations from fire scene image data and maximize its prediction accuracy and generalization ability, as detailed below:

[0224] ① Data Splitting: During the dataset construction phase, the total dataset is divided proportionally into a training set (80%) and a validation set (20%). The training set is used to update model parameters and help the model learn feature representations from the data. The validation set is used to evaluate the model's performance on unseen data to avoid overfitting.

[0225] ② Definition of Loss Function: Mean Squared Error (MSE) is one of the most commonly used loss functions in regression models. It is used to measure the difference between predicted and actual values. Its calculation formula is as follows:

[0226]

[0227] Among them, y i This is the actual value. denoted as the predicted value, and n as the number of samples. The advantage of MSE is that it has a high penalty for large errors because it squares the error. Large errors will significantly affect the value of MSE, so it can effectively guide the model to focus on large prediction biases during training.

[0228] In summary, this invention first performs necessary preprocessing operations (transforms layer) on the fire scene images to improve data quality and consistency. The preprocessed image sequence is used as input, and the spatiotemporal features of the image sequence are extracted sequentially through a Bi-LSTM layer. An attention mechanism (Attention layer) is introduced at the output of the Bi-LSTM to enhance the temporal information of the target and identify key features of the fire scene images at different time points, thereby improving the prediction accuracy of the future long-term heat release rate (HRR). At the same time, this method makes full use of the features and temporal information of the fire scene images, and can achieve accurate prediction of HRR during the fire combustion process without the need for additional sensors or equipment. This provides strong technical support for fire monitoring and early warning, and can provide more efficient and accurate technical means for fire emergency management, which will help promote the further development of intelligent fire protection systems.

[0229] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for predicting long-term fire heat release rate based on fire scene image recognition, characterized in that: Includes the following steps: S1. Select multiple sets of fire video data from the NIST fire calorimetry database and preprocess the fire video data. S2. The key image frames corresponding to the selected fire video data are extracted by dense frame extraction, and the heat release rate is labeled on the corresponding key image frames to form HRR labels. Multiple pairs of fire scene images and HRR labels are obtained, and the obtained data are used to build a training dataset. S3. A deep learning model is built based on Bi-LSTM and Attention mechanism to capture the temporal dependence of key image frames and the relationship between HRR labels; S4. Input the extracted key image frames into the deep learning model. The deep learning model extracts spatiotemporal features and generates HRR prediction values ​​corresponding to future fire images. S5. The deep learning model is trained in multiple rounds, and its generalization ability is tested through various fire scenarios to verify its high prediction accuracy and stability in high-brightness scenarios and complex combustible material conditions. After verification, the model is trained and optimized.

2. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The preprocessing steps for the fire video data in S1 are as follows: ① Standardize video format; ② Decompose the video frames and detect and remove abnormal frames; ③ Preprocess the image frames of the fire scene; ④ Enhancement and classification of fire scene image frames; ⑤ Data storage and management.

3. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The steps for forming the HRR tag in S2 are as follows: ① Synchronous acquisition: Select fire video data containing HRR curves from the NIST database; ② Timestamp alignment: The video frames and HRR data are calibrated to use seconds as the unit of timestamp. ③ Label generation: Automatically assign the corresponding HRR value as a regression label to each image frame; ④ Data format storage: Write the images and labels to a CSV file or a JSON file.

4. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The steps for establishing a deep learning model in S3 are as follows: ① Model architecture design: An Att-BiLSTM model based on a bidirectional long short-term memory network and an attention mechanism is used to capture the temporal dependence and complex nonlinear features in fire scene image sequences. ② Input and Output Design: The processed sequence of fire scene images is input, and the image frames are converted into multidimensional tensors and then linearly transformed to fit the input layer of the model. In addition, HRR labels are used as supervision signals to guide the model to learn the mapping relationship between image sequences and HRR values. Output data: The model output is the predicted HRR value for a future time period; ③ Design of the key layer: The key layer consists of an upper path and a lower path. The upper path takes the HRR label as input and guides the network to learn the key features of HRR during training. The lower path is responsible for processing the pre-processed fire scene image sequence. The outputs of the upper and lower paths are then stitched together in the Connect layer. ④ Activation functions and regularization strategies.

5. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 4, characterized in that: The steps for concatenating the outputs of the upper and lower paths in the Connect layer are as follows: (1) HRR label information of the upper path: heat release rate label information of the upper path output; (2) Image sequence features of the next path: The output features are usually a multi-dimensional vector; (3) Concatenation operation: Concatenate the HRR label information with the image sequence features along the feature dimension; (4) Generate multidimensional feature vectors: The spliced ​​feature vectors will contain image spatial features, temporal features and heat release rate label information, forming a new feature vector.

6. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The timing steps for capturing key image frames in S3 are as follows: The bidirectional long short-term memory network is a network structure composed of forward LSTM and backward LSTM; The steps of forward LSTM capturing the forward temporal dependent features of image frames: ① Input Construction: Arrange the fire scene image sequence in chronological order as a three-dimensional tensor input; ② Feature extraction: Each image frame can first be processed by a network such as CNN to extract spatial features and convert them into a two-dimensional feature vector sequence; ③ Forward LSTM input: The feature vector sequence is input into the LSTM unit in chronological order. The LSTM receives the current input frame by frame and updates it by combining the hidden state of the previous frame. ④ State update and memory transfer: The feedforward LSTM uses a gating mechanism to determine which historical information to retain; ⑤ Output hidden state sequence: The output hidden state at each step serves as the "temporal context enhancement feature" of that frame, representing the semantic position and dependencies of the image in the forward temporal flow at that moment; The steps of backward LSTM to capture inverse time-dependent features are as follows: ① Input preparation: Sort the image sequence in reverse chronological order and construct a temporally reversed tensor; ② Spatial feature extraction: Spatial features of each image frame can be extracted first through CNN to form a reversed feature sequence; ③ Reverse LSTM input processing: Input the reversed features into the LSTM unit in sequence, and the LSTM calculates the hidden state step by step from time T to time 1; ④ Gating mechanism for modeling future information: LSTM uses the hidden states of the current frame and future frames to extract the inverse time dependency information of fire information through forget gate, input gate, and output gate mechanisms; ⑤ Output hidden state sequence: Reverse the output hidden state sequence back to its original time order so that it corresponds to the "future-aware features" at each original time point of the frame.

7. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The steps in S5 for testing the generalization ability of the model through various fire scenarios are as follows: ① Input preparation: The original image sequence collected at the fire scene is preprocessed; the processed image sequence is converted into a multidimensional tensor input model, and then input into the model batch by batch for processing. After each batch is input into the deep learning model, the output is calculated step by step. ② Feature extraction: The fire scene image sequence is first processed by a Bi-LSTM layer, and the forward LSTM captures the positive time-dependent features of the image frames; Backward LSTM captures inverse temporal dependency features, and the two are combined to form complete temporal context information; Key feature weighting; Bi - The features output by LSTM are weighted through an Attention layer; the Attention mechanism calculates the importance of each image frame in the prediction task and assigns higher weights to highly relevant frames. ③ Optimization process: (1) Forward propagation: The sequence of fire scene images is used as input and passes through the Bi-LSTM layer, Attention layer and fully connected layer of the model in sequence to generate future transient or long-term HRR prediction values. (2) Loss calculation and gradient backpropagation: Calculate the loss between the predicted value and the actual HRR value, and update the model weights through the backpropagation algorithm; ④ Training result analysis: Plot the curves of training and validation loss as a function of rounds, and analyze the convergence of the model and the risk of overfitting.

8. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 7, characterized in that: The steps for weight allocation of the features output by the Bi-LSTM through the Attention layer are as follows: (1) Input features: The temporal features output by Bi-LSTM are used as the input of Attention; (2) Calculate attention score: evaluate the relevance of each image frame to the prediction task through a scoring function; (3) Normalized score: The attention score is converted into probability weights using the Softmax function; (4) Weighted summation: The features of each frame are weighted according to the attention weights to form a global feature representation.

9. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 7, characterized in that: The Attention mechanism calculates the importance of each image frame in the prediction task as follows: (1) Input feature preparation: Obtain the temporal feature sequence from the feature extraction module, where each element is the feature vector of the corresponding image frame; (2) Calculate Attention weights: Use the Attention mechanism to calculate the importance of each frame in the prediction task, calculate the score by the similarity between the query and the key, and then normalize the score to obtain the weight of each frame. (3) Weighted summation: The input features are weighted and summed using the obtained weights to obtain the weighted feature representation of each frame; (4) Generate final prediction: Input the weighted features into the subsequent prediction module to generate the final prediction result.

10. The method for predicting long-term fire heat release rate based on fire scene image recognition according to claim 1, characterized in that: The steps for training and optimizing the model in S5 are as follows: ① Data splitting: During the dataset construction phase, the total dataset is divided into training set and validation set according to the proportion. The training set is used to update the model parameters, and the validation set is used to evaluate the model's performance on unseen data. ② Definition of loss function: Mean squared error is one of the most commonly used loss functions in regression models, used to measure the difference between predicted and actual values.

Images