A rail transit fire detection and positioning method based on computer vision

Abstract

The application discloses a rail transit fire detection and positioning method based on computer vision. The method comprises the following steps: acquiring a video data stream of a scene to be detected; inputting the video data stream into a trained fire detection and positioning model to obtain a fire detection result and a positioning result, wherein the fire detection and positioning model comprises an event detection model and a target positioning model; the event detection model takes the video data stream as input, detects whether the scene is in a suspected fire state, and detects a video frame corresponding to the suspected fire state; in the case of determining that the scene is in the suspected fire state, the target positioning model locks position information of a fire occurrence based on a detection result of the event detection model. The application has strong real-time performance, high accuracy and strong generalization capability, and can be deployed in fire detection and positioning of a rail transit station scene or other scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and more specifically, to a computer vision-based method for detecting and locating fires in rail transit. Background Technology

[0002] Fire incidents occur frequently in rail transit systems. Existing fire detection methods are usually based on physical characteristics such as heat or smoke, and are detected by sensors installed on the ceiling of the station. However, when a fire occurs in an area close to the ground, smoke and flames need a certain time delay to reach the sensor location, resulting in untimely fire detection.

[0003] Traditional flame and smoke detection methods typically involve manually designing extraction rules to extract color, shape, motion, and texture features of flames and smoke. For example, smoke detection methods based on manually designed color and motion features assume that the three channel values ​​are almost identical, but they can only detect smoke, and due to the irregular shape and fluid properties of smoke, they have a high false alarm rate. Existing flame detection methods based on manually designed color, shape changes, and motion features, while also utilizing the MIVIA fire detection dataset, can only detect flames and not smoke in the image. Therefore, traditional flame and smoke detection methods are inefficient, struggle to automatically extract flame and smoke features, and suffer from low accuracy and slow speed.

[0004] In recent years, with the development of deep learning, object detection algorithms in the field of computer vision have been gradually applied to research related to fire detection. For example, the lightweight FireNet-v2 model has high accuracy and can be deployed on low-cost devices, but it can only detect flames. Another example is existing fire smoke detection algorithms based on video content analysis, which use CNN convolutional neural networks to detect smoke in the image frames to be inspected, but they can only detect smoke and not flames.

[0005] The above studies typically detect flames and smoke separately, but flame detection and smoke detection are equally important and must be performed simultaneously during a fire. Therefore, Zhao et al. proposed a fire detection framework FSDF (ZHAO HY, JIN J, LIU Y, et al. FSDF: A high-performance fire detection framework[J]. Expert Systems with Applications, 2023, 238(A):121665.), which weaves YOLOv8 and Vector Quantization Variational Autoencoder (VQ-VAE) into the framework. This unsupervised fire detection has high accuracy, but when deployed to more complex scenarios, the model's real-time performance needs further improvement. Some researchers have also proposed convolutional neural network models based on attention mechanisms, using the Grad-CAM method to visualize and locate fire images, but this method is only applicable to fire detection in static images.

[0006] Currently, most research focuses on outdoor fires or forest fires, with limited research on fire detection in transportation scenarios. To address this, Ma Qinglu et al. proposed a deep learning-based fire and smoke detection method for highway tunnels (a YOLOv4 multi-scale tunnel fire detection algorithm embedded with DenseNet [J]. Computer Simulation, 2023, 40(04): 120-127.). By embedding DenseNet into the CSP structure of YOLOv4, it can fuse multi-scale shallow features and meet the real-time detection requirements in tunnel scenarios. However, existing methods are typically applied to road traffic scenarios, with limited research on flame and smoke detection in rail transit stations. Furthermore, current fire detection in rail transit stations mainly relies on traditional smoke sensors, which have a certain time delay.

[0007] Analysis reveals that existing fire detection models for rail transit stations have the following main problems:

[0008] 1) Since the detection of flames or smoke by sensors requires a certain time delay to trigger the alarm, there is a problem with poor real-time performance;

[0009] 2) Traditional fire detection models require manually designed algorithms to extract the features of flames and smoke, resulting in low accuracy and slow detection speed;

[0010] 3) Deep learning-based models can automatically extract flame and smoke features, but some existing models cannot detect both flame and smoke simultaneously; some models are too complex, have high equipment requirements, poor real-time performance, and are difficult to process video data; while relatively simple models have low accuracy. In summary, existing deep learning models have poor generalization ability or robustness when dealing with fire detection in rail transit. Summary of the Invention

[0011] The purpose of this invention is to overcome the shortcomings of the prior art and provide a computer vision-based method for detecting and locating fires in rail transit. This method includes the following steps:

[0012] Acquire the video data stream of the scene to be detected;

[0013] The video data stream is input into a trained fire detection and localization model to obtain fire detection and localization results.

[0014] The fire detection and localization model includes an event detection model and a target localization model. The event detection model takes the video data stream as input and detects whether the scene is in a suspected fire state and the corresponding video frame. If it is determined to be in a suspected fire state, the target localization model locates the location information of the fire based on the detection results of the event detection model.

[0015] Compared with existing technologies, the advantages of this invention lie in proposing a real-time, end-to-end refined fire location model framework (Fire-Detect). This framework can process each frame of image sequence obtained from video surveillance in real time, achieving real-time fire detection within stations without needing to be near the fire source. The model was tested on a self-built dataset, and experimental results show that it has high detection accuracy, low memory usage, and is suitable for monitoring equipment within rail transit stations. This invention has significant reference value for the safe operation of urban rail transit.

[0016] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0018] Figure 1 This is a flowchart of a computer vision-based method for detecting and locating fires in rail transit according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the framework of a fire detection and localization model Fire-Detect according to an embodiment of the present invention;

[0020] Figure 3 This is a schematic diagram of an in-station fire incident detection model according to an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a refined fire target location model according to an embodiment of the present invention;

[0022] Figure 5 This is a schematic diagram of flame and smoke detection results according to an embodiment of the present invention. Detailed Implementation

[0023] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0024] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0025] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0026] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0027] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0028] See Figure 1 As shown, the provided computer vision-based method for detecting and locating rail transit fires generally includes the following steps: Step S110, acquiring a video data stream of the scene to be detected; Step S120, inputting the video data stream into a trained fire detection and location model to obtain fire detection results and location results. The fire detection and location model includes an event detection model and a target location model. The event detection model uses the video data stream as input to detect whether the scene is in a suspected fire state and the video frames corresponding to the suspected fire state. If the scene is determined to be in a suspected fire state, the target location model locks the location information of the fire based on the detection results of the event detection model.

[0029] The following sections first elaborate on the problem this invention aims to solve. Then, the structure of each part of the model is briefly described. Finally, the application results of the model on a dataset of fires within rail transit stations are presented.

[0030] (1) Problem Statement

[0031] Fires are not a common occurrence, and detecting whether a fire has occurred in the surveillance video scene within the station requires continuous detection of every frame of the video. To conserve computing resources, this invention constructs a station fire detection algorithm with low computational resource consumption per unit time, detects the occurrence of fires within the station, and, under the premise that a fire has occurred, activates a highly accurate refined fire localization algorithm to regress the location of flames and smoke.

[0032] The station fire detection algorithm (or fire detection model or event detection model) detects each frame of the video. If the detection result is "normal", the scene in the rail transit station is in a "normal state". If the image category detection result of a frame in the monitoring video is "flame", "smoke", or "flame and smoke", the scene is in an abnormal "suspected fire" scene. In real time, Faster-RCNN is used to build a refined fire localization algorithm (or target localization model) to locate the flame and smoke rectangles at the location of the fire, so as to determine the specific location of the fire and the scale of the spread of flame and smoke as quickly as possible.

[0033] In one embodiment, the fire detection algorithm (i.e., event detection model) built based on convolutional neural networks, residual structures, and channel attention mechanisms is represented as g1(Y), and the expression of the refined fire localization algorithm (i.e., target localization model) based on Faster-RCNN target detection is simplified to g2(Y). Using real-time monitoring videos of different scenes within rail transit stations for fire detection and localization, the overall fire detection and localization model Fire-Detect designed in this invention can be represented as:

[0034]

[0035] Where Y represents the monitoring video data stream in the scene of the station, Z is the output of the Detect-Predict model, g1 represents the category to which each frame of the video belongs when the fire detection algorithm processes the monitoring video data, and g2 represents the output of the rectangular box of flame and smoke based on the refined fire localization result in the fire scene if the category of a certain frame image is one of flame, smoke, or flame and smoke.

[0036] The dataset for the fire detection algorithm within a fire station, based on convolutional neural networks, residual structures, and channel attention mechanisms, is labeled in the form of image tags. Since the detection results are divided into four categories, the labeling results are based on the One-Hot vector V. j Specifically, it means:

[0037]

[0038] Where n represents the maximum number of images in the dataset (e.g., 10834), j represents the image index in the dataset (maximum 10834), and V j This represents the annotation vector corresponding to each image in the dataset used to train the fire detection algorithm. The vector values ​​correspond to whether there is flame or smoke in the image; a value of 1 indicates flame, and a value of 0 indicates smoke. [0,0] represents the image category "normal", [1,0] represents the image category "flame", [0,1] represents the image category "smoke", and [1,1] represents the image category "flame and smoke".

[0039] The training set for the Faster-RCNN object detection refined fire localization algorithm g2 consists of Fire-Rail data with labeled flame and smoke bounding boxes. The labeled results are as follows:

[0040]

[0041] Where j represents the index of the image in the input training set, and the maximum total number of images is n, F j Let be the total number of detected targets labeled as flames on the j-th image. Represents the j-th image and the F-th image. j The center point coordinates of the flame-marked target For the j-th image, the F-th image j The width and height of the flame-marked target detection bounding box, S j This represents the total number of detected targets labeled as smoke on the j-th image. For the j-th image, the S-th image j The center point coordinates of the target marked by smoke. Represents the j-th image and the S-th image. j The width and height of the target detection box marked by smoke.

[0042] Finally, the annotation results of all images are input into algorithm g'2. After training, the Faster-RCNN refined fire localization algorithm g2 is obtained, which can be used to locate flames and smoke after a fire. The algorithm before and after training can be represented as follows:

[0043]

[0044] Here, g'2 represents the Faster-RCNN refined fire localization algorithm before training, and g2 represents the Faster-RCNN refined fire localization algorithm after training. By inputting real-time monitoring video, the model outputs the monitoring video's operating status, including "normal state" and "suspected fire." When a "suspected fire" state appears, the target localization model outputs refined bounding box location information for the flames or smoke.

[0045] (2) Fire-Detection Model Example

[0046] Figure 2 This is a schematic diagram of the Fire-Detect model. The model takes real-time passenger flow monitoring video as input and generally includes an in-station fire detection process and a refined fire location process. The in-station fire detection algorithm categorizes each frame of the monitoring video into one of four categories: normal, flame, smoke, or a combination of both. If the video frame detection result is consistently normal, a "suspected fire" alarm is not triggered. If a "suspected fire" state is detected (i.e., one of the other three states is present), an alarm is immediately alerted to the crew, and the refined fire location process is initiated. In the refined fire location process, the suspected fire video frame is input into the refined fire location algorithm based on the Faster-RCNN object detection algorithm to regress the specific pixel coordinates of the flame or smoke. If the corresponding coordinates cannot be detected, manual verification is performed to confirm whether a false alarm has occurred. If a rectangular frame of flame or smoke is detected, the coordinates of the flame and smoke are immediately regressed, and the image with the rectangular frame is returned to the in-station crew for processing.

[0047] 1) Station fire incident detection model

[0048] The purpose of the fire detection algorithm within the station is to detect whether a fire has occurred in each frame of the video stream based on surveillance videos of different scenarios within the rail transit station. Since the probability of a fire occurring in surveillance video is low, in one embodiment, a fire event detection model with low computational resource consumption is constructed based on convolutional neural networks, residual structures, and channel attention mechanisms. Its structure is as follows: Figure 3 As shown, images and video data of a fire in a rail transit station are created using Unity simulation data and Internet image data. Each frame of the input real-time monitoring video is preprocessed and classified.

[0049] The fire incident detection model within the station comprises an input module, a backbone network, and an output module. The input module processes the input images to a uniform size and standardizes them. Specifically, each frame of the input image is processed to 224×224 pixels, and the three-channel tensor of the input image is standardized based on the Z-score. Z-score standardization is calculated based on the value of each pixel in each channel, and the standardization formula is as follows:

[0050]

[0051] Where Z represents the standardized value of the pixel in the channel; x represents the original value of the pixel in the channel; μ represents the average value of the pixel in the channel; and σ represents the standard deviation of the pixel in the channel.

[0052] The backbone network extracts features from the processed image. The basic part of the backbone network is built on a convolutional neural network, using LeakyReLU activation and max pooling in the convolutional layers to avoid damaging neurons and reduce computational cost while preserving detailed image features. The entire backbone network employs a residual structure approach. For example, in the fire event detection algorithm, three residual layers containing multiple attention residual blocks are used. This prevents the network from mechanically deepening its layers; instead, short-circuit connections are added to some layers, essentially adding an identity mapping channel. This allows for better segmentation of the mapping relationship, enabling step-by-step transformation between input and output and simplifying the learning complexity of some network structures. A channel attention mechanism is introduced into the residual blocks. In flame detection, this mechanism compresses the feature map space, uses two-dimensional adaptive average pooling, and normalizes the data through two convolutional layers to obtain the channel attention matrix. This matrix is ​​then weighted onto the input matrix to determine the importance of different channels. For example, if the color of a flame is red and yellow, during model training, the channel attention mechanism may pay more attention to the red channel, which is similar in color to the flame, while ignoring less important information from other channels to some extent.

[0053] In one embodiment, the loss function of the fire event detection model within the station is set based on the BCEWithLogitsLoss function, and is expressed as follows:

[0054] l n =-[y n ·ln(σ(x n ))+(1-y n )·ln(1-σ(x n (6)

[0055] Loss = {l1, l2, ..., l N} (7)

[0056] Where N represents the sample size, x n y represents the probability that the nth sample is predicted to be either a fire or smoke. n Let L represent the label of the nth sample, and Loss = {l1, l2, ..., ln}. N} represents the sum of the losses of all samples and the average of the sums.

[0057] In one embodiment, the accuracy calculation formula for the fire event detection model is as follows:

[0058]

[0059] Among them, TP C To determine the number of images correctly detected as one of the three categories: flame, smoke, or flame and smoke, TN CTo determine the number of images correctly detected as normal, FP C The number of images that are detected as one of the three categories: flame, smoke, or flame and smoke, but are actually classified as normal. C The number of images that are actually classified as one of the three categories: flame, smoke, or flame and smoke, and are detected as normal.

[0060] The output module uses average pooling to retain all feature information extracted by the backbone network. After passing through a fully connected layer, the classification result is obtained. The classification result is then processed using the Sigmoid activation function, and finally, a One-Hot vector V is output. j It returns the category represented by its label, including four categories: normal state, flame, smoke, and flame and smoke. The "normal" category is not processed further, while the images of the other three categories, "flame," "smoke," and "flame and smoke," will continue to be input into the subsequent refined fire location algorithm for processing.

[0061] 2) Target location model (refined fire location algorithm)

[0062] Since the probability of abnormal events such as fires is relatively low, the computational load required by the target detection model per second is moderate, ensuring the algorithm's real-time performance. Under these conditions, it is necessary to maximize the model's detection accuracy; therefore, the Faster-RCNN target detection model is considered for refined fire localization. (See [link to relevant documentation]). Figure 4 As shown, an image of a "suspected fire" is input into a refined fire target localization model, which ultimately outputs regression bounding boxes of flames and smoke. For example, the target localization model as a whole includes a feature extraction module, a candidate region generation module, a ROI pooling layer, and a classification and regression module.

[0063] When a "suspected fire" image is input into the target localization model, it first passes through a feature extraction module, which includes, for example, 13 convolutional layers, 13 ReLU activation function layers, and 4 pooling layers, to generate a feature map of the image.

[0064] In the candidate region generation module, feature maps are used to generate candidate regions through a Region Generation Network (RPN). The RPN network consists of two branches: one for calculating the regression bounding box offset of the anchor boxes to obtain accurate candidate regions, and the other for classifying the anchor boxes using Softmax to determine whether a target exists in the image. The generated candidate regions are then refined through a Proposal layer, including steps such as correcting anchor box offsets, deleting out-of-bounds candidate regions, and deleting candidate regions that are too small.

[0065] ROI (Region of Interest) pooling layers are used to capture image regions containing the target to be detected. ROI pooling layers map the generated candidate regions to the corresponding locations on the feature map, dividing the ROI into equal-sized local regions and performing max pooling to generate a fixed-size feature map.

[0066] For the classification and regression module, fully connected layers and softmax layers are used to calculate the category and probability values ​​of all candidate regions. Simultaneously, positional offset information is used again to regress the target detection bounding boxes to improve detection accuracy.

[0067] In one embodiment, the loss function of the target localization model is divided into two parts: RPN loss and Fast-RCNN loss (or target detection loss), for example, a weighted sum of RPN loss and Fast-RCNN loss. The RPN loss function is further divided into two parts: classification loss and regression loss. The classification loss is the binary classification process of foreground and background of the image during the generation of candidate boxes by RPN.

[0068] For example, the RPN loss function formula is:

[0069]

[0070] Where, N cls p represents the sample size of the first mini-batch. i Predict the probability that anchor box i is the true label; Positive samples take a value of 1, and negative samples take a value of 0; λ is the balance coefficient; N reg The total number of anchor box samples used to train the RPN network; t i The parameterized coordinates of the bounding box predicted for anchor frame i; The parameterized coordinates are the true bounding boxes of the detected targets in anchor frame i.

[0071] For classification loss, L cls The function is defined as:

[0072]

[0073] For bounding box regression loss, L reg The function is defined as:

[0074]

[0075]

[0076] t i and The calculation of each component in the vector is as follows:

[0077]

[0078] Wherein, components x, y, w, and h represent the coordinates and dimensions of the center point of the bounding box predicted by anchor frame i, respectively; component x * ,y * ,w * ,h * ... a ,y a ,w a ,h a This represents the coordinates and dimensions of the center point of anchor frame i.

[0079] For Fast-RCNN loss, it is also divided into two parts: classification loss and regression loss. Here, classification refers to the final classification of candidate regions. The formula for calculating the loss function is:

[0080] L(p,u,t u ,v)=L cls (p,u)+λ[u≥1]L loc (t u ,v) (15)

[0081] Where p is the predicted softmax probability distribution during classification; u is the true class label of the target; [u≥1] is an Iverson bracket, which is 1 when u≥1, and 0 otherwise; t u The regression parameters for the predicted class u are specifically: v is the regression parameter for the true target rectangle, specifically (v x ,v y ,v w ,v h ).

[0082] L cls (p,u) is the classification loss, and its calculation formula is:

[0083] L cls (p,u)=-ln(p u (16)

[0084] In the formula, p u This represents the probability that the predicted category label u is the true one.

[0085] L loc (t u ,v) is the regression loss, and its calculation formula is:

[0086]

[0087] (3) Experimental verification

[0088] To further verify the effectiveness of the present invention, an experiment was conducted, and the experimental setup is as follows.

[0089] 1) Dataset

[0090] A fire detection dataset for rail transit stations, Fire-Rail, was constructed based on real fire accident images downloaded from the internet and smoke images of station fires simulated using Unity. The dataset contains 706 real-world fire smoke images, which were augmented to obtain 4842 images. It also includes 1964 simulated images, which were augmented to obtain 5892 images. The dataset contains a total of 10834 images, covering scenes such as inside and outside train carriages, transfer passages, station halls, escalators and stairwells, near vending machines, at train carriage doors, and where flames and smoke are obscured.

[0091] The dataset consists of videos, images, and two annotation formats. The first part is the Fire-Rail1 dataset, which uses image-based label vectors to annotate image categories and is used to train the fire detection algorithm within the station. The second part is the Fire-Rail2 dataset, which uses bounding boxes annotated with the locations of fires and smoke in the images and is used to train a refined fire localization algorithm. The third part is video data used to test the performance of the Fire-Detect model.

[0092] Fire-Rail1 data is based on the collected image dataset, with each image labeled to its category. The image category is indicated by its flame and smoke label vectors, and the generated label file format is shown in Table 1. The four example JPG images are named with serial numbers and labeled using One-Hot vectors. Therefore, the image categories represented by the four image labels are "Smoke," "Normal," "Flame and Smoke," and "Flame," respectively, corresponding to the video monitoring statuses "Suspected Fire," "Normal Status," "Suspected Fire," and "Suspected Fire."

[0093] Table 1. Examples of Fire-Rail1 data annotation files

[0094]

[0095] The Fire-Rail2 dataset labels the bounding boxes of flames and smoke in each image based on the collected image dataset. Labeling is done using the open-source tool LabelImg, which labels both flames and smoke as objects in the image. The maximum bounding box of the detected objects is drawn based on the crosshair positioning of the labeling tool, and the labeled flame and smoke objects are labeled as "flame" and "smoke," respectively. The labeling results for all detected objects in each dataset image are stored in a separate txt file, containing the pixel coordinates of the four vertices of the bounding boxes for all flames and smoke in the image.

[0096] There are 5 video datasets used for testing the Fire-Detect model, each 5 minutes long with a PFS of 25. They include various scenes such as inside a train car, on a platform, on escalators and stairs, and in transfer passages, and also cover the entire fire lifecycle from the start to the end of the fire.

[0097] 2) Fire detection algorithm experiment

[0098] ① Model training and validation

[0099] An in-station fire detection algorithm was trained on the Fire-Rail1 dataset. 80% of the data was used as the training set, and the remaining 20% ​​as the test set. The learning rate was 0.001, the number of epochs was 20, and the Adam optimizer was used. After training, the algorithm achieved a loss function of 0.0746 and an accuracy of 96.75% on the training set, and a loss function of 0.0854 and an accuracy of 95.12% on the test set, demonstrating good detection performance.

[0100] The constructed fire detection algorithm for the station was tested based on simulated video data to verify whether the algorithm can process monitoring video data in real time. The experimental results are shown in Table 2. The algorithm processes video frames at a rate of 58 frames per second, which meets the requirements for real-time detection of fire occurrence in monitoring videos of different scenarios in rail transit stations.

[0101] Table 2. Experimental results of CNN-based fire detection algorithm on animated video data.

[0102]

[0103] ② Convolutional Neural Network Hierarchy Experiment

[0104] To determine the optimal number of layers in the convolutional neural network, experiments were conducted on the fire detection algorithm within the station using networks with varying numbers of convolutional layers. The experimental results are shown in Table 3. Table 3 shows that when using 5 layers of convolutional modules, the accuracy on the training set was 96.75%, and the accuracy on the test set was 95.12%. When the number of convolutional layers was reduced to 4, the accuracy on the training and test sets decreased by 2.27% and 1.98%, respectively. When the number of convolutional layers was reduced to 3, the accuracy on the training and test sets decreased by 5.50% and 4.56%, respectively, compared to the 5-layer convolutional network. This indicates that increasing the number of convolutional layers significantly improves the algorithm's accuracy. However, when the number of convolutional modules was further increased from 5 to 7 layers, the accuracy only improved by 0.11% and 0.22%, respectively. Compared to the increase in computational resources, the improvement in accuracy was relatively small. Therefore, an in-station fire detection algorithm is constructed based on 12 convolutional layers, namely 5 multi-convolutional modules, so as to maximize the detection accuracy of the algorithm with minimal computational resource consumption.

[0105] Table 3: Experimental Results for Number of Convolutional Layers

[0106]

[0107] ③ Ablation test

[0108] The in-station fire detection algorithm is implemented based on three parts: convolutional neural network, residual structure, and attention mechanism. To examine the impact of each component's structure on the algorithm's performance, an ablation experiment was conducted. The algorithm's performance was tested using the basic convolutional neural network algorithm, the convolutional neural network combined with the residual structure, the convolutional neural network combined with the attention mechanism, and the combination of the convolutional neural network, residual structure, and attention mechanism. The test results are shown in Table 4. The accuracy of the in-station fire detection algorithm is improved by 1.91%, 0.29%, and 1.23% compared to the other sub-algorithms, respectively. Combining the convolutional neural network, residual structure, and attention mechanism maximizes the performance of the in-station fire detection algorithm.

[0109] Table 4. Results of ablation experiments for fire detection algorithms within the station.

[0110]

[0111] ④ Robustness test

[0112] In practical applications of the algorithm, real-time monitoring video data in different scenarios within the station may be affected by various sources such as shooting scene interference, monitoring equipment failure, and passenger behavior, generating various types of noise that affect the quality of monitoring video frames and the accuracy of the station fire detection algorithm. Therefore, considering this reality, to evaluate the algorithm's robustness when images are affected by noise, 50 images were randomly selected from the collected Fire-Rail image dataset, edited, and then tested using the station fire detection algorithm.

[0113] First, flames and smoke were removed from 50 randomly selected images in the dataset, making them classified as normal images. The algorithm's accuracy was 97.10%. Second, removing either flames or smoke further improved the accuracy to 95.82%. Next, flames and smoke were obscured by foreign objects, achieving an accuracy of 94.35%, demonstrating the algorithm's good performance in these cases. Finally, artifact, Poisson, Gaussian, and salt-and-pepper noise were added to the 50 images, and the fire detection algorithm was used again. The results are shown in Table 5. Compared to other interference conditions, the detection accuracy was lower under noise conditions. This is presumably because the added noise reduced image clarity to varying degrees, or because some noise resembled the color and shape of smoke. Overall, the on-site fire detection algorithm can handle most noise conditions.

[0114] Table 5. Robustness test results of the fire detection algorithm within the station.

[0115]

[0116] 3) Fire-Detect Model Experiment

[0117] A refined fire localization algorithm based on the Fire-Rail2 dataset, trained using Faster R-CNN, regresses the bounding boxes of flames and smoke. A complete Fire-Detect model is then constructed based on the in-station fire detection algorithm and the refined fire localization algorithm. The Faster R-CNN refined fire localization algorithm is trained with a batch size of 64, a maximum training iteration count of 40,000, and a learning rate of 0.001. The algorithm achieves a mean average accuracy (mAP) of 77.3% for fire localization. Experiments on the Fire-Detect model using Unity-based simulated video data are presented, and the visualization results are shown below. Figure 5 As shown, where, Figure 5 (a) Detection results of flames and smoke in the animation scene. Figure 5 (b) shows the detection results of flames and smoke in a real-world scene. A label of "0" for the detection box indicates that the detection result is "flame" and its confidence level, while a label of "1" indicates that the detection result is "smoke" and its confidence level. Experimental results show that the Fire-Detect model performs well in detecting flames and smoke, accurately regressing the pixel coordinates of flames and smoke, and visualizing them in video frames.

[0118] To highlight the advantages of the Fire-Detect model, a comparative experiment was conducted with currently performing algorithms, and the results are shown in Table 6. GFLOPs measures model complexity, FPS is the number of frames processed per second, and the number of parameters measures model size. Faster-RCNN has a slower detection speed and consumes more resources, but boasts the highest accuracy. YOLOv5s has the fastest detection speed, but its accuracy is lower, and its performance is not ideal when used alone. Combining Faster-RCNN with a fire detection algorithm to form the Fire-Detect model achieves the same detection accuracy (95.12%) as the fire detection algorithm, with an average fire localization precision (mAP) of 77.3% and an FPS of 54. Although the overall model complexity and number of parameters are greater than Faster-RCNN, fires are not a common occurrence. Under normal circumstances, the model only uses the fire detection algorithm to detect each frame of the surveillance video and determine whether a fire has occurred. Only when a fire occurs is a refined localization algorithm activated. This ensures both accuracy and high detection speed without consuming excessive resources, enabling real-time detection of fire occurrences in different scenes within rail transit stations, thus achieving a reasonable allocation and use of computing resources.

[0119] Table 6 Comparison of Experimental Results of Fire Video Data

[0120]

[0121] It should be noted that, without departing from the spirit and scope of this invention, those skilled in the art can make appropriate changes or modifications to the above embodiments. For example, the number of layers, activation function type, loss function type, etc., of each model can be set according to actual needs.

[0122] In summary, compared with the prior art, the present invention has the following advantages:

[0123] 1) This invention proposes for the first time the framework structure of the Fire-Detect fire detection and localization model. It uses an in-station fire detection algorithm to detect whether each frame of the video is in a "suspected fire" state. If a fire occurs, a refined fire localization algorithm is then activated to obtain its location information. The provided framework structure has strong real-time performance, high accuracy, and strong generalization ability, and can be deployed in various in-station scenarios or other scenarios.

[0124] 2) The fire detection algorithm for stations proposed in this invention, based on convolutional neural networks, residual structures, and channel attention mechanisms, is the first of its kind in the field of fire detection in rail transit scenarios. It can be used to detect whether a fire has occurred in each frame of a surveillance video in real time. This design effectively reduces the complexity of the model, improves the accuracy of the fire detection algorithm for stations, and can process video data at a rate of 58 frames per second, thus solving the model deployment problem to some extent. Therefore, this fire event detection algorithm can maximize detection accuracy with minimal computational resource consumption, enabling rail transit operators to quickly locate the fire and take appropriate measures.

[0125] 3) Experimental results show that the fire detection algorithm in the Fire-Detect model can monitor the occurrence of fires in rail transit stations in real time. The model only activates the refined fire location algorithm when a fire occurs, without constantly occupying a large amount of computing resources. The trained model can meet the requirements of real-time performance and accuracy after deployment.

[0126] 4) Existing technologies typically focus on outdoor fire detection. This paper proposes a specific model architecture and fire detection localization process to address the characteristics and requirements of fire detection in rail transit stations, filling the gap in fire detection in rail transit scenarios and also applicable to other scenarios.

[0127] 5) This invention constructs a computer vision-based fire detection and localization model framework for rail transit stations. Using video collected by station monitoring equipment as input, a fire detection algorithm is first built based on convolutional neural networks, residual structures, and channel attention mechanisms to determine in real time whether each frame of the video is in a "suspected fire" state. Then, a refined fire localization algorithm is used to regress the positional information of flames and smoke in the "suspected fire" images. This framework addresses the non-routine nature of fires, enabling continuous and real-time detection of fires in station monitoring video scenes. Only when a fire occurs is the resource-intensive refined fire localization algorithm activated, thus achieving both accurate and efficient processing of video data and rational allocation and utilization of resources.

[0128] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0129] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0130] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0131] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, Python, etc., and conventional procedural programming languages ​​such as "C" or similar languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0132] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0133] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0134] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0136] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A computer vision-based method for detecting and locating fires in rail transit, comprising the following steps: Acquire the video data stream of the scene to be detected; The video data stream is input into a trained fire detection and localization model to obtain fire detection and localization results. The fire detection and localization model includes an event detection model and a target localization model. The event detection model takes the video data stream as input and detects whether the scene is in a suspected fire state and the corresponding video frame. If it is determined to be in a suspected fire state, the target localization model locates the location information of the fire based on the detection results of the event detection model. The event detection model is constructed based on convolutional neural networks, residual structures, and channel attention mechanisms. The fire detection and location model is represented as follows: in, This indicates the surveillance video data stream in this scenario. It is a target localization model. This is the output of the fire detection and location model. This represents the surveillance video data and outputs the category to which each frame of the video belongs. This means that if a certain frame of image belongs to one of the three categories of flame, smoke, or flame and smoke, then the bounding boxes of flame and smoke will be output based on the fire location results in the fire scene.

2. The method according to claim 1, characterized in that, The event detection model is trained according to the following steps: Construct a dataset that reflects the correspondence between known video image frames and event categories; The event detection model is trained using the dataset and outputs the event category to which the video frame image belongs. The event category includes normal, flame, smoke, and flame and smoke. When an event category of a certain frame of an image is detected to be one of flame, smoke, or a combination of flame and smoke, it is determined to be a suspected fire situation.

3. The method according to claim 2, characterized in that, The event detection model includes an input module, a backbone network, and an output module. The input module processes the input image to the same size and performs image standardization. The backbone network extracts features from the standardized image. The output module uses average pooling to retain all feature information extracted by the backbone network, passes it through a fully connected layer to obtain the event classification result, processes the classification result with a sigmoid activation function, and uses a one-hot encoded vector to represent the event category.

4. The method according to claim 3, characterized in that, The backbone network is constructed based on a convolutional neural network and adopts a residual structure containing multiple residual blocks, with a channel attention mechanism introduced in each residual block.

5. The method according to claim 1, characterized in that, During training, the loss function of the event detection model is set as follows: in, Indicates sample size. Indicates the first n The probability that each sample is predicted to be either a fire or smoke. Indicates the first n The label of each sample, This represents the sum of the losses of all samples, calculated as the average.

6. The method according to claim 1, characterized in that, The target localization model includes a feature extraction module, a candidate region generation module, a ROI pooling layer, and a classification and regression module. The feature extraction module comprises convolutional layers, ReLU activation function layers, and pooling layers, outputting a feature map of the image. The candidate region generation module generates candidate regions based on the feature map using a Region Generation Network (RPN). The RPN includes a first branch and a second branch. The first branch calculates the regression bounding box offset of the anchor boxes to obtain candidate regions, while the second branch classifies the anchor boxes using Softmax to determine if a target exists in the image. The ROI pooling layer maps the candidate regions to corresponding positions on the feature map, divides the region of interest into equal-sized local regions, and performs max pooling to generate a fixed-size feature map. The classification and regression module uses fully connected layers and Softmax layers to calculate the category and probability values ​​of all candidate regions and uses the positional offset information to regress the target detection bounding box.

7. The method according to claim 6, characterized in that, The loss function for training the target localization model includes the Region Generation Network (RPN) loss and the target detection loss, wherein the RPN loss is set as follows: in: in, It is the sample size. For anchor frame The probability of predicting the true label; Positive samples are assigned a value of 1, and negative samples are assigned a value of 0. For balance coefficient, The total number of anchor box samples during training. For anchor frame Predicted bounding box parametric coordinates, For anchor frame The parametric coordinates of the ground truth bounding box of the detected target. It is classification loss. It is the bounding box regression loss. and The components of the vector are represented as follows: Among them, components Representing anchor frames Coordinates and dimensions (width and height) of the predicted bounding box center point; components Representing anchor frames The coordinates of the center point of the actual annotation box and the width and height components of the annotation box. Indicates anchor frame The coordinates and dimensions of the center point.

8. The method according to claim 7, characterized in that, The target detection loss is set as follows: in, For classifying losses, To regress the loss, This represents the probability distribution predicted during classification. Provide the true category label for the target; For Iverson, brackets, when hour The value is 1 for the whole, otherwise it is 0; For the predicted category The regression parameters; The regression parameters are the true target rectangle. Indicates the category label that is predicted to be true. The probability of.

9. A computer-readable storage medium having a computer program stored thereon, wherein, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 8.

10. A computer device comprising a memory and a processor, wherein a computer program capable of running on the processor is stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

Images