Fire passage detection method based on lightweight architecture

Abstract

The application discloses a fire passage detection method based on a lightweight architecture, and comprises the following steps: acquiring fire passage data, marking and constructing a fire passage data set; constructing a lightweight fire passage detection model; iteratively training the fire passage data set in the lightweight fire passage detection model to obtain an optimal model, and outputting a fire passage detection result in real time through the optimal model; and constructing an alarm mechanism and giving a warning according to the detection result. The application solves the technical problem that the existing fire passage detection method of workers is not convenient and accurate enough.

Description

Technical Field

[0001] This invention relates to the field of deep learning and object detection technology, and in particular to a fire lane detection method based on a lightweight architecture. Background Technology

[0002] Due to the relatively high density of businesses and the large volume of materials flowing through the industrial park, as well as the abundance of flammable and combustible substances, fires are highly likely to occur. At the same time, fire lanes within the park are often blocked by cluttered items, preventing fire trucks from quickly reaching the scene in the event of a fire. This severely hinders firefighting efforts, evacuation, and rescue operations, leading to even greater losses. Ensuring the timely arrival of fire trucks and minimizing economic losses and casualties during a fire is a pressing issue that needs to be addressed.

[0003] Currently, existing fire lane detection methods primarily rely on manual assessment by personnel using remote video monitoring. Due to the distance from the site, this method is prone to missed or false alarms regarding obstructions. Furthermore, the workload for personnel is high, and relying solely on video to determine occupancy or violations is susceptible to subjective biases and work conditions, leading to inaccurate fire lane detection. Therefore, there is an urgent need to develop a fire lane detection method based on a lightweight architecture to address the inconvenience and inaccuracy of existing manual fire lane detection methods. Summary of the Invention

[0004] The main objective of this invention is to provide a fire lane detection method based on a lightweight architecture, which aims to solve the technical problems that existing methods for fire lane detection by operators are not convenient and accurate enough.

[0005] To achieve the above objectives, the present invention provides a fire lane detection method based on a lightweight architecture, wherein the fire lane detection method based on a lightweight architecture includes the following steps:

[0006] S1. Obtain fire lane data, annotate it, and construct a fire lane dataset;

[0007] S2. Construct a lightweight fire lane detection model;

[0008] S3. Iteratively train the fire lane dataset on the lightweight fire lane detection model to obtain the optimal model, and output the fire lane detection results in real time through the optimal model.

[0009] S4. Establish an alarm mechanism and issue warnings based on the detection results.

[0010] One preferred embodiment, step S1, specifically comprises:

[0011] S11. Acquire monitoring data of fire escape routes from several cameras, and extract key frames and filter the monitoring data.

[0012] S12. Set a mask that matches each camera individually to obtain the mask image;

[0013] S13. Use annotation tools to annotate the masked image to construct a fire lane dataset.

[0014] One preferred embodiment is that step S2, which involves constructing a lightweight fire escape route model, specifically:

[0015] S21. Construct a lightweight backbone network and perform effective feature extraction;

[0016] S22. Construct a feature fusion network, perform secondary feature extraction, and fuse it with the effective features extracted from the lightweight backbone network to obtain a feature map.

[0017] S23. Construct a detection head network and input the feature map into the detection head network for target classification and regression.

[0018] One preferred embodiment is that, in step S3, the fire lane dataset is iteratively trained on the lightweight fire lane detection model to obtain the optimal model, specifically as follows:

[0019] Initialize parameters, set batch size, read batch size data, perform equal height and equal ratio transformation, and input the data into the lightweight fire lane detection model for iterative training;

[0020] The transformed data is then processed through a lightweight backbone network to extract effective features.

[0021] The effective features extracted by the lightweight backbone network are input into the feature fusion network for secondary feature extraction to obtain the feature map.

[0022] The feature map is input into the detection head network for target classification and regression.

[0023] Construct a loss function, and use backpropagation to adjust the model network parameters using the loss function to obtain the optimal model.

[0024] One preferred embodiment is that the loss function is:

[0025]

[0026] in, For loss function, The number of detection layers, For bounding box regression loss, The weights of the bounding box regression loss are... For the loss of the target object, The weights for the loss of the target object. For classifying losses, The weights are used for the classification loss.

[0027] In one preferred embodiment, after iteratively training the fire lane dataset in the lightweight fire lane detection model to obtain the optimal model in step S3, the method further includes:

[0028] The optimal model weight file is converted, and the optimal model is deployed to the edge.

[0029] One preferred approach is to convert the optimal model weight file and deploy the optimal model to the edge, specifically by designing a custom operator to convert the .pt format weight file generated by the optimal model into a .onnx format weight file, and then deploying the converted optimal model to the edge.

[0030] One preferred embodiment is that in step S3, the fire lane detection results are output in real time using the optimal model, specifically as follows:

[0031] Acquire data streams from several cameras in the fire escape route, perform data protocol parsing, and capture frame-by-frame image data;

[0032] Multiply the image data with the corresponding mask of the camera to eliminate non-fire passage areas;

[0033] The image data is then subjected to channel conversion, cropping, and normalization.

[0034] The optimal model is used to output the fire lane detection results, and the detection results are stored locally.

[0035] In one preferred embodiment, step S4 establishes an alarm mechanism and issues an early warning based on the detection results, specifically as follows:

[0036] S41. Initialize the alarm mechanism status;

[0037] S42. Set the minimum area and minimum side length thresholds for obstacles, and perform obstacle area threshold discrimination;

[0038] S43. Set the minimum overlap area threshold for obstacles and perform obstacle confidence judgment;

[0039] S44. If the detection result indicates the presence of an obstacle, an alarm will be issued.

[0040] In one preferred embodiment, after issuing an alarm in step S44 if the detection result indicates the presence of an obstacle, the method further includes:

[0041] The proportion of obstacles in the entire map is calculated, and the blockage status of fire lanes is prioritized according to the proportion. The blockage status includes severe blockage, moderate blockage, and slight blockage.

[0042] The fire lane detection method based on a lightweight architecture, as described above, includes the following steps: acquiring and labeling fire lane data to construct a fire lane dataset; constructing a lightweight fire lane detection model; iteratively training the fire lane dataset on the lightweight fire lane detection model to obtain an optimal model, and outputting the fire lane detection results in real time using the optimal model; and constructing an alarm mechanism to issue warnings based on the detection results. This invention can detect the occupancy of fire lanes in real time and accurately, solving the technical problems of the inconvenience and inaccuracy of existing methods for fire lane detection by personnel.

[0043] In this invention, a mask is assigned to each camera, and non-fire lane areas are removed by using the mask, which improves the applicability and detection accuracy of the model and reduces detection time.

[0044] In this invention, the detection results of fire lanes are output through the model, and an alarm mechanism is set up. By performing area threshold discrimination and confidence threshold discrimination on obstacles, the situation of missed alarms and false alarms is further reduced. Furthermore, the priority of the blockage status is divided according to the proportion of obstacles in the current image, providing a reference for operators to prioritize handling. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of a fire escape detection method based on a lightweight architecture according to an embodiment of the present invention;

[0047] Figure 2 This is a schematic diagram illustrating the acquisition and annotation of fire lane data in step S1 of an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the lightweight fire lane detection model according to an embodiment of the present invention;

[0049] Figure 4 This is a schematic diagram of the lightweight feature extraction module according to an embodiment of the present invention;

[0050] Figure 5 This is a schematic diagram of the attention module in an embodiment of the present invention;

[0051] Figure 6 This is a schematic diagram of the pooling module in an embodiment of the present invention;

[0052] Figure 7 This is a schematic diagram of the CARAFE module according to an embodiment of the present invention;

[0053] Figure 8 This is a schematic diagram illustrating the iterative training of a lightweight fire lane detection model according to an embodiment of the present invention;

[0054] Figure 9 This is a schematic diagram illustrating the changes in indicators during the training process of the lightweight fire lane detection model according to an embodiment of the present invention.

[0055] Figure 10 This is a schematic diagram of the PR curve of the lightweight fire lane detection model according to an embodiment of the present invention.

[0056] The realization of the objective, functional characteristics and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It should be noted that all directional indicators (such as up, down, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicator will also change accordingly.

[0059] Furthermore, in this invention, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0060] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0061] See Figures 1-6 According to one aspect of the present invention, the present invention provides a fire lane detection method based on a lightweight architecture, wherein the fire lane detection method based on a lightweight architecture includes the following steps:

[0062] S1. Obtain fire lane data, annotate it, and construct a fire lane dataset;

[0063] S2. Construct a lightweight fire lane detection model;

[0064] S3. Iteratively train the fire lane dataset on the lightweight fire lane detection model to obtain the optimal model, and output the fire lane detection results in real time through the optimal model.

[0065] S4. Establish an alarm mechanism and issue warnings based on the detection results.

[0066] Specifically, in this embodiment, the present invention takes an industrial park as an example. The present invention is not specifically limited. Step S1 is specifically as follows:

[0067] S11. Obtain fire lane monitoring data from several cameras within the industrial park, extract keyframes and filter data from the monitoring data, and remove blurry and duplicate data; the monitoring data is image data.

[0068] S12. Set a mask that matches each camera. Each camera has a corresponding mask. The object placement area and other areas in the image data are removed by the mask, leaving only the fire escape area, to obtain the mask image. The mask image is then used as the model input.

[0069] S13. The mask image is annotated using an annotation tool to construct a fire escape dataset; the annotation tool is the CVAT annotation tool, which is not specifically limited in this invention and can be set as needed; the fire escape dataset includes a training set and a test set, and the fire escape dataset is divided into a training set and a test set in an 8:2 ratio, which is not specifically limited in this invention and can be set as needed.

[0070] Specifically, in this embodiment, the lightweight fire lane detection model includes a lightweight backbone network, a feature fusion network, and a detection head network; step S2, constructing the lightweight fire lane model, specifically involves:

[0071] S21. A lightweight backbone network is constructed based on a lightweight feature extraction module, an attention module, and a pooling module to perform effective feature extraction. The lightweight backbone network includes two pooling modules, six lightweight feature extraction modules, and three attention modules. This invention does not impose specific limitations and can be set according to needs. The two pooling modules are a convolutional pooling module and a spatial pyramid pooling module. The lightweight feature extraction module treats each channel as a group and models the spatial relationships within the group, including three structures. , , Lightweight module , By submodule Composition; Lightweight modules It includes two branches, branch one consists of one With one composition, This is a layer combining convolution, batch normalization, and ReLU activation function. For a combination of depthwise convolution and batch normalization layers, the hierarchical structure of branch one is as follows: , Branch two consists of two With one The composition, the layer structure of branch two is as follows: , , , In The input feature map size can be halved; for lightweight applications... Input Its size is , will input By inputting branch one and branch two respectively, inputting branch one, we get Input branch two, get The outputs of the two branches are concatenated in spatial dimension to obtain a lightweight module. Output Its size is , It can be represented as:

[0072]

[0073] in, express The output, express Input, express The output of branch one, express The output of branch two, This represents a layer combining convolution, batch normalization, and ReLU activation functions. This represents a combination layer of depthwise convolution and batch normalization. This indicates that the input will be concatenated along the channel dimension;

[0074] submodule It includes two branches: branch one is a short connection that performs an identity mapping, and branch two consists of two... With one The composition, the layer structure of branch two is as follows: , , For submodules Input Its size is First, channel separation is performed, splitting the input according to the channel dimension to obtain... , Their dimensions are all ,Will Input branch one, get Since branch one and branch two are short connections ,Will Input branch two, get The outputs of branch one and branch two are concatenated in spatial dimension to obtain the substructure. Output Its size is , and input Consistent It can be represented as:

[0075]

[0076] in, Represents submodule The output, Represents submodule The input for branch one, Represents submodule The input for branch two, Represents submodule The output of branch one, Represents submodule The output of branch two, This represents a layer combining convolution, batch normalization, and ReLU activation functions. This represents a combination layer of depthwise convolution (DWConv) and batch normalization. This indicates that the input will be concatenated along the channel dimension;

[0077] Lightweight module Composed of two Submodules are composed serially, for Input Its size is Its output The size is , can be represented as:

[0078]

[0079] in, Indicates lightweight module Input, Indicates lightweight module The output;

[0080] Lightweight module Five Submodules are composed serially, for Input Its size is Its output The size is , can be represented as follows:

[0081]

[0082] In the formula Indicates lightweight module Input, Indicates lightweight module The output;

[0083] The attention module includes a spatial attention module and a channel attention module. The channel attention module uses global average pooling to obtain channel statistics, embeds global information, and then creates compact features to guide accurate and adaptive selection. The spatial attention module performs group normalization on the input, obtains spatial statistics, embeds spatial information, and then uses shape factors to enhance the feature representation of spatial information.

[0084] The attention network module comprises two branches. The input to the attention module is fed into each branch. Branch one consists of short connections. Branch two first groups the input into features according to the channel dimension, and then processes them in parallel. For each sub-feature, the attention module captures pixel-level pairwise relationships and channel dependencies in both spatial and channel dimensions. Then, all sub-features are aggregated to obtain the output of branch two. The output of branch one is then added element-wise to the output of branch two. The input to branch two... Its dimensions are denoted as Along the channel dimension Divided into Groups, obtained Sub-characteristics , Each sub-feature size is During training, sub-features gradually capture specific semantic feature responses, and an attention module generates an importance coefficient for each sub-feature; grouping the above features into G yields feature maps. Divided into two groups along the channel dimension, each feature size is [missing information]. The first set of feature maps is input into the channel attention module, and the second set of feature maps is input into the spatial attention module.

[0085] For the G input feature maps of the channel attention module Their dimensions are all First, global average pooling is used to obtain channel statistics and embed global information. Then, compact features are created to guide accurate and adaptive selection. The g-th input feature map of the channel attention module... Its output feature map It can be represented as:

[0086]

[0087] in, The channel attention module is represented by the first one. One input feature map, The channel attention module is represented by the first one. Each output feature map This represents the Sigmoid activation function. Represents global information. A scaling factor representing global information. Movement factor representing global information, global information , This indicates a global average pooling operation;

[0088] For the G input feature maps of the spatial attention module Their dimensions are all First, the input is group normalized to obtain spatial statistics and embed spatial information. Then, the shape factor is used to enhance the feature representation of the spatial information.

[0089] Specifically, the first spatial attention Input feature maps Its output feature map for:

[0090]

[0091] in, Representation Spatial Attention Module One input feature map, The channel attention module is represented by the first one. Each output feature map This represents the Sigmoid activation function. Representing spatial information, and Shape factor representing spatial information, spatial information , This indicates a grouping normalization operation;

[0092] The outputs of the channel attention module and the spatial attention module are aggregated along the channel dimension to obtain the output of branch two. Its size is , and the output of branch two Consistent;

[0093] Add the output of branch one to the output of branch two element by element to obtain the output feature map of the attention network. ;

[0094] The pooling module includes a convolutional pooling module and a spatial pyramid pooling module; the convolutional pooling module includes a... Layer and one layer, The layer is a max pooling layer, and the layer structure of the convolutional pooling module is as follows: , ,in, Layers are used to change the input channel dimension. Layers are used to compress image size; the spatial pyramid pooling module includes two... Layers and three The layer structure of the spatial pyramid pooling module is as follows: , , , , For the input of the spatial pyramid pooling module Its output It can be represented as:

[0095]

[0096] in, This represents the output of the spatial pyramid pooling module. Indicates the second layer, Indicates the first The output of the layer, Indicates the first The output of the layer, Indicates the second The output of the layer, Indicates the third The output of the layer, This indicates that elements will be merged along the channel dimension, specifically:

[0097]

[0098]

[0099]

[0100]

[0101] in, This represents the input to the spatial pyramid pooling module.

[0102] The components of the lightweight backbone network, from top to bottom, are the convolutional pooling module and the lightweight module. Attention module, lightweight module Lightweight modules Attention module, lightweight module Lightweight modules Attention module, lightweight module The spatial pyramid pooling module records the input of the backbone network as... The input image size is denoted as The backbone network The feature map output by each component is denoted as The size of the feature map is denoted as , ;

[0103] S22. Construct a feature fusion network to perform secondary feature extraction, and fuse it with the effective features extracted from the lightweight backbone network to obtain a feature map; the feature fusion network has fourteen components, including four... Layers, two CARAFE modules, four Concat layers, and four CSP modules; see also Figure 7 The CARAFE module includes an upsampling kernel prediction network and a feature reconstruction network; the upsampling kernel prediction network includes three parts: feature channel compression, content encoding and upsampling kernel prediction, and upsampling kernel normalization; the feature channel compression utilizes... indivual The convolutional kernels compress their channel dimensions, reducing the computational cost of subsequent steps; for the input feature map of the CARAFE module Its dimensions are denoted as The output of feature channel compression is denoted as Its size is The content encoding and upsampling kernel prediction are applied to the compressed input feature map. ,use indivual The convolutional kernel is used to predict the upsampling kernel. This represents the upsampling factor, used to control the sampling size. Indicates the coding factor. The larger the receptive field, the more computation is required. Then, the channel dimension of the feature map is expanded in the spatial dimension to obtain the upsampling kernel; for a size of... Input feature map First, perform upsampling kernel prediction, and denote the output feature map as... Its size is Then, the `nn.functional.pixel_shuffle` function in PyTorch is used to rearrange the elements, expanding the channel dimension of the feature map in the spatial dimension, resulting in a shape of... The upsampling kernel is obtained; the upsampling kernel is normalized by normalizing each channel of the obtained upsampling kernel using softmax, so that the sum of the convolution kernel weights is 1. Using the `unfold` and `reshape` functions in Python, the resulting shape is... The feature map is then processed using the permute function to perform channel transformation, resulting in a shape of... Feature map;

[0104] For each location in the output feature map, the feature reconstruction network maps it back to the input feature map and extracts the region centered at that location. The region is then padded with the upsampled kernel of the predicted point to obtain the output value; for the input feature map of the CARAFE module, it is first padded around its perimeter. The data is divided into 100 pixels, and then processed using the `unfold` and `reshape` functions in Python to obtain a shape of 100 pixels. The feature map is transformed using the permute function to obtain a shape of... The feature map is then multiplied by the output feature map of the upsampled kernel prediction network to obtain a shape of... The feature map is processed using the reshape and permute functions to obtain a shape of... The feature map is then processed using the `nn.functional.pixel_shuffle` function in PyTorch to rearrange the elements, expanding the channel dimension of the feature map in the spatial dimension to obtain a shape of... The upsampling kernel is finally used indivual The convolution kernels are processed to obtain the output of the CARAFE module. The output feature map size is ;

[0105] The CSP module consists of seven It consists of a layer and a Concat layer, for the input feature map of each CSP module. First, input it into two branches. The first branch has five... It is composed of layers connected in series, and the second branch is a single layer. The layer uses a Concat layer to concatenate the results of the two branches along the channel dimension, and then goes through a... Layer, to obtain the output feature map of the CSP module. For the input feature map Its shape is denoted as Each of the two branches All layers use Each convolutional kernel processes the feature map, and the output feature map of both branches has a size of [size missing]. After concatenation using the Concat layer, the feature map size is... Finally, using one The layer processes the concatenated feature maps to obtain the output feature map of the CSP module. The feature map size is , express The number of convolutional kernels in a layer;

[0106] The components of the feature fusion network, from top to bottom, are as follows: Layer, CARAFE module, Concat layer, CSP module, Layer, CARAFE module, Concat layer, CSP module, Layer, Concat layer, CSP module Layer, Concat layer, CSP module, which integrates the feature fusion network The feature map output by each component is denoted as The size of the feature map is denoted as , ;

[0107] S23. Construct a detection head network and input the feature map into the detection head network for target classification and regression.

[0108] Specifically, in this embodiment, see Figure 8 In step S3, the fire lane dataset is iteratively trained on the lightweight fire lane detection model to obtain the optimal model, specifically as follows:

[0109] Initialize parameters, set batch size, read data of batch size and perform equal height and equal ratio transformation, and input the data into the lightweight fire lane detection model for iterative training; specifically: initialize the weights of the lightweight backbone network, feature fusion network and detection head network, set the number of iterations, batch size, learning rate and momentum, randomly read data of the fire lane dataset of batch size, obtain a batch of data samples for this round of training, perform equal height and equal ratio transformation, and input the data into the lightweight fire lane detection model for iterative training;

[0110] The transformed data is then used to extract effective features through a lightweight backbone network; specifically, for any channel C, the size is... Input data First, it is input into the lightweight backbone network, and then passed through the eleven components mentioned above in sequence, making the backbone network the first... The feature map output by each component is denoted as ;

[0111] The first component is the convolutional pooling module, with the input being... Its output feature map The shape is The second component is a lightweight module. The input is Its output feature map The shape is The third component is the attention module, with the input being... Its output feature map The shape is The fourth component is a lightweight module. The input is Its output feature map The shape is The fifth component is a lightweight module. The input is Its output feature map The shape is The sixth component is the attention module, and the input is... Its output feature map The shape is The seventh component is a lightweight module. The input is Its output feature map The shape is The eighth component is a lightweight module. The input is The shape of its output feature map is The ninth component is the attention module, and its input is... Its output feature map The shape is The tenth component is a lightweight module. The input is Its output feature map The shape is The eleventh component is the spatial pyramid pooling module, with the input being... Its output feature map The shape is ;

[0112] The effective features extracted by the lightweight backbone network are input into the feature fusion network for secondary feature extraction to obtain a feature map; specifically: the feature map is... , , The input feature fusion network is passed through the fourteen components mentioned above in sequence. The feature map output by the j-th component of the feature fusion network is denoted as... The feature map size is , The first component is Layer, input is ,

[0113] Its output feature map The shape is The second component is the CARAFE module, whose input is... Its output feature map The shape is The third component is the Concat layer, and its input is... and Its output feature map The shape is The fourth component is the CSP module, and its input is... Its output feature map The shape is The fifth component is Layer, input is Its output feature map The shape is The sixth component is the CARAFE module, with the following input: Its output feature map The shape is The seventh component is the Concat layer, and its input is... and Its output feature map The shape is The eighth component is the CSP module, and its input is... Its output feature map The shape is The ninth component is Layer, input is Its output feature map The shape is The tenth component is the Concat layer, and its input is... and Its output feature map The shape is The eleventh component is the CSP module, and its input is... Its output feature map The shape is The twelfth component is Layer, input is Its output feature map The shape is The thirteenth component is the Concat layer, and its input is... and Its output feature map The shape is The fourteenth component is the CSP module, and its input is... Its output feature map The shape is ;

[0114] Taking any channel C as 3 as an example, this invention does not impose specific limitations, and the specific settings can be made according to needs; the size is... Input data First, input it into the lightweight backbone network, and then input the backbone network's... The feature map output by each component is denoted as The first component is the convolutional pooling module, and the input is...

[0115] Its output feature map The shape is The second component is a lightweight module. The input is Its output feature map The shape is The third component is the attention module, with the input being... Its output feature map The shape is The fourth component is a lightweight module. The input is Its output feature map The shape is The fifth component is a lightweight module. The input is Its output feature map The shape is The sixth component is the attention module, and the input is... Its output feature map The shape is The seventh component is a lightweight module. The input is Its output feature map The shape is The eighth component is a lightweight module. The input is The shape of its output feature map is The ninth component is the attention module, and its input is... Its output feature map The shape is The tenth component is a lightweight module. The input is Its output feature map The shape is The eleventh component is the spatial pyramid pooling module, with the input being... Its output feature map The shape is , feature map , , Input the feature fusion network, and then input the feature fusion network's first... The feature map output by each component is denoted as The size of the feature map is denoted as , The first component is Layer, input is Its output feature map The shape is The second component is the CARAFE module, whose input is... Its output feature map The shape is The third component is the Concat layer, and its input is... and Its output feature map The shape is The fourth component is the CSP module, and its input is... Its output feature map The shape is The fifth component is Layer, input is Its output feature map The shape is The sixth component is the CARAFE module, with the following input: Its output feature map The shape is The seventh component is the Concat layer, and its input is... and Its output feature map The shape is The eighth component is the CSP module, and its input is... Its output feature map The shape is The ninth component is Layer, input is Its output feature map The shape is The tenth component is the Concat layer, and its input is... and Its output feature map The shape is The eleventh component is the CSP module, and its input is... Its output feature map The shape is The twelfth component is Layer, input is Its output feature map The shape is The thirteenth component is the Concat layer, and its input is... and Its output feature map The shape is The fourteenth component is the CSP module, and its input is... Its output feature map The shape is ;

[0116] The feature map is input into the detection head network for target classification and regression; specifically, the feature map is... , , The input is fed into the detection head network for target classification and regression, and the prediction results are output. For details on the changes in various metrics during model training, please refer to [link to relevant documentation]. Figure 9 The horizontal axis represents the number of training iterations;

[0117] A loss function is constructed, and backpropagation is used to adjust the model network parameters to obtain the optimal model. Based on the prediction results and ground truth labels, the loss is calculated using the loss function. Backpropagation is then performed based on the loss to update the weights, specifically the weights of the lightweight backbone network, feature fusion network, and detector head network, thus obtaining the optimal model. The PR curve of the optimal model is shown below. Figure 10As shown, the original YOLOv5M model has a mean average precision of 0.904, a model size of 42.2 MB, 20.86 million parameters, and a computational cost of 47.9 million GFLOPs. The model described in this invention has a mean average precision of 0.911, a model size of 20.3 MB, 9.67 million parameters, and a computational cost of 19.2 million GFLOPs. Compared with the original YOLOv5M model, the mean average precision is improved by 0.77%, the number of parameters is reduced by 53.6%, and the number of parameters is reduced by 59.9%.

[0118] The loss function is:

[0119]

[0120] in, For loss function, The number of detection layers, For bounding box regression loss, The weights of the bounding box regression loss are... For the loss of the target object, The weights for the loss of the target object. For classifying losses, The weights for the classification loss;

[0121] The bounding box regression loss uses the SIoU loss function, which includes intersection-over-union (IoU) loss, angle loss, distance loss, and shape loss. The SIoU loss function is as follows:

[0122]

[0123] Where IoU represents the crossover ratio loss, Indicates distance loss. Indicates shape loss;

[0124] The cross-union ratio loss is:

[0125]

[0126] in, , These represent the prediction box and the truth box, respectively.

[0127] The angle loss is:

[0128]

[0129] in, express and The ratio of the distance to the center point's ordinate to the total distance to the center point:

[0130]

[0131]

[0132]

[0133] in, express and The distance from the center point's ordinate. express and Distance from the center point express and The angle between the line connecting the center points and the horizontal axis. express and The angle formed by the line connecting the center points and the longitudinal axis. , This indicates the coordinates of the center point of the truth box. , Indicates the coordinates of the center point of the prediction box;

[0134] The distance loss is:

[0135]

[0136] in, Represents the distance metric on the horizontal axis. Represents the distance metric on the ordinate. Indicates the angle measurement factor;

[0137]

[0138]

[0139]

[0140] in, Indicated by and A rectangle with its center point as the vertex. Represents a rectangle width, Represents a rectangle of high, , This indicates the coordinates of the center point of the truth box. , This indicates the coordinates of the center point of the prediction box. Indicates angle loss;

[0141] The shape loss is:

[0142]

[0143] in, This represents the width measurement value. This represents a height measurement value. This represents the shape loss factor, used to measure the importance of shape loss.

[0144]

[0145]

[0146] in, , This represents the width and height of the truth box. , Indicates the width and height of the prediction box;

[0147] Specifically, in this embodiment, after iteratively training the fire lane dataset in the lightweight fire lane detection model to obtain the optimal model in step S3, the method further includes: converting the weight file of the optimal model and deploying the optimal model to the edge. Specifically, this involves designing a custom operator to convert the .pt format weight file generated by the optimal model into a .onnx format weight file. During the conversion process, operations such as operator merging are performed to speed up the model. Then, the converted optimal model is deployed to the edge.

[0148] Specifically, in this embodiment, step S3, which outputs the fire lane detection results in real time through the optimal model, involves: acquiring several camera stream data of the fire lanes in the industrial park, parsing the data protocol, and capturing frame-by-frame image data; for each input of the model, multiplying the image data with the mask corresponding to the camera to eliminate non-fire lane areas; then performing channel conversion, cropping, and normalization on the image data; finally, sending the data into the inference engine for forward inference, obtaining and decoding the inference results, outputting the fire lane detection results through the optimal model, and storing the detection results locally.

[0149] Specifically, in this embodiment, step S4 establishes an alarm mechanism and issues an early warning based on the detection results, specifically as follows:

[0150] S41. Initialize the alarm mechanism status; set the initial state of the alarm mechanism to off. If the counter is greater than the set target threshold or the alarm mechanism is in the on state, then activate the alarm mechanism.

[0151] S42. Set the minimum area and minimum side length thresholds for obstacles, and perform obstacle area threshold discrimination; if the model detects that the current frame contains obstacles, calculate the obstacle area in the model output result one by one. If the obstacle area is greater than the set minimum area threshold and the shorter side is greater than the minimum side length threshold, then retain the obstacle.

[0152] S43. Set the minimum overlap area threshold for obstacles and perform obstacle confidence judgment; calculate the overlap area of ​​the current frame obstacle saved in step S42 with the detection results of multiple frames saved in the previous minute, retain the maximum overlap area in each frame result, calculate the average overlap ratio, and if the average overlap area is greater than the minimum overlap area threshold, then retain the obstacle.

[0153] S44. If the detection result shows an obstacle, an alarm is issued; calculate the proportion of the obstacle in the whole picture, and classify the blockage status of the fire lane according to the proportion. The blockage status includes severe blockage, moderate blockage and slight blockage.

[0154] The above are merely preferred embodiments of the present invention and do not limit the patent scope of the present invention. All equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.

Claims

1. A fire escape detection method based on a lightweight architecture, characterized in that, Includes the following steps: S1. Obtain fire lane data, annotate it, and construct a fire lane dataset; S2. Construct a lightweight fire lane detection model; specifically: S21. Construct a lightweight backbone network and perform effective feature extraction; The lightweight backbone network consists of 11 components, including cascaded convolutional pooling modules and lightweight modules. Attention module, lightweight module Lightweight modules Attention module, lightweight module Lightweight modules Attention module, lightweight module Spatial pyramid pooling module; The lightweight module , By submodule Composition; Lightweight modules It includes two branches, branch one consists of one With one composition, This is a layer combining convolution, batch normalization, and ReLU activation function. This is a combination layer of depthwise convolution and batch normalization; The submodule It includes two branches: branch one is a short connection that performs an identity mapping, and branch two consists of two... With one The composition, the layer structure of branch two is as follows: , , ; The attention module includes a spatial attention module and a channel attention module; S22. Construct a feature fusion network, perform secondary feature extraction, and fuse it with the effective features extracted from the lightweight backbone network to obtain a feature map. The feature fusion network has 14 components, including those cascaded sequentially. Layer, CARAFE module, Concat layer, CSP module, Layer, CARAFE module, Concat layer, CSP module, Layer, Concat layer, CSP module Layer, Concat layer, CSP module; The CARAFE module includes an upsampling kernel prediction network and a feature recombination network; The CSP module consists of seven It consists of a layer and a Concat layer; The lightweight backbone network The feature map output by each component is denoted as Let the feature map output by the j-th component of the feature fusion network be denoted as . The fourth component of the lightweight backbone network outputs a feature map. The seventh component outputs a feature map. The eleventh component outputs a feature map. The feature map , , The seventh, third, and first components of the feature fusion network are input respectively for feature fusion, and the eighth component of the feature fusion network outputs a feature map. The eleventh component outputs a feature map. The fourteenth component outputs a feature map. ; S23. Construct a detection head network and integrate the feature maps. , , The data is input into the detection head network for target classification and regression. S3. Iteratively train the fire lane dataset on the lightweight fire lane detection model to obtain the optimal model, and output the fire lane detection results in real time through the optimal model. In step S3, the fire lane dataset is iteratively trained on the lightweight fire lane detection model to obtain the optimal model, specifically as follows: Initialize parameters, set batch size, read batch size data, perform equal height and equal ratio transformation, and input the data into the lightweight fire lane detection model for iterative training; The transformed data is then processed through a lightweight backbone network to extract effective features. The effective features extracted by the lightweight backbone network are input into the feature fusion network for secondary feature extraction to obtain the feature map. The feature map is input into the detection head network for target classification and regression. Construct a loss function, and use backpropagation to adjust the model network parameters using the loss function to obtain the optimal model; S4. Establish an alarm mechanism and issue warnings based on the detection results; Specifically: S41. Initialize the alarm mechanism status; S42. Set the minimum area and minimum side length thresholds for obstacles, and perform obstacle area threshold discrimination; S43. Set the minimum overlap area threshold for obstacles and perform obstacle confidence judgment; S44. If the detection result indicates the presence of an obstacle, an alarm will be issued.

2. The fire escape detection method based on a lightweight architecture according to claim 1, characterized in that, Step S1 specifically includes: S11. Acquire monitoring data of fire escape routes from several cameras, and extract key frames and filter the monitoring data. S12. Set a mask that matches each camera individually to obtain the mask image; S13. Use annotation tools to annotate the masked image to construct a fire lane dataset.

3. A fire escape detection method based on a lightweight architecture according to any one of claims 1-2, characterized in that, The loss function is: in, For loss function, The number of detection layers, For bounding box regression loss, The weights of the bounding box regression loss are... For the loss of the target object, The weights for the loss of the target object. For classifying losses, The weights are used for the classification loss.

4. A fire escape detection method based on a lightweight architecture according to any one of claims 1-2, characterized in that, After iteratively training the fire lane dataset in the lightweight fire lane detection model to obtain the optimal model in step S3, the method further includes: The optimal model weight file is converted, and the optimal model is deployed to the edge.

5. The fire escape detection method based on a lightweight architecture according to claim 4, characterized in that, The optimal model weight file is converted and then deployed to the edge. Specifically, a custom operator is designed to convert the .pt format weight file generated by the optimal model into a .onnx format weight file, and the converted optimal model is deployed to the edge.

6. A fire escape detection method based on a lightweight architecture according to any one of claims 1-2, characterized in that, In step S3, the fire lane detection results are output in real time using the optimal model, specifically as follows: Acquire data streams from several cameras in the fire escape route, perform data protocol parsing, and capture frame-by-frame image data; Multiply the image data with the corresponding mask of the camera to eliminate non-fire passage areas; The image data is then subjected to channel conversion, cropping, and normalization. The optimal model is used to output the fire lane detection results, and the detection results are stored locally.

7. A fire escape detection method based on a lightweight architecture according to any one of claims 1-2, characterized in that, If the detection result indicates the presence of an obstacle, step S44, after issuing an alarm, further includes: The proportion of obstacles in the entire map is calculated, and the blockage status of fire lanes is prioritized according to the proportion. The blockage status includes severe blockage, moderate blockage, and slight blockage.

Images