Fire monitoring method and device based on improved Cascade RCNN model, medium and equipment

Abstract

The application discloses a fire monitoring method and device based on an improved Cascade RCNN model, a medium and equipment, wherein the fire monitoring method comprises the following steps: acquiring a picture of a monitoring area; based on the improved Cascade RCNN model, picture features of the picture are extracted, and whether the monitoring area is on fire and the severity are determined according to the picture features. The application can extract picture features of the picture, and determine whether the monitoring area is on fire and the severity according to the picture features.

Description

Technical Field

[0001] This invention relates to a fire monitoring method, apparatus, medium, and equipment based on an improved Cascade RCNN model, belonging to the fields of computer vision and image processing technology. Background Technology

[0002] Object detection, as one of the fundamental problems in computer vision, has wide applications in daily life, including face detection, anomaly monitoring, industrial inspection, text detection, remote sensing detection, and pedestrian detection. Fire monitoring in industrial parks is a crucial component of smart parks, playing a significant role in promoting the park's intelligence and informatization, and greatly protecting public property and the safety of people's lives.

[0003] Existing object detection methods are mainly divided into traditional methods and deep learning-based methods. Traditional methods rely on carefully designed handcrafted features, which are not robust and lack flexibility.

[0004] With the advent of deep learning, object detection has gradually entered a new stage. Deep learning-based object detection has been widely applied. However, while these object detection methods perform well for objects with relatively regular shapes, they struggle to maintain excellent performance when the object's shape is varied. For fire detection, the shapes of fire and smoke are uncertain, and smoke also has a certain degree of transparency, easily blending with the background, which increases the difficulty of fire detection.

[0005] Therefore, this application proposes a fire monitoring method, device, medium, and equipment based on an improved Cascade RCNN model. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the prior art and provide a fire monitoring method, device, medium and equipment based on an improved Cascade RCNN model, which can determine whether a fire has occurred in the monitored area and the severity of the fire.

[0007] To achieve the above objectives, the present invention is implemented using the following technical solution:

[0008] On the one hand, this invention provides a fire monitoring method based on an improved Cascade RCNN model, comprising:

[0009] Obtain images of the monitored area;

[0010] Based on the improved Cascade RCNN model, image features are extracted from the images, and the presence and severity of fire in the monitored area are determined according to these features.

[0011] Furthermore, the improved Cascade RCNN model includes a backbone network, a feature pyramid network, a region proposal network, and a global semantic detection head network arranged sequentially.

[0012] The backbone network extracts image features from the images;

[0013] The feature pyramid network transforms image features into multi-scale image features;

[0014] The region proposal network determines the feature regions of objects with varying shapes in the image based on multi-scale image features;

[0015] The global semantic detection head network determines the bounding box coordinates and category of the shape-variable object based on the feature region.

[0016] Furthermore, the backbone network includes ResNeXt-101, which includes four stages arranged in sequence. The latter three stages are equipped with improved deformable convolutions to enhance the backbone network's ability to capture the shape of objects with varied forms.

[0017] The improved deformable convolution uses grouped convolution to divide image features into multiple groups, and normalizes the image features of each group according to preset depth-aware modulation factors and sampling point-aware modulation factors.

[0018] Furthermore, the global semantic detection head network includes multiple cascaded global semantic detection heads;

[0019] Multiple global semantic detection heads include a first global semantic detection head, at least one intermediate global semantic detection head, and a last global semantic detection head set in sequence;

[0020] The input of the first global semantic detection head is connected to the region proposal network. The output of the first global semantic detection head is connected to the input of the middle global semantic detection head. The output of the middle global semantic detection head is connected to the input of the last global semantic detection head. The output of the last global semantic detection head is connected to the output of the improved Cascade RCNN model.

[0021] Furthermore, each global semantic detection head includes a sequentially configured ROI Pooling module, a detection head, and a global context module;

[0022] The global context module includes a 3×3 convolutional layer, a pooling layer, and a normalization layer arranged in sequence;

[0023] The 3×3 convolutional layer reduces the channels in the feature region;

[0024] The pooling layer includes a parallel max pooling layer and a mean pooling layer, wherein: the max pooling layer extracts local information of the feature region, and the mean pooling layer extracts global information of the feature region;

[0025] The normalization layer is based on channel-wise multiplication, which uses extracted local and global information to improve the feature quality of the feature region and determine the rectangular coordinates of the feature region and the type of objects with varied shapes in the feature region.

[0026] Furthermore, the step of extracting image features from the image based on the improved Cascade R-CNN model and determining whether the monitored area is on fire and the severity of the fire based on these features includes:

[0027] Image features are extracted from images using the backbone network of an improved Cascade RCNN model;

[0028] The feature pyramid network of the improved Cascade RCNN model is used to transform image features into multi-scale image features;

[0029] The improved Cascade RCNN model's region proposal network is used to determine the feature regions of objects with varying shapes in images based on multi-scale image features;

[0030] A global semantic detection head network using an improved Cascade RCNN model determines the bounding box coordinates and category of morphologically variable objects based on feature regions.

[0031] The extent of the fire in the monitored area and its severity are determined by the coordinates of the rectangular frame of the object with the variable shape and the category of the object.

[0032] Furthermore, the category includes empty sets, or smoke and / or fire;

[0033] The rectangular frame coordinates include the empty set, or the rectangular frame coordinates of smoke and / or the rectangular frame coordinates of fire. Each rectangular frame coordinate includes the horizontal and vertical coordinates of the upper left corner of the rectangle, the length of the rectangle, and the width of the rectangle.

[0034] The process of determining whether a fire has broken out in the monitored area and its severity based on the coordinates of the rectangular frame of the object with variable shape and the category of the object with variable shape includes:

[0035] If the category is an empty set, it is determined that there is no fire in the monitored area;

[0036] Furthermore, if the category only includes smoke, it is determined that there is a fire hazard in the monitored area, and an alarm signal is issued;

[0037] Furthermore, in response to categories including smoke and fire, a fire is identified in the monitored area, and the area of ​​the fire's rectangular frame is determined based on the coordinates of the fire's rectangular frame. The ratio of the smoke's rectangular frame area to the image area is then calculated, and the ratio is compared with a preset threshold: when the ratio is greater than the preset threshold, the fire in the monitored area is determined to be spreading; when the ratio is less than or equal to the preset threshold, the fire in the monitored area is determined to be severe.

[0038] On the other hand, the present invention provides a campus fire monitoring device based on an improved Cascade RCNN model, comprising:

[0039] The acquisition module is used to acquire images of the monitored area;

[0040] The monitoring module is used to extract image features from images based on an improved Cascade RCNN model, and to determine whether there is a fire in the monitored area and the severity of the fire based on the image features.

[0041] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that, when the computer program is executed by a processor, it implements the above-mentioned fire monitoring method based on the improved Cascade RCNN model.

[0042] On the other hand, the present invention provides a device characterized in that it comprises:

[0043] Memory, used to store instructions;

[0044] A processor is configured to execute the instructions, causing the device to perform operations that implement the fire monitoring method based on the improved Cascade RCNN model described above.

[0045] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0046] This invention improves the Cascade RCNN model to enhance the ability to recognize and capture objects with varied shapes, enabling the correct capture and identification of fire and smoke that do not have a specific structure, thereby achieving fire detection. Attached Figure Description

[0047] Figure 1 The diagram shown is a flowchart of an embodiment of the fire monitoring method based on the improved Cascade RCNN model of the present invention.

[0048] Figure 2 The diagram shown is a schematic representation of an embodiment of the improved Cascade RCNN model of this invention.

[0049] Figure 3 The diagram shown is a schematic representation of an embodiment of the backbone network of the present invention.

[0050] Figure 4 The diagram shown is a structural schematic of one embodiment of the global semantic detection head of the present invention;

[0051] Figure 5 The diagram shown is a structural schematic of one embodiment of the global context module of the present invention;

[0052] Figure 6 The diagram shown is a structural schematic of one embodiment of the fire monitoring device for industrial parks according to the present invention. Detailed Implementation

[0053] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0054] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0055] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0056] Example 1

[0057] This embodiment introduces a fire monitoring method based on an improved Cascade RCNN model.

[0058] The fire monitoring method in this embodiment includes:

[0059] Obtain images of the monitored area;

[0060] Based on the improved Cascade RCNN model, image features are extracted from the images, and the presence and severity of fire in the monitored area are determined according to these features.

[0061] This invention improves the Cascade RCNN model to enhance the ability to recognize and capture objects with varied shapes, enabling the correct capture and identification of fire and smoke that do not have a specific structure, thereby achieving fire detection.

[0062] Example 2

[0063] Based on Example 1, this example details a fire monitoring method based on an improved Cascade RCNN model.

[0064] The improved Cascade RCNN model in this embodiment includes a backbone network, a feature pyramid network, a region proposal network, and a global semantic detection head network arranged sequentially.

[0065] In application, the backbone network extracts image features; the feature pyramid network converts image features into multi-scale image features; the region proposal network determines the feature regions of objects with varying shapes in the image based on the multi-scale image features; and the global semantic detection head network determines the bounding box coordinates and category of the objects with varying shapes based on the feature regions.

[0066] The backbone network in this embodiment includes ResNeXt-101, which includes four stages arranged in sequence. The last three stages are equipped with improved deformable convolutions to enhance the backbone network's ability to capture the shape of objects with varied forms.

[0067] In application, the improved deformable convolution includes DCN V3. The improved deformable convolution uses grouped convolution to divide image features into multiple groups, and normalizes the image features of each group according to preset depth-aware modulation factors and sampling point-aware modulation factors.

[0068] In practical applications, the shared projection of the depth-wise and point-wise modulation factors is used as the convolution weights of the improved deformable convolution.

[0069] The feature pyramid network and region proposal network structure settings in this embodiment are consistent with those of the existing Cascade RCNN model.

[0070] The global semantic detection head network in this embodiment includes multiple cascaded global semantic detection heads, which can progressively improve the feature quality of feature regions to improve the accuracy of monitoring results.

[0071] When applied, multiple global semantic detection heads include a first global semantic detection head, at least one intermediate global semantic detection head, and a last global semantic detection head set in sequence.

[0072] The input of the first global semantic detection head is connected to the region proposal network. The output of the first global semantic detection head is connected to the input of the middle global semantic detection head. The output of the middle global semantic detection head is connected to the input of the last global semantic detection head. The output of the last global semantic detection head is connected to the output of the improved Cascade RCNN model.

[0073] Each global semantic detection head in this embodiment includes a sequentially arranged ROI Pooling module, a detection head, and a global context module. The ROI Pooling module performs the ROI Pooling operation found in existing Cascade R-CNN models. Furthermore, the structure of the detection head is consistent with that of the detection head in existing Cascade R-CNN models.

[0074] The global context module in this embodiment includes a 3×3 convolutional layer, a pooling layer, and a normalization layer arranged sequentially.

[0075] In application, 3×3 convolutional layers reduce the number of channels in the feature regions. Pooling layers include parallel max-pooling and mean-pooling layers, where max-pooling extracts local information from the feature regions, and mean-pooling extracts global information. Normalization layers, based on channel-wise multiplication, utilize the extracted local and global information to improve the feature quality of the feature regions.

[0076] This embodiment is based on an improved Cascade RCNN model to extract image features from the image and determine whether the monitored area is on fire and the severity of the fire based on the image features.

[0077] Specifically, the following steps are included:

[0078] S21 uses the backbone network of an improved Cascade RCNN model to extract image features from images.

[0079] S22 utilizes a feature pyramid network of an improved Cascade RCNN model to transform image features into multi-scale image features.

[0080] S23 utilizes the improved Cascade RCNN model's region proposal network to determine the feature regions of objects with varying shapes in an image based on multi-scale image features.

[0081] S24 utilizes a global semantic detection head network based on an improved Cascade RCNN model to determine the bounding box coordinates and category of morphologically variable objects according to feature regions.

[0082] S25 determines whether a fire has broken out in the monitored area and the severity of the fire based on the rectangular coordinates of the object with a variable shape and the category of the object with a variable shape.

[0083] When applying the application, the categories include an empty set, or smoke and / or fire. The bounding box coordinates include the bounding box coordinates of the empty set, or smoke, and / or fire. Each bounding box coordinate includes the x and y coordinates of the top-left corner of the rectangle, the length of the rectangle, and the width of the rectangle.

[0084] In practical applications, step S25 specifically includes the following steps:

[0085] S251 responds to an empty set as a response, determining that there is no fire in the monitored area;

[0086] S252 responds to the fact that the category only includes smoke, then determines that there is a fire hazard in the monitored area and issues an alarm signal;

[0087] S253 responds to categories including smoke and fire, determines that a fire has occurred in the monitored area, and determines the area of ​​the fire's rectangular frame based on the coordinates of the fire's rectangular frame. It then calculates the ratio of the smoke's rectangular frame area to the image area and compares the ratio with a preset threshold: when the ratio is greater than the preset threshold, it determines that the fire in the monitored area is spreading; when the ratio is less than or equal to the preset threshold, it determines that the fire in the monitored area is severe.

[0088] This invention fully mines the feature information of images by introducing global context and adding contextual semantics to each local information; it improves the model's ability to recognize and capture objects with varied shapes in images by improving deformable convolution; and it improves the quality of image features by improving the Cascade RCNN model, thus obtaining more accurate monitoring results.

[0089] Example 3

[0090] Based on Example 1 or 2, this example details a fire monitoring method based on an improved Cascade RCNN model for monitoring fire conditions in a park.

[0091] The fire monitoring method in this embodiment includes:

[0092] Step a: Acquisition and preprocessing of images of fires in the park.

[0093] Images and information about the fire in the park were collected from the internet, and the LabelImg tool was used to annotate the images, specifically labeling the coordinates of the rectangles and the type of fire and smoke in the images.

[0094] Step b: Use data augmentation to expand the sample of images of fires in the park.

[0095] By applying random flipping, random lighting changes, and color jitter to the labeled images to expand the image samples, we can not only increase the diversity and quantity of image samples, but also effectively suppress overfitting during model training and effectively improve the performance of the model.

[0096] In addition, images of fire and smoke with transparent backgrounds are randomly copied to random positions within the augmented image to obtain new annotations and images.

[0097] Step c: Build the improved Cascade RCNN model.

[0098] To improve the accuracy of fire detection, the Cascade R-CNN model was improved. This mainly includes improvements to the backbone network and the addition of global context to ROI features.

[0099] The fire detection network model includes a backbone network, a feature pyramid network, a region proposal network, and a global semantic detection head network.

[0100] refer to Figure 3 The backbone network uses ResNeXt-101, which is combined with improved deformable convolution to enhance the network model's ability to model targets with varied shapes.

[0101] ResNeXt-101 comprises four stages, and the improved deformable convolution in this embodiment is applied only to the last three stages. To reduce the computational complexity of deformable convolution, this embodiment introduces depth-separable convolution, adjusting the original convolution weights to shared projection weights of depth-wise and point-wise modulation factors. To learn richer information from the subspaces of each channel of the image features, group convolution (Group Conv) is applied to divide the image features into 32 groups by channel, each group having different sampling and scale offsets. Furthermore, to make the improved deformable convolution more stable during training, softmax normalization is performed point-by-point.

[0102] This embodiment combines ResNeXt-101 with improved deformable convolution, which can enhance the model's ability to capture the shape of fire and smoke.

[0103] The feature pyramid network in this embodiment extracts multi-scale feature maps of images, which can improve the model's ability to model multi-scale image features.

[0104] In this embodiment, the region suggestion network extracts feature regions from images. The feature regions are rectangular boxes laid over multi-scale image features.

[0105] In this embodiment, the feature pyramid network and region proposal network directly apply the structure of the feature pyramid network and region proposal network in the existing technology Cascade RCNN, and input the extracted feature regions into the global semantic detection head network.

[0106] The global semantic detection head network in this embodiment is a cascaded architecture. It improves the accuracy of the final monitoring results by continuously improving the quality of feature regions. (Refer to...) Figure 4 The global semantic detection head network in this embodiment includes three cascaded global semantic detection heads.

[0107] Each global semantic detection head adds a global context module (GC) to the detection head to obtain the GC-ROI-Head structure.

[0108] refer to Figure 5 The Global Context (GC) module first sets up a 3×3 convolutional layer to reduce the number of channels in the feature region by half. Then, it sets up max pooling and average pooling layers respectively. The max pooling layer extracts local information of the feature region, while the average pooling layer extracts global information of the feature region. That is, connecting at the channel dimension can simultaneously take into account both local and global information. Finally, after passing through a sigmoid function, a channel-wise multiplication is performed with the feature region. The output signals of the max pooling and average pooling layers are normalized to improve the feature quality of the feature region.

[0109] When applying, refer to Figure 4 B0 is the feature region output by the Region Proposal Network (RPN), H1, H2, and H3 are detection heads, and Pool is the ROI Pooling module, which performs the ROI Pooling operation found in Cascade RCNN. First, feature region B0 undergoes ROI Pooling. Then, feature region B0 is fed into the GC-ROI-Head1 structure to obtain the category C1 and the bounding box coordinates B1 after feature region regression. Next, the obtained feature region B1 is fed into the second global semantic detection head to obtain the category C2 and the bounding box coordinates B2 after feature region regression. Finally, the obtained feature region B2 is fed into the third global semantic detection head to obtain the final category C3 and the bounding box coordinates B3.

[0110] In practical applications, during training, positive samples fed into GC-ROI-Head1 are defined as those where the IOU threshold between all feature regions in B0 and the labeled ground truth is greater than 0.5; similarly, positive samples fed into GC-ROI-Head2 are defined as those where the IOU threshold between all feature regions in B1 and the labeled ground truth is greater than 0.7; and positive samples fed into GC-ROI-Head3 are defined as those where the IOU threshold between all feature regions in B2 and the labeled ground truth is greater than 0.8. By gradually increasing the IOU threshold of each GC-ROI-Head, the feature quality of the positive samples is continuously improved.

[0111] Step d: Improve the training of the Cascade RCNN model.

[0112] This example uses PyTorch to build an improved Cascade R-CNN model. The model's loss function mainly consists of two parts: classification loss and regression loss. The classification loss uses the cross-entropy loss function, and the regression loss uses the smoothL1 loss function, as detailed below:

[0113] L cls = -plogq-(1-p)log(1-q)

[0114]

[0115]

[0116]

[0117] Among them, L cls For classification loss, L reg For regression loss, x is an intermediate parameter, p is the true class, q is the model's predicted class, and log refers to the logarithmic operation. v = (v x ,v y ,v w ,v h ) represents the coordinates of the actual rectangle, t u =(t u x ,t u y ,t u w ,t u h ) represents the coordinates of the predicted rectangle, where x is the x-coordinate of the top-left corner of the rectangle, y is the y-coordinate of the top-left corner, w is the width of the rectangle, h is the height of the rectangle, and the top-left corner of the image is the origin.

[0118] In this embodiment, the improved total loss function of the Cascade RCNN model is as follows:

[0119] L = L cls1 +L reg1 +L cls2 +L reg2 +L cls3 +L reg3

[0120] Where L is the total loss function, L cls1 L cls2 L cls3 Let L be the classification loss function corresponding to the three global semantic detection heads respectively. reg1 L reg2 L reg3 The regression loss corresponds to the three global semantic detection heads respectively.

[0121] In application, images of historical fires are input into the improved Cascade R-CNN model for training. The training process mainly consists of forward propagation, loss calculation, gradient backpropagation, and parameter update. This process is repeated until the loss converges to a small value and no longer changes, at which point the corresponding model weights are saved.

[0122] In this embodiment, the obtained weights are reloaded, and a weighted average of the model weights is performed, which is a secondary training and optimization. In the optimization process of deep learning, the model weights will fluctuate at the actual optimal point in the last n steps, so taking the average of the last n steps can make the model more robust.

[0123] When applying the model, the fire detection network model in the park is first initialized with model weights, and then fine-tuned for 12 epochs. One fine-tuned weight is saved for each epoch, thus obtaining 12 fine-tuned weights.

[0124] In practical applications, the learning rate (lr) for each epoch will vary from its maximum to its minimum according to a cosine function, as shown in the following formula:

[0125] lr=lr end +0.5*(lr start -lr end )*[cos(π*factor)+1]

[0126]

[0127] Among them, lr start The initial learning rate, lr endThe termination learning rate in this embodiment is 0.01 times the initial learning rate. `factor` is a scaling factor, which in this embodiment ranges from [0, 1) and changes continuously with each epoch. `total_step` is the current cumulative number of iterations, and `step` is the number of iterations in the current epoch process, ranging from [0, total_step].

[0128] In this embodiment, after fine-tuning for 12 epochs, the 12 fine-tuning weights are added together and divided by 12 to obtain the final weights, which are then saved.

[0129] Step e: Installation and deployment of equipment in the park to be monitored for fire.

[0130] Install cameras, gateway servers, edge embedded devices, and a main control computer in the park where fire monitoring is required, as per [reference]. Figure 6 Different cameras send image streams to the gateway server. Edge devices receive the image data from the gateway server and process it using an improved Cascade R-CNN model. Furthermore, the gateway server collects the processing results from all the edge embedded devices and sends them to the main control computer.

[0131] Step f: Obtaining fire detection results in the park.

[0132] To accelerate and improve the inference of the Cascade R-CNN model, TensorRT was used to transform the model, and INT8 quantization was used. After quantization, the image to be detected from the camera was input into the trained campus fire detection model and deployed to an edge embedded device to monitor campus fires.

[0133] This invention provides a fire detection method based on an improved Cascade RCNN model. The model is quantized and deployed to embedded devices, which improves operating efficiency and reduces deployment costs.

[0134] Example 4

[0135] This embodiment introduces a fire monitoring device based on an improved Cascade RCNN model.

[0136] The fire monitoring device based on the improved Cascade RCNN model in this embodiment includes:

[0137] The acquisition module is used to acquire images of the monitored area;

[0138] The monitoring module is used to extract image features from images based on an improved Cascade RCNN model, and to determine whether there is a fire in the monitored area and the severity of the fire based on the image features.

[0139] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0140] Example 5

[0141] This embodiment describes a computer-readable storage medium storing a computer program that, when executed by a processor, implements a fire monitoring method based on an improved Cascade RCNN model as described in any of Embodiments 1-3.

[0142] Example 6

[0143] This embodiment describes a device.

[0144] The device in this embodiment includes:

[0145] Memory, used to store instructions;

[0146] A processor is configured to execute the instructions, causing the device to perform operations implementing the fire monitoring method based on the improved Cascade RCNN model as described in any of Embodiments 1-3.

[0147] As can be seen from the above embodiments, due to the uncertainty of the shape of flames and smoke, and the fact that smoke also has a certain degree of transparency, it is easy to mix with the background, which makes the existing detection technology unable to guarantee high detection accuracy. This invention improves the ability of this application to capture, recognize and model objects with variable shapes by combining an improved backbone network with an improved deformable convolution.

[0148] This invention adds a global context module to the improved Cascade RCNN model, providing contextual information for each feature region. By continuously improving the feature quality of the feature regions through cascaded global semantic detection heads, more accurate monitoring results are ultimately obtained.

[0149] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0151] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0152] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0153] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A fire monitoring method based on an improved Cascade RCNN model, characterized in that, include: Obtain images of the monitored area; Based on the improved Cascade R-CNN model, image features are extracted from the images, and the presence and severity of a fire in the monitored area are determined based on these features. The determination of the presence and severity of a fire in the monitored area based on image features includes: Image features are extracted using the backbone network of an improved Cascade R-CNN model. The backbone network includes ResNeXt-101, which comprises four sequentially arranged stages. The latter three stages each have improved deformable convolutions to enhance the backbone network's ability to capture the shapes of objects with varying forms. The improved deformable convolutions apply grouped convolutions to divide the image features into multiple groups, and normalize the image features of each group according to preset depth-aware modulation factors and sampling point-aware modulation factors. The feature pyramid network of the improved Cascade RCNN model is used to transform image features into multi-scale image features; The improved Cascade RCNN model's region proposal network is used to determine the feature regions of objects with varying shapes in images based on multi-scale image features; A global semantic detection head network, utilizing an improved Cascade R-CNN model, determines the bounding box coordinates and category of shape-variable objects based on feature regions. This network comprises multiple cascaded global semantic detection heads, each consisting of a sequentially arranged ROI Pooling module, a detection head, and a global context module. The global context module includes sequentially arranged 3×3 convolutional layers, pooling layers, and normalization layers. The 3×3 convolutional layers reduce the number of channels in the feature regions. The pooling layers include parallel max-pooling and mean-pooling layers, where max-pooling extracts local information from the feature regions, and mean-pooling extracts global information. The normalization layer, based on channel-wise multiplication, utilizes the extracted local and global information to improve the feature quality of the feature regions and determine the bounding box coordinates and the type of shape-variable objects within them. The presence and severity of a fire in the monitored area are determined based on the rectangular coordinates of the object with a variable shape and the category of the object with a variable shape. The categories include empty sets, or smoke and / or fire. The rectangular frame coordinates include the empty set, or the rectangular frame coordinates of smoke and / or the rectangular frame coordinates of fire. Each rectangular frame coordinate includes the horizontal and vertical coordinates of the upper left corner of the rectangle, the length of the rectangle, and the width of the rectangle. The process of determining whether a fire has broken out in the monitored area and its severity based on the coordinates of the rectangular frame of the object with variable shape and the category of the object with variable shape includes: If the category is an empty set, it is determined that there is no fire in the monitored area; And / or, in response to a category that includes only smoke, an alarm signal is issued to determine that there is a fire hazard in the monitored area; And / or, in response to categories including smoke and fire, determine that a fire has broken out in the monitored area, and determine the area of ​​the fire's rectangular frame based on the coordinates of the fire's rectangular frame, to calculate the ratio of the area of ​​the smoke's rectangular frame to the area of ​​the image, and compare the ratio with a preset threshold: when the ratio is greater than the preset threshold, determine that the fire in the monitored area is spreading, and when the ratio is less than or equal to the preset threshold, determine that the fire in the monitored area is severe.

2. The fire monitoring method based on the improved Cascade RCNN model according to claim 1, characterized in that, The improved Cascade RCNN model includes a backbone network, a feature pyramid network, a region proposal network, and a global semantic detection head network arranged sequentially. The backbone network extracts image features from the images; The feature pyramid network transforms image features into multi-scale image features; The region proposal network determines the feature regions of objects with varying shapes in the image based on multi-scale image features; The global semantic detection head network determines the bounding box coordinates and category of the shape-variable object based on the feature region.

3. The fire monitoring method based on the improved Cascade RCNN model according to claim 2, characterized in that, Multiple global semantic detection heads include a first global semantic detection head, at least one intermediate global semantic detection head, and a last global semantic detection head set in sequence; The input of the first global semantic detection head is connected to the region proposal network. The output of the first global semantic detection head is connected to the input of the middle global semantic detection head. The output of the middle global semantic detection head is connected to the input of the last global semantic detection head. The output of the last global semantic detection head is connected to the output of the improved Cascade RCNN model.

4. A campus fire monitoring device based on an improved Cascade RCNN model, characterized in that, include: The acquisition module is used to acquire images of the monitored area; The monitoring module is used to extract image features from images based on an improved Cascade R-CNN model, and determine whether a fire has occurred in the monitored area and its severity based on these features. The determination of whether a fire has occurred and its severity based on image features includes: Image features are extracted using the backbone network of an improved Cascade R-CNN model. The backbone network includes ResNeXt-101, which comprises four sequentially arranged stages. The latter three stages each have improved deformable convolutions to enhance the backbone network's ability to capture the shapes of objects with varying forms. The improved deformable convolutions apply grouped convolutions to divide the image features into multiple groups, and normalize the image features of each group according to preset depth-aware modulation factors and sampling point-aware modulation factors. The feature pyramid network of the improved Cascade RCNN model is used to transform image features into multi-scale image features; The improved Cascade RCNN model's region proposal network is used to determine the feature regions of objects with varying shapes in images based on multi-scale image features; A global semantic detection head network, utilizing an improved Cascade R-CNN model, determines the bounding box coordinates and category of shape-variable objects based on feature regions. This network comprises multiple cascaded global semantic detection heads, each consisting of a sequentially arranged ROI Pooling module, a detection head, and a global context module. The global context module includes sequentially arranged 3×3 convolutional layers, pooling layers, and normalization layers. The 3×3 convolutional layers reduce the number of channels in the feature regions. The pooling layers include parallel max-pooling and mean-pooling layers, where max-pooling extracts local information from the feature regions, and mean-pooling extracts global information. The normalization layer, based on channel-wise multiplication, utilizes the extracted local and global information to improve the feature quality of the feature regions and determine the bounding box coordinates and the type of shape-variable objects within them. The presence and severity of a fire in the monitored area are determined based on the rectangular coordinates of the object with a variable shape and the category of the object with a variable shape. The categories include empty sets, or smoke and / or fire. The rectangular frame coordinates include the empty set, or the rectangular frame coordinates of smoke and / or the rectangular frame coordinates of fire. Each rectangular frame coordinate includes the horizontal and vertical coordinates of the upper left corner of the rectangle, the length of the rectangle, and the width of the rectangle. The process of determining whether a fire has broken out in the monitored area and its severity based on the coordinates of the rectangular frame of the object with variable shape and the category of the object with variable shape includes: If the category is an empty set, it is determined that there is no fire in the monitored area; And / or, in response to a category that includes only smoke, an alarm signal is issued to determine that there is a fire hazard in the monitored area; And / or, in response to categories including smoke and fire, determine that a fire has broken out in the monitored area, and determine the area of ​​the fire's rectangular frame based on the coordinates of the fire's rectangular frame, to calculate the ratio of the area of ​​the smoke's rectangular frame to the area of ​​the image, and compare the ratio with a preset threshold: when the ratio is greater than the preset threshold, determine that the fire in the monitored area is spreading, and when the ratio is less than or equal to the preset threshold, determine that the fire in the monitored area is severe.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the fire monitoring method based on the improved Cascade RCNN model as described in any one of claims 1-3.

6. A device, characterized in that, include: Memory, used to store instructions; A processor for executing the instructions, causing the device to perform operations implementing the fire monitoring method based on the improved Cascade RCNN model as described in any one of claims 1-3.

Images