Fire passage monitoring method, fire passage monitoring device, and storage medium

Abstract

The application discloses a fire-fighting passage monitoring method, a fire-fighting passage monitoring device and a computer storage medium, and relates to the technical field of fire-fighting passage monitoring. The fire-fighting passage monitoring method comprises the following steps: acquiring a preset fire-fighting passage area; acquiring a passage monitoring video, wherein the passage monitoring video comprises a plurality of monitoring images; acquiring a change area of the plurality of monitoring images, and acquiring overlap information of the change area of each monitoring image and the fire-fighting passage area; determining a congestion image representing a fire-fighting passage congestion situation in the plurality of monitoring images based on the overlap information; monitoring a frame number of continuous multiple frames of the congestion image in the passage monitoring video; and outputting early warning information of passage congestion when the frame number is greater than or equal to a preset frame number. Through the above method, the application can determine whether the passage is congested by calculating the overlap degree of the change area of the real-time passage monitoring video and the passage area when the passage is not congested and the congestion time, so that a better monitoring effect of the passage congestion can be obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing and computer vision, in particular to a fire passage monitoring method, a fire passage monitoring device and a storage medium. BACKGROUND

[0002] With the continuous development of computer image processing technology, the application field of image is also more and more extensive, involving biomedical, military, machine vision and other application fields. Image processing technology is a technology for processing image information by computer, mainly including image enhancement and restoration, image recognition, image segmentation, image coding and the like.

[0003] In an application scenario, when an emergency such as fire occurs in a high-rise building, an important factor affecting the difficulty of rescue of firefighters is the blockage of the fire passage. The passage blockage mainly shows as accumulation of sundries, occupation of goods, blockage of garbage, random parking of vehicles and the like. Once the fire passage is occupied, it will lead to difficulty in evacuation of people when fire occurs, and will bring great hidden danger to the life safety and property safety of the people. At present, the management of the fire passage is mostly manually and regularly carried out on-site inspection. This way consumes a lot of manpower and is low in efficiency, and at the same time cannot keep the fire passage unblocked for a long time. The existing monitoring method for the blockage of the fire passage has poor monitoring effect due to the interference of many irrelevant factors. SUMMARY

[0004] The technical problem solved by the present application is how to improve the monitoring effect of the blockage of the fire passage. To this end, the present application provides a fire passage monitoring method, a fire passage monitoring device and a storage medium.

[0005] To solve the above technical problem, one technical solution adopted by the present application is to provide a fire passage monitoring method, which comprises: acquiring a preset fire passage area; acquiring a passage monitoring video, the passage monitoring video comprising a plurality of monitoring images; acquiring a change area of the plurality of monitoring images, and acquiring overlap information of the change area of each monitoring image and the fire passage area; determining a blockage image representing a blockage situation of the fire passage in the plurality of monitoring images based on the overlap information; monitoring the number of frames of the blockage image appearing continuously in the passage monitoring video, and outputting a pre-warning information of the blockage of the passage when the number of frames is greater than or equal to a preset frame number.

[0006] The ratio of the area of the change area of the blockage image to the area of the fire passage area is greater than or equal to a first preset threshold.

[0007] The change region of the plurality of monitoring images is obtained, including: performing change detection on each monitoring image to obtain a plurality of initial change regions of each monitoring image; and defining an initial change region with a center point located in the fire channel region as the change region of the monitoring image.

[0008] The position change value of the change region of the plurality of continuous blocking images is less than or equal to a second preset threshold value, and the second preset threshold value is used to determine whether the change region of the plurality of continuous blocking images is the same change region.

[0009] After the change region of the plurality of monitoring images is obtained, the fire channel monitoring method further includes: obtaining a human body frame of the plurality of monitoring images by using a human body detection model; calculating a first intersection-over-union of the human body frame and the change region of the corresponding frame; and defining the monitoring image as a non-blocking image when the first intersection-over-union is greater than or equal to a third preset threshold value.

[0010] After the change region of the plurality of monitoring images is obtained, the fire channel monitoring method further includes: obtaining a target tracking output frame of the plurality of monitoring images by using a target tracking model; calculating a second intersection-over-union of the target tracking output frame and the change region of the corresponding frame; and defining the monitoring image as a non-blocking image when the second intersection-over-union is greater than or equal to a fourth preset threshold value.

[0011] After the change region of the plurality of monitoring images is obtained, the fire channel monitoring method further includes: obtaining a motion region of the plurality of monitoring images by using a motion detection model; calculating an intersection pixel value of the motion region and the change region of the corresponding frame; and defining the monitoring image as a non-blocking image when the ratio of the intersection pixel value to the change region is greater than or equal to a fifth preset threshold value.

[0012] The change detection is performed on each monitoring image to obtain a plurality of initial change regions of each monitoring image, including: performing feature extraction on the fire channel region image to obtain a first feature map; performing feature extraction on each monitoring image to obtain a second feature map set; performing feature extraction on the fire channel region image and each frame of connection image of each monitoring image to obtain a third feature map set; performing feature fusion on the first feature map, the second feature map set and the third feature map set, and inputting the fusion result into a feature prediction head; and performing change detection by using the feature prediction head to obtain a plurality of initial change regions of each monitoring image.

[0013] The first feature map is obtained by performing feature extraction on the fire passage area image, including: constructing a feature extraction backbone network to perform feature extraction on the fire passage area image; adding a channel and spatial attention module in the feature extraction backbone network; generating a first attention feature map for the fire passage area image by using the module; and correcting the feature extraction result by using the first attention feature map to obtain the first feature map.

[0014] The second feature map set is obtained by performing feature extraction on each frame of the monitoring image, including: constructing a feature extraction backbone network to perform feature extraction on each frame of the monitoring image; adding a channel and spatial attention module in the feature extraction backbone network; generating a corresponding second attention feature map for each frame of the monitoring image by using the module; and correcting the feature extraction result by using the corresponding second attention feature map to obtain the second feature map set.

[0015] The third feature map set is obtained by performing feature extraction on each frame of the connection image of the fire passage area image and each frame of the monitoring image, including: constructing a feature extraction backbone network to perform feature extraction on each frame of the connection image; adding a channel and spatial attention module in the feature extraction backbone network; generating a corresponding third attention feature map for each frame of the connection image by using the module; and correcting the feature extraction result by using the corresponding third attention feature map to obtain the third feature map set.

[0016] The first feature map, the second feature map set, and the third feature map set are fused, and the fusion result is input into a feature prediction head, including: dividing the fusion result into two channels for convolution operation; and inputting the convolution operation result into the feature prediction head.

[0017] To solve the above technical problems, another technical solution adopted by the present application is to provide a fire passage monitoring device, which comprises a processor and a memory, the memory is coupled with the processor, the memory stores program data, and the processor is used to execute the program data to realize the fire passage monitoring method as described above.

[0018] To solve the above technical problems, another technical solution adopted by the present application is to provide a computer readable storage medium, which stores program data, and the program data is used to realize the fire passage monitoring method when executed.

[0019] The beneficial effects of the present application are: different from the prior art, the fire passage monitoring method provided by the present application is applied to a fire passage monitoring device, the fire passage monitoring device acquires a preset fire passage area; acquires a passage monitoring video, the passage monitoring video includes a plurality of frames of monitoring images; acquires a change area of the plurality of frames of monitoring images, and acquires overlap information of the change area of each frame of monitoring image and the fire passage area; determines a blocked image indicating a fire passage blocking condition in the plurality of frames of monitoring images based on the overlap information; monitors the number of frames in which the continuous multiple frames of blocked images appear in the passage monitoring video, and when the number of frames is greater than or equal to a preset number of frames, outputs early warning information of passage blocking. Through the above-mentioned manner, compared with the conventional fire passage monitoring method, the method adopted by the present application can compare the real-time condition of the passage with the normal condition, more accurately and efficiently detect the position of the blocking condition in the passage monitoring video and timely give an early warning, and can also avoid the misjudgment condition caused by human activities or temporary image changes. The fire passage monitoring method can extract features from the monitoring video and the normal image of the passage and construct a deep learning detection model to more accurately obtain the change area of the passage monitoring video, exclude the interference conditions in the change area by using a plurality of detection models to obtain the final passage blocking condition, timely and accurately give an early warning to the blocking condition of the passage, and solve the hidden dangers and problems such as passage blocking and illegal occupation. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0021] Among them:

[0022] Figure 1 is a flowchart of an embodiment of the fire passage monitoring method provided by the present application;

[0023] Figure 2 is a flowchart of the fire passage monitoring method executed by the fire passage monitoring device provided by the present application;

[0024] Figure 3 is a network structure diagram for change detection in the fire passage monitoring device provided by the present application;

[0025] Figure 4 is a structure diagram of the feature prediction head in the fire passage monitoring device provided by the present application;

[0026] Figure 5 The schematic diagram of the passage blockage early warning rule in the fire passage monitoring method provided by the present application is shown in the following figure:

[0027] Figure 6 The structural schematic diagram of the first embodiment of the fire passage monitoring device provided by the present application is shown in the following figure:

[0028] Figure 7 The structural schematic diagram of the second embodiment of the fire passage monitoring device provided by the present application is shown in the following figure:

[0029] Figure 8 The structural schematic diagram of an embodiment of the computer readable storage medium provided by the present application is shown in the following figure. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0031] Referring to Figure 1 and Figure 2 , Figure 1 The flow schematic diagram of an embodiment of the fire passage monitoring method provided by the present application is shown in the following figure: Figure 2 The flow schematic diagram of the fire passage monitoring device provided by the present application executing the fire passage monitoring method is shown in the following figure.

[0032] The fire passage refers to the passage for the firemen to implement rescue and the trapped personnel to evacuate. When a fire occurs, if the fire passage is blocked or occupied, the rescue opportunity of the firemen will be delayed, which will cause a great threat to the property and life safety of the people. The fire passage monitoring method provided by the present application can be applied to the monitoring of all road blockage situations in various buildings or outdoors, and is not limited to the fire passage.

[0033] As shown in Figure 1 and Figure 2 , the fire passage monitoring method of the present application embodiment comprises:

[0034] Step 11: acquiring a preset fire passage area.

[0035] Specifically, the fire passage area can be an area in a facility in the real world, such as a passage in a supermarket, a parking lot, a shopping mall, an airport, and the like. The setting of the fire passage area can be selected by a user, or can be obtained by identifying the acquired image of the unblocked fire passage area. The device for acquiring the image of the fire passage area can be a camera, a monitoring camera, a robot with a camera, a drone, a mobile phone, or the like.

[0036] Step 12: acquiring a passage monitoring video.

[0037] Specifically, the passage monitoring video is a real-time monitoring video of the preset fire passage area, including a plurality of frames of monitoring images of the preset fire passage area.

[0038] Step 13: acquiring a change area of the plurality of frames of monitoring images, and acquiring overlap information of the change area of each frame of monitoring image and the fire passage area.

[0039] Specifically, the change area is an area in which the pixel value of the monitoring image is different from that of the preset unblocked fire passage area image. The overlap information of the change area of each frame of monitoring image and the fire passage area includes the overlap area of the change area and the fire passage area. The fire passage monitoring device compares each frame of monitoring image with the preset fire passage image to obtain the change area of each frame of monitoring image. The fire passage monitoring device then compares the area of the change area with the area of the fire passage area to calculate the overlap area, and compares the overlap area with the first preset threshold value. When the overlap area of the change area and the fire passage area is greater than the first preset threshold value, it is considered that the fire passage area is blocked.

[0040] Referring to Figure 3 , Figure 3 A network structure diagram for change detection in the fire passage monitoring device provided in the present application.

[0041] Specifically, the fire access monitoring device detects changes in each frame of the monitoring image in the monitoring video to obtain a plurality of initial change regions of each frame of the monitoring image. The fire access monitoring device defines an initial change region with a center point located in the fire access region as a change region of the monitoring image from the plurality of initial change regions. The position change value of the change region of the continuous multiple frames of the blocked image is less than or equal to a second preset threshold. The second preset threshold is used to determine whether the change region of the continuous multiple frames of the blocked image is the same change region. If the position change value is greater than the second preset threshold, it is considered that the change region of the multiple frames of the blocked image is not the same change region. The second preset threshold can be set by the user, which is not limited herein. When the position change value of the change region of the continuous multiple frames of the blocked image is less than or equal to the second preset threshold, it can be considered that the change region of the continuous multiple frames of the blocked image is the same change region, and thus the duration of the change region can be calculated by the number of frames of the video.

[0042] In an embodiment of the present application, the fire access monitoring device extracts features from the input image 1 in the fire access region image, i.e. Figure 3 to obtain a first feature map; extracts features from the input image 2 in each frame of the monitoring image, i.e. Figure 3 to obtain a second feature map set; extracts features from the image concat in the fire access region image and each frame of the monitoring image, i.e. Figure 3 to obtain a third feature map set; performs feature fusion on the first feature map, the second feature map set, and the third feature map set, and inputs the fusion result into a feature prediction head; performs change detection using the feature prediction head to obtain a plurality of initial change regions of each frame of the monitoring image.

[0043] Specifically, the fire access monitoring device constructs a feature extraction backbone network to extract features from the fire access region image, adds a channel and spatial attention module in the feature extraction backbone network, generates a first attention feature map from the fire access region image using the module, and corrects the feature extraction result using the first attention feature map to obtain a first feature map; generates a corresponding second attention feature map from each frame of the monitoring image using the module, and corrects the feature extraction result using the corresponding second attention feature map to obtain a second feature map set; generates a corresponding third attention feature map from each frame of the connected image using the module, and corrects the feature extraction result using the corresponding third attention feature map to obtain a third feature map set. Adding the attention module during feature extraction can enable the fire access monitoring device to obtain responses focused on different channels during feature extraction, and adjust the original feature extraction image accordingly to enhance the feature extraction capability of the fire access monitoring device.

[0044] Continuing to refer to Figure 3The fire passage monitoring device fuses the first feature map, the second feature map set, and the third feature map set. The fire passage monitoring device adds and subtracts the results of the convolution operation on the first feature map and the second feature map set at the pixel level to realize the fusion of the images, and then connects the SE attention module to obtain the response of different channel features. The fire passage monitoring device connects the fusion results of the first feature map and the second feature map set with the third feature map set to more adaptively represent the change features of the foreground image and the background image, thereby further improving the expression ability of the fire passage monitoring device for the change features.

[0045] Specifically, the fire passage monitoring device divides the connected results into two channels for convolution operation, and transmits the convolution operation results to the feature prediction head. The fire passage monitoring device divides the input fusion results into two channels, so that the image transmitted to the feature prediction head has the same size as the original input monitoring video image, and can extract more detailed features and prevent network degradation.

[0046] Referring to Figure 4 , Figure 4 The structure diagram of the feature prediction head in the fire passage monitoring device provided in the present application can be divided into three branches, including classification, regression, and confidence, etc. The number of channels of the classification branch at the end is the number of detection categories, the number of channels of the regression branch at the end is 4, corresponding to the [x, y, w, h] information of the position coordinates, and the confidence branch outputs the judgment of whether there is a change. Multiple feature prediction heads can predict the classification, regression, and confidence of the image respectively, fully utilize the feature information of the image, and improve the overall performance of the model.

[0047] Referring to Figure 5 , Figure 5 The schematic diagram of the passage blockage early warning rule in the fire passage monitoring method provided in the present application.

[0048] Specifically, after obtaining the change regions of a plurality of frames of monitoring images, the fire passage monitoring device can also use a human body detection model, a target tracking model, and a motion detection model to exclude non-blocking conditions of the obtained monitoring images.

[0049] Specifically, the fire passage monitoring device obtains human body boxes of a plurality of frames of monitoring images by using a human body detection model, calculates a first intersection-over-union of each human body box and a change region of the corresponding frame. When the first intersection-over-union is greater than or equal to a third preset threshold, the frame of monitoring image is defined as a non-blocking image. The first intersection-over-union reflects the confidence of the human body box to the change region, that is, the greater the confidence, the more it can represent that the image change region is a human body. When the first intersection-over-union is greater than or equal to the third preset threshold, it can be considered that the image change of the frame is caused by a human body and does not belong to the passage blocking situation. The third preset threshold can be set by the user, which is not limited here. For example, the first intersection-over-union calculated by the fire passage monitoring device is IoU_1, when IoU_1 is less than the third preset threshold thresh1, that is, flag1 = IoU_1 < thresh1, it can be considered that the image change region of the frame of monitoring image is not caused by a human body.

[0050] Specifically, the fire passage monitoring device obtains human body boxes of a plurality of frames of monitoring images by using a human body detection model, calculates a first intersection-over-union of each human body box and a change region of the corresponding frame. When the first intersection-over-union is greater than or equal to a third preset threshold, the frame of monitoring image is defined as a non-blocking image. The first intersection-over-union reflects the confidence of the human body box to the change region, that is, the greater the confidence, the more it can represent that the image change region is a human body. When the first intersection-over-union is greater than or equal to the third preset threshold, it can be considered that the image change of the frame is caused by a human body and does not belong to the passage blocking situation. The third preset threshold can be set by the user, which is not limited here. For example, the first intersection-over-union calculated by the fire passage monitoring device is IoU_1, when IoU_1 is less than the third preset threshold thresh1, that is, flag1 = IoU_1 < thresh1, it can be considered that the image change region of the frame of monitoring image is not caused by a human body.

[0051] Specifically, the fire passage monitoring device obtains a motion region of a plurality of frames of monitoring images by using a motion detection model; calculates an intersection pixel value of the motion region and a change region corresponding to the frame; and defines the frame of monitoring images as a non-blocking image when a ratio of the intersection pixel value to the change region is greater than or equal to a fifth preset threshold value. When the ratio of the intersection pixel value to the total pixel value of the change region is greater than or equal to the fifth preset threshold value, it can be considered that the image change of the frame is in a motion state and does not belong to the passage blocking condition. The fifth preset threshold value can be set by the user, and is not limited herein. For example, the fire passage monitoring device calculates a ratio of the intersection pixel value to the change region as ratio_1, and when ratio_1 is less than the fifth preset threshold value thresh3, i.e., flag3 = ratio_1 < thresh3, it can be considered that the image change region of the frame of monitoring images is not in motion.

[0052] In an embodiment of the present application, the fire passage monitoring device can first obtain a human body frame of each frame of monitoring images by using a human body detection model, and then filter out the human body frame before performing motion detection, so as to reduce the calculation time of the fire passage monitoring device.

[0053] After the fire passage monitoring device excludes the non-blocking condition from the acquired monitoring images by using the human body detection model, the target tracking model and the motion detection model, it can more accurately obtain the condition that the change of the fire passage image is caused by the passage blocking, so as to avoid the influence of human activity and other interference factors.

[0054] In an embodiment of the present application, after the fire passage monitoring device performs human body detection, target tracking and motion detection, when the three judgment conditions are all met, i.e., flag1 & flag2 & flag3, the calculation of the fire passage blocking area is performed. The flag1 is a condition that the confidence IoU_1 of the human body frame to the change region is less than the third preset threshold value thresh1, the flag2 is a condition that the confidence IoU_2 of the target tracking output frame to the change region is less than the fourth preset threshold value thresh2, and the flag3 is a condition that the ratio of the intersection pixel value of the motion region to the change region corresponding to the frame to the total pixel value of the change region is ratio_1, which is less than the fifth preset threshold value thresh3. The flag1 & flag2 & flag3 indicates that the above three conditions are all met at the same time.

[0055] Step 14: determining a blocking image representing the fire passage blocking condition in the plurality of frames of monitoring images based on the coincidence information.

[0056] Specifically, when the ratio of the area area_1 of the changed region of the image acquired by the passage monitoring device to the area area_0 of the fire passage region is greater than or equal to a first preset threshold ratio, the frame monitoring image is defined as a jam image in the case of passage jam.

[0057] Step 15: Monitor the number of frames of continuous multiple frames of jam images in the passage monitoring video, and output a pre-warning information of passage jam when the number of frames is greater than or equal to a preset frame number.

[0058] Continuing to refer to Figure 5 , the initial time of the first frame jam image is set as t_i, the jam time of the passage is calculated by the number of frames of continuous multiple frames of jam images, and is recorded as t. When t is greater than a preset time T corresponding to the preset frame number, the fire passage monitoring device sends a pre-warning information to the user about the passage jam situation.

[0059] Different from the prior art, the fire passage monitoring method provided by the application is applied to a fire passage monitoring device, the fire passage monitoring device acquires a preset fire passage region; acquires a passage monitoring video, the passage monitoring video includes a plurality of frame monitoring images; acquires a changed region of the plurality of frame monitoring images, and acquires overlap information of the changed region of each frame monitoring image and the fire passage region; determines a jam image representing a fire passage jam situation in the plurality of frame monitoring images based on the overlap information; monitors the number of frames of continuous multiple frames of jam images in the passage monitoring video, and outputs a pre-warning information of passage jam when the number of frames is greater than or equal to a preset frame number. Through the above-mentioned manner, compared with the conventional fire passage monitoring method, the method adopted by the application uses the fire passage monitoring device to acquire the image changed region based on the real-time monitoring video of the passage and the image when the passage is not jammed, and uses a plurality of detection models to exclude interference factors in the image changed region, which can compare the real-time situation of the passage with the normal situation, more accurately and efficiently detect the position of the jam situation in the passage monitoring video and timely pre-warning, and can also avoid the misjudgment situation caused by human activities or temporary image changes. The fire passage monitoring method can extract features from the monitoring video and the normal image of the passage and construct a deep learning detection model to more accurately obtain the changed region of the passage monitoring video, use a plurality of detection models to exclude interference in the changed region to obtain the final passage jam situation, timely and accurately pre-warning the jam situation of the passage, and solve the hidden dangers and problems such as passage jam, illegal occupation, etc.

[0060] The method of the above-mentioned embodiment can be realized by using a fire passage monitoring device, which will be described below in combination with Figure 6 Figure 6 is a structural schematic diagram of the first embodiment of the fire passage monitoring device provided by the application.

[0061] As Figure 6 ​As shown, the fire lane monitoring device 60 of this application embodiment includes an acquisition module 61, a change detection module 62, a blockage determination module 63, and an early warning output module 64.

[0062] The acquisition module 61 is used to acquire preset fire lane areas and lane monitoring videos, and the lane monitoring videos include several frames of monitoring images.

[0063] The change detection module 62 is used to acquire the change area of ​​several frames of monitoring images and to acquire the overlap information between the change area of ​​each frame of monitoring images and the fire lane area.

[0064] The blockage determination module 63 is used to determine the blockage image representing the blockage of the fire lane in several frames of monitoring images based on the overlap information.

[0065] The early warning output module 64 is used to monitor the number of consecutive blocked images in the monitoring video of the monitoring channel. When the number of frames is greater than or equal to the preset number of frames, it outputs an early warning message of channel blockage.

[0066] The method described in the above embodiments can be implemented using a fire escape monitoring device, as described below. Figure 7 , Figure 7 This is a schematic diagram of the structure of the second embodiment of the fire lane monitoring device provided in this application. The fire lane monitoring device 70 includes a memory 71 and a processor 72. The memory 71 is used to store program data, and the processor 72 is used to execute the program data to implement the following method:

[0067] The system acquires a preset fire lane area; acquires monitoring video of the lane, which includes several frames of monitoring images; acquires the changing areas of the several frames of monitoring images, and acquires the overlap information between the changing areas of each frame of monitoring images and the fire lane area; based on the overlap information, it determines the blocked images in the several frames of monitoring images that represent the blockage of the fire lane; monitors the number of consecutive blocked images in the monitoring video of the lane, and outputs a warning message for lane blockage when the number of frames is greater than or equal to the preset number of frames.

[0068] See Figure 8 , Figure 8 This is a schematic diagram of an embodiment of the computer-readable storage medium 80 provided in this application. The computer-readable storage medium 80 stores program data 81, which, when executed by a processor, is used to implement the following method:

[0069] The preset fire-fighting passage area is acquired; a passage monitoring video is acquired, the passage monitoring video including a plurality of monitoring images; a change area of the plurality of monitoring images is acquired, and overlap information of the change area of each monitoring image and the fire-fighting passage area is acquired; a blocked image representing a fire-fighting passage blockage condition in the plurality of monitoring images is determined based on the overlap information; a frame number of continuous multiple frames of the blocked image in the passage monitoring video is monitored, and when the frame number is greater than or equal to a preset frame number, prewarning information of passage blockage is output.

[0070] The embodiments of the present application are realized in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the whole or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0071] The above is only an embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation based on the content of the specification and drawings, or direct or indirect application in other related technical fields, is also included in the patent protection scope of the present application.

Claims

1. A fire access way monitoring method characterized by, The method comprises: acquiring a preset fire passage area, the fire passage area being obtained based on recognition of an acquired image of an unblocked fire passage area; acquiring a passage monitoring video, the passage monitoring video comprising a plurality of monitoring images; acquiring a change area of the plurality of monitoring images, and acquiring overlap information of the change area of each of the monitoring images and the fire passage area; determining, based on the overlap information, a blocked image representing a fire passage blocking condition in the plurality of monitoring images; monitoring a frame number of the plurality of monitoring images in which a continuous plurality of blocked images appear, and outputting a warning information of passage blocking when the frame number is greater than or equal to a preset frame number; wherein the acquiring of the change area of the plurality of monitoring images comprises: performing change detection on each of the monitoring images to obtain a plurality of initial change areas of each of the monitoring images, comprising: performing feature extraction on an image of the fire passage area to obtain a first feature map; performing feature extraction on each of the monitoring images to obtain a second feature map set; performing feature extraction on each of the fire passage area image and a connection image of each of the monitoring images to obtain a third feature map set; performing feature fusion on the first feature map, the second feature map set, and the third feature map set, and inputting a fusion result into a feature prediction head; and performing change detection by using the feature prediction head to obtain the plurality of initial change areas of each of the monitoring images; defining an initial change area with a center point located in the fire passage area in the plurality of initial change areas as the change area of the monitoring image.

2. The fire passage monitoring method according to claim 1, wherein a ratio of an area of the change area of the blocked image to an area of the fire passage area is greater than or equal to a first preset threshold.

3. The fire passage monitoring method according to claim 1, wherein a position change value of the change area of the continuous plurality of blocked images is less than or equal to a second preset threshold, and the second preset threshold is used to determine whether the change area of the continuous plurality of blocked images is a same change area.

4. The fire passage monitoring method according to any one of claims 1-3, wherein after the acquiring of the change area of the plurality of monitoring images, the fire passage monitoring method further comprises: obtaining a human body frame of the plurality of monitoring images by using a human body detection model; calculating a first intersection-over-union of the human body frame and a change area of a corresponding frame of the human body frame; when the first intersection-over-union is greater than or equal to a third preset threshold, defining the frame of the monitoring image as a non-blocked image.

5. The fire passage monitoring method according to any one of claims 1-3, wherein after the acquiring of the change area of the plurality of monitoring images, the fire passage monitoring method further comprises: obtaining a target tracking output frame of the plurality of monitoring images by using a target tracking model; calculating a second intersection-over-union of the target tracking output frame and a change area of a corresponding frame of the target tracking output frame; when the second intersection-over-union is greater than or equal to a fourth preset threshold, defining the frame of the monitoring image as a non-blocked image. 6.The fire passage monitoring method of any one of claims 1-3, wherein, after the change region of the plurality of frames of monitoring images is obtained, the fire passage monitoring method further comprises: obtaining a motion region of the plurality of frames of monitoring images by using a motion detection model; calculating an intersection pixel value of the motion region and the change region of the corresponding frame; when a ratio of the intersection pixel value to the change region is greater than or equal to a fifth preset threshold, defining the frame of monitoring image as a non-blockage image. 7.The fire passage monitoring method of claim 1, wherein, the feature extraction on the fire passage region image to obtain a first feature map comprises: constructing a feature extraction backbone network to perform feature extraction on the fire passage region image; adding a channel and spatial attention module in the feature extraction backbone network; generating a first attention feature map of the fire passage region image by using the module; correcting the feature extraction result by using the first attention feature map to obtain the first feature map. 8.The fire passage monitoring method of claim 1, wherein, the feature extraction on each frame of monitoring image to obtain a second feature map set comprises: constructing a feature extraction backbone network to perform feature extraction on each frame of monitoring image; adding a channel and spatial attention module in the feature extraction backbone network; generating a corresponding second attention feature map of each frame of monitoring image by using the module; correcting the feature extraction result by using the corresponding second attention feature map to obtain the second feature map set. 9.The fire passage monitoring method of claim 1, wherein, the feature extraction on the fire passage region image and each frame of connection image of each frame of monitoring image to obtain a third feature map set comprises: constructing a feature extraction backbone network to perform feature extraction on each frame of connection image; adding a channel and spatial attention module in the feature extraction backbone network; generating a corresponding third attention feature map of each frame of connection image by using the module; correcting the feature extraction result by using the corresponding third attention feature map to obtain the third feature map set. 10.The fire passage monitoring method of claim 1, wherein, the feature fusion on the first feature map, the second feature map set, and the third feature map set, and the fusion result is transmitted to a feature prediction head, comprises: performing feature fusion on the first feature map, the second feature map set, and the third feature map set; dividing the fusion result into two channels to perform convolution operation respectively; transmitting the convolution operation result to the feature prediction head.

11. A fire access way monitoring device, characterized by, The fire passage monitoring device comprises a memory and a processor coupled with the memory; wherein the memory is used to store program data, and the processor is used to execute the program data to realize the fire passage monitoring method of any one of claims 1-10.

12. A computer storage medium, characterized in that, The computer storage medium is used to store program data, which when executed by a computer, implements the fire passage monitoring method according to any one of claims 1 to 10.

Images