Fire fighting access monitoring method, device and equipment and storage medium

Abstract

The invention relates to a fire fighting access monitoring method, device and equipment and a storage medium. The method comprises the steps of collecting image data containing color and depth information and passing standard parameters; analyzing the color information, and generating a semantic mask of the channel region and an independent identifier of each occupied object; fusing the semantic mask and the depth information, reconstructing a three-dimensional scene point cloud of the channel, and separating a three-dimensional point cloud cluster of each object; analyzing the point cloud to determine the main direction and key obstacle points of the channel, constructing a dynamic evaluation cross section, and delimiting a standard traffic area; and comprehensively calculating the projection occupation condition of each object on the cross section, and generating a quantitative channel comprehensive availability index and an evaluation result. According to the method, the limitation that in the prior art, only whether occupation or not can be carried out is overcome, real-time and accurate three-dimensional space availability quantitative evaluation on the fire fighting access is achieved, and visual traffic capacity data support can be provided for fire fighting emergency decision making.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image recognition technology, specifically to a method, device, equipment, and storage medium for monitoring fire exits. Background Technology

[0002] Fire lanes are critical infrastructure ensuring unobstructed access for fire and rescue operations, and their constant availability directly impacts the efficiency of firefighting and emergency response. To achieve automated monitoring of fire lane status, computer vision-based intelligent monitoring methods are widely used. These methods typically analyze surveillance video or images to automatically identify obstructions such as vehicles or obstacles within the lane area, issuing alarms upon detection. This aims to replace inefficient manual patrols and improve the timeliness of monitoring.

[0003] However, existing monitoring methods primarily focus on addressing the fundamental question of "is it occupied?". They are essentially binary judgments: once a target object is detected within a passageway, an occupancy alarm is triggered. This approach has significant limitations. First, it cannot provide a refined assessment of occupancy events. For example, the degree of obstruction to passageway capacity differs drastically between a small electric bicycle and a large, horizontally parked van, yet existing technologies consistently assign the same "occupancy" conclusion, resulting in overly general warnings that fail to differentiate the urgency of the risk. Second, these methods rely on two-dimensional image analysis, making it difficult to perceive the true three-dimensional size and spatial layout of objects. They are prone to misjudgments due to perspective and obstruction, and completely fail to assess the remaining vertical passageway capacity. Furthermore, their output is disconnected from the practical decision-making needs of fire and rescue operations. Command centers cannot quickly determine, based on simple "occupancy / unoccupied" information, which passageways around a fire scene can still accommodate fire trucks and how much safe space remains to plan the optimal route. Summary of the Invention

[0004] Based on this, the purpose of the present invention is to provide a method, device, equipment and storage medium for monitoring fire lanes that can go beyond simple road occupancy identification and achieve accurate quantitative assessment of the real-time availability of fire lanes.

[0005] The objective of this invention is achieved through the following solution:

[0006] In a first aspect, the present invention provides a method for monitoring fire escape routes, comprising the following steps:

[0007] S1: Perform multimodal data acquisition on the fire lane monitoring area, simultaneously acquire image data containing color and depth information, obtain predefined standard parameters for lane passage, and generate the original dataset and standard parameter set;

[0008] S2: Perform intelligent parsing and processing on the color information in the original dataset, identify and separate the channel regions and occupying objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object.

[0009] S3: Perform 3D spatial fusion processing on the 2D semantic mask and the depth information in the original dataset, map the pixels in the 2D semantic mask to the 3D space, generate a 3D scene point cloud describing the true geometric shape of the channel, and separate the 3D point cloud clusters corresponding to each occupying object from the 3D scene point cloud based on the identifier of each independent 2D instance.

[0010] S4: Perform bottleneck analysis on the 3D scene point cloud and each 3D point cloud cluster, determine the main extension direction of the channel, locate the key obstacle point that most affects the passage, construct a spatial cross section for quantitative evaluation, delineate the theoretical passage area on the spatial cross section, and generate a dynamic evaluation cross section and a standard passage area.

[0011] S5: Perform comprehensive availability calculations on the dynamic assessment section, standard passage area, and 3D point cloud clusters; analyze the projection occupancy of each obstacle on the spatial section; comprehensively calculate the remaining proportion of passage capacity in width and height; and generate fire lane assessment results containing a comprehensive access availability index.

[0012] Secondly, the present invention provides a fire escape monitoring device, which is equipped with the following modules:

[0013] The multimodal data acquisition module is used to acquire multimodal data of the fire lane monitoring area, simultaneously acquire image data containing color and depth information, and obtain predefined standard parameters for passage, generating the original dataset and standard parameter set;

[0014] The color information parsing module is used to intelligently parse and process the color information in the original dataset, identify and separate the channel regions and occupies the objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object.

[0015] The 3D spatial fusion module is used to perform 3D spatial fusion processing on the depth information in the 2D semantic mask and the original dataset, mapping the pixels in the 2D semantic mask to the 3D space, generating a 3D scene point cloud that describes the true geometric shape of the channel, and separating the 3D point cloud clusters corresponding to each occupying object from the 3D scene point cloud based on the identifier of each independent 2D instance.

[0016] The bottleneck analysis module is used to perform bottleneck analysis on the 3D scene point cloud and each 3D point cloud cluster, determine the main extension direction of the channel, locate the key obstacle point that most affects the passage, construct a spatial cross section for quantitative evaluation, delineate the theoretical passage area on the spatial cross section, and generate a dynamic evaluation cross section and a standard passage area.

[0017] The access road availability assessment module is used to perform comprehensive availability calculations on dynamic assessment sections, standard passage areas, and 3D point cloud clusters. It analyzes the projection occupancy of each obstacle on the spatial section, comprehensively calculates the remaining proportion of passage capacity in width and height, and generates fire access road assessment results containing a comprehensive access road availability index.

[0018] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the above-mentioned fire escape monitoring methods.

[0019] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned fire escape monitoring methods.

[0020] In summary, the fire lane monitoring method provided in this application effectively overcomes the core shortcomings of existing fire lane monitoring schemes, as pointed out in the background art, which can only make binary judgments of "occupancy" and lack three-dimensional spatial perception capabilities, as well as being disconnected from emergency decision-making. By constructing a complete technical chain of "two-dimensional analysis - three-dimensional reconstruction - bottleneck analysis - quantitative evaluation," a leap from qualitative judgment to quantitative analysis can be achieved. Specifically, this method can achieve accurate three-dimensional spatial measurement of fire lane capacity. By fusing depth information to reconstruct the real geometric scene, it can achieve immunity to inherent defects of two-dimensional vision such as parallax and occlusion. Furthermore, by dynamically locating the bottleneck section and calculating the projected occupied area, a comprehensive and continuous quantitative evaluation of the remaining capacity in both the width and height dimensions of the lane can be achieved, generating an intuitive availability index. Ultimately, this quantitative evaluation result can directly serve fire emergency command and decision-making, enabling optimal path planning and risk assessment based on the real-time capacity of the lane, thereby significantly improving the accuracy and practicality of intelligent fire lane monitoring.

[0021] To better understand and implement this invention, the following detailed description is provided in conjunction with the accompanying drawings. Attached Figure Description

[0022] Figure 1 A flowchart illustrating a fire escape monitoring method provided in an embodiment of this application;

[0023] Figure 2This is a structural schematic diagram of a fire lane monitoring device provided in another embodiment of this application. Detailed Implementation

[0024] To facilitate understanding of the present invention, a more complete description will be given below with reference to the accompanying drawings. Preferred embodiments of the invention are shown in the drawings. However, the invention can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a thorough and complete understanding of the disclosure of the invention.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0026] In one embodiment, such as Figure 1 As shown, a method for monitoring fire exits is provided. This embodiment illustrates the method applied to a terminal, but it is understood that the method can also be applied to a server, or to a device including both a terminal and a server, and implemented through interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0027] S1: Perform multimodal data acquisition on the fire lane monitoring area, simultaneously acquire image data containing color and depth information, obtain predefined standard parameters for lane passage, and generate the original dataset and standard parameter set.

[0028] Specifically, the system uses an industrial-grade RGB-D camera as the core acquisition device. This device integrates a visible light imaging module and a depth sensing module, enabling synchronous frame acquisition of color and depth information. The visible light imaging module is equipped with a CMOS sensor to acquire RGB images containing color and texture features, while the depth sensing module, based on the time-of-flight principle, acquires depth images representing the three-dimensional distance information of the scene. During acquisition, the system uses the device's built-in synchronization trigger interface to achieve frame synchronization between the RGB and depth images, avoiding subsequent data fusion deviations caused by time differences. The acquisition range covers the entire extension area and edge areas of the fire lane.

[0029] The predefined passageway standard parameters are determined by the system based on building fire protection design codes and local fire management regulations. The system encapsulates these parameters into a standard parameter set using a structured data format. This standard parameter set includes core parameters such as passageway type, standard net width threshold, standard net height threshold, turning radius threshold, and allowable area ratio threshold. It also includes attributes such as parameter identifier, parameter name, threshold range, applicable scenario, and update timestamp, supporting dynamic retrieval or updates based on actual application scenarios. The system stores synchronously acquired RGB and depth images in a time-stamped association to form a raw dataset. Each data record in the raw dataset includes metadata such as acquisition time, device number, RGB image data, depth image data, and acquisition location coordinates. Simultaneously, the system organizes the predefined passageway standard parameters into a structured document to generate the standard parameter set. Both the raw dataset and the standard parameter set are stored on industrial-grade solid-state drives. The system uses a RAID5 array to ensure data security and supports subsequent fast retrieval.

[0030] S2: Perform intelligent parsing and processing on the color information in the original dataset, identify and separate the channel regions and occupying objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object.

[0031] Specifically, the system preprocesses the RGB images in the original dataset to eliminate environmental interference. The preprocessing process involves sequentially performing Gaussian filtering for denoising, image enhancement, and scale normalization. During Gaussian filtering, the system uses a Gaussian kernel to perform convolution operations on the image to remove interference from ambient light variations and sensor noise. During image enhancement, the system uses adaptive histogram equalization to adjust image contrast and enhance the grayscale difference between the enhanced channel region and the surrounding environment. During scale normalization, the system uniformly scales the image size to facilitate subsequent model inference. The scaling process uses a bilinear interpolation algorithm to ensure that image details are not lost. The system uses an improved U-Net++ semantic segmentation model to parse the preprocessed RGB images. This model is equipped with a backbone network to enhance feature extraction capabilities.

[0032] During the model training phase, the system uses annotated images of fire lanes from different scenarios as the training set. The annotations cover three semantic information categories: lane area, object, and background. The system uses a weighted sum of cross-entropy and Dice loss functions as the loss function to optimize the model's recognition accuracy for small object occupancy. The system uses the trained model to infer initial semantic segmentation results from the images. Subsequently, the system fills small holes within the segmented regions using morphological closing operations and removes small noise points at the region edges using morphological opening operations, ultimately generating a two-dimensional semantic mask. Different pixel values ​​are used to identify the background of the object occupancy in the lane area within the two-dimensional semantic mask. Based on semantic segmentation, the system employs a Mask R-CNN instance segmentation model to perform instance-level separation of occupancy objects. This model uses the occupancy object regions obtained from semantic segmentation as candidate regions, generates candidate boxes through a region proposal network, and classifies and generates masks for the features within each candidate box to distinguish different occupancy objects. The system assigns a unique instance identifier to each identified independent object and generates an instance identifier map with the same size as the RGB image. The value of each pixel in the instance identifier map corresponds to the instance identifier of the object at that location. The pixel values ​​in areas without objects are set to specific values. Finally, the system outputs an independent two-dimensional instance identifier for each object.

[0033] S3: Perform 3D spatial fusion processing on the 2D semantic mask and the depth information in the original dataset, map the pixels in the 2D semantic mask to the 3D space, generate a 3D scene point cloud describing the true geometry of the channel, and separate the 3D point cloud clusters corresponding to each occupying object from the 3D scene point cloud based on the independent 2D instance identifiers.

[0034] Specifically, before data fusion, the system performs intrinsic parameter calibration on the RGB-D camera, using the Zhang Zhengyou calibration method to obtain the camera's intrinsic parameter matrix and distortion coefficients. Based on the calibration results, the system performs distortion correction on the depth image to eliminate depth measurement errors caused by lens distortion. Simultaneously, the system removes invalid values ​​from the depth image, marking pixels corresponding to invalid depth values ​​as invalid and using neighborhood interpolation to fill in invalid pixels to ensure the integrity of the depth image. Based on the camera imaging principle, the system maps pixels within a two-dimensional semantic mask to a three-dimensional world coordinate system. During the mapping process, the system completes the conversion from two-dimensional pixel coordinates to three-dimensional world coordinates using a preset mapping formula, which is related to the two-dimensional pixel coordinate depth value and the camera intrinsic parameters.

[0035] Furthermore, the system iterates through all non-zero pixels in the two-dimensional semantic mask, calculates the three-dimensional coordinates corresponding to each pixel using a mapping formula, and associates them with their color information in the RGB image to generate a three-dimensional scene point cloud. The three-dimensional scene point cloud includes the three-dimensional spatial information of fixed structures such as channels, floors, walls, and columns, as well as all occupying objects. Each point cloud data point contains three-dimensional coordinates and color information. The system establishes a mapping relationship between two-dimensional pixels and three-dimensional point clouds based on the independent two-dimensional instance identifiers of each occupied object generated earlier. Specifically, for each point in the three-dimensional scene point cloud, the system queries the instance identifier in the instance identifier image based on its corresponding two-dimensional pixel coordinates, and groups all three-dimensional points corresponding to the same instance identifier into one category to form a three-dimensional point cloud cluster corresponding to that occupied object. To improve the purity of the point cloud clusters, the system performs noise filtering on each initial point cloud cluster. The noise filtering process sequentially executes a statistical filtering algorithm and a radius filtering algorithm. The statistical filtering algorithm removes isolated noise points by selecting points whose neighboring point distances meet preset conditions, while the radius filtering algorithm further optimizes the point cloud quality by selecting points whose number of neighboring points within a specific range meets preset conditions. Finally, the system outputs a pure 3D point cloud cluster corresponding to each occupied object, and each point cloud cluster is associated with a unique instance identifier and a 2D instance identifier to form a one-to-one correspondence.

[0036] S4: Perform bottleneck analysis on the 3D scene point cloud and each 3D point cloud cluster to determine the main extension direction of the channel, locate the key obstacle points that most affect passage, construct a spatial cross section for quantitative evaluation, delineate the theoretical passage area on the spatial cross section, and generate a dynamic evaluation cross section and a standard passage area.

[0037] Specifically, the system employs principal component analysis (PCA) to analyze the channel ground point cloud within the 3D scene point cloud. The channel ground point cloud is obtained through semantic masking, retaining only point clouds within a specific height range within the channel region. The system first calculates the covariance matrix of the ground point cloud, obtaining three eigenvalues ​​and corresponding eigenvectors through eigenvalue decomposition. The eigenvector corresponding to the largest eigenvalue is the main extension direction of the channel, representing the longest extension axis of the channel. To verify the accuracy of the direction, the system calculates the projected length of the ground point cloud along this direction and the projected width perpendicular to it. The ratio of the projected length to the projected width is used to determine the accuracy of the direction. If the ratio meets a preset condition, the direction is confirmed as the main extension direction of the channel; otherwise, the system re-filters the ground point cloud and performs PCA again. The system slices the 3D scene point cloud along the main extension direction of the channel at a preset step size, obtaining multiple sliced ​​point clouds parallel to and perpendicular to this direction. The system calculates the effective passage width of the channel for each sliced ​​point cloud, which is the maximum lateral distance of the channel region within the slice. Simultaneously, the system records the position coordinates of each slice.

[0038] Preferably, the system sorts the effective passage widths of all slices and selects slices with smaller effective passage widths as candidate bottleneck slices. For each candidate bottleneck slice, the system extracts the vertices and edge points of the object point cloud clusters that occupy them; these points are the key obstacle points affecting passage. The system calculates the occlusion ratio of the key obstacle points on the passage width and height, and finally determines the obstacle point corresponding to the slice with the largest occlusion ratio as the core key obstacle point. The location of this slice is the passage bottleneck position. The system constructs a plane perpendicular to the main extension direction of the passage as a dynamic evaluation section centered on the passage bottleneck position. The range of the dynamic evaluation section covers the full width and standard clearance of the passage. Based on the standard clearance width and standard clearance height parameters of the passage in the standard parameter set, the system delineates the theoretical passage area on the dynamic evaluation section. This area is the area after deducting the fixed walls, columns, and other immovable structures on both sides of the passage, and is defined as the maximum passage area under the ideal state of the passage. The boundary of the standard passage area is represented by a polygon coordinate set. The coordinates of each vertex are obtained by the system through fitting the fixed structure point cloud in the 3D scene point cloud to ensure consistency with the actual passage structure.

[0039] S5: Perform comprehensive availability calculations on the dynamic assessment section, standard passage area, and 3D point cloud clusters; analyze the projection occupancy of each obstacle on the spatial section; comprehensively calculate the remaining proportion of passage capacity in width and height; and generate fire lane assessment results containing a comprehensive access availability index.

[0040] Specifically, the system projects the 3D point cloud clusters corresponding to each occupying object onto the dynamic evaluation section along the main extension direction of the channel, obtaining a 2D projection area of ​​each occupying object on the section. The system uses a polygon scanning algorithm to calculate the area of ​​each projection area and accumulates the areas of all projection areas to obtain the total occupied area. Simultaneously, the system uses an edge detection algorithm to extract the lateral and longitudinal boundaries of each projection area, determining the lateral and longitudinal occupancy range of each obstacle on the section. The total lateral width of the standard passage area is taken from a standard parameter set. The system calculates the maximum continuous lateral width not occupied by obstacle projections on the dynamic evaluation section to obtain the width surplus value. The width surplus ratio is calculated by the ratio of the width surplus value to the total lateral width. The total longitudinal height of the standard passage area is taken from a standard parameter set. The system calculates the maximum continuous longitudinal height not occupied by obstacle projections within the width surplus area to obtain the height surplus value. The height surplus ratio is calculated by the ratio of the height surplus value to the total longitudinal height. If multiple width surplus areas exist, the system selects the area with the largest width surplus value as the core passage area and calculates the height surplus value based on this area. The system uses a weighted summation algorithm to calculate the comprehensive availability index of the channel. The weight coefficients are set according to the actual needs of fire rescue operations. The comprehensive availability index is obtained by adding the product of the remaining width ratio and the corresponding weight to the product of the remaining height ratio and the corresponding weight.

[0041] Furthermore, the system adjusts the comprehensive availability index based on obstacle type. Large obstacles and easily removable obstacles will lower the comprehensive availability index. The comprehensive availability index has a range of values, and the system classifies the passage status into different levels based on these values: fully available, basically available, restricted access, and impassable. The system generates structured data for fire lane assessments, including assessment time, lane number, dynamic assessment cross-section coordinates, standard access area parameters, obstacle instance identifiers and occupancy information, width remaining percentage, height remaining percentage, comprehensive availability index, and passage status level. The assessment results can be output to the fire monitoring platform in JSON format and can also generate visual charts.

[0042] In summary, the fire lane monitoring method provided in this application effectively overcomes the core shortcomings of existing fire lane monitoring schemes, as pointed out in the background art, which can only make binary judgments of "occupancy" and lack three-dimensional spatial perception capabilities, as well as being disconnected from emergency decision-making. By constructing a complete technical chain of "two-dimensional analysis - three-dimensional reconstruction - bottleneck analysis - quantitative evaluation," a leap from qualitative judgment to quantitative analysis can be achieved. Specifically, this method can achieve accurate three-dimensional spatial measurement of fire lane capacity. By fusing depth information to reconstruct the real geometric scene, it can achieve immunity to inherent defects of two-dimensional vision such as parallax and occlusion. Furthermore, by dynamically locating the bottleneck section and calculating the projected occupied area, a comprehensive and continuous quantitative evaluation of the remaining capacity in both the width and height dimensions of the lane can be achieved, generating an intuitive availability index. Ultimately, this quantitative evaluation result can directly serve fire emergency command and decision-making, enabling optimal path planning and risk assessment based on the real-time capacity of the lane, thereby significantly improving the accuracy and practicality of intelligent fire lane monitoring.

[0043] In one embodiment, S1 of the fire escape monitoring method provided by the present invention specifically includes the following steps:

[0044] S11: Synchronously collect data on the fire lane monitoring area using the calibrated stereo vision equipment, and simultaneously acquire the left-view color image, right-view color image, and depth image generated in real time by the stereo vision equipment of the fire lane monitoring area, generating an original dataset containing the left-view color image, right-view color image, and depth image.

[0045] Specifically, the stereo vision device consists of two imaging units (left and right) and a corresponding control and computing unit. It features parallel dual-channel image acquisition and real-time depth calculation. The system sends acquisition commands to the stereo vision device via the device control interface, triggering both imaging units to simultaneously acquire left and right view color images of the fire escape monitoring area. Simultaneously with image acquisition, the system triggers the stereo vision device's built-in depth calculation module. This module performs pixel-level matching calculations on the synchronously acquired left and right view color images based on the principle of binocular vision matching, generating a real-time depth image of the corresponding monitoring area by calculating the disparity information of the corresponding pixels.

[0046] To ensure spatiotemporal consistency of the data, the system extracts the acquisition timestamps of the left-view color image, right-view color image, and depth image. A timestamp alignment algorithm is used to synchronize and associate the three types of image data. During the synchronization process, invalid data with mismatched timestamps is discarded. After synchronization, the system integrates and stores the left-view color image, right-view color image, and corresponding depth image according to preset file naming rules and data storage formats, generating a raw dataset containing the left-view color image, right-view color image, and depth image. Each data entry in the raw dataset contains a unique data identifier and the corresponding three types of image data. The system stores the raw dataset in a designated storage unit to support subsequent steps.

[0047] S12: Perform intrinsic and extrinsic parameter calibration on the binocular stereo vision device to obtain the camera intrinsic parameter matrix characterizing the relationship between the lens optical characteristics and spatial position and the extrinsic parameter matrix describing the relative pose of the left and right cameras, and generate a camera calibration parameter set.

[0048] Specifically, before calibration begins, the system controls the binocular stereo vision device to align with a standard calibration board, which is a pre-defined regular pattern board. The system uses the device driver module to control the imaging unit to adjust the focal length and exposure parameters to ensure clear imaging of the calibration board. During calibration, the system calls a pre-defined calibration algorithm. First, it controls the binocular stereo vision device to acquire multiple sets of left and right view images of the standard calibration board from different angles and positions. After acquisition, the system performs corner detection on each set of images and extracts the pixel coordinates of pre-defined feature points on the standard calibration board. Based on the extracted feature point pixel coordinates, the system calculates the camera intrinsic parameter matrix, which characterizes the optical properties of the lens, using the calibration algorithm. The intrinsic parameter matrix contains core parameters such as lens focal length and principal point coordinates. These parameters describe the projection relationship of light from three-dimensional space to the two-dimensional image plane.

[0049] Furthermore, the system analyzes the pixel coordinate differences of the same feature point in the left and right view images to obtain the extrinsic parameter matrix describing the relative pose relationship between the left and right cameras. The extrinsic parameter matrix includes a rotation matrix and a translation vector. The rotation matrix represents the rotational relationship between the left and right camera coordinate systems, and the translation vector represents the translational relationship between them. After the intrinsic and extrinsic parameter matrices are calculated, the system verifies their validity using a reprojection error verification method. The calculated intrinsic and extrinsic parameters are substituted into the projection formula to calculate the deviation between the reprojected coordinates of the feature point and the actual detected coordinates. If the deviation meets the preset verification conditions, the calibration is deemed valid. After successful verification, the system encapsulates the intrinsic and extrinsic parameter matrices in a structured data format. During encapsulation, attribute information such as parameter type and meaning is added to each parameter, generating a camera calibration parameter set and storing it in the parameter database.

[0050] S13: Call the preset passage standard threshold, obtain and load the minimum rated passage width and minimum rated passage height of fire trucks as passage standard parameters, and integrate the camera calibration parameter set to generate a standard parameter set.

[0051] Specifically, the system calls the preset access control threshold interface, accesses the built-in threshold library, and queries and retrieves the corresponding access control standard parameters for the fire lane. These standard parameters are the minimum rated width and minimum rated height for fire trucks, which are the core benchmarks for determining whether a fire lane is passable. After obtaining these parameters, the system performs standardization processing, converting the units of the parameters to spatial coordinate units consistent with subsequent 3D spatial calculations and image pixel mapping. This ensures the consistency of the parameters in subsequent calculations and avoids calculation errors caused by unit differences.

[0052] Furthermore, the system retrieves the camera calibration parameter set and integrates the standardized minimum rated passage width and minimum rated passage height of fire trucks with the camera calibration parameter set. During the integration process, the system uses the unique identifier of the stereo vision device and the unique code of the fire lane monitoring area as association keywords to establish a correspondence between the standard passage parameters and the camera calibration parameters, ensuring that the parameters accurately match the corresponding monitoring area and acquisition equipment. This integration process generates a standard parameter set, which includes the standard passage parameters, the camera calibration parameter set, and related metadata. The metadata covers information such as parameter acquisition time, standard reference identifier, stereo vision device identifier, and monitoring area code. The standard reference identifier indicates the specific fire protection code upon which the standard passage parameters are based, providing support for the authority and compliance of the parameters.

[0053] In one embodiment, step S2 of the fire escape monitoring method provided by the present invention specifically includes the following steps:

[0054] S21: Perform feature encoding processing on the left-view color images in the original dataset based on a deep convolutional neural network. Extract multi-level visual features of the images through multi-layer convolution and pooling operations to generate high-dimensional feature maps.

[0055] Specifically, the system first retrieves the left-view color image from the original dataset. A data preprocessing module then converts the image into an input format recognizable by the deep convolutional neural network. After preprocessing, the system inputs the processed left-view color image into the pre-defined encoding module of the deep convolutional neural network. The encoding module consists of multiple layers of alternating convolutional and pooling layers, and the system controls the encoding module to perform convolution and pooling operations sequentially.

[0056] During convolution, the system performs sliding calculations on image pixels using convolution kernels to extract local feature information. After each convolution operation, the system calls an activation function to perform a non-linear transformation on the convolution result, enhancing feature representation capabilities. Pooling operations are performed alternately with convolution operations to downsample the feature maps extracted by convolution, reducing the size and number of parameters of the feature maps while retaining key feature information, achieving feature reduction and abstraction. As the number of network layers increases, convolution and pooling operations iterate continuously, and the system gradually extracts multi-level visual features of the image from low to high levels. Low-level features include basic information such as image edges and textures, while high-level features include abstract information such as the semantic structure of the image. After multiple convolution and pooling operations, the system generates a high-dimensional feature map representing the overall semantic information of the left-view color image. Each feature point in the high-dimensional feature map corresponds to a specific region of the original image, integrating multi-level feature information of that region.

[0057] S22: Perform channel region semantic parsing on the high-dimensional feature map, map the features back to the original image resolution through the decoder network structure, and perform binary classification on each pixel to determine whether it is a fire lane ground, generating a pixel-level two-dimensional semantic mask.

[0058] Specifically, the system retrieves high-dimensional feature maps from the feature cache unit and inputs them into a pre-defined decoder network structure. The decoder network structure employs a progressive upsampling approach, using deconvolution operations to restore the size of the high-dimensional feature maps, mapping the abstract high-dimensional features back to a resolution consistent with the original left-view color image. During upsampling, the system uses a skip connection mechanism to fuse feature maps extracted from different levels of the encoding module with feature maps from corresponding levels of the decoder, supplementing detailed feature information and improving the accuracy of semantic parsing. After feature mapping is complete, the system performs pixel-level classification processing on the mapped feature maps through the output layer of the decoder network. The classification task is binary classification, i.e., determining whether the area corresponding to each pixel is the ground of a fire lane.

[0059] Furthermore, the system calculates the probability value of each pixel belonging to the fire lane ground category using a classification function. Based on a preset classification threshold, it determines the classification result for each pixel, classifying pixels with probability values ​​greater than the threshold as fire lane ground pixels and pixels with probability values ​​less than or equal to the threshold as non-fire lane ground pixels. After classification, the system generates a pixel-level two-dimensional semantic mask based on the classification results. The two-dimensional semantic mask has the same size as the original left-view color image and distinguishes between fire lane ground areas and non-fire lane ground areas by different pixel values. The system performs post-processing on the generated two-dimensional semantic mask, using morphological processing to eliminate small noise and holes in the mask, optimizing the mask's integrity and accuracy.

[0060] S23: Perform occupancy object instance parsing on the high-dimensional feature map, distinguish and locate different occupancy objects in the image through the region proposal and mask prediction branches in the network, and generate category labels for each occupancy object and independent two-dimensional instance identifiers of the bounding contours of each occupancy object.

[0061] Specifically, the system retrieves a high-dimensional feature map and activates the region proposal branch and mask prediction branch in the preset instance segmentation network. These two branches work in parallel to distinguish and locate occupiers. The region proposal branch first performs a sliding window scan of the high-dimensional feature map, generating multiple candidate regions that may contain occupiers. Each candidate region is represented by its bounding box coordinates. The system calculates the confidence level of each candidate region containing an occupier using a region scoring function, selecting candidate regions with confidence levels meeting preset conditions as valid candidate regions and discarding invalid candidate regions with low confidence levels, thus achieving preliminary screening of candidate regions. The mask prediction branch, for the selected valid candidate regions, performs refined feature extraction on the corresponding regions in the high-dimensional feature map and generates a binary mask corresponding to each valid candidate region through a mask generation network. The binary mask is used to accurately describe the bounding contour of the occupier.

[0062] Furthermore, the system identifies the category of occupied objects within each valid candidate region through a classification branch, generating category labels for each occupied object. These category labels identify the specific type of the occupied object. After completing candidate region filtering, mask generation, and category identification, the system uses a non-maximum suppression algorithm to remove duplicates from overlapping candidate regions, retaining the candidate region with the highest confidence level and its corresponding mask and category label. Subsequently, the system assigns a unique identifier to each deduplicated occupied object and associates this identifier with the corresponding bounding contour mask to generate an independent two-dimensional instance identifier for each occupied object's bounding contour. This independent two-dimensional instance identifier has the same size as the original left-view color image, and different identifier values ​​distinguish different occupied objects.

[0063] In one embodiment, step S3 of the fire escape monitoring method provided by the present invention specifically includes the following steps:

[0064] S31: Perform mask filtering on the depth images in the original dataset based on two-dimensional semantic masks, retain only the depth pixel values ​​corresponding to the channel regions marked in the two-dimensional semantic masks, and generate a filtered depth map of the channel regions.

[0065] Specifically, the system extracts a two-dimensional semantic mask from the semantic data unit and simultaneously extracts the corresponding depth image from the original dataset. By comparing the acquisition timestamps and device acquisition parameters of the two-dimensional semantic mask and the depth image, it confirms that the monitoring time and monitoring range are completely consistent, ensuring a precise one-to-one correspondence between pixel coordinates. After successful verification, the system constructs mask filtering rules, specifying that pixels in the two-dimensional semantic mask used to mark channel regions are valid identifiers, while other pixels are invalid identifiers.

[0066] During the filtering phase, the system traverses each pixel of the depth image in row-major order, associating the pixel coordinates with the corresponding identifier information in the two-dimensional semantic mask, and selectively retaining the depth value: when the associated identifier is valid, the depth value of the current pixel is retained; when the associated identifier is invalid, the depth value of the current pixel is discarded and marked as blank. Preferably, to address the potential edge data breakage problem in the filtered depth image, the system performs edge smoothing processing. A neighboring pixel interpolation algorithm is used to fill blank pixels at the edges of the effective region. The mean of the effective depth values ​​within a preset range around the blank pixels is calculated and used as the fill value to supplement the blank pixel position, ensuring the continuity of the channel region depth information. After the smoothing processing is completed, the system generates a filtered depth map containing only channel region depth information, which maintains the same size as the original depth image.

[0067] S32: Perform 3D coordinate inversion processing on the selected depth map of the channel region. Based on the coordinates and depth value of each effective pixel, combined with the intrinsic and extrinsic parameter matrices in the device calibration parameters of the standard parameter set, calculate the 3D coordinates of the spatial points corresponding to the effective pixels in the world coordinate system, and aggregate all points to generate a 3D scene point cloud.

[0068] Specifically, the system retrieves the filtered depth map from the depth data unit and extracts the intrinsic and extrinsic parameter matrices from the standard parameter set, which are included in the device calibration parameters. These two types of parameters are then loaded into the coordinate transformation module and initialized. The system initiates the coordinate inversion process, traversing each valid pixel in the filtered depth map in a preset order. It extracts the two-dimensional coordinates of each valid pixel on the image plane and its corresponding depth value, using these two types of data as the basic input data for coordinate inversion. The coordinate transformation module, based on the pinhole model of camera imaging, constructs a complete coordinate transformation formula by combining the loaded intrinsic and extrinsic parameter matrices. The intrinsic parameter matrix corrects projection deviations caused by lens optical characteristics, adjusting the mapping relationship between pixel coordinates and the camera coordinate system through focal length and principal point coordinate parameters. The extrinsic parameter matrix realizes the pose transformation from the camera coordinate system to the world coordinate system, using a rotation matrix to represent the rotation angle of the camera coordinate system relative to the world coordinate system and a translation vector to represent the positional offset between the two coordinate systems.

[0069] Furthermore, the system calculates the input data for each valid pixel using coordinate transformation formulas. First, it converts the pixel coordinates to 3D coordinates in the camera coordinate system, and then converts them to 3D coordinates in the world coordinate system using an extrinsic parameter matrix, completing a precise mapping from the image plane to 3D space. After traversal, the system aggregates the 3D coordinates in the world coordinate system corresponding to all valid pixels, forming a 3D point set containing all spatial points in the channel region. To optimize data volume and improve subsequent processing efficiency, the system performs redundant point removal on this 3D point set, deleting redundant points with completely identical coordinates by comparing the overlap of 3D coordinates. After redundant point removal, the system encapsulates the processed 3D point set into a 3D scene point cloud, which contains the world coordinate system 3D coordinate information of all spatial points in the channel region.

[0070] S33: Based on the independent two-dimensional instance identifiers, perform two-dimensional to three-dimensional instance matching on the three-dimensional scene point cloud. For the contour of each instance, find all three-dimensional points in the three-dimensional scene point cloud that fall within the contour after being projected onto the image plane, forming a candidate three-dimensional point set for each instance.

[0071] Specifically, the system retrieves the identifiers of each independent 2D instance and the 3D scene point cloud. Based on the intrinsic and extrinsic parameter matrices, it constructs a projection mapping relationship between the image plane and the 3D world coordinate system. This mapping relationship enables the back projection calculation of 3D points in the world coordinate system onto the pixel coordinates of the image plane, ensuring that 3D points and image pixels can be accurately associated. The system performs a matching operation on each independent 2D instance identifier sequentially, extracting the object contour range corresponding to the current instance identifier. This contour range is represented by the set of all pixel coordinates corresponding to the same identifier value in the instance identifier image. Further, the system traverses each 3D point in the 3D scene point cloud, calculates the pixel coordinates of the 3D point projected onto the image plane using the preset projection mapping relationship, and compares the calculated pixel coordinates with the contour range of the current instance identifier to determine whether the pixel coordinates fall within the contour range. If the pixel coordinates fall within the contour range, the system marks the 3D point as an associated point of the current instance; if the pixel coordinates do not fall within the contour range, the system skips the 3D point and continues traversing. After the matching of a single instance identifier is completed, the system aggregates all 3D points marked as associated points to form a candidate 3D point set corresponding to the instance.

[0072] Preferably, the system sequentially completes the matching operation of all independent two-dimensional instance identifiers according to the above process, generating candidate three-dimensional point sets corresponding one-to-one with each instance. Each candidate three-dimensional point set is associated with a unique instance identifier. The system binds and stores each candidate three-dimensional point set with its corresponding instance identifier, and records the projection calculation error of each three-dimensional point. This error is the deviation between the projected pixel coordinates and the actual contour pixel coordinates, which is used to provide a reference for subsequent point set optimization steps. After storage, the system performs a matching integrity check, which determines whether the candidate three-dimensional point set completely covers the three-dimensional spatial region corresponding to the instance by checking whether the number of points in each candidate three-dimensional point set matches the number of pixels in the corresponding instance contour.

[0073] S34: Perform spatial outlier filtering and clustering on the candidate 3D point sets of each instance. Based on the Euclidean distance between 3D points, aggregate the spatially closely distributed points into a cluster, while removing sparse noise points, and generate a 3D point cloud cluster that precisely corresponds to each occupied object.

[0074] Specifically, the system processes each candidate 3D point set sequentially, using a density-based outlier detection algorithm. This algorithm calculates the Euclidean distance between each 3D point and other 3D points in its neighborhood, and statistically analyzes the distribution of 3D points within a preset neighborhood for each point. Based on a preset density threshold, the system assesses the number of neighboring points for each 3D point, identifying and removing those with fewer neighboring points than the threshold. The system retains the core 3D points whose neighboring points meet the density threshold, completing the initial outlier filtering. After outlier filtering, the system initiates a clustering unit, employing an Euclidean distance-based clustering algorithm to cluster the remaining core 3D points. The system sets a threshold for the Euclidean distance between 3D points, iterates through all core 3D points, and aggregates 3D points with Euclidean distances less than the threshold into a cluster, achieving precise aggregation of spatially densely distributed points.

[0075] During clustering, the system monitors the distribution density of 3D points within each cluster in real time to ensure that all 3D points in each cluster meet the preset spatial compactness requirements. After clustering, the system performs a secondary screening of the generated clusters, identifying sparse clusters containing fewer than a preset threshold of 3D points as incompletely filtered noise points and removing them, while retaining valid clusters that meet the quantity requirements. After processing a single candidate 3D point set, the system uses the final valid clusters as the 3D point cloud clusters that precisely correspond to the corresponding occupied objects. The system sequentially completes the processing operations of all candidate 3D point sets according to the above process, generating a 3D point cloud cluster that precisely corresponds to each occupied object. Each 3D point cloud cluster is associated with a unique instance identifier, forming a precise correspondence with the corresponding occupied object. After processing, the system performs integrity and accuracy checks. By comparing the spatial contour of the 3D point cloud cluster with the contour of the corresponding 2D instance identifier, it determines whether the 3D point cloud cluster completely and accurately represents the spatial morphology of the occupied object; simultaneously, by detecting the uniformity of the distribution of 3D points within the 3D point cloud cluster, it determines whether there are any missing or redundant local data.

[0076] In one embodiment, step S4 of the fire escape monitoring method provided by the present invention specifically includes the following steps:

[0077] S41: Extract the main direction of spatial distribution of the point cloud in the 3D scene, calculate the covariance matrix of the point cloud data by principal component analysis and obtain the eigenvector of the covariance matrix, and determine the direction of the eigenvector corresponding to the largest eigenvalue as the main extension direction of the channel.

[0078] Specifically, the system retrieves the 3D scene point cloud from the point cloud cache unit, first preprocessing it to remove isolated and redundant points. After preprocessing, the system calculates the mean coordinates of all data points in the 3D scene point cloud, then calculates the deviation of each data point based on the mean coordinates and constructs a covariance matrix. The calculation formula is:

[0079]

[0080] in, This represents the 3D coordinates of the i-th data point in the world coordinate system within the 3D scene point cloud. This represents the mean of the three-dimensional coordinates of all data points. This represents the total number of data points in the point cloud of a 3D scene. This represents the generated covariance matrix. The system performs eigenvalue decomposition on the covariance matrix to obtain three eigenvalues ​​and their corresponding eigenvectors. The direction of the eigenvector corresponding to the largest eigenvalue is determined as the main extension direction of the channel.

[0081] S42: Based on the main extension direction, calculate the projection distance of each 3D point cloud cluster. Treat the main extension direction as a one-dimensional coordinate axis, calculate the projection coordinate value of all points in each point cloud cluster on this axis, and perform minimum value traversal processing on the projection coordinate values ​​of all point cloud clusters to find the global minimum coordinate value and locate the corresponding 3D spatial point, which will generate the key obstacle point.

[0082] Specifically, the system retrieves the feature vector parameters of the main extension direction and each 3D point cloud cluster, treats the main extension direction as a one-dimensional coordinate axis, and constructs a projection calculation model. The system sequentially traverses all 3D points for each 3D point cloud cluster, calculating the projected coordinate values ​​using the vector dot product principle. The formula for calculating the projected coordinate values ​​is:

[0083]

[0084] in, This represents the projected coordinates of the i-th 3D point on the main extension direction coordinate axis. Represents the coordinates of the origin of the projected coordinate system. This represents the unit eigenvector along the main extension direction. After all point cloud cluster projection calculations are completed, the system iterates through all projected coordinate values, filters out the global minimum value, locates the corresponding 3D spatial point, and marks it as a key obstacle point.

[0085] S43: Based on the key obstacle points and the main extension direction, an orthogonal plane is constructed. The key obstacle points are used as plane passing points, and the main extension direction is used as the plane normal vector to construct an infinitely extending two-dimensional plane and generate a dynamic evaluation section.

[0086] Specifically, the system retrieves the 3D coordinates of key obstacle points and the feature vector parameters of the main extension direction, initiates the plane construction process, and constructs an infinitely extending 2D plane with the key obstacle points as plane passages and the main extension direction as the plane normal vector. The general equation of the plane is:

[0087]

[0088] in, The unit eigenvector representing the main extension direction. Represents the three-dimensional coordinates of any point on the plane in the world coordinate system. The system represents the three-dimensional coordinates of the key obstacle points in the world coordinate system. The system verifies the accuracy of the plane using spatial geometry verification methods, calculating the angle between the eigenvector of the main extension direction and the plane normal vector to confirm it is a right angle, and simultaneously verifying that the coordinates of the key obstacle points satisfy the plane equations.

[0089] S44: Generate a standard rectangular area on the dynamic evaluation section. Using the position of the key obstacle point on the section as the reference origin, extend the minimum rated passage width and minimum rated passage height of the standard parameter centralized fire truck along two orthogonal directions of the section to generate a standard passage area.

[0090] Specifically, the system retrieves dynamic evaluation section parameters, three-dimensional coordinates of key obstacle points, and minimum rated passage width and height parameters of the fire truck. Using the projection point of the key obstacle point on the section as the reference origin, it transforms the point to obtain its two-dimensional coordinates on the section. The system determines two directions perpendicular to the main extension direction and orthogonal to each other on the section, corresponding to the passage width and height directions respectively. The calculation formulas for extending the boundary along the width direction and extending the boundary along the height direction are as follows:

[0091]

[0092]

[0093] in, The two-dimensional coordinates of the cross section representing the reference origin, This indicates the minimum rated passage width for fire trucks. Indicates the minimum rated clearance height for fire trucks. , These represent the boundary coordinates of the standard passage area in the width and height directions, respectively.

[0094] In one embodiment, step S5 of the fire escape monitoring method provided by the present invention specifically includes the following steps:

[0095] S51: For each 3D point cloud cluster, construct a 3D bounding box and perform vertex projection processing. First, fit the smallest volume hexahedron that can completely enclose each point cloud cluster. Then, project all vertices of each hexahedron vertically onto the dynamic evaluation section to generate a 2D projection vertex set of each object on the section.

[0096] Specifically, the system first retrieves all 3D point cloud clusters from the point cloud database, and simultaneously retrieves dynamic evaluation section parameters to complete data synchronization verification. Preferably, the system can use an axis-aligned bounding box algorithm to fit the minimum volume hexahedron that can completely enclose each point cloud cluster. The fitting process involves traversing the 3D point coordinates of each point cloud cluster, determining the extreme values ​​in each coordinate axis direction, and constructing the minimum volume hexahedron, with the following constraints:

[0097]

[0098]

[0099]

[0100] in, , These are the minimum and maximum coordinates of the point cloud cluster along the x-axis, respectively. , These are the extreme values ​​along the y-axis. , These represent the extreme values ​​along the z-axis. After fitting, the system extracts the coordinates of the eight vertices of each hexahedron and performs vertical projection processing based on the plane equation of the dynamic evaluation section. The projection direction is the opposite direction of the normal vector of the dynamic evaluation section, and the projection formula is:

[0101]

[0102] in, These are the coordinates of the projected vertex. These are the original coordinates of the vertices of the hexahedron. The normal vector of the cross section. The coordinates of the points passing through the cross section are given. Further, the system aggregates all projected vertices corresponding to each object, generating a two-dimensional projection vertex set for each object on the cross section.

[0103] S52: Based on the two-dimensional projection vertex set of each object, calculate the projection area occupied by each object, connect the projection vertices of each object to form a convex polygon, and calculate the area of ​​each convex polygon on the dynamic evaluation section to generate the projection area occupied by each object.

[0104] Specifically, the system retrieves the two-dimensional projection vertex sets of each object from the projection data unit, sorts each vertex set, and arranges the vertex order in a clockwise or counterclockwise direction. The system uses the Graham scan method to connect the projection vertices of each object to form convex polygons, ensuring that the polygons completely enclose the projection area. Preferably, the area of ​​each convex polygon on the dynamic evaluation section can be calculated using the shoelace formula. The calculation formula is:

[0105]

[0106] in, The number of vertices of the convex polygon. Let be the two-dimensional coordinates of the i-th vertex. The system verifies the validity of the calculated area values, excludes invalid data with an area of ​​zero, and generates the projected area occupied by each object.

[0107] S53: Calculate the total occupied area and width availability index within the standard passage area. Merge the polygons corresponding to the projected occupied areas of all objects, calculate the total area of ​​the merged area, and calculate the ratio of the difference between the standard passage area and the total area of ​​the merged area to the standard passage area to generate the width availability index.

[0108] Specifically, the system retrieves the projected area occupied by each object and the coordinates of its corresponding convex polygon from the area data unit, and retrieves the boundary coordinates of the standard passage area from the standard area unit. The system uses the Sutherland-Hodgman algorithm to perform an intersection operation between the projected convex polygons of all objects and the standard passage area, retaining the polygon portions located within the standard passage area. Then, a region merging algorithm is used to merge the overlapping polygon regions, calculating the total area of ​​the merged region, i.e., the total occupied area. Preferably, the system retrieves the area parameters of the standard passage area to calculate the width availability index, using the following formula:

[0109]

[0110] in, Width availability index, The standard passage area is [area]. This represents the total area occupied after the merger.

[0111] S54: Based on each 3D point cloud cluster and the minimum rated passage height, calculate the height availability factor. Extract the coordinates of the highest point in the vertical direction from all point cloud clusters and determine the global maximum value. Calculate the height difference between the minimum rated passage height and the global maximum value. Combine the minimum rated passage height to calculate the ratio of the height difference to the minimum rated passage height. Constrain the result to the range of 0 to 1 to generate the height availability factor.

[0112] Specifically, the system retrieves all 3D point cloud clusters from the point cloud database, iterates through the 3D point coordinates of each cluster, extracts the coordinates of the highest point in the vertical direction (z-axis direction), and records the z-value of the highest point in each cluster. The system then performs a global traversal of all the highest z-values, selecting the global maximum value, which is the highest height of the object occupying the channel. The calculation formula is as follows:

[0113]

[0114] in, The number of 3D point cloud clusters, Let z be the highest point value of the i-th point cloud cluster. The system retrieves the minimum rated clearance height for fire trucks from the standard parameter set. The formula for calculating the height difference is as follows:

[0115]

[0116] Furthermore, the system calculates the high availability factor using the following formula:

[0117]

[0118] in, As a high availability factor, the result is constrained to the range of 0 to 1 by using the max function.

[0119] S55: Perform a comprehensive evaluation of the width availability index and the high availability factor, combine the width availability index and the high availability factor to obtain the comprehensive availability index of the passage, match the corresponding warning level to the comprehensive availability index of the passage according to the preset warning threshold range, and generate a fire passage evaluation result that includes the comprehensive availability index of the passage and the warning level. The fire passage evaluation result is used to indicate the real-time availability status and risk level of the fire passage.

[0120] Specifically, the system retrieves the width availability index $$I_w$$ and the height availability factor $$I_h$$ from the availability data unit, and calculates the channel comprehensive availability index using a weighted summation algorithm. The calculation formula is as follows:

[0121]

[0122] in, As a comprehensive availability index, The weighting coefficients for the width availability index are... is the weighting coefficient of the high availability factor, and The system retrieves preset warning threshold ranges, matches the comprehensive availability index with these ranges, and determines the corresponding warning level. The system generates a fire lane assessment result that includes the comprehensive availability index of the passage, the warning level, the assessment time, the passage number, and information on any obstructing objects. The assessment result is stored in a structured format and output to the fire monitoring platform to indicate the real-time availability status and risk level of the fire lane, providing data support for fire rescue decision-making.

[0123] After the assessment results are output, firefighters and maintenance personnel will perform corresponding actions based on the warning level: If the warning level is high-risk (corresponding to a comprehensive availability index below the preset low threshold), firefighters will immediately dispatch nearby forces to the scene, verify the type and scale of the obstructing object, formulate a clearing plan and execute it quickly, and simultaneously plan alternative rescue routes; maintenance personnel will arrive at the scene simultaneously to assist in clearing the obstruction and set up warning signs to prevent secondary obstruction. If the warning level is medium-risk, maintenance personnel will arrange a special inspection, inform the entity to which the obstructing object belongs, urge them to clear it within a specified period, and after clearing, report back to the fire monitoring platform, where the system will reassess the passage. If the warning level is low-risk, maintenance personnel will include the passage status in their daily inspection checklist and periodically check its accessibility.

[0124] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0125] Based on the same inventive concept, this application also provides a fire lane monitoring device for implementing the fire lane monitoring method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations of one or more fire lane monitoring device embodiments provided below can be found in the limitations of the fire lane monitoring method described above, and will not be repeated here.

[0126] Preferably, such as Figure 2 As shown, the present invention provides a fire escape monitoring device 600, which is configured with the following modules:

[0127] The multimodal data acquisition module 610 is used to acquire multimodal data of the fire lane monitoring area, simultaneously acquire image data containing color and depth information, acquire predefined standard parameters for passage, and generate raw datasets and standard parameter sets.

[0128] The color information parsing module 620 is used to intelligently parse and process the color information in the original dataset, identify and separate the channel regions and occupying objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object.

[0129] The 3D spatial fusion module 630 is used to perform 3D spatial fusion processing on the depth information in the 2D semantic mask and the original dataset, mapping the pixels in the 2D semantic mask to the 3D space, generating a 3D scene point cloud that describes the true geometric shape of the channel, and separating the 3D point cloud clusters corresponding to each occupying object from the 3D scene point cloud based on each independent 2D instance identifier.

[0130] The bottleneck analysis module 640 is used to perform bottleneck analysis on the 3D scene point cloud and each 3D point cloud cluster, determine the main extension direction of the channel, locate the key obstacle point that most affects the passage, construct a spatial cross section for quantitative evaluation, delineate the theoretical passage area on the spatial cross section, and generate a dynamic evaluation cross section and a standard passage area.

[0131] The access availability assessment module 650 is used to perform comprehensive availability calculations on dynamic assessment sections, standard passage areas, and three-dimensional point cloud clusters. It analyzes the projection occupancy of each obstacle on the spatial section, comprehensively calculates the remaining proportion of passage capacity in width and height, and generates fire access assessment results containing a comprehensive access availability index.

[0132] Preferably, the multimodal data acquisition module 610 provided in this application is configured with the following units:

[0133] The stereo vision data acquisition unit is used to synchronously acquire data of the fire lane monitoring area through the calibrated stereo vision equipment. At the same time, it acquires the left view color image, the right view color image, and the depth image generated in real time by the stereo vision equipment of the fire lane monitoring area, and generates a raw dataset containing the left view color image, the right view color image, and the depth image.

[0134] The camera parameter calibration unit is used to perform intrinsic and extrinsic parameter calibration processing on the binocular stereo vision device, obtain the camera intrinsic parameter matrix that characterizes the relationship between the optical characteristics of the lens and the spatial position, and the extrinsic parameter matrix that describes the relative pose of the left and right cameras, and generate a camera calibration parameter set.

[0135] The standard parameter integration unit is used to call the preset passage standard threshold, obtain and load the minimum rated passage width and minimum rated passage height of fire trucks as passage standard parameters, and integrate the camera calibration parameter set to generate a standard parameter set.

[0136] Preferably, the color information parsing module 620 provided in this application is configured with the following units:

[0137] The image feature encoding unit is used to perform feature encoding processing on the left-view color image in the original dataset based on a deep convolutional neural network. It extracts multi-level visual features of the image through multi-layer convolution and pooling operations to generate a high-dimensional feature map.

[0138] The channel semantic mask generation unit is used to perform channel region semantic parsing on the high-dimensional feature map. The features are mapped back to the original image resolution through the decoder network structure, and each pixel is classified into two categories: whether it is a fire lane ground or not, to generate a pixel-level two-dimensional semantic mask.

[0139] The Occupied Object Instance Parsing Unit is used to parse the high-dimensional feature map into occupies objects. It distinguishes and locates different occupies in the image through the region proposal and mask prediction branches in the network, and generates category labels for each occupies and independent two-dimensional instance identifiers of the bounding contours of each occupies.

[0140] Preferably, the three-dimensional spatial fusion module 630 provided in this application is configured with the following units:

[0141] The channel depth map filtering unit is used to perform mask filtering on the depth images in the original dataset based on the two-dimensional semantic mask, retaining only the depth pixel values ​​corresponding to the channel regions marked in the two-dimensional semantic mask, and generating a filtered depth map of the channel regions.

[0142] The 3D coordinate inversion unit is used to perform 3D coordinate inversion processing on the selected depth map of the channel region. Based on the coordinates and depth value of each effective pixel, combined with the intrinsic and extrinsic parameter matrices in the device calibration parameters of the standard parameter set, the 3D coordinates of the spatial points corresponding to the effective pixels in the world coordinate system are calculated, and all points are aggregated to generate a 3D scene point cloud.

[0143] The instance point cloud matching unit is used to perform two-dimensional to three-dimensional instance matching on the three-dimensional scene point cloud based on each independent two-dimensional instance identifier. For the contour of each instance, it searches for all three-dimensional points in the three-dimensional scene point cloud that fall within the contour after being projected onto the image plane, forming a candidate three-dimensional point set for each instance.

[0144] The point cloud clustering optimization unit is used to filter out spatial outliers and cluster candidate 3D point sets for each instance. Based on the Euclidean distance between 3D points, it aggregates spatially closely distributed points into a cluster, while removing sparse noise points, generating a 3D point cloud cluster that precisely corresponds to each occupying object.

[0145] Preferably, the bottleneck analysis module 640 provided in this application is configured with the following units:

[0146] The channel main direction extraction unit is used to extract the main direction of spatial distribution of point cloud in 3D scene. It calculates the covariance matrix of point cloud data by principal component analysis and obtains the eigenvector of covariance matrix. The direction of the eigenvector corresponding to the largest eigenvalue is determined as the main extension direction of the channel.

[0147] The obstacle point localization calculation unit is used to calculate the projection distance of each three-dimensional point cloud cluster based on the main extension direction. The main extension direction is regarded as a one-dimensional coordinate axis. The projected coordinate values ​​of all points in each point cloud cluster on this axis are calculated. The minimum value traversal processing of the projected coordinate values ​​of all point cloud clusters is performed to find the global minimum coordinate value and locate the corresponding three-dimensional spatial point to generate key obstacle points.

[0148] The dynamic evaluation section construction unit is used to construct an orthogonal plane based on key obstacle points and the main extension direction. It constructs an infinitely extending two-dimensional plane with key obstacle points as plane passage points and the main extension direction as plane normal vector, thus generating the dynamic evaluation section.

[0149] The standard passage area generation unit is used to generate a standard rectangular area on the dynamic evaluation section. Taking the position of the key obstacle point on the section as the reference origin, it extends the minimum rated passage width and minimum rated passage height of the standard parameter centralized fire truck along two orthogonal directions of the section to generate the standard passage area.

[0150] Preferably, the channel availability assessment module 650 provided in this application is configured with the following units:

[0151] The point cloud cluster projection vertex generation unit is used to construct a 3D bounding box and process vertex projection for each 3D point cloud cluster. First, it fits the smallest volume hexahedron that can completely enclose each point cloud cluster. Then, it projects all vertices of each hexahedron vertically onto the dynamic evaluation section to generate a 2D projection vertex set of each object on the section.

[0152] The projection area calculation unit is used to calculate the projection area based on the two-dimensional projection vertex set of each object. It connects the projection vertices of each object to form a convex polygon, calculates the area of ​​each convex polygon on the dynamic evaluation section, and generates the projection area of ​​each object.

[0153] The width availability index calculation unit is used to calculate the total occupied area and width availability index within the standard passage area. It merges the polygons corresponding to the projected occupied area of ​​all objects, calculates the total area of ​​the merged area, and calculates the ratio of the difference between the standard passage area and the total area of ​​the merged area to the standard passage area area to generate the width availability index.

[0154] The high availability factor calculation unit is used to calculate the high availability factor based on each 3D point cloud cluster and the minimum rated passage height. It extracts the coordinates of the highest point in the vertical direction from all point cloud clusters and determines the global maximum value. It calculates the height difference between the minimum rated passage height and the global maximum value, and calculates the ratio of the height difference to the minimum rated passage height. It then constrains the result to the range of 0 to 1 to generate the high availability factor.

[0155] The comprehensive availability assessment unit for fire lanes is used to comprehensively assess the width availability index and the high availability factor. The width availability index and the high availability factor are combined to obtain the comprehensive availability index of the fire lanes. According to the preset warning threshold range, the comprehensive availability index of the fire lanes is matched with the corresponding warning level, and the fire lane assessment result containing the comprehensive availability index of the fire lanes and the warning level is generated. The fire lane assessment result is used to indicate the real-time availability status and risk level of the fire lanes.

[0156] In one embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described fire escape monitoring method.

[0157] In one embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described fire escape monitoring method.

[0158] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0159] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0160] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope disclosed in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for monitoring fire escape routes, characterized in that, Includes the following steps: S1: Perform multimodal data acquisition on the fire lane monitoring area, simultaneously acquire image data containing color and depth information, obtain predefined standard parameters for lane passage, and generate the original dataset and standard parameter set; S2: Perform intelligent parsing processing on the color information in the original dataset, identify and separate the channel regions and occupying objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object. S3: Perform three-dimensional spatial fusion processing on the two-dimensional semantic mask and the depth information in the original dataset, map the pixels in the two-dimensional semantic mask to three-dimensional space, generate a three-dimensional scene point cloud describing the true geometric shape of the channel, and separate the three-dimensional point cloud clusters corresponding to each occupying object from the three-dimensional scene point cloud according to each independent two-dimensional instance identifier. S4: Perform traffic bottleneck analysis on the three-dimensional scene point cloud and each of the three-dimensional point cloud clusters, determine the main extension direction of the channel, locate the key obstacle point that most affects traffic, construct a spatial cross section for quantitative evaluation, delineate the theoretical traffic area on the spatial cross section, and generate a dynamic evaluation cross section and a standard traffic area. S5: Perform a comprehensive availability calculation on the dynamic evaluation section, the standard passage area, and the three-dimensional point cloud cluster; analyze the projection occupancy of each obstacle on the spatial section; comprehensively calculate the remaining proportion of passage capacity in width and height; and generate a fire lane evaluation result containing a comprehensive access availability index.

2. The method according to claim 1, characterized in that, S1 includes: S11: Synchronously collect data on the fire lane monitoring area using the calibrated stereo vision device, and simultaneously acquire the left-view color image, the right-view color image, and the depth image generated in real time by the stereo vision device of the fire lane monitoring area, to generate an original dataset containing the left-view color image, the right-view color image, and the depth image. S12: Perform intrinsic and extrinsic parameter calibration on the binocular stereo vision device to obtain the camera intrinsic parameter matrix characterizing the relationship between the lens optical characteristics and spatial position and the extrinsic parameter matrix describing the relative pose of the left and right cameras, and generate a camera calibration parameter set. S13: Call the preset passage standard threshold, obtain and load the minimum rated passage width and minimum rated passage height of the fire truck as the passage standard parameters, and integrate the camera calibration parameter set to generate a standard parameter set.

3. The method according to claim 1, characterized in that, S2 includes: S21: Perform feature encoding processing on the left-view color image in the original dataset based on a deep convolutional neural network, extract multi-level visual features of the image through multi-layer convolution and pooling operations, and generate a high-dimensional feature map. S22: Perform channel region semantic parsing processing on the high-dimensional feature map, map the features back to the original image resolution through the decoder network structure, and perform binary classification on each pixel to determine whether it is a fire lane ground, and generate a pixel-level two-dimensional semantic mask. S23: Perform object instance parsing processing on the high-dimensional feature map, distinguish and locate different objects in the image through region proposal and mask prediction branches in the network, and generate category labels for each object and independent two-dimensional instance identifiers of the bounding contours of each object.

4. The method according to claim 1, characterized in that, S3 includes: S31: Based on the two-dimensional semantic mask, perform mask filtering on the depth image in the original dataset, retain only the depth pixel values ​​corresponding to the channel regions marked in the two-dimensional semantic mask, and generate a filtered depth map of the channel regions; S32: Perform three-dimensional coordinate inversion processing on the filtered depth map of the channel area. Based on the coordinates and depth value of each effective pixel, and combined with the intrinsic and extrinsic parameter matrices in the device calibration parameters of the standard parameter set, calculate the three-dimensional coordinates of the spatial points corresponding to the effective pixels in the world coordinate system, and aggregate all points to generate a three-dimensional scene point cloud. S33: Based on the independent two-dimensional instance identifiers of each instance, perform two-dimensional to three-dimensional instance matching on the three-dimensional scene point cloud. For the contour of each instance, find all three-dimensional points in the three-dimensional scene point cloud that fall within the contour after being projected onto the image plane, and form a candidate three-dimensional point set for each instance. S34: Perform spatial outlier filtering and clustering on the candidate 3D point sets of each instance. Based on the Euclidean distance between the 3D points, aggregate the spatially closely distributed points into a cluster, while removing sparse noise points, and generate a 3D point cloud cluster that precisely corresponds to each occupying object.

5. The method according to claim 1, characterized in that, S4 includes: S41: Extract the main direction of spatial distribution of the point cloud of the three-dimensional scene, calculate the covariance matrix of the point cloud data by principal component analysis and obtain the eigenvector of the covariance matrix, and determine the direction of the eigenvector corresponding to the largest eigenvalue as the main extension direction of the channel. S42: Based on the main extension direction, perform projection distance calculation on each of the three-dimensional point cloud clusters, regard the main extension direction as a one-dimensional coordinate axis, calculate the projection coordinate value of all points in each point cloud cluster on this axis, and perform minimum value traversal on the projection coordinate values ​​of all point cloud clusters to find the global minimum coordinate value and locate the corresponding three-dimensional spatial point, thereby generating the key obstacle point. S43: Based on the key obstacle point and the main extension direction, an orthogonal plane is constructed. The key obstacle point is used as the plane passing point, and the main extension direction is used as the plane normal vector to construct an infinitely extending two-dimensional plane and generate a dynamic evaluation section. S44: Generate a standard rectangular area on the dynamic evaluation section. Using the position of the key obstacle point on the section as the reference origin, extend the minimum rated passage width and minimum rated passage height of the standard parameter centralized fire truck along two orthogonal directions of the section to generate a standard passage area.

6. The method according to any one of claims 1-5, characterized in that, S5 includes: S51: For each of the three-dimensional point cloud clusters, construct a three-dimensional bounding box and perform vertex projection processing. First, fit the smallest volume hexahedron that can completely enclose each point cloud cluster. Then, project all vertices of each hexahedron vertically onto the dynamic evaluation section to generate a two-dimensional projection vertex set of each object on the section. S52: Based on the two-dimensional projection vertex set of each object, perform projection area calculation processing, connect the projection vertices of each object to form a convex polygon, and calculate the area of ​​each convex polygon on the dynamic evaluation section to generate the projection area of ​​each object. S53: Calculate the total occupied area and width availability index within the standard passage area. Merge the polygons corresponding to the projected occupied areas of all objects, calculate the total area of ​​the merged area, and calculate the ratio of the difference between the standard passage area and the total area of ​​the merged area to the standard passage area to generate the width availability index. S54: Based on each of the three-dimensional point cloud clusters and the minimum rated passage height, perform a height availability factor calculation process. Extract the coordinates of the highest point in the vertical direction from all point cloud clusters and determine the global maximum value. Calculate the height difference between the minimum rated passage height and the global maximum value. Combine the minimum rated passage height to calculate the ratio of the height difference to the minimum rated passage height. Constrain the result to the range of 0 to 1 to generate a height availability factor. S55: Perform a comprehensive evaluation of the width availability index and the high availability factor, combine the width availability index and the high availability factor to obtain the comprehensive availability index of the passage, match the corresponding warning level to the comprehensive availability index of the passage according to the preset warning threshold range, and generate the fire passage evaluation result containing the comprehensive availability index of the passage and the warning level. The fire passage evaluation result is used to indicate the real-time availability status and risk level of the fire passage.

7. A fire escape monitoring device, characterized in that, The device includes: The multimodal data acquisition module is used to acquire multimodal data of the fire lane monitoring area, simultaneously acquire image data containing color and depth information, and obtain predefined standard parameters for passage, generating the original dataset and standard parameter set; The color information parsing module is used to intelligently parse and process the color information in the original dataset, identify and separate the channel regions and occupying objects in the image, and generate a two-dimensional semantic mask for the channel regions and an independent two-dimensional instance identifier for each occupying object. The three-dimensional spatial fusion module is used to perform three-dimensional spatial fusion processing on the two-dimensional semantic mask and the depth information in the original dataset, map the pixels in the two-dimensional semantic mask to three-dimensional space, generate a three-dimensional scene point cloud describing the true geometric shape of the channel, and separate the three-dimensional point cloud clusters corresponding to each occupying object from the three-dimensional scene point cloud according to each independent two-dimensional instance identifier. The bottleneck analysis module is used to perform bottleneck analysis on the three-dimensional scene point cloud and each of the three-dimensional point cloud clusters, determine the main extension direction of the channel, locate the key obstacle points that most affect the passage, construct a spatial cross section for quantitative evaluation, delineate the theoretical passage area on the spatial cross section, and generate a dynamic evaluation cross section and a standard passage area. The access availability assessment module is used to perform comprehensive availability calculations on the dynamic assessment section, the standard passage area, and the three-dimensional point cloud cluster. It analyzes the projection occupancy of each obstacle on the spatial section, comprehensively calculates the remaining proportion of passage capacity in width and height, and generates fire access assessment results containing a comprehensive access availability index.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images