Fire passage dynamic occupation detection method and system based on video image analysis

Abstract

The application provides a fire-fighting passage dynamic occupation detection method and system based on video image analysis, and relates to the technical field of intelligent fire-fighting security monitoring.The method first acquires a monitoring video, delimits a fire-fighting passage region, and acquires an obstacle category and a boundary box through single-frame target detection;then, pixel-level light flow field deformation features are used to filter non-rigid false interference such as light and shadow, and the effective stay duration of the obstacle is accurately accumulated;subsequently, the obstacle danger weight, the passage space duty ratio, and the effective stay duration are fused, a real-time occupation risk value is calculated through cross-dimensional nonlinear coupling, and finally, the illegal occupation is determined according to the nonlinear penalty boundary, an evidence chain is synchronously generated, an edge alarm is triggered, and semantic re-examination is completed relying on a cloud multi-modal large model.The application solves the problems of high false detection rate, single risk assessment, and poor determination accuracy of traditional detection technologies, effectively improves the accuracy and compliance of fire-fighting passage occupation detection in complex scenes, and adapts to the intelligent fire-fighting control requirements.

Description

Technical Field

[0001] This invention relates to the field of intelligent fire protection and security monitoring technology, and in particular to a method and system for detecting dynamic occupancy of fire lanes based on video image analysis. Background Technology

[0002] Fire lanes are vital lifelines for emergency rescue, and real-time, accurate detection of their occupancy status is a core aspect of fire safety management. Currently, most mainstream fire lane occupancy detection methods utilize traditional image-based target detection technology.

[0003] CN110659606A discloses a method for determining the occupancy of fire lanes by comparing corresponding tiles of a video image under test with a reference image and determining whether the total actual space area corresponding to the difference tiles reaches a set threshold.

[0004] CN121640390A discloses a fire lane monitoring scheme with functions of occupancy intent time-series analysis, low-light enhancement, and feedback optimization. The above technology has made corresponding improvements in fire lane occupancy identification, but there is still room for further optimization when facing complex scenarios such as changes in lighting, swaying tree shadows, reflections from water on the ground, and temporary movement of people.

[0005] Specifically, detection methods based on image differences and area thresholds struggle to distinguish between physical obstacles and non-physical interference such as changes in light and shadow at the motion morphology level, easily misidentifying non-rigid visual changes within the detection area as occupancy targets. While existing time-series analysis schemes can differentiate between temporary passage and continuous occupancy based on the target's motion state or dwell time, they haven't yet constructed pixel-level optical flow fields constrained by physical boundaries for the obstacle's physical region. The motion vectors of ground shadows and background pixels may interfere with the judgment of the obstacle's motion state. Furthermore, existing detection methods typically use the presence of a target, the occupied area, or the dwell time as the primary criteria, failing to correlate the degree of danger corresponding to the obstacle category, the degree of occupancy of the passage space, and the effective continuous dwell time after interference filtering. Consequently, it is difficult to differentiate and quantify the risks of obstacles of different types, occupancy levels, and durations.

[0006] Meanwhile, existing technologies mostly use fixed area thresholds or fixed dwell time thresholds for violation determination, which have limited adaptability to dynamic and complex scenarios. After the initial determination is completed at the edge, there is usually a lack of a secondary review mechanism based on the semantics of the complete scene, making it difficult to effectively identify non-violation situations with specific contextual semantics, such as temporary handling and water reflection. Some solutions also fail to link alarm screenshots, video clips before and after the event, and integrity verification information to form a traceable chain of evidence, which limits the application of detection results in subsequent verification and management.

[0007] In view of this, there is an urgent need for a dynamic occupancy detection method and system for fire lanes based on video image analysis, in order to at least address the above-mentioned shortcomings. Summary of the Invention

[0008] One of the objectives of this invention is to provide a method and system for dynamic occupancy detection of fire lanes based on video image analysis, in order to solve the problems pointed out in the background art.

[0009] The dynamic occupancy detection method for fire lanes based on video image analysis provided in this invention includes: Step S1: Acquire surveillance video and determine the fire lane area; perform target detection on single-frame images to obtain the category and bounding box of each obstacle; Step S2: Extract the pixel-level optical flow field of the obstacle between consecutive frames. If the pixel-level optical flow field exhibits non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it exhibits rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated. Step S3: Query and extract the corresponding hazard weight from the preset category hazard weight configuration table, calculate the space occupancy ratio of the bounding box in the fire lane area; multiply the hazard weight by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle. Step S4: When the real-time dynamic occupancy risk value is greater than or equal to the preset violation risk threshold, it is determined to be a static violation occupancy; backtrack and extract screenshots and continuous video clips related to the violation occupancy event, generate an evidence chain package, trigger an alarm on the local edge device, and asynchronously upload the alarm screenshot and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decide whether to retain or cancel the alarm record based on the review conclusion.

[0010] Preferably, determining the fire lane area in step S1 specifically includes: Extract image feature points from the initial unoccupied frames and determine the initial coordinates of the fire lane area; Periodically calculate the global affine transformation matrix between the current video frame and the initial unoccupied frame; The translation component is solved from the global affine transformation matrix to obtain the global translation offset. If the global translation offset is greater than the set tolerance, the coordinates of the fire lane area are dynamically compensated and updated using the inverse affine transformation matrix to offset the physical jitter of the camera equipment.

[0011] Preferably, the target detection in step S1 specifically includes: Video frames are input into the object detection network to extract features, and a spatial attention mechanism module is introduced into the feature extraction layer; The spatial attention mechanism module is used to enhance the feature weights of low-lying pixel regions on the ground, thereby improving the sensitivity of bounding box extraction for low-lying obstacles and outputting the enhanced obstacle bounding boxes.

[0012] Preferably, step S2 involves extracting the pixel-level optical flow field of obstacles between consecutive frames, specifically including: Based on the bounding box coordinates of the obstacle, crop out the target image patch containing the obstacle and the adjacent ground area at the bottom of the obstacle from the current frame and the previous frame; The spatial high-frequency gradient magnitude is calculated for the target image patch. A binary mask is generated according to the preset spatial gradient threshold. Morphological closing operation is performed on the binary mask to fill the internal holes. Connected component analysis is performed on the mask after the closing operation to filter out isolated regions with connected component areas smaller than the preset area threshold, and a solid structure mask that closely fits the physical obstacle is generated. Corner points are extracted only within the connected regions of the solid structure mask as structural anchor points. The motion vectors of the structural anchor points between adjacent frames are calculated using the pyramid optical flow algorithm to obtain a sparse anchor point optical flow set. Using sparse anchor point optical flow sets as seed elements, a bilateral interpolation algorithm with pixel color difference as guiding weight is used to diffuse the optical flow field. During the diffusion process, a solid structure mask is used as a physical boundary constraint to prevent the optical flow vector from diffusing into the background and shadow pixel regions outside the solid structure mask. The pixel optical flow value outside the physical structure mask is forced to zero, and a dense optical flow field that is truncated and diffused through the physical boundary is output as the pixel-level optical flow field of the obstacle between consecutive frames.

[0013] Preferably, step S2 involves extracting pixel-level optical flow fields and determining non-rigid body deformation features, specifically including: Extract the motion vectors of all non-zero pixels within the pixel-level optical flow field; Calculate the directional divergence and magnitude variance of the motion vector; when the directional divergence exceeds the preset directional divergence threshold, or the magnitude variance exceeds the preset magnitude variance threshold, it indicates that the obstacle's appearance has broken or flickered irregularly, and is judged as a non-rigid body deformation feature.

[0014] Preferably, the cumulative effective continuous dwell time in step S2 further includes a dynamic anti-disconnection mechanism: Assign persistent tracking identifiers to obstacles identified as rigid body translations; When the detection confidence of the obstacle corresponding to the persistent tracking identifier drops below the set confidence threshold due to brief occlusion and the duration does not exceed the preset maximum occlusion tolerance time, the trajectory association of the obstacle is maintained by linear extrapolation prediction of the historical frame displacement of the obstacle. During this period, the effective continuous dwell time is continuously accumulated without interruption. When the duration of occlusion exceeds the preset maximum occlusion tolerance time, stop accumulating the effective continuous dwell time and clear the persistent tracking identifier record.

[0015] Preferably, step S3 involves cross-dimensional nonlinear coupling and calculating the real-time dynamic occupancy risk value, specifically including: Substitute the effective continuous dwell time into the exponential growth function to calculate the generation time penalty; Multiply the hazard weight by the space duty cycle to calculate the space hazard base; Multiplying the spatial hazard baseline by the time penalty yields the real-time dynamic occupancy risk value.

[0016] Preferably, the generation of the evidence chain package in step S4 specifically includes: The system continuously caches the monitoring video stream using a circular memory loop; When an unauthorized occupation determination is triggered, the following content is automatically extracted from the circular memory and combined: a screenshot of the moment when the real-time dynamic occupation risk value reaches the preset violation risk threshold, and continuous video clips containing set durations before and after the unauthorized occupation event; SHA-256 hash check values ​​are calculated for the screenshots and continuous video clips, and the hash check values ​​are associated with the corresponding files and stored to generate an evidence chain package that can be verified for integrity.

[0017] Preferably, in step S4, the cloud-based multimodal visual large model performs semantic review, specifically including: The cloud-based multimodal visual big model receives alarm screenshots and video clips of preset duration before and after the alarm time from the local edge device, and performs global scene semantic inference on the alarm screenshots and video clips. If the cloud-based multimodal vision big data model identifies temporary occupation characteristics of personnel and container collaborative movement in the current scene, or identifies the specular reflection characteristics caused by water accumulation on the ground, it sends an alarm cancellation command to the edge device and property security platform to cancel the alarm record and violation work order that have been generated.

[0018] The dynamic occupancy detection system for fire lanes based on video image analysis provided in this embodiment of the invention includes: The obstacle recognition module is used to acquire surveillance video and determine the fire lane area, perform target detection on single frame images, and obtain the category and bounding box of each obstacle; The false interference filtering module is used to extract the pixel-level optical flow field of obstacles between consecutive frames. If the pixel-level optical flow field exhibits non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it exhibits rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated. The real-time dynamic occupancy risk value calculation module is used to query and extract the corresponding hazard weights from the preset category hazard weight configuration table, calculate the space occupancy ratio of the boundary box in the fire lane area, multiply the hazard weights by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle. The detection module is used to determine static illegal occupation when the real-time dynamic occupation risk value is greater than or equal to the preset illegal risk threshold; it backtracks and extracts screenshots and continuous video clips related to the illegal occupation event, generates an evidence chain package, triggers an alarm on the local edge device, and asynchronously uploads the alarm screenshot and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decides whether to retain or cancel the alarm record based on the review conclusion.

[0019] The beneficial effects of this invention are as follows: This invention effectively addresses the shortcomings of existing fire lane detection technologies, such as poor anti-interference capabilities, incomplete risk assessment, and low accuracy. First, it filters non-rigid light and shadow interference through pixel-level optical flow field analysis, accurately distinguishing between background interference and low-speed lingering obstacles, significantly improving detection anti-interference capabilities and reducing false detections at the source. Second, it constructs a cross-dimensional dynamic risk function, integrating multi-dimensional parameters such as obstacle hazard weight, lane space occupancy ratio, and effective dwell time, to achieve non-linear risk quantification assessment, breaking through the limitations of traditional single-duration judgment and accurately distinguishing the degree of occupancy hazard of different obstacles. Finally, it sets up a non-linear violation penalty boundary judgment mechanism, combining a dual verification mode of real-time edge alarms and cloud-based large-scale model semantic review, significantly improving the accuracy of violation judgment and significantly reducing missed detections and false alarms. Simultaneously, it can automatically backtrack and generate violation evidence chains, balancing detection real-time performance and result compliance, adapting to various complex scenarios, and effectively improving the intelligent management and control level of fire lanes.

[0020] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in this application.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0022] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of a dynamic occupancy detection method for fire lanes based on video image analysis in an embodiment of the present invention; Figure 2 This is a schematic diagram of a dynamic occupancy detection system for fire lanes based on video image analysis, as described in an embodiment of the present invention. Detailed Implementation

[0023] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0024] This invention provides a method for dynamic occupancy detection of fire lanes based on video image analysis, such as... Figure 1 As shown, it includes: Step S1: Acquire surveillance video and determine the fire lane area. Perform target detection on a single frame image to obtain the category and bounding box of each obstacle.

[0025] The system pulls real-time video streams from the camera via the RTSP (Real Time Streaming Protocol), and uses FFmpeg or GStreamer to decode the H.264 / H.265 video streams to obtain raw frame images in RGB format with a resolution of 1920×1080 pixels.

[0026] During the initial system deployment, the property administrator marks the polygonal areas (typically irregular quadrilaterals with 4 to 8 vertices, covering the physical boundaries of the fire escape routes) on the screen via a web management interface. The system records this frame as the initial unoccupied frame and extracts its image feature points, preferentially using ORB (Oriented Fast and Rotated BRIEF) or SIFT (Scale-Invariant Feature Transform) feature points as the reference for global registration. These initial coordinates are stored in a local configuration file in JSON format.

[0027] During system operation, cameras may experience minor physical vibrations due to factors such as wind, ground vibrations caused by passing vehicles, and loose brackets, resulting in translation, rotation, or slight perspective distortion of the image content. To ensure that the fire lane area is always accurately aligned with its actual physical position, the system periodically performs global registration correction (e.g., every 10 seconds, or when a decrease in the matching quality of image feature points is detected).

[0028] Specifically, the system extracts feature points from the current video frame and performs RANSAC (RANdom Sampling Consensus) robust matching with feature points from the initial unoccupied frame to estimate the global affine transformation matrix between the two frames. This matrix is ​​a 2x3 parameter matrix containing geometric transformation parameters such as rotation, scaling, translation, and shearing. The two elements in the third column correspond to the pixel translation amounts in the horizontal and vertical directions of the image, denoted as . and .

[0029] The system for translation vectors Calculate the Euclidean modulus, that is, for The square of and The global translation offset is obtained by summing the squares and taking the square root. This reflects the overall displacement of the current frame relative to the reference frame. The system has a preset tolerance threshold. (In this embodiment) (pixels). If If the jitter is considered negligible, no correction will be made; The inverse of the affine matrix is ​​then calculated, and the vertex coordinates of the polygon in the fire lane area of ​​the current frame are substituted into the inverse transformation to obtain the compensated new coordinates. This dynamically corrects the detection area and offsets the physical jitter of the camera equipment. This mechanism ensures that even in windy weather or construction vibration environments, the detection area can still be stably locked on the actual physical location of the fire lane.

[0030] Before performing target detection, the system performs adaptive image enhancement preprocessing on the original video frames, and performs pre-repair for three types of severe scenes at fire scenes, replacing the traditional fixed parameter image enhancement algorithm.

[0031] The system extracts three major image quality features of the current frame in real time: global average brightness. (Obtained by averaging the grayscale values ​​of all pixels in the frame, with values ​​ranging from 0 to 255), global contrast. (Obtained by calculating the standard deviation of the pixel grayscale values ​​across the entire frame, reflecting the dispersion of brightness differences in the image), and an image sharpness estimate based on the Laplacian operator. (Obtained by calculating the variance of the Laplacian filter response map; a larger value indicates a clearer image). Based on these three feature values, the system automatically classifies the current frame into one of the following three operating conditions: when (Corresponding to the highlight area of ​​an 8-bit grayscale image) and When the system detects a strong daylight condition, it determines that the image is in a bright daylight environment. At this point, the system uses Contrast Limited Adaptive Histogram Equalization (CLAHE) and the Retinex algorithm to suppress highlights to enhance local contrast in the image. Simultaneously, it performs gamma correction on overexposed areas to restore edge features obscured by strong light.

[0032] when When the image shows a dark area (corresponding to the dark region of an 8-bit grayscale image), it is determined to be a low-light nighttime condition. The system calls the Multi-Scale Retinex (MSR) illumination compensation algorithm to estimate and compensate for the illumination components of the scene, increasing the brightness of the low-light image to the normal detection range. At the same time, it calls an adaptive noise filtering algorithm based on bilateral filtering or non-local mean filtering to eliminate noise particles in the low-light environment while preserving the edge texture features of obstacles.

[0033] when Below the preset resolution threshold (In this embodiment) When the image is foggy (the specific camera model can be used for calibration), it is determined to be a rain / fog blurring condition. The system calls a defogging algorithm based on dark channel priors or an improved atmospheric scattering model to defog and restore the image, improving the visibility and texture clarity of distant obstacles.

[0034] After the three enhancement modules have finished processing, the system outputs a standardized 640×640 pixel RGB image as the input for the subsequent object detection network.

[0035] This embodiment uses an improved YOLOv8s (You Only Look Once version 8 small, a lightweight version of the eighth-generation single-look object detection network) object detection network.

[0036] Training Dataset: The system constructed a dedicated training dataset for fire lane scenarios, totaling 8600 valid image samples, including 7310 images for training, 860 for validation, and 430 for testing. The dataset covers five on-site conditions: strong daylight / backlight, low nighttime illumination, rain / fog, camera shake, and dense pedestrian interference. It includes full-scene samples of vehicle traffic, pedestrian crossings, long-term illegal parking, piled-up debris, and interference from birds and fallen leaves. All data uses supervised learning annotation, with annotations including obstacle bounding box coordinates (in words). The format represents the x and y coordinates of the top left and bottom right corners of the bounding box, the object category label, and the mask of the effective fire lane ROI region. During the global offline training phase of the deep learning model, to ensure the synchronous optimization of spatial recognition and model training classification, the model utilizes a weighted composite loss function to calculate the backpropagation gradient feedback mechanism. Its total loss function consists of two concatenated parts: one is the complete intersection-union loss function (CIoU Loss) to ensure accurate regression of the bounding box spatial location, and the other is the binary cross-entropy classification loss function (BCE Loss) responsible for distinguishing the five specific fire obstacle types, thus achieving stable weight distribution and correction.

[0037] For fire lane scenarios, the system removes general detection categories that are not related to fire lane occupancy detection and retains only the following 5 fire lane-specific detection targets: motor vehicles (sedan_car), non-motorized vehicles / electric vehicles (e_bike), debris piles (debris_pile), construction materials (construction_material), and boxed goods (cargo_box).

[0038] The preset category hazard weight configuration table assigns values ​​to the degree of hazard posed by various targets to the fire lane's passage capacity: Hazard weights corresponding to motor vehicles. (Highest), corresponding construction materials The corresponding storage of miscellaneous items Non-motorized vehicles / electric vehicles correspond Corresponding to container cargo The aforementioned weight values ​​can be customized and adjusted by the property administrator through the back-end management interface according to the actual needs of the scenario.

[0039] To address the issue that low-lying obstacles in fire lanes (such as cardboard boxes, tires, and cones piled on the ground) have small projected areas and sparse features when viewed from above, making them easy to miss, this invention introduces a Spatial Attention Pooling Module for fire lanes (SPAM) in series after the backbone feature extraction layer (C2f module of CSPDarknet) of the YOLOv8s network and before the multi-scale feature fusion layer (PANet).

[0040] The SPAM module works as follows: It receives the global image feature map output from the backbone network and performs global average pooling and global max pooling to generate two spatial response maps of the same size. These two maps are then concatenated along the channel dimension and passed through a 7×7 convolutional layer and a sigmoid activation function to obtain a spatial attention weight map with values ​​ranging from 0 to 1. Each position in this weight map corresponds to a spatial region of the original input image; a higher value indicates that the features in that region should be emphasized.

[0041] The SPAM module performs targeted optimization based on the geometric characteristics of fire lanes: since low-lying obstacles in fire lanes are typically located in the lower half of the frame (occupying 60% to 85% of the vertical area from a top-down camera perspective), SPAM applies a gain factor of 1.5 to 2.0 to the values ​​of the corresponding ground areas (determined based on the calibrated ROI coordinates of the fire lane) in the spatial attention weight map during its generation. This actively amplifies the feature responses of low-lying targets while suppressing the feature weights of invalid background areas such as the sky, vegetation, and distant buildings. The gain-adjusted attention weight map is then multiplied element-wise with the backbone feature map to obtain an enhanced local feature map, which is then input into the subsequent PANet multi-scale feature fusion layer.

[0042] In addition, the system adds a dedicated Small Object Detection Branch after PANet. By introducing a higher resolution feature map (downsampling step size of 4, i.e. 1 / 4 resolution of the original image), it specifically handles small obstacle targets with an area of ​​less than 32×32 pixels, thus solving the problem that the general YOLO network has a weak ability to extract low-lying small targets on the ground.

[0043] Meanwhile, the system fixes the Region of Interest (ROI) of the fire lane and performs detection head calculations only on the feature map within the designated lane area, shielding interference objects outside the lane and reducing invalid calculations.

[0044] Inference Output: The object detection network outputs information on all detected obstacles within the fire escape ROI region for each frame of image, including bounding box coordinates. The system includes obstacle category labels and detection confidence scores. It filters out objects with confidence scores below a preset confidence threshold. (In this embodiment) The candidate detection boxes (configurable) are passed to step S2 for time-series dynamic analysis.

[0045] Step S2: Extract the pixel-level optical flow field of the obstacle between consecutive frames. If the pixel-level optical flow field exhibits non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it exhibits rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated.

[0046] The bounding box output in step S1 inevitably includes the obstacle itself and the adjacent ground area at its bottom (including ground projection and shadow). If the optical flow field is extracted directly from the entire bounding box, the moving shadow on the ground will severely contaminate the motion vector of the obstacle itself, causing the subsequent rigid body determination to be inaccurate.

[0047] This step utilizes three sub-mechanisms—optical flow extraction based on solid structure masks, motion vector statistical analysis, and dynamic anti-disconnection—to identify and filter false interference and accurately accumulate the effective continuous dwell time of obstacles.

[0048] The system uses the bounding box coordinates of the obstacles From the current frame With the previous frame The corresponding image patches are cropped out from the bounding box. To include ground context information at the bottom of the obstacle (for subsequent mask segmentation and ground shadow recognition), the cropped region is extended downwards by 10% to 15% of its height to obtain the target image patch. and .

[0049] This step fully utilizes the physical uniqueness of the fire lane scene: physical obstacles (such as the edges of boxes, vehicle outlines, and seams of metal barrels) have rich high-frequency spatial texture features, which manifest as high-amplitude edge responses in the image gradient domain; while ground projections and gradient lighting present low-frequency smooth transitions in space, with significantly lower gradient amplitudes. The system uses this physical difference to separate the two.

[0050] The system targets image patches Perform the following five steps: The first step is to perform Gaussian smoothing preprocessing on the image patch (the spatial standard deviation of the Gaussian kernel is 1.0 pixel; this parameter only affects noise suppression in this preprocessing sub-step and is not related to the spatial bandwidth parameter in the subsequent bilateral interpolation diffusion formula). Completely independent in symbol and purpose, with no relation between the two, to suppress the impact of image acquisition noise and compression artifacts on subsequent gradient calculations.

[0051] The second step involves using the Sobel operator (a commonly used differential operator for image edge detection, which performs horizontal and vertical convolution operations on the image) to calculate the gradient component of each pixel in the horizontal direction. gradient components in the vertical direction Then, the sum of the squares of the gradient components in both directions is taken as the square root to obtain the comprehensive gradient magnitude of each pixel. This comprehensively reflects the edge response strength of the pixel in all directions. , and Each element is a two-dimensional matrix of the same size as the input image, and each element corresponds to the gradient information of a pixel position.

[0052] The third step is to set a preset spatial gradient threshold. (In this embodiment) Based on the 0 to 255 grayscale levels of an 8-bit grayscale image, the calibration can be configured according to the ambient lighting conditions. medium to low The pixels marked as background (corresponding to low-frequency ground shadow areas) are higher than The pixels are marked as foreground (corresponding to the high-frequency edge region of the physical entity structure), and a preliminary binarized mask is obtained.

[0053] The fourth step is to perform a morphological closing operation (dilation followed by erosion, using a 5×5 rectangular structuring element) on the binarized mask to fill the small holes inside the obstacle caused by uneven texture distribution, so that the mask's connected regions can more completely cover the surface of the obstacle.

[0054] The fifth step is to perform connected component analysis on the mask after the closing operation, filtering out isolated regions whose connected component area is smaller than a preset threshold (50 pixels in this embodiment). These isolated noise points usually come from interference sources such as small stones on the ground and the edges of water stains. Their area is much smaller than the connected component area of ​​the real obstacle, and they can be reliably filtered out by the area threshold.

[0055] After the above five steps, the final solid structure mask is generated. In this mask, areas with a value of 1 correspond precisely to the physical entities of obstacles, while areas with a value of 0 correspond to the ground, shadows, and background.

[0056] Traditional dense optical flow algorithms (such as the Farneback algorithm) require iterative optimization of each pixel within an image patch, resulting in high computational costs and susceptibility to low-texture noise in shadow regions, making them difficult to run in real-time on edge devices. This invention employs a hybrid strategy combining sparse anchor point tracking with constrained dense diffusion, achieving high-quality, clean optical flow estimation while maintaining computational efficiency.

[0057] The system only works on solid structure masks. Feature point detection is performed within connected regions with a value of 1, using the Shi-Tomasi corner detection algorithm (which evaluates corner quality by calculating the minimum eigenvalue within a local image region), and extracting the feature points with the highest response values. Feature points (in this embodiment) An adaptive value ranging from 50 to 200, dynamically adjusted based on the area of ​​the mask's connected regions, serves as the set of structural anchor points, denoted as... Each of them For the first The two-dimensional coordinates of each anchor point in the image. These anchor points are naturally located at the corners of obstacles and on textured surfaces, providing good inter-frame traceability.

[0058] Using the Lucas-Kanade (LK) pyramid optical flow algorithm, in and The displacement of the anchor points of the aforementioned structure is tracked. The pyramid LK algorithm constructs a pyramid containing... Layered image pyramid (in this embodiment) The image resolutions of each layer are, in order, the original image, half the original image, and a quarter of the original image. Starting from the coarsest top layer of the pyramid, the optical flow vector of each anchor point is iteratively solved in the local neighborhood using the assumption of brightness invariance (i.e., the pixel brightness of the same physical point remains consistent between adjacent frames). The calculation results of the coarse layers are passed to the fine layers as initial values, and the process is refined layer by layer. Finally, the precise displacement vector of each anchor point is output at the original resolution layer, and its horizontal component is denoted as... The vertical component is denoted as .

[0059] The algorithm simultaneously calculates the forward and reverse reprojection errors for each anchor point (i.e., first tracing in the direction from the current frame to the previous frame, then tracing back to the current frame in the reverse direction, and calculating the initial and final position deviations), filtering out reprojection errors exceeding a threshold. (In this embodiment) (pixel) abnormal tracking points to ensure the reliability of tracking points.

[0060] Retain after filtration One effective anchor point ( ), forming a sparse anchor point optical flow set: ; in, For the first The two-dimensional coordinates of each effective anchor point in the image. For the horizontal coordinate components, These are the vertical coordinate components; For the first The two-dimensional motion vector corresponding to each effective anchor point This represents the horizontal pixel displacement. This represents the vertical pixel displacement. This represents the total number of valid anchor points after filtering for forward and reverse reprojection errors. .

[0061] Sparse anchor point optical flow collection Only the surface of the obstacle was covered. To obtain a complete dense optical flow field covering the entire obstacle body, the system needs to diffuse sparse vectors to all pixels within the mask to obtain motion information at the locations of key feature points.

[0062] This invention uses a solid structure mask As a physical boundary constraint, a restricted bilateral interpolation diffusion algorithm with mask boundary truncation was designed. Its core idea is that the optical flow vector can only diffuse in homogeneous regions with similar appearance and spatial proximity, and the diffusion direction is strictly limited to the inside of the mask, thereby ensuring that the zero-value optical flow in the shadow area is not contaminated by the obstacle motion vector, and at the same time, the vectors between different color blocks inside the obstacle do not merge incorrectly.

[0063] For masks Any target pixel to be diffused within The diffused optical flow vector By all The weighted normalized interpolation of the effective anchor seed elements is determined as follows: ; in, The coordinates of the target pixel to be diffused. The horizontal coordinate is... Vertical coordinates; For target pixel The two-dimensional optical flow vector obtained after interpolation and diffusion; The total number of effective anchor points for sparse anchor point optical flow concentration; For the first Anchor points Two-dimensional motion vector (from sparse anchor point optical flow set) ); It is the first Anchor points for target pixels The overall interpolation weight is obtained by multiplying the spatial distance weight and the pixel color difference weight, i.e. .

[0064] Spatial distance weight Based on the Gaussian kernel function, reflecting the target pixel point With the Anchor points Spatial proximity between them, with closer distances carrying greater weight: ; in, For target pixel Image coordinates, The horizontal coordinate is... Vertical coordinates; For the first Anchor points Image coordinates, The horizontal coordinate is... Vertical coordinates; It is the spatial distance Gaussian kernel bandwidth parameter (unit: pixels, in this embodiment) (Pixels, configurable) controls how quickly spatial weights decay with increasing distance; The larger the value, the wider the spatial influence range of each anchor point, and the smoother the diffusion effect.

[0065] Pixel color difference weight Based on Euclidean color difference in the RGB color space, reflecting the target pixel point With the Anchor points In terms of color similarity, the closer the colors are, the greater the weight, ensuring that light flow diffusion only occurs within areas of similar appearance, avoiding incorrect vector blending that crosses color boundaries inside obstacles. ; in, For target pixel RGB three-channel color vector; For the first Anchor points RGB three-channel color vector; yes and In the RGB color space, the square of the Euclidean color difference is equal to the sum of the squares of the differences between the three color channels, i.e. ; The color difference Gaussian kernel bandwidth parameter (unit: gray level, in this embodiment) Based on 8-bit channels with 0 to 255 gray levels (configurable), it controls how quickly the color difference weight decays as the color difference increases; The smaller the value, the stricter the requirement for color similarity, and the stronger the ability to maintain the color boundary of diffusion.

[0066] It should be noted that in the above formula and These are both kernel bandwidth parameters specific to this bilateral interpolation diffusion sub-step. They are independent of each other and are used only within this sub-step, and are not related to image quality features appearing elsewhere in the specification. , , (etc.) have no connection in terms of symbolism or meaning.

[0067] Key constraint: Throughout the diffusion process, the system will mask the solid structure. As a physical boundary constraint. Any target pixel. If it is located For regions with a value of 0 (i.e., ground, shadow, and background pixels outside the mask), their optical flow vectors are forcibly set to zero, and they do not participate in interpolation calculations or accept any diffusion. At the same time, the weight propagation starting from each anchor point seed element is immediately truncated when it encounters the mask boundary, and diffusion outward across the mask boundary is prohibited.

[0068] Ultimately, the system outputs a dense optical flow field that is truncated and diffused through the physical boundary. This serves as the pixel-level optical flow field of the obstacle across consecutive frames. Inside the mask (the obstacle itself), this optical flow field has a complete dense non-zero vector, while outside the mask (the ground shadow), the optical flow value is all zero, achieving truly "pure" optical flow extraction and completely isolating the ground shadow motion from interfering with the obstacle motion analysis.

[0069] System extracts dense optical flow field The set of motion vectors of all non-zero pixels, totaling One, denoted as Each vector ,in and The first The system measures the horizontal and vertical motion components of each non-zero pixel. Calculate the following two statistics for each vector: Directional divergence First, calculate the motion direction angle of each vector. ,get One direction angle; then for The circular standard deviation is calculated for each direction angle to obtain the directional divergence. Its unit is degrees (°). The larger the value, the more dispersed and inconsistent the movement directions of each pixel.

[0070] Modulus variance Calculate the magnitude of each vector. ,get Each modulus value; then for Calculate the statistical variance of each modulus value to obtain the modulus variance. . The larger the value, the greater the difference in the movement speed of each pixel.

[0071] when Exceeding the rigid divergence threshold (In this embodiment) (configurable), or Exceeding the rigid variance threshold (In this embodiment) Based on image resolution and frame rate calibration (configurable), the system determines that the optical flow field of the current obstacle exhibits non-rigid deformation characteristics with internal vector discreteness.

[0072] Non-rigid deformation features physically mean that different areas within the obstacle's bounding box move in drastically different directions or at vastly different speeds. This feature should not appear on real physical obstacles (rigid bodies) and is typically caused by the following: flickering shadows cast by moving tree shadows on stationary boxes, reflections from ripples on the ground, or wind-blown leaves obscuring the detection frame. Once a non-rigid deformation feature is identified, the system marks the obstacle detection frame as "false interference," suppresses dwell time on the obstacle, and the results of this round of detection are not included in the cumulative effective dwell time to avoid triggering alarms for false targets.

[0073] Conversely, if and If so, the current optical flow field is determined to exhibit rigid body motion characteristics (internal vector directions are highly consistent and velocity differences are small).

[0074] Furthermore, the system calculates all Mean of the magnitudes of non-zero vectors Its calculation method is based on Modulus value Find the arithmetic mean. With the preset low speed threshold (In this embodiment) Pixels per frame (corresponding to an extremely low speed of approximately 0.05 m / s in a real-world scene, which can be compared based on the camera's installation height and focal length calibration): like If an obstacle is determined to be in significant motion (such as a pedestrian passing by or a vehicle moving), the effective dwell time is not counted. like If the obstacle is determined to be in a rigid body translation state and its motion vector magnitude is below the low-speed threshold, the obstacle is considered to be in a stationary or near-stationary state, triggering the effective continuous dwell time. The cumulative total.

[0075] System settings for transient interference filtering threshold (In this embodiment) (seconds). If the duration of a target's continuous appearance within the channel is less than [a certain number of seconds]. If the target disappears, the system will directly identify it as interference noise (such as a bird flying by quickly or a leaf falling instantly), clear all historical records of the target, exclude it from the dwell time, and issue no invalid alarms throughout the process.

[0076] Taking a bird flying quickly across a fire lane as an example: the target detection network detects a small target (the bird), and optical flow field analysis identifies it. Much larger directional divergence Exceeding The dwell time is 1.2 seconds. Seconds. The system identifies this as brief, rapid interference noise, filters it directly, and does not activate it. Cumulative analysis is performed without the need to upload large models for re-evaluation, and there are no invalid alarm outputs throughout the process.

[0077] In real-world fire lane scenarios, stationary obstacles may be briefly obscured by pedestrians, causing the confidence score output by the object detection network to temporarily drop to the confidence threshold during the obstruction period. If the accumulation of valid continuous stay time is interrupted at this time, the time statistics will be inaccurate, and real violations will be incorrectly filtered out.

[0078] Therefore, the system determines each object to be a rigid body translation (and A globally unique persistent tracking identifier (PTID) is assigned to stationary obstacles, in the format of "{obstacle category}{timestamp}{random checksum}".

[0079] When the obstacle corresponding to PTID is briefly obscured, causing the detection confidence to drop to [a certain level], In the following situation, the system does not immediately clear the tracking record of the PTID, but instead enters the "trajectory prediction hold" state: The system reads the historical sequence of the bounding box center coordinates of the obstacle in the 5 most recent frames before the confidence level decreased, and uses linear extrapolation (i.e., using the average displacement between historical frames to estimate the predicted position of the obstacle in the current frame and subsequent frames) to predict the current position of the obstacle. Since the obstacle is stationary ( Its historical displacement is extremely small, and the position error predicted by linear extrapolation is usually within 1 to 3 pixels, with high reliability.

[0080] The system sets a prediction search window (with sides 1.5 times the original bounding box width and height) centered on the predicted location, and continuously waits for the obstacle corresponding to the PTID to reappear in subsequent frames. If the maximum occlusion tolerance time is within the preset range... (In this embodiment) Within seconds, the confidence level of the detection box recovers to [a certain value]. The system will then associate the new detection bounding box with the PTID, restoring normal tracking; throughout the occlusion period (confidence level below...). (All time periods), effective continuous stay duration The accumulation continues uninterrupted, accumulating at a constant rate based on the frame rate. If the occlusion duration exceeds... If detection still fails to resume, it is determined that the obstacle has truly left the passage. The PTID record is then cleared, and the process is stopped immediately. Grand total.

[0081] Step S3: Query and extract the corresponding hazard weight from the preset category hazard weight configuration table, calculate the space occupancy ratio of the bounding box in the fire lane area; multiply the hazard weight by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle.

[0082] In step S1, the object detection network has output the category label for each obstacle. Step S3 queries and extracts the corresponding hazard weight from the preset category hazard weight configuration table. (Dimensionless, range of values) This configuration table is stored in a local configuration file in key-value pair format: vehicle corresponding to Construction materials The corresponding storage of miscellaneous items Non-motorized vehicles / electric vehicles correspond Corresponding to container cargo .

[0083] The system calculates the space occupancy ratio of the obstacle boundary box within the fire lane area. Its definition is: ; in, Space duty cycle (dimensionless, range of values) ), representing the degree to which obstacles obstruct the space of fire lanes, This indicates that the obstacle is completely outside the passage area. This indicates that the obstacle completely covers the fire lane area; The area of ​​the intersection between the obstacle bounding box and the polygon of the fire lane area (unit: pixel²) is obtained by calculating the intersection of the two polygons and counting the number of pixels. It is the total area of ​​the polygon of the fire lane area (unit: pixels²), which is calculated and stored during the initial system calibration and is recalculated only when the coordinates of the fire lane area are updated.

[0084] The larger the value, the greater the degree to which the obstacle obstructs the fire lane, and the more severe the impairment of the passage's unobstructed flow.

[0085] This invention employs a three-step multiplication coupling model to calculate the real-time dynamic occupancy risk value.

[0086] Step 1: Calculate the spatial hazard baseline: Obstacle category hazard weight With space duty cycle Multiplying these values ​​yields a spatial hazard baseline, which comprehensively reflects the hazard level of the obstacle itself, multiplied by the actual space it occupies. For example, motor vehicles ( ) Fully occupy the channel ( The spatial hazard baseline for small cardboard boxes is 1.0, while that for small cardboard boxes is 1.0. Occupying 20% ​​of the passage area ( The spatial hazard baseline is only 0.1.

[0087] Step 2: Calculate the time penalty based on the exponential growth function: Effective continuous stay duration Substituting the result into an exponential growth function with the natural constant e as the base, the time penalty is calculated. As... As the dwell time increases, the time penalty increases exponentially: when the dwell time is short, the time penalty is close to 1, contributing little to the risk value; once the dwell time exceeds a certain limit, the time penalty rises sharply, driving the risk value to quickly exceed the violation penalty boundary. This exponential growth characteristic is consistent with the actual threat pattern of fire safety (short-term stops pose a limited threat to fire lanes, while long-term obstruction of lanes leads to a sharp increase in danger).

[0088] Step 3: Calculate the real-time dynamic occupancy risk value: Multiplying the spatial hazard baseline by the time penalty yields the real-time dynamic occupancy risk value at the current moment. : ; in, for The real-time dynamic occupancy risk value of obstacles (dimensionless) comprehensively reflects the superimposed threat of three dimensions: obstacle category risk level, space occupancy degree, and dwell time; Category hazard weights (dimensionless, range of values) The risk weight configuration table is used to obtain the risk weight. Space duty cycle (dimensionless, range of values) ); is the natural constant, approximately equal to 2.718, and is the base of the natural exponential function; The time penalty growth factor (unit: 1 / second, in this embodiment) (This can be configured by the property manager through the backend interface according to the scenario) and controlled. The exponential growth rate with the duration of stay; The larger the value, the faster the risk value increases for the same length of stay, and the more sensitive the system becomes. It is the effective continuous dwell time (unit: seconds), that is, the dwell time of the obstacle in the fire lane that is continuously accumulated by the dynamic anti-disconnection mechanism in step S2.

[0089] To illustrate the function's working characteristics, let's take a specific numerical example: when a motor vehicle ( Completely block the fire escape route. And remain there for 60 seconds. ; When staying for 180 seconds, The risk value increased more than sixfold; while when the same motor vehicle only stopped temporarily for 30 seconds, If the value is below the violation penalty threshold, the system will not trigger an alarm. For small cardboard boxes ( , Even if it stays for 180 seconds, The risk value remains low, and the system does not trigger an alarm, demonstrating the effective suppression capability of this invention against minor interferences.

[0090] The above three-dimensional coupling model makes Simultaneously responding to changes in three dimensions—"obstacle category hazard level," "space occupancy ratio," and "stay time"—the risk value increases sharply when all three are high, and is effectively suppressed when any one dimension is low, thus avoiding false alarms and missed alarms caused by a single dimension judgment.

[0091] Step S4: When the real-time dynamic occupancy risk value is greater than or equal to the preset violation risk threshold, it is determined to be a static violation occupancy; backtrack and extract screenshots and continuous video clips related to the violation occupancy event, generate an evidence chain package, trigger an alarm on the local edge device, and asynchronously upload the alarm screenshot and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decide whether to retain or cancel the alarm record based on the review conclusion.

[0092] The system presets non-linear violation penalty boundaries. (In this embodiment) (This can be configured by the property manager through the backend interface according to the needs of the scenario). When the real-time dynamic occupancy risk value output in step S3... breakthrough When the system determines that the current obstacle constitutes a static unauthorized occupation, it triggers the subsequent alarm process.

[0093] The penalty boundary is not a simple linear time threshold (such as "alarm after 180 seconds"), but a non-linear boundary that dynamically adjusts based on obstacle type and occupied area: for high-risk obstacles (such as vehicles completely blocking the passage), the system can break through in a shorter dwell time. For low-risk obstacles (such as small cardboard boxes occupying 20% ​​of the passage area), a longer dwell time is required to trigger an alarm, thus avoiding overreaction to minor disturbances.

[0094] The system continuously uses a circular buffer (in this embodiment, the buffer capacity covers the video frame data of the most recent 5 minutes) to cache the original frame sequence of the monitoring video stream.

[0095] When an unauthorized use is detected, the system automatically extracts and combines the following content from the circular memory: Real-time dynamic occupancy risk value A screenshot of the moment when the preset violation risk threshold is reached, overlaid with the current risk value and violation judgment status label; The system includes continuous video clips of set durations (5 seconds before and after in this embodiment, configurable) before and after the unauthorized occupation event, packaged in MP4 format, covering the key processes before and after the system determines the violation.

[0096] The system calculates the SHA-256 hash checksum for the above materials and stores it accordingly, generating an evidence chain package capable of integrity verification. The evidence chain package is named "{PTID}_{violation timestamp}", stored in a local encrypted storage partition, and simultaneously backed up to the property security platform's cloud storage.

[0097] Throughout the entire image acquisition, judgment, and evidence extraction process, this technical solution is deployed using a standalone video intelligent analysis architecture on the edge side. All analysis algorithms are directly embedded and run in existing surveillance cameras or edge AI gateways to perform ontology reasoning. The system only sends back the generated, extremely small-volume evidence-collecting video clips and result text to the platform, without continuously and in real-time transmitting large-capacity, high-definition, full-volume basic video streams to the cloud server. This decentralized deployment completely avoids the high-bandwidth communication challenges of on-site construction and simultaneously protects residents' privacy.

[0098] After the evidence chain is generated, the system immediately triggers an alarm on the local edge device and executes a level 4 coordinated response: Level 1: Edge devices drive the on-site audible and visual alarm to output sound prompts and flashing lights to drive away people or vehicles obstructing the road. Level 2: Push the evidence chain package to the property security platform, generate a violation work order, and push differentiated alarm notifications (motor vehicles occupying the road trigger a red alarm of "severe congestion", and small debris occupying the road triggers a yellow alarm of "minor obstruction"). Level 3: The alarm frame (boundary box screenshot and 5-second video clip) is uploaded asynchronously to the cloud multimodal visual large model for semantic review without blocking the normal execution of the local alarm process; Level 4: Based on the review results of the cloud-based big data model, dynamically decide whether to retain or revoke the generated violation work orders.

[0099] Cloud-based multimodal visual large model (in this embodiment, open-source multimodal large language models such as InternVL2, Qwen-VL, or LLaVA-1.6 are preferred).

[0100] The adaptation of large-scale fire escape scenarios is achieved through the following incremental fine-tuning process: Fine-tuning Data Construction: Based on the ActivityNet and Something-Something V2 public action understanding datasets, approximately 4,200 video clips with semantic similarities to fire lane interference scenarios, including personnel carrying, goods moving, and water reflection, were selected. At the same time, approximately 3,100 short video clips covering four scenarios—real illegal occupation of the road, temporary carrying and passing, tree shadow interference, and ground water reflection—were manually labeled and selected from the actual fire lane monitoring videos collected at the system deployment site. A dedicated fine-tuning dataset for fire lane semantic review was constructed, totaling approximately 7,300 video samples, which were divided into training, validation, and test sets in a 7:1:2 ratio.

[0101] Model Input: The input for each inference consists of two parts: ① a static alarm screenshot (RGB image, resolution no less than 640×640 pixels, including overlaid information such as obstacle bounding boxes, category labels, risk values, and other visual annotations); ② short video clips (frame rate no less than 10FPS, packaged in MP4 format) 5 seconds before and after the alarm time. The two are combined and fed into the visual encoder of the large model as multimodal input. Meanwhile, to ensure that the multimodal large model can accurately focus on the rule logic of fire lanes, the system synchronously sends specific natural language prompts to the large model for combined input. The pre-built task instructions are as follows: "Task Role: You are a professional fire monitoring and safety audit AI. Based on the input static images and video clip sequences, please determine whether there are any static illegal road occupancy behaviors that truly block the passage of people and vehicles. Constraints: It is imperative to intelligently identify and automatically ignore interference caused by environmental factors (such as dynamic tree shadows caused by wind, and mirror reflections caused by strong light and water accumulation on a stationary road surface), and at the same time, it is necessary to ignore temporary and brief legal passage behaviors of people directly pushing or lifting cargo boxes. Output control requirements: Please return the corresponding data in JSON structured field format only." Model output: The large model outputs a structured binary classification review conclusion: {"result": "real violation" | "invalid interference", "reason": "<scene description>", "action": "keep" | "cancel"}, where the reason field records the key semantic features identified (such as "personnel collaborative handling action" and "ground mirror reflection area"), and the action field serves as the control signal for downstream alarm processing logic.

[0102] Fine-tuning steps and key parameters: Low-rank adaptive (LoRA) method was used to incrementally fine-tune the vision-language alignment layer of the large model, freezing the backbone weights of the visual encoder and updating only the LoRA adaptation layer (rank r=16, scaling factor alpha=32); AdamW optimizer was used with an initial learning rate of 1e-4, cosine annealing scheduling, batch size of 8, and a total training time of 20 epochs; the classification accuracy (F1-Score) of the fire escape scene was used as the main validation metric, and training was stopped early when the F1-Score on the validation set no longer improved for 3 consecutive epochs.

[0103] Scenario Integration: The innovation of this application lies in the system architecture design that combines the semantic inference capability of the large-scale model with the fire escape detection and re-inspection logic: a lightweight edge detection network is responsible for real-time judgment, while the cloud-based large-scale model is responsible for high-level semantic confirmation of suspected alarms. The two collaborate in an asynchronous, non-blocking manner, balancing real-time performance and re-inspection accuracy. The large-scale model receives alarm screenshots and video clips reported by local edge devices in an asynchronous, non-blocking manner.

[0104] After receiving alarm screenshots and short video clips 5 seconds before and after, the large model performs global scene semantic inference on the alarm frames. Unlike the lightweight network at the edge, which only focuses on local bounding box features, the cloud-based large model can understand the semantic relationships of the entire scene, such as perceiving high-level semantic information such as "whether there are multiple workers carrying boxes and coordinating to move them" or "whether there are large areas of water stains on the ground causing mirror reflections".

[0105] The large model internally constructs a scene semantic judgment rule library, covering the following typical legal or non-threatening occupation scenarios: Temporary Occupation Scenario: If the large model identifies that the current scene has temporary occupation characteristics of personnel and container collaborative moving movements (such as multiple people moving around an obstacle in the picture, and the position of the obstacle constantly changing), it is determined to be normal temporary passage behavior, and a reverse interception command is issued.

[0106] Environmental interference scenario: If the large model recognizes the specular reflection feature caused by water on the ground (there is a large area of ​​bright reflection on the ground, which is a mirror image of the building above or the sky), or recognizes that the ground shadow caused by strong light is misjudged as an obstacle (the shadow edge lacks three-dimensionality and projection depth), then a reverse interception command will be issued.

[0107] Once an alarm cancellation command is sent, the system cancels the alarm record, cancels the corresponding violation work order, and notifies the property security platform. Simultaneously, the false alarm sample (alarm screenshot and large model review conclusion) is recorded in the local false alarm sample library for subsequent incremental fine-tuning training of the edge network, enabling continuous self-optimization of the system. If the large model confirms a genuine violation, the alarm record is retained, and the violation work order continues to be pushed.

[0108] To verify the actual implementation effect of the technical solution of the present invention, the applicant deployed a control experiment in real fire passage scenarios including residential communities, industrial parks, and underground garages, and uniformly deployed an NVIDIA Jetson NX architecture edge AI gateway box with onboard GPU acceleration as the common hardware test platform for the three sets of comparison algorithms.

[0109] Test data source: RGB video frames separately collected from independent security monitoring in the above-mentioned fire lane deployment scenario, totaling 3200 static image samples and 720 hours of continuous video recording; all test samples were collected in real time after deployment and are independent of the 8600 image samples used in the training phase in terms of collection time, scene distribution and annotation personnel; the test conditions cover four scenarios: strong daylight, low illumination at night (<50 lux), rainy days and high-density pedestrian traffic.

[0110] The comparison objects include: existing single-frame static detection algorithms (Prior Technology 1), existing general-purpose lightweight YOLO algorithms (Prior Technology 2 is used as the baseline for ablation comparison, and the same architecture is used on the same NX edge platform during testing, but the post-optical flow timing and large model processing are disabled in cascade), and the dynamic timing detection algorithm of this invention. The data comparison of various core test indicators is shown in Table 1 below: Table 1. Comparison of core test metrics for different algorithms Existing single-frame algorithms often trigger false alarms (with a false alarm rate as high as 50%) when detecting pedestrians pushing carts or vehicles reversing, due to the lack of temporal judgment. This invention introduces optical flow temporal discrimination and a large-model semantic re-judgment mechanism. Baseline ablation comparison verification confirms that if only the improved spatial detection is loaded and temporal optical flow judgment is removed, the static judgment architecture cannot avoid dynamic action environments such as pedestrians passing by. During testing, the detection misleading rate in nighttime and shadow-occluded areas still fluctuates by 17.5% or more. However, this patent, by associating spatial attention mechanism with pixel-level optical flow temporal sequence for collaborative computation, eliminates the separation of spatial location. Both mechanisms effectively prevent transient environmental changes without incurring stuttering or delay penalties. Data from the actual test environment shows that the pre-processed optical flow tracking rigorously calculates obstacles. By adjusting the dwell time, the large-scale model accurately filters out "transportation operations" and "tree shadow reflections," and its high level of underlying coordination reduces the daily false alarm rate from 50% to below 4.2%. Security personnel save approximately 90% of the time they spend processing invalid work orders each day.

[0111] Existing lightweight algorithms over-prune to meet computational power requirements and lack environmental adaptability, resulting in a persistently high false negative rate (28.5%) at night. This invention combines adaptive Retinex illumination compensation and other front-image enhancement techniques with SPAM spatial attention weight enhancement, reducing the false negative rate to 6.3% in adverse weather conditions without requiring replacement with expensive full-color night vision hardware.

[0112] Some existing patents employ a dual-source solution of "camera + geomagnetic / infrared sensor" to address blind spots, resulting in high installation costs and significant power consumption. This invention adopts a pure vision architecture, eliminating redundant classification and sensor communication modules. As shown in the table, the algorithm's targeted pruning allows the system relying solely on pure vision to achieve an inference frame rate as high as 25 FPS, while reducing overall device power consumption to 52% of the benchmark. This demonstrates that the marginal cost and engineering adaptability of this invention far surpass existing technologies in practical application.

[0113] This invention provides a dynamic occupancy detection system for fire lanes based on video image analysis, such as... Figure 2 As shown, it includes: Obstacle recognition module 1 is used to acquire surveillance video and determine the fire lane area, perform target detection on single frame images, and obtain the category and bounding box of each obstacle; The false interference filtering module 2 is used to extract the pixel-level optical flow field of the obstacle between consecutive frames. If the pixel-level optical flow field shows non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it shows rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated. The real-time dynamic occupancy risk value calculation module 3 is used to query and extract the corresponding hazard weight from the preset category hazard weight configuration table, calculate the space occupancy ratio of the boundary box in the fire lane area, multiply the hazard weight by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle. Detection module 4 is used to determine static illegal occupation when the real-time dynamic occupation risk value is greater than or equal to the preset illegal risk threshold; it backtracks and extracts screenshots and continuous video clips related to the illegal occupation event, generates an evidence chain package, triggers an alarm on the local edge device, and asynchronously uploads the alarm screenshots and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decides whether to retain or cancel the alarm record based on the review conclusion.

[0114] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A method for dynamic occupancy detection of fire lanes based on video image analysis, characterized in that, include: Step S1: Acquire surveillance video and determine the fire lane area; perform target detection on single-frame images to obtain the category and bounding box of each obstacle; Step S2: Extract the pixel-level optical flow field of the obstacle between consecutive frames. If the pixel-level optical flow field exhibits non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it exhibits rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated. Step S3: Query and extract the corresponding hazard weight from the preset category hazard weight configuration table, calculate the space occupancy ratio of the bounding box in the fire lane area; multiply the hazard weight by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle. Step S4: When the real-time dynamic occupancy risk value is greater than or equal to the preset violation risk threshold, it is determined to be a static violation occupancy; backtrack and extract screenshots and continuous video clips related to the violation occupancy event, generate an evidence chain package, trigger an alarm on the local edge device, and asynchronously upload the alarm screenshot and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decide whether to retain or cancel the alarm record based on the review conclusion.

2. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, Step S1 involves identifying the fire lane area, specifically including: Extract image feature points from the initial unoccupied frames and determine the initial coordinates of the fire lane area; Periodically calculate the global affine transformation matrix between the current video frame and the initial unoccupied frame; The translation component is solved from the global affine transformation matrix to obtain the global translation offset. If the global translation offset is greater than the set tolerance, the coordinates of the fire lane area are dynamically compensated and updated using the inverse affine transformation matrix to offset the physical jitter of the camera equipment.

3. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, The target detection in step S1 specifically includes: Video frames are input into the object detection network to extract features, and a spatial attention mechanism module is introduced into the feature extraction layer; The spatial attention mechanism module is used to enhance the feature weights of low-lying pixel regions on the ground, thereby improving the sensitivity of bounding box extraction for low-lying obstacles and outputting the enhanced obstacle bounding boxes.

4. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, Step S2 extracts the pixel-level optical flow field of obstacles between consecutive frames, specifically including: Based on the bounding box coordinates of the obstacle, crop out the target image patch containing the obstacle and the adjacent ground area at the bottom of the obstacle from the current frame and the previous frame; The spatial high-frequency gradient magnitude is calculated for the target image patch. A binary mask is generated according to the preset spatial gradient threshold. Morphological closing operation is performed on the binary mask to fill the internal holes. Connected component analysis is performed on the mask after the closing operation to filter out isolated regions with connected component areas smaller than the preset area threshold, and a solid structure mask that closely fits the physical obstacle is generated. Corner points are extracted only within the connected regions of the solid structure mask as structural anchor points. The motion vectors of the structural anchor points between adjacent frames are calculated using the pyramid optical flow algorithm to obtain a sparse anchor point optical flow set. Using sparse anchor point optical flow sets as seed elements, a bilateral interpolation algorithm with pixel color difference as guiding weight is used to diffuse the optical flow field. During the diffusion process, a solid structure mask is used as a physical boundary constraint to prevent the optical flow vector from diffusing into the background and shadow pixel regions outside the solid structure mask. The pixel optical flow value outside the physical structure mask is forced to zero, and a dense optical flow field that is truncated and diffused through the physical boundary is output as the pixel-level optical flow field of the obstacle between consecutive frames.

5. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 4, characterized in that, Step S2 involves extracting pixel-level optical flow fields and determining non-rigid body deformation features, specifically including: Extract the motion vectors of all non-zero pixels within the pixel-level optical flow field; Calculate the directional divergence and magnitude variance of the motion vector; when the directional divergence exceeds the preset directional divergence threshold, or the magnitude variance exceeds the preset magnitude variance threshold, it indicates that the obstacle's appearance has broken or flickered irregularly, and is judged as a non-rigid body deformation feature.

6. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, Step S2 accumulates the effective continuous dwell time, and further includes a dynamic anti-disconnection mechanism: Assign persistent tracking identifiers to obstacles identified as rigid body translations; When the detection confidence of the obstacle corresponding to the persistent tracking identifier drops below the set confidence threshold due to brief occlusion and the duration does not exceed the preset maximum occlusion tolerance time, the trajectory association of the obstacle is maintained by linear extrapolation prediction of the historical frame displacement of the obstacle. During this period, the effective continuous dwell time is continuously accumulated without interruption. When the duration of occlusion exceeds the preset maximum occlusion tolerance time, stop accumulating the effective continuous dwell time and clear the persistent tracking identifier record.

7. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, Step S3 involves cross-dimensional nonlinear coupling and calculating the real-time dynamic occupancy risk value, specifically including: Substitute the effective continuous dwell time into the exponential growth function to calculate the generation time penalty; Multiply the hazard weight by the space duty cycle to calculate the space hazard base; Multiplying the spatial hazard baseline by the time penalty yields the real-time dynamic occupancy risk value.

8. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, Step S4 generates the evidence chain package, which specifically includes: The system continuously caches the monitoring video stream using a circular memory loop; When an unauthorized occupation determination is triggered, the following content is automatically extracted from the circular memory and combined: a screenshot of the moment when the real-time dynamic occupation risk value reaches the preset violation risk threshold, and continuous video clips containing set durations before and after the unauthorized occupation event; SHA-256 hash check values ​​are calculated for the screenshots and continuous video clips, and the hash check values ​​are associated with the corresponding files and stored to generate an evidence chain package that can be verified for integrity.

9. The method for dynamic occupancy detection of fire lanes based on video image analysis as described in claim 1, characterized in that, In step S4, the cloud-based multimodal visual large model performs semantic review, specifically including: The cloud-based multimodal visual big model receives alarm screenshots and video clips of preset duration before and after the alarm time from the local edge device, and performs global scene semantic inference on the alarm screenshots and video clips. If the cloud-based multimodal vision big data model identifies temporary occupation characteristics of personnel and container collaborative movement in the current scene, or identifies the specular reflection characteristics caused by water accumulation on the ground, it sends an alarm cancellation command to the edge device and property security platform to cancel the alarm record and violation work order that have been generated.

10. A dynamic occupancy detection system for fire lanes based on video image analysis, characterized in that, The system performs the detection method as described in any one of claims 1 to 9, comprising: The obstacle recognition module is used to acquire surveillance video and determine the fire lane area, perform target detection on single frame images, and obtain the category and bounding box of each obstacle; The false interference filtering module is used to extract the pixel-level optical flow field of obstacles between consecutive frames. If the pixel-level optical flow field exhibits non-rigid deformation characteristics with discrete internal vectors, it is determined to be background interference with light and shadow changes, and the dwell time of the obstacle is suppressed. If it exhibits rigid translation and the mean magnitude of the motion vector of non-zero pixels in the pixel-level optical flow field is calculated, when the mean magnitude is lower than the set low speed threshold, the effective continuous dwell time of the obstacle is accumulated. The real-time dynamic occupancy risk value calculation module is used to query and extract the corresponding hazard weights from the preset category hazard weight configuration table, calculate the space occupancy ratio of the boundary box in the fire lane area, multiply the hazard weights by the space occupancy ratio to obtain the space hazard base, substitute the effective continuous stay time into the exponential growth function to obtain the time penalty, and multiply the space hazard base by the time penalty to calculate the real-time dynamic occupancy risk value of the obstacle. The detection module is used to determine static illegal occupation when the real-time dynamic occupation risk value is greater than or equal to the preset illegal risk threshold; it backtracks and extracts screenshots and continuous video clips related to the illegal occupation event, generates an evidence chain package, triggers an alarm on the local edge device, and asynchronously uploads the alarm screenshot and video clips of preset duration before and after the alarm time to the cloud multimodal visual big model for semantic review, and decides whether to retain or cancel the alarm record based on the review conclusion.

Images