Fire passage occupation visual inspection method and system

Abstract

The application discloses a fire passage occupation visual inspection method and system, and belongs to the field of computer vision monitoring and fire safety management. In order to solve the problems of lagging behind in manual inspection, and misjudgment caused by personnel, light and background sundries, the application separates personnel and non-personnel foreground from a multi-light background prototype subjected to closed-loop gating, and confirms the continuous occupation residual after the personnel exit by bidirectional time sequence recovery. The same residual is mapped to a passage graph and an identification line-of-sight graph, and events and evidence are output according to the connected state, the bottleneck remaining net width and the continuous visible proportion, so that continuous inspection of the passage and the evacuation identification line-of-sight function of the fire passage is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision monitoring and fire safety management, and in particular to a method and system for visual inspection of fire lane occupancy. Background Technology

[0002] Building fire lanes serve the functions of personnel evacuation, fire rescue, and emergency material passage. Their clear width, continuous accessibility from entrance to exit, and the effectiveness of visibility when observing evacuation signs along the lanes all directly affect fire safety. Existing buildings typically use fixed monitoring equipment to obtain images of the lanes, but daily inspections still mainly rely on personnel checking the monitoring images at set times or conducting on-site inspections, making it difficult to continuously monitor the obstruction caused by vehicles, goods, and temporary construction materials.

[0003] Existing solutions based on image difference or object detection easily misidentify normal pedestrian passage, changes in local lighting, short-term occupancy, and changes in background clutter as occupancy. If images containing occupants or obstructions are directly written into the background, it can lead to subsequent missed detections. Furthermore, providing only the foreground area or object category cannot determine whether the foreground cuts off the path from the entrance to the exit, creates a local bottleneck in net width, or obstructs the effective graphic area of ​​evacuation signs, thus making it difficult to form continuous evidence corresponding to the loss of fire protection functionality.

[0004] Therefore, there is a need for a visual inspection method and system for fire lane occupancy that can overcome the shortcomings of the existing technologies. Summary of the Invention

[0005] One objective of this invention is to propose a visual inspection method and system for fire lane occupancy. Addressing the problems of existing technologies that rely on manual, periodic inspections, making it difficult to detect lane occupancy promptly, and the susceptibility to misjudgments due to personnel passage, lighting changes, obstructions, and background clutter, this invention proposes a method that utilizes a multi-light background prototype with closed-loop gating for foreground separation. After personnel exit, the method confirms the persistent occupancy residual by combining continuous observation and lighting interpretation results from bidirectional temporal recovery of keyframes before and after entry. The same residual is then bound to the encroached unit on the ground passage map and the obstructed line of sight on the signage sight map. Furthermore, events and evidence are output based on the connectivity status from entrance to exit, the remaining net width of the bottleneck, and the continuous visibility ratio of evacuation signs. The background update and freeze state are controlled by dual-function compliance results, transforming appearance changes into losses of passage and visibility functions verified by personnel exit.

[0006] This invention provides a visual inspection method for fire lane occupancy, comprising:

[0007] S1. Collect continuous video of the ground area of ​​the fire lane and the evacuation sign area. Select the background prototype that matches the current frame from the multi-light background prototype library that is written to the gate in the closed loop. Perform photometric correction, foreground detection and personnel tracking on the current frame to obtain the time sequence of personnel occlusion area and non-personnel foreground.

[0008] S2. During the period of personnel occupancy, freeze the cumulative occupancy and background update of the area, and after the personnel leave, perform bidirectional temporal recovery based on the keyframes before the personnel enter and after the personnel leave. Non-personnel foreground that is consistent in bidirectional recovery, inconsistent with continuous observation after the personnel leave, and cannot be explained by changes in illumination are identified as persistent occupancy residuals and assigned residual labels.

[0009] S3. Project the persistent occupancy residual onto the walkable units divided along the longitudinal direction of the fire lane to determine the connectivity from the entrance to the exit and the remaining net width of the bottleneck.

[0010] S4. Establish a line of sight from the preset observation position to the effective graphic area of ​​the evacuation sign, update the occlusion state of the line of sight with the spatial projection of the continuous occupation residual, determine the continuous visibility ratio of the evacuation sign, and associate the occupied walkable unit and the occluded line of sight with the residual sign.

[0011] S5. Output an occupancy event or a marker occupancy event when the connectivity is disconnected, the remaining net width of the bottleneck does not reach the specified net width threshold, or the continuous visibility ratio after personnel exit does not reach the visibility ratio threshold; only update the multi-light background prototype library when there is no continuous occupancy residual and the connectivity, the remaining net width of the bottleneck, and the continuous visibility ratio all meet the standards; otherwise, keep the corresponding area frozen.

[0012] Optionally, S1 includes: reading channel geometric calibration data and a judgment threshold, wherein the channel geometric calibration data includes ground calibration points, entrance boundary, exit boundary, effective graphic area of ​​evacuation signs and preset observation position zone, and the judgment threshold includes a specified net width threshold and a visibility ratio threshold;

[0013] Illumination descriptors are extracted from frames with different illumination states where the clearance has been confirmed, and background prototypes are established. The background prototype is selected based on the distance between the illumination descriptor of the current frame and the illumination descriptors of each background prototype.

[0014] The current frame is partitioned and normalized according to the local brightness mapping relationship corresponding to the selected background prototype. The normalized current frame is then differentiated from the selected background prototype and combined with the target trajectory category to output the temporal sequence of the personnel occlusion area and the non-personnel foreground.

[0015] Optionally, S2 includes: saving a pre-entry keyframe when the personnel target trajectory enters the ground area of ​​the fire lane, and marking the area covered by the personnel target trajectory as the personnel occlusion area;

[0016] Save the key frame after exiting when the personnel target trajectory exits and the inter-frame change amount continuously fails to reach a stable threshold and reaches a preset number of frames.

[0017] Starting from the entry keyframe and the exit keyframe respectively, pixels or features are propagated to the person during the occlusion period using the forward motion field and the backward motion field to obtain the forward restoration result and the backward restoration result.

[0018] The recovery difference between the forward recovery result and the backward recovery result is calculated. In the region where the recovery difference does not exceed the recovery consistency threshold, the arithmetic mean of the corresponding pixel values ​​or feature vectors of the forward recovery result and the backward recovery result is taken to obtain the fused recovery result. The observation difference between the fused recovery result and the continuous observation after exiting the observation, as well as the illumination difference between the fused recovery result and the illumination change prediction are calculated.

[0019] Regions where the recovery difference does not exceed the recovery consistency threshold and the observation difference does not exceed the observation consistency threshold, as well as regions where the recovery difference does not exceed the recovery consistency threshold, the observation difference exceeds the observation consistency threshold and the illumination difference does not exceed the illumination interpretation threshold, are all classified as recoverable regions. Regions where the recovery difference does not exceed the recovery consistency threshold, the observation difference exceeds the observation consistency threshold and the illumination difference exceeds the illumination interpretation threshold are classified as persistently occupied residuals. The remaining regions are classified as unknown occlusion regions.

[0020] For recoverable areas, the cumulative freeze of occupation is lifted and the background write status is determined by closed-loop write gating. For persistently occupied residuals, the cumulative freeze of occupation and the cumulative event duration are lifted and the background write freeze is maintained. For unknown occupancy areas, the cumulative freeze of occupation and the background write freeze are maintained.

[0021] Furthermore, a verification waiting time is set for the unknown occupancy area. During the verification waiting time, the cumulative occupancy freeze and background writing freeze are maintained, and observation frames without personnel target trajectories are collected and bidirectional time-series recovery is repeated.

[0022] When repeated recovery meets the condition of continuous occupancy residual, the unknown occupancy residual is converted into a continuous occupancy residual, the cumulative occupancy freeze and cumulative event duration are released, and the background write freeze is maintained;

[0023] When repeated recovery meets the conditions of the recoverable area, the accumulated freeze of the occupied area is released and the background write status is determined by the closed-loop write gating.

[0024] When the review waiting time expires and the area is still an unknown occupancy zone, the cumulative freezing of occupancy and background writing are maintained, and a record awaiting manual review is generated.

[0025] Optionally, S3 includes: establishing the projection relationship between the image coordinates and the ground coordinates of the fire lane using ground calibration points, and expanding the projection area of ​​the continuous occupancy residual in the ground coordinates according to the calibration error zone;

[0026] The ground area is divided into slices according to the longitudinal position of the passage, and the area in each slice that is not occupied by the expanded projection area and is located within the boundary of the passage is taken as a walkable unit.

[0027] Connect the walkable units that are in contact with each other in adjacent slices to form a ground access map, and determine the entrance-exit path from the corresponding node of the entrance boundary to the corresponding node of the exit boundary in the ground access map;

[0028] For each inlet-outlet path, the lateral width of the walkable units that the path passes through in each slice is obtained to form a path width set. The lower bound of the path width set is used as the path bottleneck net width of the corresponding inlet-outlet path. The path bottleneck net widths of each inlet-outlet path are then used to form a path bottleneck set. The upper bound of the path bottleneck set is used as the remaining net width of the bottleneck.

[0029] When an inlet-outlet path exists, the connectivity state is set to connected; when no inlet-outlet path exists, the connectivity state is set to disconnected and the remaining net width of the bottleneck is set to zero.

[0030] Optionally, S4 includes: using camera calibration parameters to transform the observation positions selected at preset intervals within the preset observation position band and the graphic sampling points within the effective graphic area of ​​the evacuation marker to the current frame image coordinate domain, using the line segments between each observation position and each graphic sampling point as the line edges to form a marker line map, and using the persistent occupancy residual mask in the current frame as the spatial projection in the line coordinate domain.

[0031] The line-of-sight edges that intersect with the continuously occupied residual mask are marked as obscured line-of-sight edges, and the line-of-sight edges that do not intersect are marked as visible line-of-sight edges;

[0032] Along the extension direction of the preset observation position band, adjacent observation positions where the number of visible line edges reaches or exceeds the single-position visibility threshold are merged into continuous visible segments. When continuous visible segments exist, the ratio of the supremum of the length set of each continuous visible segment to the length of the preset observation position band is used as the continuous visibility ratio. When there are no continuous visible segments, the continuous visibility ratio is set to zero.

[0033] Optionally, S5 includes: establishing an event status record indexed by the residual identifier, wherein the event status record includes the first confirmation time, the most recent confirmation time, the duration, the invaded unit, the edge of the obscured line of sight, the connectivity status, the remaining net width of the bottleneck, the continuous visibility ratio, and the before and after comparison keyframes.

[0034] The event status record allows for the parallel recording of functional loss types. When the remaining net width of the bottleneck does not reach the specified net width threshold, a net width bottleneck label is added. When the connectivity status is disconnected, a path disconnection label is added. When the continuous visibility ratio does not reach the visibility ratio threshold, an identification occlusion label is added. When an identification occlusion label has been added and either a path disconnection label or a net width bottleneck label has been added, a dual risk label is added while retaining the existing labels.

[0035] Set the closed-loop write gate to background update interlock. Only when the continuous occupancy residual corresponding to the event state record disappears, the connectivity state is connected, the remaining net width of the bottleneck reaches or exceeds the specified net width threshold, and the continuous visibility ratio reaches or exceeds the visibility ratio threshold, the event state record is set to the deactivated state and the background prototype of the corresponding lighting state is allowed to receive the current frame.

[0036] Furthermore, according to the residual identifier, the keyframe before personnel enter, the keyframe after personnel exit, the continuously occupied residual mask, the invaded unit, the obscured line edge, as well as each connectivity state, the remaining net width of the bottleneck, and the continuous visibility ratio are written into the same evidence chain.

[0037] In subsequent frames, the residual identifier is maintained based on the spatial overlap relationship of the residual region and the distance of the appearance feature. When the residual splits or merges continuously, the parent residual identifier and the child residual identifier are recorded, and the event location, duration and functional loss type are updated with the evidence chain.

[0038] Furthermore, a coordinate interval that maintains the order of boundary points is constructed based on the positioning error range of the ground calibration points. The coordinate interval is propagated to the boundary of the invaded unit and the boundary of the continuously occupied residual mask. For each line of sight, all intersecting segments formed by the intervalized continuously occupied residual mask and the line of sight are determined. The union of all intersecting segments is used as the blocking range of the line of sight. The lower and upper bounds of the remaining net width of the bottleneck and the lower and upper bounds of the continuous visibility ratio are calculated based on the intervalized boundary of the invaded unit and the blocking range of each line of sight.

[0039] When the lower bound of the remaining net width of the bottleneck does not reach the specified net width threshold and its upper bound reaches or exceeds the specified net width threshold, or when the lower bound of the continuous visible ratio does not reach the visible ratio threshold and its upper bound reaches or exceeds the visible ratio threshold, the background writing is frozen and a record awaiting manual review is generated. When the upper bound of the remaining net width of the bottleneck does not reach the specified net width threshold or the upper bound of the continuous visible ratio does not reach the visible ratio threshold, the corresponding alarm status is maintained.

[0040] On the other hand, the present invention also provides a visual inspection system for fire lane occupancy, comprising:

[0041] The video and background processing module is used to acquire continuous video, match multi-light background prototypes, and output the time sequence of personnel occlusion areas and non-personnel foreground areas. The residual confirmation module is used to freeze the cumulative occupancy of the corresponding area during the personnel occupancy period and update the background, and perform bidirectional time sequence recovery after the personnel leave to determine the continuous occupancy residual with residual identifier. The passage function determination module is used to project the continuous occupancy residual onto the walkable unit and determine the connectivity status from the entrance to the exit and the remaining net width of the bottleneck. The identification recognition determination module is used to update the occupancy status of the line of sight edge with the spatial projection of the continuous occupancy residual, determine the continuous visibility ratio, and associate the intruded walkable unit and the occluded line of sight edge with the residual identifier. The closed-loop gating and event output module is used to output occupancy events or occupancy marker events according to the connectivity status, the remaining net width of the bottleneck, and the continuous visibility ratio, and control the update of the multi-light background prototype library and the freezing status of the corresponding area.

[0042] The beneficial effects of this invention are:

[0043] 1. By freezing the cumulative occupancy and background writing during the period of personnel coverage, and using the keyframes before and after personnel exit for bidirectional timing recovery, and combining the results of continuous observation and illumination change interpretation after exit to confirm the continuous occupancy residual, misjudgments caused by personnel passage, short-term occupancy and illumination changes can be reduced.

[0044] 2. By projecting the same persistent occupancy residual onto walkable units divided along the longitudinal direction of the passage, and calculating the connectivity and remaining net width of the bottleneck on the complete entrance-exit path, unknown objects or local small-area encroachment can be converted into verifiable loss of passage function.

[0045] 3. By binding the persistent occupancy residual to the obscured edge of the evacuation sign sight map and calculating the continuous visibility ratio, and writing the joint gating background prototype with passage results, visual recognition results and residual status, the homologous association between events, locations, durations, functional loss types and before-and-after comparison evidence can be maintained. Attached Figure Description

[0046] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0047] Figure 1 This is a flowchart of the visual inspection method for fire lane occupancy according to the present invention.

[0048] Figure 2 This is a flowchart of the continuous occupancy residual confirmation sub-process based on bidirectional timing recovery in S2 of the present invention. Detailed Implementation

[0049] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0050] refer to Figures 1-2 Visual inspection methods for fire lane obstruction include:

[0051] S1. Collect continuous video of the ground area of ​​the fire lane and the evacuation sign area. Select the background prototype that matches the current frame from the multi-light background prototype library that is written to the gate in the closed loop. Perform photometric correction, foreground detection and personnel tracking on the current frame to obtain the time sequence of personnel occlusion area and non-personnel foreground.

[0052] S2. During the period of personnel occupancy, freeze the cumulative occupancy and background update of the area, and after the personnel leave, perform bidirectional temporal recovery based on the keyframes before the personnel enter and after the personnel leave. Non-personnel foreground that is consistent in bidirectional recovery, inconsistent with continuous observation after the personnel leave, and cannot be explained by changes in illumination are identified as persistent occupancy residuals and assigned residual labels.

[0053] S3. Project the persistent occupancy residual onto the walkable units divided along the longitudinal direction of the fire lane to determine the connectivity from the entrance to the exit and the remaining net width of the bottleneck.

[0054] S4. Establish a line of sight from the preset observation position to the effective graphic area of ​​the evacuation sign, update the occlusion state of the line of sight with the spatial projection of the continuous occupation residual, determine the continuous visibility ratio of the evacuation sign, and associate the occupied walkable unit and the occluded line of sight with the residual sign.

[0055] S5. Output an occupancy event or a marker occupancy event when the connectivity is disconnected, the remaining net width of the bottleneck does not reach the specified net width threshold, or the continuous visibility ratio after personnel exit does not reach the visibility ratio threshold; only update the multi-light background prototype library when there is no continuous occupancy residual and the connectivity, the remaining net width of the bottleneck, and the continuous visibility ratio all meet the standards; otherwise, keep the corresponding area frozen.

[0056] In this specific embodiment, S1 includes:

[0057] In this specific embodiment, the video and background processing module is connected to a network camera fixedly installed at the top or side of the fire lane. The camera outputs color video frames with monotonically increasing timestamps at 1920×1080 pixels and 25 frames / second. The module forms a current frame record according to the camera number and acquisition time, and reads the ground area of ​​the fire lane, evacuation sign area, entrance boundary, exit boundary, ground calibration point, preset observation position zone, and effective graphic area of ​​evacuation sign from the calibration configuration library. The configuration record also saves the metric ground coordinates, pixel coordinates, calibration version, and effective time. In this specific embodiment, the specified net width threshold is set to 1.10m and the visibility ratio threshold is set to 0.80. These two are written into the judgment threshold table as site configuration fields. The threshold table uses the building number, floor number, and passage number as the joint search key. When the video stream lacks a timestamp, the calibration version is inconsistent, or the joint search key does not match, the current frame is marked as an invalid configuration frame and is not included in the occupancy accumulation and background writing.

[0058] The video and background processing module collects no fewer than 300 frames of images during the initial period after the fire escape route has been manually confirmed to be clear. Background prototypes are established for four lighting conditions: natural light in the early morning, mixed daytime light, constant nighttime lighting, and emergency lighting. For each frame, a lighting description vector is extracted after excluding personnel detection boxes and moving foreground elements. ,in, Indicates time The illumination description vector, Indicates the time index of the current frame. This represents the normalized mean of the brightness channel in the ground area, and its value range is... , Indicates the brightness channel number The pixel proportion of each histogram interval, with a value range of [value missing]. , Represent the histogram interval index and satisfy... , This represents the dimensionless contrast quantity obtained after truncating the ratio of the mean brightness of the evacuation marking area to the ground area, and its value range is [value missing]. Before calculating the average brightness ratio, the normalized average brightness of the ground area, which serves as the denominator, is checked. In this specific implementation, the effective threshold of the denominator, calibrated by the black field, lens occlusion, and severely underexposed verification segments, is set to 0.02. If the average value is not greater than 0.02, division is not performed, the current frame is recorded as an invalid brightness frame, and it is prohibited from entering subsequent prototype distance calculation, occupancy accumulation, and background writing. The most recent valid prototype is kept read-only. Only when the average value is greater than 0.02 is the average brightness ratio calculated and truncated to the above range. The module uses mini-batch clustering to group similar illumination descriptions into the same prototype. The prototype record includes prototype number, description mean, description standard deviation, pixel-wise background median, local brightness mapping parameters, clearance confirmation flag, and version number.

[0059] After extracting the illumination description vector for the current frame, the module follows... Calculate its standardized distance from each background prototype, where, Indicates the current frame and the first frame. The dimensionless distance between background prototypes, with smaller values ​​indicating better lighting matching. Indicates the background prototype index. The current lighting description vector is represented by the first... One portion, Indicates the first The background prototype in the first The mean of each component This represents the standard deviation of the component. This represents a dimensionless smoothing quantity to prevent the standard deviation from being zero, and in this specific embodiment it is set to 0.01. In this formula, the component index of the 18-dimensional illumination description vector is specifically represented and taken as an integer from 1 to 18. The letter j is only retained as the index of the first sixteen histogram intervals. The module selects the prototype with the smallest distance. When the smallest distance exceeds the illumination acceptance threshold of 6.5 calibrated by the 99th percentile of the clearance verification set, the frame is recorded as an unknown illumination frame and the most recently accepted prototype is used for read-only detection. At the same time, the unknown illumination frame is prohibited from being written to the prototype library.

[0060] The module divides the ground area of ​​the fire lane into local brightness zones using a 64×64 pixel grid and performs the zone mapping according to the zone mapping coefficients saved by the selected background prototype. ,in, Indicates the pixel after photometric correction The brightness value and the unit is gray level. Indicates the pixel coordinates within the current partition. Indicates the pixels before correction The brightness value and the unit is gray level. Indicates the first The background prototype in the first Dimensionless gain coefficient of each partition This indicates the grayscale offset of the corresponding partition. Indicates a local brightness partition index. This means that the result is truncated to a given closed interval; the gain coefficient and bias are obtained by robust linear regression between the net sample and the corresponding prototype and are limited to the range of 0.70 to 1.30 and -35 to 35 gray levels. Partitions with mapping residuals exceeding 20 gray levels for three consecutive frames are marked as illumination-uninterpretable partitions and the background write is frozen.

[0061] After photometric correction, the module follows Form an initial foreground mask, wherein, Represents pixels At any moment Binary foreground labeling, This is an indicator function that takes the value 1 if the condition is true and 0 otherwise. Indicates the first A background prototype in pixels The median background brightness at that location, expressed in grayscale levels. This represents the foreground difference threshold of the pixel, expressed in gray levels. The foreground difference threshold is obtained by multiplying the median absolute deviation of the absolute difference of the net sample by 3 and adding 5 gray levels. The value is limited to 12 to 45 gray levels. Then, 3×3 opening operation is used to remove isolated noise, 5×5 closing operation is used to fill small holes, and connected components with an area of ​​less than 80 pixels that do not appear consecutively for three frames are deleted. This yields a foreground record with connected component number, bounding box, pixel area, and number of consecutive frames.

[0062] The module uses 640×640 pixel scaled frames as input to the personnel detection network. It uses a convolutional target detector finely tuned with personnel samples from fire escape routes to output personnel categories, detection boxes, and confidence scores. It performs frame-by-frame trajectory association by combining cross-union ratio (CUNR) with Kalman motion prediction. The training samples are personnel boxes collected by similar cameras and manually labeled. The training set, validation set, and test set are divided into three sets in an 8:1:1 ratio. During inference, the personnel confidence threshold is set to 0.55 and the non-maximum suppression cross-union ratio threshold is set to 0.50. Detection boxes with a CUNR of less than 0.55 but with a cross-union ratio of 0.70 with existing trajectory prediction boxes are only used to continue the trajectory for two frames without creating new personnel labels. The personnel detection boxes are expanded outward by 8 pixels according to the human contour segmentation results to form a conservative personnel occlusion region. Foreground pixels with an intersection ratio of 0.20 with the personnel occlusion region are included in the personnel occlusion mask, and the remaining foreground pixels are written into the non-personnel foreground mask.

[0063] The module uses the camera hardware timestamp as the main timeline and writes video frames into the frame sequence at 40ms intervals. When the interval between adjacent timestamps exceeds 80ms, a missing frame mark is inserted instead of using interpolated images to replace the real observation. When more than 5 frames are missing consecutively, the occupancy accumulation of the corresponding partition and the background are written and frozen at the same time. After the acquisition is restored, 8 consecutive frames of images with increasing timestamps and qualified illumination distance are required to remove the acquisition anomaly mark, ensuring that the key frames used by S2 before entry, after exit, and continuous observations come from the same time reference.

[0064] The background prototype configuration table is generated uniformly by the clearance sample construction task. Each record in the table uses the camera number and the lighting prototype number as keys and stores the 18-dimensional description mean, description standard deviation, gain and bias for each zone, background median for each pixel, sample start and end time, clearance reviewer number, and version summary. The construction task is re-executed every quarter or after the camera location changes. The switch will only take effect when the new version has a foreground false trigger rate of no more than 0.5% on an independent clearance verification segment and the prototype selection accuracy reaches 98%. Records with unmatched keys, expired versions, or missing clearance confirmation flags shall not be used for background writing.

[0065] After the initial foreground connected components are associated with the personnel trajectory, the module maintains the same non-personnel foreground sequence identifier for connected components that do not intersect with the personnel occlusion area in three consecutive frames, based on the condition that the centroid distance does not exceed 20 pixels and the appearance histogram distance does not exceed 0.25. When there are multiple associated candidates, they are first sorted by centroid distance from smallest to largest, and then by appearance distance from smallest to largest. When both sorting keys are the same, the sequence with the earlier first appearance time is selected. A new sequence is established for unmatched connected components. The sequence that is unmatched for five consecutive frames ends, but its historical mask is retained for S2 recovery comparison.

[0066] The video and background processing module outputs a foreground timing record for each frame. The recording fields include frame timestamp, selected background prototype number, illumination distance, photometric mapping version, personnel trajectory identifier, personnel occlusion mask, non-personnel foreground mask, illumination-unexplainable partition, and background write permission bit. When a personnel trajectory enters the ground area of ​​the fire lane, the module immediately freezes the cumulative occupancy permission position of the partition covered by the trajectory and freezes the background write permission position. Partitions not covered by personnel and without non-personnel foreground or illumination-unexplainable markers remain in read-only detection state. The resulting personnel occlusion area and non-personnel foreground timing are written to the timing buffer in the order of camera number and timestamp for S2 to read and perform bidirectional timing recovery after personnel exit.

[0067] In this specific embodiment, S2 includes:

[0068] The residual confirmation module reads the personnel trajectory, personnel occlusion mask, non-personnel foreground mask, and selected background prototype number written by S1. When the ground projection of the personnel trajectory intersects the ground area of ​​the fire lane for the first time, the module selects the image with the most recent frame before that moment from the circular frame buffer, where the personnel mask is empty and the foreground area ratio does not exceed 0.5%, as the key frame before personnel entry. The module also writes the union of the areas covered by the trajectory during the entry to exit period into the occlusion status record. The record fields include trajectory identifier, entry time, set of covered partitions, index of key frame before entry, valid key frame bit, cumulative occupancy freeze bit, and background write freeze bit. When the candidate frame set is empty, no key frame before entry is generated. The valid key frame bit is set to 0, and the cumulative occupancy of the covered partition and the background write are kept double-frozen. The module continues to wait in the read-only buffer for a reference frame that has been manually confirmed to be clear and matches the lighting prototype. If no valid reference frame is found after 10 seconds, a record for manual review is generated and bidirectional recovery is not initiated. When the same partition is covered by multiple personnel trajectories, the freeze is controlled by the count of the unexited trajectories. Entry recovery judgment is only allowed when the count is zero and all key frames are valid.

[0069] Personnel trajectories are only designated as geometric exit candidates if their ground projection continuously crosses the configured entrance or exit boundary from inside the channel, the direction of movement points out of the channel, and they remain outside the boundary for the next five consecutive frames. Trajectories are designated as disconnected only if they lack effective surface projection for five consecutive frames due to low confidence, detection mismatch, or occlusion by other personnel. Disconnected trajectory status takes precedence over exit candidates and maintains double-freezing of the coverage partition. The module performs recapture within 2 seconds using Kalman prediction boxes, color histograms, and edge orientation histograms. If recapture is successful, the original trajectory is restored. If recapture fails within the time limit, a disconnected trajectory record awaiting manual review is generated, preventing direct triggering of recovery. For trajectories that have passed the geometric exit gate, the module constructs a set of pixels to be recovered, consisting of the occluded area and its 12-pixel outward neighboring region. After confirming that the number of pixels in this set is greater than 0, calculation is performed. ,in, Indicates time Dimensionless inter-frame variation within the region to be recovered This represents the set of pixels to be recovered. This indicates the number of pixels in the pixel set. Indicates the pixels to be recovered. Indicates time Pixel brightness values ​​after photometric correction This represents the pixel brightness value after photometric correction in the previous frame; when the set of pixels to be recovered is empty or invalid, division is prohibited, the recovery input is written as invalid, double freezing is maintained, and a record of set anomalies to be reviewed is generated; in this specific embodiment, the stability threshold is set to 0.015. When the change amount does not reach this threshold for 8 consecutive frames, the 8th frame is saved as the key frame after the personnel exit. If the stability condition is not met within 10 seconds after exiting, double freezing is maintained and a record of video disturbances to be reviewed is generated.

[0070] The module uses the pre-entry keyframe and post-exit keyframe as boundaries to calculate the forward and backward motion fields of adjacent frames within the occlusion time period using a pyramid dense optical flow network. The network input is normalized to... The network outputs two frames of three-channel images as a two-dimensional vector composed of horizontal and vertical displacements per pixel. It is fine-tuned using adjacent frames and synthetic occlusion samples from similar fire escape videos, with bidirectional reprojection error and edge smoothing as training objectives. Model versions with reprojection errors exceeding 3 pixels on the validation set are not allowed to be deployed. Before entering the recovery branch, the module reads the background prototype and partition mapping version accepted by S1 for each keyframe, performs the predefined positive photometric correction for each of the three channels of the keyframe, converting the grayscale values ​​of the original observation domain to the corrected background prototype domain with a range of 0 to 255 grayscale levels. If any keyframe mapping is invalid or the version is missing, the corresponding pixel is marked as propagation invalid. The module uses the union of the personnel mask and the non-personnel foreground mask after exiting each frame as the exclusion parameter. The mask enables forward and backward retrieval to jointly estimate the background state after removing people and candidate non-people foregrounds at the exit time: the forward branch propagates the net pixels from the entry keyframe, and the motion of the occluded area is taken as the median of the effective motion vectors within its 8-pixel outer ring; the backward branch first deletes the pixels within the exclusion mask from the exit keyframe, then reads the three-channel values ​​of the same version of the read-only background prototype in the corrected background prototype domain as the initial value of the occluded area, and performs boundary constraint iterative completion with the current effective corrected pixels of the 8-pixel ring outside the exclusion mask boundary; both branches are mapped to the spatial coordinate domain of the exit keyframe but always remain in the corrected background prototype radiation domain. Pixels that go beyond the image boundary, have a forward and reverse motion field closure error of more than 2 pixels, or have an effective support pixel ratio of less than 60% are marked as propagation invalid.

[0071] For pixels that are effectively propagated, the module calculates... ,in, Represents pixels The dimensionless recovery difference between the forward and backward recovery results, with a value range of . , Indicates the forward recovery result in pixels The three-channel brightness vector at that location. Indicates the backward recovery result in pixels The three-channel brightness vector at that location. The sum of the absolute values ​​of each component of the vector is represented; the recovery consistency threshold is determined by the 95th percentile of the recovery difference in the personnel clearance verification segment, which is 0.08 in this specific implementation. Pixels whose recovery difference exceeds this threshold or whose propagation is invalid do not participate in the fusion and are temporarily assigned to the unknown occlusion area.

[0072] At pixels where the recovery difference does not exceed the recovery consistency threshold, the module follows... Generate fusion recovery results, in which, Represents pixels The three-channel fused recovery vector located in the corrected background prototype radiation domain. and These represent the forward and backward recovery vectors, which are defined and located in the same radiative domain, respectively. The module then takes the median of the three channels (without photometric correction) over 12 consecutive frames after exiting as the continuous observation data for the current original observation domain. Read the background prototype number, local brightness partition, forward mapping gain, bias and version of the currently accepted S1, and convert the continuous observation to the corrected background prototype domain according to the following forward relationship;

[0073] in, Represents pixels The module transforms the fused and restored background to the current original observation domain according to the following inverse relationship;

[0074] in, Represents pixels The three-channel background prediction vector located in the current original observation domain. and These are the current acceptance background prototypes defined by S1. The partition forward mapping of gain and bias is performed, and the inverse transformation is performed only when the gain is in the effective range of 0.70 to 1.30 and is not zero; when forward and inverse mapping are effective, the following observation differences are calculated in the corrected background prototype domain.

[0075] in, This represents the dimensionless observational difference between the fusion recovery result within the corrected background prototype domain and the corrected continuous observations; the following illumination interpretation difference is calculated in the current original observation domain;

[0076] in, This represents the dimensionless residual difference in illumination interpretation between continuous observations within the current original observation domain and the inverse transform-recovered background prediction; occupants are retained in the continuous observations. The middle enters along with the positive mapping. Only changes in illumination bring two pairs of vectors in the same domain closer together, while occupancy that cannot be explained by the same version of the forward and inverse mappings increases the corresponding differences. Both of these differences take values ​​from... ;

[0077] The observation consistency threshold is set to 0.10 based on the 95th percentile of the observation difference in the corrected background prototype domain for the segment where personnel exit the area. The illumination interpretation threshold is set to 0.08 based on the segment where only illumination switching occurs in the current original observation domain. The module first verifies that continuous observations, the current local brightness mapping, and the background prototype number and version are all valid and consistent. The forward mapping input, forward mapping output, reverse mapping input, and reverse mapping output all fall within the gray level of 0 to 255, and the gain meets the aforementioned non-zero valid range. If any item is invalid, occurs in a non-finite number, or the version is inconsistent, it is directly classified into the unknown occlusion area and cannot be judged as a recoverable area. For the remaining propagated valid pixels, a deterministic classification is performed: if the recovery difference does not exceed the recovery consistency threshold and the observation difference does not exceed the observation consistency threshold, it is classified into the recoverable area. When the observation difference exceeds the observation consistency threshold but the illumination difference does not exceed the illumination interpretation threshold, it is also classified into the recoverable area. When the recovery difference does not exceed the recovery consistency threshold, the observation difference exceeds the observation consistency threshold, and the illumination difference exceeds the illumination interpretation threshold, it is classified into the persistent occupancy residual. The remaining pixels are classified into the unknown occupancy area. In the illumination-only switching verification segment, when the recovery difference is 0.04, the same-domain observation difference is 0.06, and the illumination difference is 0.03, it is classified into the recoverable area. In the verification segment where an object is placed during the period of human occupancy, when the recovery difference is 0.05, the same-domain observation difference is 0.22, and the illumination difference is 0.19, it is classified into the persistent occupancy residual. The classification results are connected by a 3×3 closing operation to connect the discontinuous regions of the same object and delete the transient isolated regions with an area of ​​less than 100 pixels. However, the deleted regions are still retained in the unknown occupancy area and are not written into the background.

[0078] The residual confirmation module calculates the mask, bounding box, centroid, area, color histogram, and edge direction histogram for the continuously occupied residual connected components. It generates a temporary candidate identifier that is unique only in the current frame using the camera number, the current frame time, and an increasing sequence number obtained by sorting by area and centroid. For recoverable areas, it removes the cumulative freezing of occupancy and returns the background write status to the closed-loop write gate of S5. For continuously occupied residuals, it removes the cumulative freezing of occupancy, accumulates the event duration from the first confirmation time, and keeps the background write frozen. For unknown occupancy areas, it continues to keep the cumulative freezing of occupancy and the background write frozen. The above-mentioned area status, three difference statistics, threshold version, and keyframe index are written into the recovery judgment record so that the same parameter version is used in subsequent reviews.

[0079] For unknown occlusion areas, in this specific implementation, the review waiting time is set to 10 seconds. Within this time, the module only receives observation frames that do not contain personnel target trajectories and whose inter-frame changes meet the stability conditions. Every 8 stable frames are accumulated, the original key frame before entry and the latest stable frame are used to repeat the bidirectional timing recovery. When the repetition result meets the triple conditions of continuous occupancy residual, the corresponding pixel is converted into continuous occupancy residual, the occupancy cumulative freeze is lifted, the event duration is accumulated, and the background write freeze is maintained. When the repetition result meets any recoverable condition, the occupancy cumulative freeze is lifted and the background write state is handed over to the closed-loop write gate. If the review waiting time expires and it is still an unknown occupancy area, the double freeze is maintained and a record for manual review containing key frames, difference maps, and coverage partitions is generated.

[0080] Before entering, the keyframe is determined from the candidate frame set within 2 seconds before the trajectory entry time. The module first deletes candidate frames containing personnel masks, invalid configuration markers, or foreground area ratios exceeding 0.5%. Then, it sorts the frames according to the time difference from the entry time from smallest to largest. If the time differences are the same, the frame with the highest illumination distance is selected. If both sorting keys are the same, the frame with the highest sequence number is selected. After deletion, if the candidate set is empty, the keyframe invalid branch is kept double-frozen and not filled from invalid frames or frames after entry. After exiting, the keyframe is selected only from the set of stable frames that have passed the geometric exit gate and meet the stability condition of 8 consecutive frames. If there are retransmitted frames at the same timestamp, the frame with the highest network sequence number and the highest verification is selected. This ensures that the selection of keyframes before and after has a definite candidate set, sorting direction, parallel rules, and unique output.

[0081] For each adjacent frame pair, the module only accepts direct motion field sampling when the closed-loop error after combining the forward and backward motion vectors does not exceed 2 pixels, the two frames have the same lighting prototype number, and the pixel is outside the exclusion mask. Inside the exclusion mask, the appearance of people or candidate objects is not used; instead, the mask is completed according to the aforementioned 8-pixel extended ring motion median and read-only background prototype boundary constraints. If the effective ring support ratio is less than 60%, the average residual of the three channels of the completed boundary exceeds 10 gray levels, or convergence is not achieved after 30 iterations, the mask is written into the propagation invalid mask. When multiple pyramid level estimates exist for the same pixel, the estimate with the smallest closed-loop error is selected first. If the closed-loop errors are the same, the level with the highest spatial resolution is selected from high to low. If they are still the same, the estimate with the highest iteration number is selected. The effective mask and forward and backward background restoration are then propagated. Figure 1 It then enters the three-category region determination; in the verification segment where an object is placed during the period of personnel occlusion, when the difference between the forward background recovery and the backward background recovery is 0.05, the difference between continuous observation after exiting and the background recovery is 0.22, and the illumination difference is 0.19, the conditions that the recovery difference does not exceed 0.08, the observation difference exceeds 0.10 and the illumination difference exceeds 0.08 are met in sequence, and the output continuously occupies the residual area instead of the recoverable area;

[0082] After exiting, continuous observations are generated from candidate frames after the stability conditions are met for the first time. The module first sorts the frames by timestamp from early to late and deletes retransmission frames, personnel re-entry frames, unknown illumination frames, and configuration invalid frames, and then selects 12 frames continuously. If there are fewer than 12 valid candidates, the observation difference is not calculated and the area to be restored is set as an unknown occlusion area. When there are 12 valid candidates, the median of the three channels is calculated by pixel in the current original observation domain without photometric correction. Then, the median observation is positively transformed to the corrected background prototype domain using the S1 partition mapping that is accepted by the 12 frames. If the mapping version between the 12 frames is not unique or a certain pixel has no valid original value in more than 3 frames, only the corresponding pixel is set as an unknown occlusion area, and the remaining pixels continue to participate in the restoration classification.

[0083] Illumination transformation prediction only uses the background prototype currently accepted by S1 and the local brightness mapping record of the same version as the keyframe after exit. The stored mapping direction is fixed from the current original observation domain to the corrected background prototype domain. The module reads the gain and bias according to the partition key and deletes records with inconsistent version summaries. When the partition key does not match, the gain or bias exceeds the limit range, the gain is zero, or the forward or reverse operation results in a non-finite number, the illumination difference judgment of that partition is directly written as uninterpretable. Only when the record is valid will the corrected continuous observation map and the current original observation domain restored background prediction map be generated respectively. When the observation and prediction of each domain are valid, the corresponding threshold comparison is performed to avoid the mapping direction, radiation value range or parameter missing being misjudged as a recoverable area.

[0084] After classification, the module processes persistent residual candidates in descending order of connected component pixel area. If the areas are the same, they are processed in ascending order of centroid ordinate. If they are still the same, they are processed in ascending order of centroid abscissa. S2 only generates temporary candidate identifiers and candidate features for the current frame based on this. It does not perform one-to-one inheritance, splitting, or merging with historical residuals. All cross-frame persistent residual identifiers are determined by the unique versioned association table of S5. After S5 completes calibration and submits the version of the association table for this frame, it returns the final residual identifier, parent-child relationship, and corresponding mask to the scheduler. S3 and S4 are triggered sequentially with the same version. Temporary candidate identifiers are prohibited from being directly written into the pass status, recognition status, or event evidence chain.

[0085] S2 ultimately outputs a set of persistent occupancy residual candidate records. Each record includes a temporary candidate identifier, persistent occupancy residual mask, confirmation time, keyframe before personnel entry, keyframe after personnel exit, recovery difference map, observation difference map, illumination difference map, coverage partition, cumulative permission bit, and background write freeze bit. The record is written to the candidate state cache according to the current frame timestamp and handed over to S5 for initial calibration. After S5 completes the association calibration, it merges all active persistent residual masks of the same channel according to the channel number, frame timestamp, and association table version to form a unique channel frame composite mask and retains the persistent residual identifier to the reverse index of the constituent area. S3 ground projection and S4 identification line-of-sight determination read the complete composite record only once per frame and outputs the channel-level functional indicators and the intersection attribution of each persistent residual.

[0086] In this specific embodiment, S3 includes:

[0087] The passage function determination module reads the persistent residual identifier, the currently continuously occupied residual mask or the candidate empty mask that has disappeared, the depth frame synchronized with the color frame hardware, the rigid transformation of the depth unit, and the ground calibration points, passage boundaries, entrance boundaries, and exit boundaries in the S1 configuration library from the output of the unique versioned association table of S5. In this specific embodiment, no less than six non-collinear calibration points are set up on the ground of the fire passage. The metric ground coordinates of each calibration point are obtained using a total station or a calibrated laser rangefinder, and the corresponding pixel coordinates are marked in the clear image. The module uses a random sampling consistency algorithm to estimate the homography matrix from the image plane to the ground plane. It is required to verify that the root mean square reprojection error of the calibration points does not exceed 2.0 pixels, the upper limit of the depth ranging error does not exceed 0.03m, and the upper limit of the converted ground positioning error does not exceed 0.03m. If any limit is exceeded, the function determination of the current passage is frozen and a recalibration record is generated.

[0088] The module backprojects pixels with a depth confidence of at least 0.80 within the residual mask according to camera intrinsic parameters, obtains a world coordinate point cloud through rigid transformation of depth units, and orthogonally projects each effective 3D point onto the ground plane along the world coordinate gravity direction, using the 0.05m concave hull of the projected point as the original ground polygon. If the effective pixel ratio of the point cloud is less than 90% of the mask pixels, there are fewer than 30 effective points, the lowest effective point is more than 0.05m above the ground resulting in a suspended state, or the concave hull self-intersects or exceeds the calibrated effective polygon, the entire 2D contour is not used as a ground substitution projection. The passage status is recorded as projection uncertainty, the event and background are frozen, and a depth or ground contour is generated for manual verification. For ground contact boundary points whose absolute height above the ground is confirmed to be no more than 0.03m, their homogeneous image coordinates are used... still in accordance with Calculate the ground verification coordinates, where, This represents a three-dimensional homogeneous image coordinate system consisting of pixel x-coordinate, pixel y-coordinate, and a constant 1. This represents a three-dimensional homogeneous ground coordinate system consisting of the ground x-coordinate, the ground y-coordinate, and a constant 1. This represents the 3×3 homography matrix obtained by fitting ground calibration points. Indicates a non-zero homogeneous scale factor; when the homography check coordinates differ from the vertical projection coordinates of the point cloud by more than 0.05m or cannot be homogeneously normalized, it also enters the projection uncertainty branch. Only contact points located on the ground plane are confirmed to use the homography matrix, and the pixels on the side or top of the object are not directly mapped to ground points.

[0089] The module adds the upper bounds of the ground calibration error (0.03m), depth ranging error (0.03m), and point cloud voxel and concave hull discretization error (0.02m) to obtain the upper bound of the total ground projection error for this cycle, and performs the following operations on the original site polygon based on this upper bound. ,in, This represents the ground residual region after expansion according to the total error band. This represents any point on the ground plane with coordinates in meters. Point To the original ground area The minimum Euclidean distance, in meters. This represents the original occupied area formed by projecting the residual 3D point cloud along the direction of gravity. The upper limit of the total ground projection error is represented, and in this specific embodiment, it is taken as 0.08m; the circular structural element is expanded and intersected with the boundary of the fire lane to ensure that the error zone only forms a conservatively occupied area within the effective surface of the lane;

[0090] The module sets slice centers at 0.10m intervals along the centerline of the passage from the entrance to the exit, and forms longitudinal slices using intercepts perpendicular to the local tangent of the centerline and the mid-boundary lines of adjacent center positions. Each slice records the slice number, longitudinal mileage, left and right passage boundaries, effective polygon, and calibration version. The module deducts the expanded ground residual area from the effective polygon, decomposes the remaining area into walkable units according to the four-connectivity relationship, and writes the unit identifier, the slice to which it belongs, the left and right boundary points, the lateral width, the adjacent candidate, and the encroachment residual identifier for each unit. Narrow fragments with a width of less than 0.05m or isolated fragments with an area of ​​less than 0.01 square meters are deleted, and the reason for deletion is saved in the slice status record.

[0091] Undirected adjacent edges are established when the actual passable contact width of walkable cells on the common slice boundary of adjacent slices is not less than 0.03m. The length of the remaining contact segment after deducting the encroached area and the boundary error zone is defined as the graph edge capacity. ;in, This represents the index of the adjacent edge in the path. Indicates the first The actual walkable contact width of each adjacent edge is measured in meters. The graph edge capacity from the entrance unit to the virtual entrance node and from the exit unit to the virtual exit node are respectively taken as the actual walkable intersection length between the unit and the entrance boundary or exit boundary. Adjacent edges record the unit identifiers at both ends, contact segments, graph edge capacity, and associated residual identifiers. A ground access map is formed by all walkable units and adjacent edges. The module performs a maximum capacity path label search on this map, initializes the capacity label of the virtual entrance node to positive infinity, and initializes the other nodes to 0. Each time, it takes the undetermined node with the largest capacity label from the maximum heap, and uses the minimum value among the current path capacity label, the horizontal width of the target adjacent unit, and the current graph edge capacity as the value of its adjacent units. Candidate capacity is determined. If the candidate capacity is greater than the original label, the label and predecessor are updated. If the candidate capacities are the same, the predecessor ranked first in terms of cumulative turning times is prioritized. If they are still the same, the path ranked first in lexicographical order of the complete sequence composed of the unit identifier and graph edge identifier is used to obtain a unique predecessor chain. When the virtual exit node is determined, its capacity label is the maximum bottleneck value that considers both the unit width and the connection width in all inlet-outlet paths. The module writes the representative path and the bottleneck unit or bottleneck connection that reaches this value along the predecessor chain, without enumerating all simple paths. When the virtual exit node is still 0 and unreachable, the connectivity is written as disconnected, the representative path is written as empty, and the remaining net width of the bottleneck is written as 0m.

[0092] When an inlet-outlet path exists, the module follows... Calculate the remaining net width of the bottleneck, where, This indicates the remaining net width of the bottleneck in the passage, expressed in meters. This represents the set of all entrance-exit candidate paths on the ground access map. This represents an entry-exit path in the candidate path set. Representing a path Index of walkable cells traversed. Indicates the first The lateral width of each walkable unit within its corresponding slice, measured in meters. Representing a path The graph edge indexes that have been passed through. Indicates the first The actual passable contact width of the strip is shown in meters; the inner two... Obtain the lower bounds of the width of all elements and the width of all connections along the path, respectively. Take the minimum of the two. The supremacy of the bottleneck values ​​of all candidate paths is taken; the exit capacity label of the maximum capacity path number search is equal to the supremacy of this formula, and the unique predecessor chain saved when the label is updated is used as the representative path; if multiple bottleneck units or bottleneck connection widths are the same, the one that appears first in the representative path mileage order is taken first, and if they are still the same, the unique bottleneck object is determined by the lexicographical order of the unit identifier or the graph edge identifier.

[0093] The passage function determination module outputs passage status records. The record fields include residual identifier set, ground projection area, error expansion area, encroached walkable unit, ground passage map version, connectivity status, representative entrance-exit path, bottleneck unit, and bottleneck remaining net width. The encroached walkable unit is determined by the intersection of the expansion area and the original net walkable unit. When the same unit intersects with multiple residuals, all residual identifiers are retained. The record is written to the passage status cache according to the current frame timestamp, which is used by S4 to establish the same source association and by S5 to perform the specified net width threshold comparison, event tag writing, and interval uncertainty verification.

[0094] In this specific embodiment, S4 includes:

[0095] The identification and judgment module reads the persistent residual identifier, the currently occupied residual mask or the candidate empty mask for disappearance output from the unique versioned association table of S5, and the trespassed walkable unit output from S3. It also reads the camera's internal and external parameters, the rigid transformation between the depth unit and the color camera, the preset observation position zone, and the effective graphic area of ​​the evacuation marker from the calibration configuration library. In this specific embodiment, the fixed network camera is configured with a hardware-synchronized depth unit, where the depth frame and color video share a timestamp, and the effective depth range is 0.3m to 20m. The preset observation position zone is along the center of the fire escape route. The line is established at a viewpoint height of 1.60m above the ground. Observation positions are generated at intervals of 0.25m. After removing the borders from the effective graphic area of ​​the evacuation marker, graphic sampling points are generated at intervals of 8 pixels. Each observation position and graphic sampling point saves its three-dimensional coordinates in the world coordinate system, sequence number, channel number, and calibration version. The camera's intrinsic and extrinsic parameters are obtained by chessboard intrinsic parameter calibration and ground control point extrinsic parameter calibration. The rigid transformation of the depth unit is fitted by a common calibration plate. The visual recognition judgment for this cycle is stopped when the reprojection error of any verification point exceeds 2 pixels or the three-dimensional ranging error exceeds 0.03m.

[0096] The module establishes three-dimensional closed line segments between the same observation position and all graphic sampling points as line edges in a unified world coordinate system. At the same time, it uses the camera projection matrix to project the endpoints onto the current frame image coordinate domain to establish a two-dimensional coarse screening index. The node records that identify the line image include the observation position number, the longitudinal mileage of the observation position, the graphic sampling point number, the three-dimensional coordinates and pixel coordinates. The edge records include the line edge identifier, the two end nodes, the three-dimensional line segment, the two-dimensional coarse screening pixel sequence, the occlusion status, the three-dimensional occlusion section, the closest distance and the associated residual identifier. When the projected point falls outside the image or is occluded by a static wall, the graphic sampling point is marked as invalid during the calibration stage and is not counted in the total number of line edges. If the effective line edges of an observation position are less than 80% of the total number of effective graphic sampling points, the position is marked as an invalid calibration position and its continuous segment is interrupted.

[0097] The module first uses a persistent residual 2D mask to filter potentially intersecting line-of-sight edges. Then, it back-projects pixels with a depth confidence of at least 0.80 within the mask according to camera intrinsic parameters and performs a rigid transformation to obtain a residual 3D point cloud. It downsamples at 0.02m voxels, vertically fills the area between the ground contact point and the highest effective point, and expands outwards according to a 0.03m ground positioning error to form a conservative 3D envelope. When the effective depth pixels are less than 90% of the residual mask pixels, no occlusion conclusion is output, and the image remains frozen, generating a depth loss record for manual verification. For each effective line-of-sight edge that passes the coarse screening... The module samples 3D line segments in world coordinates at 0.02m intervals. When the nearest distance from a continuous sampling segment of at least 0.05m to any residual 3D envelope does not exceed 0.03m, and the envelope is located between the observation position and the marked sampling point, the edge is marked as an occluded line edge; otherwise, it is marked as a visible line edge. The 0.05m and 0.03m intervals are verified by occlusion plates with known thickness, height, and distance. When intersecting with multiple residual envelopes, each 3D occlusion segment and its corresponding residual label are retained. Two-dimensional intersections are only used to reduce the amount of 3D calculation and are not used as the final occlusion basis.

[0098] For the Each observation position, the module is arranged according to Calculate the visibility at a single location, where, Indicates the first The dimensionless single-position visibility at each observation location, with a value range of [value missing]. , This represents the index of observation positions sorted by the direction of extension of the observation position. Indicates the first The number of visible effective line-of-sight edges marked at each observation position. This indicates the total number of valid line-of-sight edges at that location. In this specific implementation, the single-location visibility threshold is set to 0.85. When the visibility of a single location reaches or exceeds this threshold, the location status is written as visible; otherwise, it is written as invisible. When the value is 0, this position is written as invalid and not incorporated into any continuous visible segment;

[0099] Before forming an observation position, the module first verifies the start and end coordinates, channel number, and total length of the preset observation position band. If the observation position band is missing, the start and end coordinates are invalid, or the total length is not greater than 0m, the current cycle's visual configuration is written as invalid, no observation position is generated, the continuous visibility ratio is not calculated, the event and background are frozen, and a configuration record awaiting manual review is generated. Only when the total length of the observation position band is greater than 0m, the module scans the position status along its mileage sequence, merging observation positions that are visible and whose adjacent mileage difference does not exceed 0.30m into a continuous visible segment. The endpoint of the segment is taken as the projected mileage of the start and end observation positions on the observation position band. When at least one continuous visible segment exists, the module follows the... Calculate the continuous viewing scale, where, The dimensionless continuous visible scale representing the evacuation signage, with a value range of [value missing]. , This represents the set of all continuously visible segments in the current period. Indicates one of the continuous visible segments. Indicates a section The length of the observation zone, in meters. This indicates the total length of the preset observation position band that has been verified to be greater than 0m, and the unit is meters. This represents the supremum of the ratio of the lengths of all continuous visible segments; when there are no continuous visible segments, the continuous visible ratio is set to 0; when multiple segments have the same length, the segment with the first starting mileage sorted from low to high is selected as the representative segment.

[0100] Before continuously occupying the residual mask and depth frame and entering the view coordinate domain, the module checks the frame size, camera number, hardware timestamp, color camera calibration version, depth calibration version, and rigid transformation version, and transforms the mask boundary according to the same distortion correction mapping table. If the time difference between the color frame and the depth frame exceeds 40ms, the time difference between any frame and the residual record exceeds 80ms, the size is inconsistent, the depth confidence is insufficient, or any calibration version is inconsistent, all view edges in this cycle are written to an invalid state, the event and background are frozen, and the process is transferred to manual review. When all checks pass, a 3D envelope is constructed and the 3D nearest distance is calculated. If the boundary distance is exactly equal to 0.03m, it is treated as occlusion.

[0101] The single-position visibility threshold and the three-dimensional line-of-sight occlusion threshold are established by the calibration and verification task using unobstructed segments, normal personnel passage segments, and occluded segments with known three-dimensional positions, thicknesses, and heights. The occluded segments cover 10%, 20%, and 30% of the effective width of the graphic, respectively, and also cover counterexamples located in front of the line of sight, behind the line of sight, and at different ground heights. The threshold record saves the depth confidence, minimum three-dimensional occlusion length, closest distance threshold, single-position visibility threshold, verification sample number, false positive rate, false negative rate, and version number. In this specific implementation, only threshold versions with an unobstructed false positive rate of no more than 1% and a 20% occluded segment detection rate of 95% are accepted. Recalibration is triggered when the camera or depth unit pose changes, the label is changed, or the false positive rate exceeds 1% for seven consecutive days. If the threshold version does not match, a definite continuous visibility ratio is not output.

[0102] The identification and judgment module summarizes all intersecting line edges according to the residual identifier, and writes the invaded walkable units with the same residual identifier in S3 into the same association record, forming a bidirectional index between the residual identifier, the invaded walkable unit, and the occluded line edge; when a line edge is jointly occluded by multiple residuals, the event layer retains all residual identifiers. Only when the edge still meets the occlusion condition after removing any single residual will the occlusion association other than that residual be maintained, so that when the residuals disappear, split, or merge, the line graph can be incrementally updated without resetting the state of other residuals;

[0103] S4 outputs a visual status record, with fields including identifier number, observation position with version, valid observation position, single position visibility, continuous visible segment, representative segment, continuous visibility ratio, obscured line edge, obscured segments on each edge, and associated residual identifier. The record is written to the visual status cache according to the current frame timestamp and aligned with the passage status record of S3 using channel number, residual identifier, and calibration version. This allows S5 to compare the visibility ratio threshold, generate obscuring events, establish evidence chains, and perform background update interlocking.

[0104] In this specific embodiment, S5 includes:

[0105] The closed-loop gating and event output module first reads the temporary candidate record of S2 according to the channel number and frame timestamp. The unique residual identifier management submodule maintains the versioned association table and completes the candidate calibration of splitting, merging, one-to-one inheritance, creation and disappearance in this frame. After the association table is atomically submitted, the module establishes a unique channel frame synthesis record according to the channel number, frame timestamp and association table version. It performs pixel-by-pixel aggregation on all active persistent residual masks of the same channel and records the composition mask, connected component index and the reverse index of persistent residual identifier returned by the synthesized pixel or geometric composition region, sorted by persistent residual identifier. The scheduler triggers one S3 passage determination and one S4 identifier recognition determination only with this complete synthesis record. S3 and S4 output the channel-level connectivity status, bottleneck remaining net width, continuous visibility ratio and the distance from the edge of the invaded unit and the occluded line of sight to each persistent residual identifier under the same synthesis version. Intersection attribution; the module then establishes event status records with persistent residual identifiers as the primary key. Each event record references the same channel frame synthesis version and saves the corresponding residual attribution. It is prohibited to repeatedly calculate and overwrite channel-level indicators for a single residual. The event status record includes the first confirmation time, the most recent confirmation time, the duration, the keyframe before personnel enter, the keyframe after personnel exit, the persistently occupied residual mask, the synthesis mask version, the intruded walkable unit, the obscured line edge, the connectivity status, the remaining net width of the bottleneck, the continuous visibility ratio, the functional loss type, the background frozen partition, the calibration version, the association table version, and the event status. The most recent confirmation time is written by overwriting the timestamp of the current valid observation frame, and the duration is obtained by subtracting the first confirmation time from the most recent confirmation time. If the calibration version, association table version, or synthesis mask version of any input record is inconsistent, it remains frozen and generates a record for review due to inconsistent versions.

[0106] When the temporary candidate set, active persistent residual set, and active event tombstone are all empty and the S1 input is valid, the closed-loop gating and event output module enters the channel-level normal clearance branch. It generates a versioned normal record with all zeros in the residual mask according to the channel number and the current complete association table version. It triggers S3 and S4 with this record instead of the persistent residual identifier to obtain the connectivity status, bottleneck remaining net width, and continuous visibility ratio of the current frame. The normal record does not establish events or write event evidence chains. It is consistent with the camera calibration version, depth calibration version, and association table version for 15 consecutive frames. Background write permission is only issued to the corresponding foregroundless partition when the connectivity status is connected, the remaining net width of the bottleneck reaches or exceeds 1.10m, the continuous visibility ratio reaches or exceeds 0.80, and there are no people covering, unknown lighting, invalid configuration, or pending manual review status in each frame; if any condition is not met, the normal continuous compliance count is reset to 0 and kept read-only or frozen; the priority of the active residual processing and active event tombstone clearing branches is higher than that of the normal net emptiness branch, and the reuse of normal records or normal compliance counts is prohibited as long as any of the three sets is not empty;

[0107] The module performs deterministic function loss determination: when the remaining net width of the bottleneck is less than the specified net width threshold of 1.10m, a net width bottleneck label is added; when the connectivity status is disconnected, a path disconnection label is added; when the continuous visibility ratio after personnel exit is less than the visibility ratio threshold of 0.80, an identification occlusion label is added; when an identification occlusion label has been added and either a path disconnection label or a net width bottleneck label has been added, a dual risk label is added while retaining the original label, and multiple labels are stored side by side without overwriting each other; if only an identification occlusion label exists, an occlusion event is output; if a path disconnection label or a net width bottleneck label exists, an occupancy event is output; if dual risk labels exist, two types of function loss fields are carried in the same event.

[0108] The event evidence chain uses residual identifiers and event versions as joint keys. It appends keyframes before personnel entry, keyframes after personnel exit, persistent occupancy residual masks, recovery difference maps, observation difference maps, illumination difference maps, intruded walkable units, obscured line edges, and all connectivity states, bottleneck remaining net width, and continuous visibility ratio to an overwhelmable evidence entry in chronological order. Each evidence entry saves the summary value of the previous entry, the summary value of the current entry, and the acquisition timestamp. Keyframes and masks are written to read-only object storage at their original resolution, while graph records and numerical records are written to the event database. This allows any alarm to be traced back to the same source data for personnel entry, exit, recovery confirmation, and the two types of fire protection function judgments through residual identifiers.

[0109] In subsequent frames, the module computes between the persistent residual set of the previous frame and the temporary candidate set of the current frame. ,in, Indicates the residual of the previous frame With current frame candidate The dimensionless correlation cost between them, and the smaller the value, the more likely the same residual identifier should be maintained. This represents the residual index of the previous frame. Indicates the candidate index of the current frame. Indicates the residual region of the previous frame. With the current frame candidate region The intersection-union ratio and its range is , The dimensionless appearance distance is obtained by truncating the cosine distance of the concatenated features of two residual color histograms and edge direction histograms, and its value range is [value missing]. The coefficient 0.7 directly represents the dimensionless weight of the spatial overlap cost term, and the coefficient 0.3 directly represents the dimensionless weight of the appearance distance term. Both weights are non-negative and their sum is 1. In this specific implementation, independent verification segments with manually labeled splitting, merging, and short-term occlusion relationships are used. With the goal of minimizing the residual identifier switching rate and ensuring that the error inheritance rate does not exceed 1%, the two are labeled as 0.7 and 0.3 on a 0.05 step grid, and the sample number, target value, and weight version are written into the associated configuration.

[0110] The association management submodule first constructs a bipartite relation graph using edges with an intersection-union ratio (IU) of 0.20: when a previous frame residual connects at least two current candidates, it is preferentially determined to be a split, generating child residual identifiers for each current candidate and recording the common parent residual identifier; when at least two previous frame residuals connect to the same current candidate, it is preferentially determined to be a merge, generating a merged residual identifier for that candidate and recording all parent residual identifiers; when the same connected component has multiple in-degrees and multiple out-degrees, a many-to-many ambiguity record is written, recording the previous frame identifier set, current candidate set, all IU ratios, association cost, frozen partitions, and pending status. These events remain frozen and are not automatically inherited until the reviewer selects the mapping, signs, and submits a new association table version; splits, merges, or ambiguities... The nodes involved in the definition are removed from the subsequent pairing set in this frame. The remaining nodes are matched one-to-one with minimum cost. When the association cost does not exceed 0.45, the old residual identifier is maintained. If the costs are the same, the node with the highest intersection-union ratio is selected. If they are still the same, the node with the earliest first occurrence time of the old residual is selected. The candidate pair, cost, sorting key, and output identifier are written into the association table. Thus, the splitting and merging determination takes precedence over one-to-one inheritance and the three are mutually exclusive. If the current candidate is not matched, a new persistent residual identifier is generated. When the mapping table version is submitted, the old identifier that has been split or merged is terminated. The child event inherits the first confirmation time of the parent event. The merged event inherits the earliest first confirmation time of all parent events. The old evidence chain remains read-only and is migrated to the new event with the parent-child digest link.

[0111] When a persistent residual from the previous frame does not match, the module immediately creates an active event tombstone for it. The tombstone retains the original residual identifier, the last valid mask, the first confirmation time, the most recent association table version, the disappearance count, and the composite mask verification version. For each subsequent missing frame, the residual corresponding to the tombstone is removed from the composite record of this frame's channel, while all other active persistent residuals are retained. When multiple tombstones exist simultaneously, their respective residuals are removed at once, and then a unique composite mask is formed again from the remaining active residual masks. Only when the remaining set is empty is a composite mask of the same size with all zeros used. After confirming that the camera, timestamp, calibration version, association table version, and composite mask version are valid, the scheduler triggers S3 and S4 only once for this complete record. Generate current-frame channel-level clearing candidate status, net width, and continuous visibility ratio records, and write the removed tombstone identifiers and the reverse indices of other still active residuals into the clearing candidate records. Reuse of the last frame's functional status when residuals still exist or repeated calculations for each tombstone are prohibited. Each tombstone only removes its own last valid mask coverage partition, while other active residuals still participate in channel-level indicators and remain frozen. If any frame input is invalid, version is inconsistent, synthesis record is incomplete, or manual review is required, the continuous valid disappearance count of the relevant tombstone is reset to 0 and the background is frozen. After obtaining valid clearing candidate records under the same version chain for 15 consecutive frames, the tombstone remains until the closed-loop interlock is completed before being converted to the released state.

[0112] To handle the synthesis errors of ground calibration, depth ranging, and point cloud discretization, the module constructs a coordinate interval for each ground boundary point by subtracting and adding the upper limit of the total ground projection error of 0.08m determined in S3 to its x and y coordinates. While maintaining the original boundary point order and polygon topology, the coordinate interval is propagated to the boundary of the encroached unit and the ground boundary of the continuously occupied residual point cloud. For each set of interval boundaries, the conservative inner envelope and the loose outer envelope of the remaining area of ​​the fire lane are calculated respectively. The slicing, adjacency, and maximum capacity inlet-exit search of S3 are repeated on the two envelopes.

[0113] The modules form the bottleneck remaining net width interval according to the following relationship;

[0114] in, This indicates the lower limit of the remaining net width of the bottleneck, in meters. This indicates the upper limit of the remaining net width of the bottleneck, in meters. This represents the set of entrance-exit paths in a conservative inner envelope ground traffic map. This represents the set of entrance-exit paths in a loosely enveloped ground access map. This represents a path within the corresponding set. Indicates the index of the walkable cell traversed by the path. This indicates the graph edge index traversed by the path. and These represent the first conserved inner envelope and the second relaxed outer envelope, respectively. The horizontal width of each unit, in meters. and They represent the first The actual passable contact width of each connection port in the conservative inner envelope and the loose outer envelope, in meters; within each envelope, the minimum value of the lower bound of the path unit width and the lower bound of the connection port width is first taken, and then the upper bound of the corresponding path set is taken; before executing the formula, the path set is checked. When the conservative path set is empty, the lower bound of the net width is directly set to 0m. When the loose path set is empty, both the upper bound and the lower bound of the net width are set to 0m and no entrance-exit path is recorded. Only when the corresponding path set is not empty is the upper bound operation in the formula performed on the set.

[0115] The module uses the complete 3D occlusion conditions of S4 in the visual interval branch, without using 2D pixel intersection to replace the final occlusion: ground positioning error, depth ranging error, color camera extrinsic parameter error, depth unit rigid transformation error, and 0.02m voxel discretization error are all propagated to the residual 3D point cloud. The outer boundary of the union of each error interval of the point coordinates forms a conservative 3D outer envelope, and the inner boundary of the intersection of each error interval forms a loose 3D inner envelope. For each world coordinate 3D line of sight, the conservative branch checks the outer envelope item by item. If the envelope is located between the observation position and the marked sampling point, the continuous occlusion length is not less than 0.05m, and the closest distance is not more than 0.03m, it is considered occluded. The loose branch is only considered occluded if all allowed realizations of the inner envelope meet the same position order, continuous length, and closest distance conditions. The two sets of edge states are calculated according to the single position visibility determination and continuous segment merging rules of S4, respectively. ,in, It represents the lower bound of the continuous visibility scale under the conservative three-dimensional envelope occlusion case and is a dimensionless quantity. This represents the upper bound of the continuous visibility ratio under the loose 3D inner envelope occlusion condition and is a dimensionless quantity; the module simultaneously calculates the nominal continuous visibility ratio by repeating S4 with the nominal calibration parameters, and verifies that the lower bound of the continuous visibility ratio is not greater than the nominal value and the nominal value is not greater than the upper bound of the continuous visibility ratio. If any 3D envelope cannot be formed, the parameter interval is empty, or the order relationship verification fails, the interval is written as invalid, the event and background are frozen, and a 3D interval record is generated for manual review; the 3D occlusion segment, the nearest distance range, the parameter error version, and the interval result are all written into the evidence chain;

[0116] When the lower bound of the net width is less than 1.10m while the upper bound of the net width reaches or exceeds 1.10m, or when the lower bound of the continuous visibility ratio is less than 0.80 while the upper bound of the continuous visibility ratio reaches or exceeds 0.80, the module writes the judgment status as "cross-threshold uncertain," keeps the background of the corresponding area frozen, and generates a record pending manual review. When the upper bound of the net width is still less than 1.10m, the net width bottleneck label is maintained; when the upper bound of the continuous visibility ratio is still less than 0.80, the sign occlusion label is maintained. Only when the ground access map with a loose outer envelope still does not have an entrance-exit path is the status maintained. The path disconnection label; when the conservative inner envelope does not have an inlet-outlet path but the loose outer envelope still has an inlet-outlet path, the connectivity state is written as interval uncertainty and enters manual review, and the deterministic path disconnection label is not maintained or a new one is added. The connectivity interval uncertainty state and the net width or visible ratio threshold crossing state are merged into the same review state and are given priority over the deterministic functional loss label output; the manual review record includes upper and lower bounds, thresholds, calibration error version, interval boundaries, intersection segment union and before and after comparison keyframes, and the interval midpoint is not used to replace the conservative judgment;

[0117] The closed-loop write gate reads the candidate records for clearing generated by the original residual identifier under the tombstone of the active event frame by frame. Only when all 15 consecutive missing frames are recorded as empty residual masks, the connectivity state is connected, the lower limit of the net width reaches or exceeds the specified net width threshold, the lower limit of the continuous visibility ratio reaches or exceeds the visibility ratio threshold, and the camera calibration version, depth calibration version, and association table version are consistent within the 15 frames, the event status record is set to the deactivated state and the corresponding partition is allowed to receive the current frame; when any frame is missing the current record of S3 or S4, an invalid input occurs, the version changes, the indicator does not meet the standard, or the manual review mark is used, the continuous valid disappearance count is reset to 0 instead of using the historical compliant value;

[0118] Press after allowing write Update the background prototype that matches the lighting conditions, where, Indicates the first A background prototype in pixels The updated background brightness value is displayed in grayscale levels. This represents the background brightness value before the update, expressed in grayscale levels. Indicates time The current pixel brightness value after S1 photometric correction, in grayscale levels. This represents the dimensionless update coefficient, which is set to 0.01 in this specific embodiment. This indicates that the background prototype index has been matched. This indicates the pixel coordinates that can be written to the partition; the corresponding area will remain frozen when there is a persistent residual, any functional indicator fails to meet the standard, the interval crosses the threshold, there is unknown lighting, personnel coverage, or the status is pending manual review.

[0119] After receiving the passage status record and the recognition status record, the module first removes completely duplicate records according to the channel number, frame timestamp, association table version, and composite mask version. Then, it deletes records that fail the digest verification or have inconsistent composite mask digests. Records that are missing bottleneck remaining net width, continuous visibility ratio, inverted index, or keyframe index are sent to the review queue. Only one set of channel-level numerical fields is accepted for the same channel frame. The invaded unit and the obscured line edge are attributed to all intersecting persistent residual identifiers according to the inverted index of the same composite record. The event record references the channel-level state without copying or overwriting the values. The tombstone removal record must specify the residual identifiers that have been removed in this frame, the set of still active residuals, and the composite mask digest. Only the state snapshot that has been cleaned and is consistent with the three is used for label comparison, evidence chain addition, tombstone removal, and background interlocking determination.

[0120] Each time an evidence entry is added, the module updates the event location, duration, and functional loss type along with the evidence chain: the event location is jointly covered by the latest residual mask centroid, the longitudinal mileage range of the invaded unit, and the corresponding identifier number of the obscured line edge; the duration is recalculated by the difference between the first confirmation time of the evidence chain and the latest valid entry time; and the functional loss type is obtained by re-executing the label rules based on the latest connectivity status, the remaining net width of the bottleneck, and the continuous visibility ratio. If a new entry is not written into the evidence chain due to verification failure, the event location, duration, and functional loss type of the previous valid entry are maintained, and the alarm status is not changed using data not included in the chain.

[0121] S5 writes occupancy events, occupancy events, dual risk labels, records awaiting manual review, and background gating results into the event output queue. The output message includes the channel number, event number, residual identifier and parent-child relationship, first confirmation time, duration, functional loss type, net width and its range, continuous visibility ratio and its range, evidence chain address, and background frozen partition. After the event is resolved, the complete evidence chain and the resolution time are still retained. The background prototype update only applies to partitions that have passed all interlocking conditions, thus completing the closed loop from personnel occupancy recovery, continuous occupancy residual confirmation, dual function determination of passage and labeling to event evidence and background update.

[0122] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0123] The multi-light background prototype matching, personnel occlusion freezing, bidirectional temporal recovery, and continuous occupancy residual confirmation adopted in this invention are applied sequentially to the same continuous video processing chain, so that normal personnel passage and appearance changes that can be explained by lighting changes do not directly enter the occupancy event, and continuous areas that cannot be explained by personnel exit recovery or lighting model are entered into subsequent fire protection function judgment.

[0124] This invention connects the ground access map and the evacuation sign line of sight with residual markers, and feeds back the compliance results of the two maps as background writing and freeze release conditions, changing the processing structure that only updates the background or outputs alarms based on the appearance of a single frame; when the positioning error causes the net width or visible ratio range to cross the threshold, the background is kept frozen and transferred to manual review, thereby maintaining the consistency of functional judgment and evidence chain.

Claims

1. A visual inspection method for fire lane obstruction, characterized in that, include: S1. Collect continuous video of the ground area of ​​the fire lane and the evacuation sign area. Select the background prototype that matches the current frame from the multi-light background prototype library that is written to the gate in the closed loop. Perform photometric correction, foreground detection and personnel tracking on the current frame to obtain the time sequence of personnel occlusion area and non-personnel foreground. S2. During the period of personnel occupancy, freeze the cumulative occupancy and background update of the area, and after the personnel leave, perform bidirectional temporal recovery based on the keyframes before the personnel enter and after the personnel leave. Non-personnel foreground that is consistent in bidirectional recovery, inconsistent with continuous observation after the personnel leave, and cannot be explained by changes in illumination are identified as persistent occupancy residuals and assigned residual labels. S3. Project the persistent occupancy residual onto the walkable units divided along the longitudinal direction of the fire lane to determine the connectivity from the entrance to the exit and the remaining net width of the bottleneck. S4. Establish a line of sight from the preset observation position to the effective graphic area of ​​the evacuation sign, update the occlusion state of the line of sight with the spatial projection of the continuous occupation residual, determine the continuous visibility ratio of the evacuation sign, and associate the occupied walkable unit and the occluded line of sight with the residual sign. S5. Output an occupancy event or a marker occupancy event when the connectivity is disconnected, the remaining net width of the bottleneck does not reach the specified net width threshold, or the continuous visibility ratio after personnel exit does not reach the visibility ratio threshold; only update the multi-light background prototype library when there is no continuous occupancy residual and the connectivity, the remaining net width of the bottleneck, and the continuous visibility ratio all meet the standards; otherwise, keep the corresponding area frozen.

2. The visual inspection method for fire lane occupancy according to claim 1, characterized in that, Step S1 further includes reading channel geometric calibration data and determining thresholds. The channel geometric calibration data includes ground calibration points, entrance boundaries, exit boundaries, effective graphic areas of evacuation signs, and preset observation position zones. The determining thresholds include a specified net width threshold and a visibility ratio threshold. Illumination descriptors are extracted from frames with different illumination states where the clearance has been confirmed, and background prototypes are established. The background prototypes are selected based on the distance between the illumination descriptors of the current frame and the illumination descriptors of each background prototype. The current frame is partitioned and normalized according to the local brightness mapping relationship corresponding to the selected background prototype. The normalized current frame is then differentiated from the selected background prototypes and combined with the target trajectory category to output the temporal sequence of the personnel occlusion area and the non-personnel foreground.

3. The visual inspection method for fire lane occupancy according to claim 1, characterized in that, S2 includes: Save the keyframe before the personnel target trajectory enters the ground area of ​​the fire lane, and mark the area covered by the personnel target trajectory as the personnel occlusion area; When the personnel target trajectory exits and the inter-frame change amount continuously fails to reach a stable threshold and reaches a preset number of frames, the exit key frame is saved; taking the entry key frame and exit key frame as the starting point, pixels or features are propagated to the personnel during the occlusion period using the forward motion field and the backward motion field, respectively, to obtain the forward recovery result and the backward recovery result. The recovery difference between the forward recovery result and the backward recovery result is calculated. In the region where the recovery difference does not exceed the recovery consistency threshold, the arithmetic mean of the corresponding pixel values ​​or feature vectors of the forward recovery result and the backward recovery result is taken to obtain the fused recovery result. The observation difference between the fused recovery result and the continuous observation after exiting the observation, as well as the illumination difference between the fused recovery result and the illumination change prediction are calculated. Regions where the recovery difference does not exceed the recovery consistency threshold and the observation difference does not exceed the observation consistency threshold, as well as regions where the recovery difference does not exceed the recovery consistency threshold, the observation difference exceeds the observation consistency threshold, and the illumination difference does not exceed the illumination interpretation threshold, are all classified as recoverable regions. Regions where the recovery difference does not exceed the recovery consistency threshold, the observation difference exceeds the observation consistency threshold, and the illumination difference exceeds the illumination interpretation threshold are classified as persistent occupancy residuals. The remaining regions are classified as unknown occupancy areas. The occupancy accumulation freeze is lifted for recoverable regions, and the background write status is determined by closed-loop write gating. The occupancy accumulation freeze and the accumulated event duration are lifted for persistent occupancy residuals, while the background write freeze is maintained. The occupancy accumulation freeze and background write freeze are maintained for unknown occupancy areas.

4. The visual inspection method for fire lane occupancy according to claim 1, characterized in that, S3 includes: establishing a projection relationship between image coordinates and fire lane ground coordinates using ground calibration points; expanding the projection area of ​​the continuous occupancy residual in the ground coordinates according to the calibration error band; dividing the ground area into slices according to the longitudinal position of the passage, and taking the area in each slice that is not occupied by the expanded projection area and is located within the passage boundary as a walkable unit; connecting the walkable units in adjacent slices to form a ground passage map, and determining the entrance-exit path from the corresponding node of the entrance boundary to the corresponding node of the exit boundary in the ground passage map; for each entrance-exit path, obtaining the lateral width of the walkable units that the path passes through in each slice to form a path width set, taking the lower bound of the path width set as the path bottleneck net width of the corresponding entrance-exit path, and then taking the path bottleneck net widths of each entrance-exit path to form a path bottleneck set, taking the upper bound of the path bottleneck set as the remaining net width of the bottleneck; setting the connection state to connected when there is an entrance-exit path, and setting the connection state to disconnected and the remaining net width of the bottleneck to zero when there is no entrance-exit path.

5. The visual inspection method for fire lane occupancy according to claim 1, characterized in that, S4 includes: using camera calibration parameters to transform the observation positions selected at preset intervals within the preset observation position band and the graphic sampling points within the effective graphic area of ​​the evacuation marker to the current frame image coordinate domain; using the line segments between each observation position and each graphic sampling point as line edges to form a marker line image; and using the persistent occupancy residual mask in the current frame as the spatial projection in the line coordinate domain; marking the line edges intersecting with the persistent occupancy residual mask as occluded line edges, and marking the non-intersecting line edges as visible line edges; merging adjacent observation positions where the number of visible line edges reaches or exceeds the single-position visibility threshold along the extension direction of the preset observation position band into continuous visible segments; when continuous visible segments exist, using the ratio of the supremum of the set of lengths of each continuous visible segment to the length of the preset observation position band as the continuous visibility ratio; and when no continuous visible segments exist, setting the continuous visibility ratio to zero.

6. The visual inspection method for fire lane occupancy according to claim 1, characterized in that, S5 includes: establishing an event state record indexed by residual identifiers, the event state record including first confirmation time, most recent confirmation time, duration, intruded unit, obscured line edge, connectivity status, bottleneck remaining net width, continuous visibility ratio, and before-and-after comparison keyframes; the event state record allows parallel recording of functional loss types, adding a net width bottleneck label when the bottleneck remaining net width does not reach a specified net width threshold, adding a path disconnection label when the connectivity status is disconnected, adding an identification occlusion label when the continuous visibility ratio does not reach a visibility ratio threshold, and adding a double risk label while retaining the existing labels when the identification occlusion label has been added and either the path disconnection label or the net width bottleneck label has been added; setting the closed-loop write gate to background update interlock, only when the continuous occupancy residual corresponding to the event state record disappears, the connectivity status is connected, the bottleneck remaining net width reaches or exceeds a specified net width threshold, and the continuous visibility ratio reaches or exceeds a visibility ratio threshold, setting the event state record to a deactivated state and allowing the background prototype of the corresponding illumination state to receive the current frame.

7. The visual inspection method for fire lane occupancy according to claim 3, characterized in that, A review waiting time is set for the unknown occupancy area. During the review waiting time, the cumulative occupancy freeze and background write freeze are maintained. Observation frames without personnel target trajectories are collected and bidirectional time-series recovery is repeated. When the repeated recovery meets the continuous occupancy residual condition, the unknown occupancy area is converted into a continuous occupancy residual, the cumulative occupancy freeze and cumulative event duration are released, and the background write freeze is maintained. When repeated recovery meets the conditions for recoverable area, the cumulative freeze of occupancy is lifted and the background write status is determined by closed-loop write gating; when the review waiting time expires and it is still an unknown occluded area, the cumulative freeze of occupancy and the background write freeze are maintained and a record to be manually reviewed is generated.

8. The visual inspection method for fire lane occupancy according to claim 6, characterized in that, The keyframes before personnel enter, the keyframes after personnel exit, the persistently occupied residual mask, the invaded unit, the edge of the obscured line of sight, the connectivity status of each time, the remaining net width of the bottleneck, and the continuous visibility ratio are written into the same evidence chain according to the residual identifier. In subsequent frames, the residual identifier is maintained according to the spatial overlap relationship of the residual area and the distance of the appearance feature. When the persistently occupied residual splits or merges, the parent residual identifier and the child residual identifier are recorded, and the event location, duration, and functional loss type are updated with the evidence chain.

9. The visual inspection method for fire lane occupancy according to claim 6, characterized in that, Based on the positioning error range of the ground calibration points, a coordinate interval is constructed to maintain the order of the boundary points. The coordinate interval is propagated to the boundary of the invaded unit and the boundary of the continuously occupied residual mask. For each line of sight, all intersecting segments formed by the intervalized continuously occupied residual mask and the line of sight are determined, and the union of all intersecting segments is used as the blocking range of the line of sight. Based on the intervalized boundary of the invaded unit and the blocking range of each line of sight, the lower and upper bounds of the remaining net width of the bottleneck and the lower and upper bounds of the continuous visibility ratio are calculated. When the lower bound of the remaining net width of the bottleneck does not reach the specified net width threshold and its upper bound reaches or exceeds the specified net width threshold, or the lower bound of the continuous visibility ratio does not reach the visibility ratio threshold and its upper bound reaches or exceeds the visibility ratio threshold, the background writing is frozen and a record awaiting manual review is generated. When the upper bound of the remaining net width of the bottleneck does not reach the specified net width threshold or the upper bound of the continuous visibility ratio does not reach the visibility ratio threshold, the corresponding alarm state is maintained.

10. A visual inspection system for fire lane occupancy, used to perform the visual inspection method for fire lane occupancy as described in any one of claims 1 to 9, characterized in that, include: The video and background processing module is used to acquire continuous video, match multi-light background prototypes, and output the time sequence of the area occluded by people and the foreground of non-people. The residual confirmation module is used to freeze the cumulative occupancy and background update of the corresponding area during the period of personnel occupancy, and to perform bidirectional temporal recovery after personnel exit to determine the continuous occupancy residual with residual identifier; the passage function determination module is used to project the continuous occupancy residual onto the walkable unit and determine the connectivity status from the entrance to the exit and the remaining net width of the bottleneck; the identifier recognition determination module is used to update the occupancy status of the line of sight edge with the spatial projection of the continuous occupancy residual, determine the continuous visibility ratio, and associate the intruded walkable unit and the occupant line of sight edge with the residual identifier; the closed-loop gating and event output module is used to output occupancy event or occupancy marker event according to the connectivity status, the remaining net width of the bottleneck, and the continuous visibility ratio, and to control the update of the multi-light background prototype library and the freezing status of the corresponding area.

Images