A method, system, medium and product for monitoring the presence of personnel in a fire control monitoring room

Abstract

A fire control monitoring room personnel on-duty monitoring method, system, medium and product, relate to image recognition field.The method comprises: in response to detecting that the to-be-monitored object completely enters the preset entrance area, extracting the visual feature vector in the unobstructed image frame as the initial reference feature; when the to-be-monitored object moves to the on-duty monitoring area, based on the overlapping relationship of the real-time detection frame and the environmental occlusion mask, the occluded part is deducted, and the exposed area is extracted; the local matching degree of the exposed area and the initial reference feature is calculated to determine the personnel on-duty state, and the update reference feature is generated when the spatial overlap degree and the texture similarity condition are met.The present application uses the complete feature obtained at the entrance as the reference, and combines the local comparison of the exposed area, effectively solves the problem that the existing general target detection model cannot identify the incomplete human body target under the static facility occlusion environment, resulting in false alarm, and improves the accuracy of on-duty monitoring.

Description

Technical Field

[0001] This application relates to the field of image recognition, and in particular to a method, system, medium, and product for monitoring personnel on duty in a fire control room. Background Technology

[0002] As the hub of building safety, the fire control room must be staffed 24 / 7 according to regulations, and leaving the post is strictly prohibited. To ensure monitoring efficiency, the industry generally adopts indoor video surveillance combined with intelligent visual analysis technology to automatically monitor the on-duty status of personnel in real time, aiming to determine whether anyone is on duty indoors through video analysis.

[0003] To improve the automation of monitoring and reduce the cost of manual inspections, a relatively mature personnel presence monitoring solution employs a general target detection model based on deep learning (such as the classic YOLO or SSD series algorithms). This technology is trained on a large-scale general human dataset (mainly including standing, walking, or standard sitting posture samples), enabling the system to identify human targets with complete anatomical structures from the monitoring background. In actual operation, when a target matching the complete human contour features appears in the monitoring screen, the system outputs a personnel presence signal. Conversely, if no human target meeting the confidence requirements is detected within a preset time period, the system determines that no one is on duty and triggers an absence alarm.

[0004] However, because fire monitoring rooms are usually equipped with tall monitoring consoles or large cabinets, and for security reasons, surveillance cameras are mostly installed from a high position looking down, when on-duty personnel are in non-standard postures (such as being significantly obscured by the back of a high chair due to fatigue, bending over to perform equipment maintenance in the blind spot behind the control console, or having most of their body hidden in the corner of the equipment), the on-site facilities will cause serious physical obstruction of the human body, resulting in only partial features being revealed in the picture. Existing algorithms that rely on the complete outline of the human body are unable to identify such incomplete targets, thus misjudging the state of someone on duty but obscured as someone leaving their post, triggering frequent false alarms and reducing the accuracy of on-duty monitoring. Summary of the Invention

[0005] This application provides a method, system, medium, and product for monitoring personnel on duty in a fire control room, which is used to improve the accuracy of on-duty monitoring.

[0006] Firstly, this application provides a method for monitoring personnel on duty in a fire control room, applied to a server of a video surveillance system, comprising: responding to the detection that a monitored object has completely entered a preset entrance area, capturing an unobstructed image frame containing the monitored object, and extracting the visual feature vector from the unobstructed image frame as an initial reference feature; continuously tracking the coordinate position of the monitored object, and when the coordinate position moves from the entrance area to the preset on-duty monitoring area, obtaining an environmental occlusion mask pre-calibrated in the on-duty monitoring area; based on the overlap relationship between the real-time detection frame of the monitored object and the environmental occlusion mask, subtracting the occluded portion from the current video frame, and extracting the remaining image area as the exposed area; calculating the exposed area... The local matching degree between the visual feature vector of the region and the initial reference feature is calculated. When the local matching degree is greater than a preset discrimination threshold, it is determined that the monitored object is in the state of being on duty. After confirming that the personnel are on duty, the spatial overlap and texture similarity between the exposed area in the current video frame and the exposed area in the previous video frame are calculated in real time. When the spatial overlap is greater than a preset continuity threshold and the texture similarity is greater than a preset update safety threshold, the initial reference feature is weighted and fused based on the visual feature vector of the exposed area in the current video frame to generate updated reference features. The local matching degree of the monitored object in subsequent video frames is calculated based on the updated reference features to maintain the monitoring of the personnel's on-duty status.

[0007] By adopting the above technical solution, when the object to be monitored enters the entrance area, the server acquires an unobstructed image frame of the object and extracts initial reference features from it, avoiding occlusion interference from the source and providing a high-confidence comparison benchmark for subsequent local matching. When the object to be monitored moves to the on-duty monitoring area, the server accurately identifies fixed obstructions (such as desks, equipment, green plants, etc.) based on the pre-calibrated environmental occlusion mask. Then, based on the overlap relationship between the real-time detection frame of the object to be monitored and the environmental occlusion mask, the obstructed part is deducted, and the exposed area is used as the feature extraction object. The local matching degree with the initial reference features is calculated, achieving the shielding of obstructed parts and focusing on the exposed area. This avoids misjudging that the personnel are not on duty or that the features do not match due to environmental occlusion. It is more in line with the actual situation of dynamic changes in personnel posture and temporary occlusion in some areas, ensuring both the accuracy of the judgment and the flexibility of the scene. The server determines whether the monitored object is on duty by comparing the local matching degree with a preset discrimination threshold. When the on-duty status is confirmed, it further monitors whether the initial reference features meet the update conditions. When the update conditions are met, the initial reference features are weighted and fused based on the visual feature vector of the exposed area in the current video frame to generate updated reference features. The on-duty status of the monitored object is then monitored using the updated reference features, thus achieving continuity and accuracy in on-duty status detection and avoiding inaccuracies caused by changes in features such as clothing changes. In summary, this solution allows the server to maintain target locking even when the on-duty personnel's body is obscured, using residual visual information such as the head or shoulders. This solves the problem of traditional whole-body detection algorithms easily failing in occluded environments and improves the accuracy of on-duty monitoring.

[0008] In conjunction with some embodiments of the first aspect, in some embodiments, in response to detecting that the object to be monitored has completely entered a preset entry area, an unobstructed image frame containing the object to be monitored is captured, and the visual feature vector in the unobstructed image frame is extracted as an initial reference feature. Specifically, this includes: after detecting that the object to be monitored has entered the entry area, continuously acquiring multiple frames of candidate image sequences containing the object to be monitored; performing foreground segmentation processing on each frame of the candidate image sequence to generate a foreground mask image that retains only the pixels of the object to be monitored; calculating the aspect ratio and effective pixel density of the effective region corresponding to the object to be monitored in the foreground mask image, and determining the candidate images whose aspect ratio is in a preset upright range and whose effective pixel density is greater than a preset integrity threshold as reference sample images; extracting the visual feature vectors in all reference sample images, and performing fusion calculation on all visual feature vectors as initial reference features.

[0009] By employing the above technical solution, the server does not end with a single capture, but continuously collects candidate sequences and uses foreground segmentation to remove background noise, retaining only the effective pixels of the human target, thus purifying the source data for feature extraction. Based on this, by calculating the aspect ratio and its conformity to the preset upright interval, interference from unconventional postures such as bending over or squatting is eliminated. Simultaneously, effective pixel density verification ensures the integrity of the human contour, achieving rigorous screening of sample quality. Finally, by fusing the feature vectors of all selected images, accidental distortions or local blurring that may exist in a single frame are eliminated. In summary, this solution constructs a highly stable initial reference feature, avoiding subsequent recognition errors caused by poor initial sample quality.

[0010] In conjunction with some embodiments of the first aspect, in some embodiments, based on the overlap relationship between the real-time detection box of the object to be monitored and the environmental occlusion mask, the occluded portion is subtracted from the current video frame, and the remaining image region is extracted as the exposed region. Specifically, this includes: mapping the real-time detection box of the object to be monitored to the coordinate system where the environmental occlusion mask is located, and determining the occluded pixel region where the real-time detection box and the environmental occlusion mask intersect; subtracting the occluded pixel region from the pixel set of the real-time detection box to obtain a pixel difference set; performing connected component labeling on the pixel difference set to obtain several independent connected components; extracting the color histograms of the reference sample image corresponding to the initial reference feature and the corresponding regions of the connected components as reference histograms; calculating the correlation coefficient between the color histogram of each connected component and the reference histogram respectively; and selecting the connected pixel block with the largest correlation coefficient as the exposed region.

[0011] By adopting the above technical solution, the video surveillance system server solves the problem of image fragmentation after directly subtracting the occlusion mask. After obtaining the connected components of the pixel difference set, the server uses the color histogram of the reference sample image obtained at the entrance and the corresponding region of the connected component as the retrieval basis to calculate the correlation coefficient with each connected component. Utilizing the consistency of clothing color for the same person, the server can filter out the connected pixel blocks that truly belong to the body part of the monitored object from multiple fragmented regions as the exposed area. Furthermore, by using a local matching strategy of the color histogram (i.e., only taking the local part of the reference image corresponding to the exposed area), the server effectively avoids the problem of inconsistent overall feature distribution caused by the occlusion of the lower body of the monitored object, ensuring the accuracy of exposed area selection. Therefore, this solution ensures that the image region input to the subsequent feature extraction network contains effective human visual information, rather than background noise or other objects, improving the purity of feature extraction and the reliability of matching.

[0012] In conjunction with some embodiments of the first aspect, in some embodiments, calculating the local matching degree between the visual feature vector of the exposed area and the initial reference features specifically includes: extracting the luminance component of the exposed area in a preset color space and calculating the mean of the luminance component; calculating the luminance reference mean corresponding to the reference sample image, and generating a luminance compensation coefficient based on the ratio of the mean of the luminance component to the luminance reference mean; adjusting the pixel values ​​of the exposed area using the luminance compensation coefficient to obtain the compensated exposed area; extracting the visual feature vector of the compensated exposed area and calculating the cosine similarity between it and the vector elements of the corresponding dimension in the initial reference features as the local matching degree.

[0013] By adopting the above technical solution, considering the potential difference in light intensity between the entrance area and the on-duty monitoring area, direct comparison of feature vectors by the server is easily affected by brightness distribution. The server first extracts the brightness component of the exposed area and calculates its ratio to the average brightness of the reference sample, generating a brightness compensation coefficient. The server uses this coefficient to adjust the pixel values ​​of the exposed area to ensure that the lighting conditions of the test area and the reference area are consistent. After eliminating the interference of ambient light differences, the server then performs visual feature vector extraction and cosine similarity calculation, so that the local matching degree can truly reflect the similarity of human appearance texture, improving the accuracy of personnel on-duty determination across areas and under different lighting conditions, and reducing false alarms caused by ambient light interference.

[0014] In conjunction with some embodiments of the first aspect, in some embodiments, an updated reference feature is generated by weighted fusion of the initial reference feature based on the visual feature vector of the exposed area in the current video frame. Specifically, this includes: weighted summing of the visual feature vector of the exposed area in the current video frame with the initial reference feature according to a first preset weight to obtain a temporary reference feature; calculating the vector Euclidean distance between the temporary reference feature and the initial reference feature; if the vector Euclidean distance is less than a preset drift limit value, and the area ratio of the exposed area in the current video frame is greater than a preset update validity threshold, then the temporary reference feature is determined as the updated reference feature; if the vector Euclidean distance is greater than or equal to the preset drift limit value, or the area ratio of the exposed area is less than the preset update validity threshold, then the update is abandoned, and the initial reference feature remains unchanged.

[0015] By adopting the above technical solution, the server first ensures the smoothness of feature evolution through a weighted fusion strategy, avoiding abrupt feature mutations. Secondly, by calculating the Euclidean distance of vectors and limiting it within a drift boundary, the server forces new features to retain the core attributes of the original identity, preventing the model from deviating from the true target due to long-term error accumulation. Simultaneously, the server eliminates the risk of updating when there is insufficient visual information by verifying the proportion of the exposed area. In summary, this solution enables the server to adapt to feature fine-tuning caused by changes in clothing or posture, while maintaining the stability of identity recognition, achieving a balance between adaptability and accuracy during long-term monitoring.

[0016] In conjunction with some embodiments of the first aspect, in some embodiments, after selecting the connected pixel block with the largest correlation coefficient as the exposed region, the method further includes: calculating the sparse optical flow field between the exposed region in the previous video frame and the exposed region in the current video frame; statistically analyzing the proportion of dynamic pixels in the sparse optical flow field whose optical flow modulus is greater than a preset static threshold; when the proportion of dynamic pixels is less than a preset micro-motion judgment value, determining that the exposed region meets the stability requirements, and performing subsequent steps to calculate spatial overlap and texture similarity.

[0017] By adopting the above technical solution, after determining the exposed area, the server introduces sparse optical flow field technology to detect the microscopic motion state inside the exposed area; by statistically analyzing the proportion of dynamic pixels and comparing it with the micro-motion judgment value, the server can distinguish whether a person is in a static state or in situ limb movement state; only when the proportion of dynamic pixels meets the stable condition of the micro-motion judgment value is the feature update process allowed. Thus, the server effectively intercepts the interference of blurred images caused by violent movement on the reference feature library, ensuring that the generated updated reference features are clear and reliable, thereby avoiding the situation of mistaking noise or blurred textures as human features for learning, and ensuring the quality and accuracy of the reference features in the feature library.

[0018] In conjunction with some embodiments of the first aspect, in some embodiments, before subtracting the occluded portion from the current video frame and extracting the remaining image region as the exposed region based on the overlap relationship between the real-time detection box of the object to be monitored and the environmental occlusion mask, the method further includes: calculating the displacement of the center point of the real-time detection box of the object to be monitored between the current video frame and the previous video frame; when the displacement of the center point indicates that the object to be monitored is in a moving state, expanding the boundary of the real-time detection box according to the moving direction to obtain an expanded detection box; and using the expanded detection box as the real-time detection box for calculating the overlap relationship with the environmental occlusion mask.

[0019] By adopting the above technical solution, the server perceives the target's movement trend by calculating the displacement of the center point. When it confirms that the target is moving, it expands the boundary of the real-time detection box according to the direction of movement, thus offsetting the target position lag caused by camera latency or the computation time of the detection algorithm. This solution ensures that the expanded real-time detection box can completely cover the moving human body and edge limbs. The expanded detection box containing complete information is used for subsequent environmental occlusion masking calculations, preventing the artificial omission of exposed areas due to detection box positioning deviations and improving the completeness of exposed area extraction in dynamic scenes.

[0020] Secondly, this application provides a server for a video surveillance system, the server comprising: one or more processors and a memory; the memory is coupled to one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, and the one or more processors invoking the computer instructions to cause the server of the video surveillance system to perform the method described in the first aspect and any possible implementation thereof.

[0021] Thirdly, this application provides a computer-readable storage medium including instructions that, when executed on a server of a video surveillance system, cause the server of the video surveillance system to perform the method described in the first aspect and any possible implementation thereof.

[0022] Fourthly, this application provides a computer program product that, when run on a server of a video surveillance system, causes the server of the video surveillance system to execute the method described in the first aspect and any possible implementation thereof.

[0023] Understandably, the server of the video surveillance system provided in the second aspect, the computer program product provided in the third aspect, and the computer storage medium provided in the fourth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here.

[0024] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0025] 1. By employing an initial reference feature extraction method based on the detection of the monitored object fully entering the preset entrance area, and extracting the exposed area based on the overlap relationship between the real-time detection box and the environmental occlusion mask, and calculating the local matching degree between the visual feature vector of the exposed area and the initial reference feature, the video surveillance system server can no longer rely on the complete human outline when the monitored object is occluded by static facilities in the on-duty monitoring area. Instead, it compares the visual features of the unoccluded part with the high-quality features obtained at the entrance, effectively solving the problem that related technologies rely on general target detection algorithms to have difficulty identifying incomplete human targets, thus leading to false alarms. This enables accurate monitoring of personnel on-duty status in complex occlusion environments.

[0026] 2. By performing connected component labeling on the pixel difference set and selecting the connected pixel block with the highest correlation coefficient between the color histogram of each connected component and the reference histogram of the reference sample image corresponding to the initial reference feature as the exposed area, the video surveillance system server can accurately filter out the effective area belonging to the human body from multiple fragmented areas after removing occlusion by utilizing the consistency of the clothing color of the monitored object. This effectively solves the problem of image area extraction errors caused by background noise or interference residues after simple occlusion removal in related technologies, thereby improving the purity of feature extraction and the accuracy of subsequent matching calculations.

[0027] 3. By calculating the vector Euclidean distance between the temporary reference feature and the initial reference feature, as well as the area ratio of the exposed region, when generating updated reference features, and updating only when the preset drift limit value and update validity threshold are met, the video surveillance system server implements a progressive feature update strategy with dual constraints. This strategy allows the model to adapt to changes in the appearance of the monitored object, while preventing features from deviating from identity attributes by limiting the magnitude of changes and the amount of visual information. This effectively solves the problem of model drift caused by the failure of fixed features or incorrect learning during long-term monitoring, thereby achieving the continuity and stability of monitoring during long-term monitoring periods. Attached Figure Description

[0028] Figure 1 This is a flowchart illustrating a method for monitoring the on-duty status of personnel in a fire control room, as described in this application.

[0029] Figure 2 This is another flowchart illustrating a method for monitoring the on-duty status of personnel in a fire control room, as described in this application embodiment;

[0030] Figure 3 This is a schematic diagram of the physical device structure of a server for a video surveillance system in an embodiment of this application. Detailed Implementation

[0031] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification of this application, the singular expressions “a,” “an,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in this application refers to any or all possible combinations including one or more of the listed items.

[0032] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0033] To facilitate understanding, the application scenarios of the embodiments of this application are described below.

[0034] In related technologies, a general object detection model based on deep learning can be used to identify human targets and determine their presence in video footage. The following describes a scenario using this technology for monitoring personnel presence in a fire control room. When personnel on duty in the fire control room are behind a tall monitoring console or around a large cabinet, the static equipment on site can physically obstruct their view, as security cameras are often installed from a high, downward angle. General object detection models typically rely on complete human anatomical structures for identification. However, when personnel slump due to fatigue and are obstructed by the back of a high chair, or when they are leaning over equipment in a blind spot, only partial features are visible in the image. Algorithms relying on complete outlines struggle to identify such incomplete targets, leading to misjudgments of an obstructed presence as absenteeism and triggering an alarm.

[0035] The fire control room personnel on-duty monitoring method described in this application, applied to the server of a video surveillance system, extracts visual feature vectors from unobstructed image frames as initial reference features in response to the detection that the monitored object has fully entered a preset entrance area. When the monitored object moves to the on-duty monitoring area, the exposed area is extracted based on the environmental occlusion mask, and its local matching degree with the initial reference features is calculated. This enables the identification of the monitored object in an occluded state, utilizing not only the complete features at the entrance as a benchmark but also adapting to the occlusion environment of the work area. The following describes a scenario using one of the fire control room personnel on-duty monitoring methods from this application. When a duty personnel passes through the doorway, the server acquires their complete clothing texture and appearance features. Subsequently, when the personnel sit behind the control console, with only their head and shoulders exposed, the server automatically subtracts the occluded portion of the control console and compares the extracted head and shoulder exposed area features with the initial reference features acquired at the entrance. As long as the local matching degree meets the requirements, the server confirms the personnel's on-duty status and can dynamically update the reference features based on subsequent spatial overlap and texture similarity.

[0036] As can be seen, by adopting the local matching degree calculation mechanism based on the visual feature vector of the exposed area and the initial reference feature in the embodiments of this application, while realizing the cross-space and cross-occlusion degree feature association, it can also effectively solve the problem that the general target detection model cannot identify the incomplete human target due to the occlusion of static facilities, and thus realize the accurate monitoring of the on-duty status of personnel in the fire monitoring room in complex occlusion environment.

[0037] To facilitate understanding, the method provided in this implementation will be described in detail below, using the above scenario as an example. Please refer to [link / reference]. Figure 1 This is a flowchart illustrating a method for monitoring the on-duty status of personnel in a fire control room, as described in this application.

[0038] S101. In response to detecting that the object to be monitored has completely entered the preset entrance area, capture an unobstructed image frame containing the object to be monitored, and extract the visual feature vector in the unobstructed image frame as the initial reference feature.

[0039] The monitored object refers to the individual on-duty personnel entering the fire control room, a moving target with human features identified in the video stream by a target detection algorithm; the preset entrance area refers to a rectangular or polygonal area pre-defined in the monitoring screen, located at the entrance or passageway of the control room, ensuring that personnel passing through can be fully captured without being obstructed by control consoles, cabinets, or other facilities; complete entry indicates that the detection bounding box of the monitored object is completely within the entrance area, meaning that the coordinates of all four vertices of the detection box are within the boundary of the entrance area; an unobstructed image frame refers to a video frame captured at that moment in which the human outline of the monitored object is fully visible and not obstructed by any static facilities or other objects; the visual feature vector represents a high-dimensional numerical vector extracted from the image through a deep learning feature extraction network, which contains human appearance information such as clothing color, texture features, and body shape features, and is usually a 512-dimensional or 1024-dimensional floating-point array; the initial reference feature is used to represent the feature vector that serves as the benchmark for subsequent identity recognition and is stored in the server memory for subsequent matching.

[0040] Specifically, after receiving the real-time video stream transmitted from the surveillance camera, the server of the video surveillance system (hereinafter referred to as the server) first performs human target detection on each frame of the image using a target detection model (such as YOLOv8). When a human target is detected, the server obtains the bounding box coordinates of the target and determines whether the bounding box is completely located within the preset entrance area. Once it is confirmed that the object to be monitored has completely entered the entrance area, the server immediately captures the current frame as an unobstructed image frame and crops out the image region containing the complete human body from the frame. Subsequently, the server inputs the cropped human image into a pre-trained feature extraction network (such as a ReID network based on ResNet). This network calculates and outputs a fixed-length feature vector through forward propagation. The server stores this feature vector as an initial reference feature in the feature library in memory, and at the same time assigns a unique tracking ID to the object to be monitored, establishing a mapping relationship between the feature vector and the tracking ID.

[0041] Optionally, in some embodiments, the server can also perform integrity checks on the captured image frames by combining human morphological features to exclude dynamic occlusion (such as occlusion by others) or non-standard poses. Specifically, after capturing the image frame, the server calculates the aspect ratio of the bounding box of the object to be monitored and calls a lightweight human keypoint detection model to identify skeletal keypoints (such as head, shoulders, hips, knees, and ankles) in the image. Only when the aspect ratio of the bounding box is within a preset standard range for upright human bodies (e.g., between 0.25 and 0.45), and the proportion of the number of high-confidence keypoints identified to the total number of standard human keypoints exceeds a preset integrity threshold (e.g., more than 80% of keypoints are detected), does the server finally confirm the frame as a qualified unoccluded image frame and extract features; if the check fails, the server will discard the current frame and continue to detect the next frame until an image that meets the integrity requirements is captured in the entry area.

[0042] S102. Continuously track the coordinate position of the object to be monitored. When the coordinate position moves from the entrance area to the preset on-duty monitoring area, obtain the environmental occlusion mask that is pre-calibrated in the on-duty monitoring area.

[0043] Among them, continuous tracking means that the server uses a multi-target tracking algorithm (such as DeepSORT) to continuously track the position of the identified objects to be monitored between frames, maintaining the consistency of the target ID; coordinate position refers to the center point coordinates or bounding box coordinates of the object to be monitored in the current video frame, which is represented in pixels as the position information in the image coordinate system; the preset on-duty monitoring area refers to the pre-marked work area in the monitoring screen, which covers the work seat of the on-duty personnel, the area in front of the console and the possible range of activity around it; the environmental occlusion mask is a binary image with the same resolution as the video frame, where the area with a pixel value of 1 represents the position of fixed facilities such as consoles, cabinets, and chair backs, and the area with a pixel value of 0 represents the visible space. This mask is pre-generated and stored in the server through background modeling or manual annotation.

[0044] Specifically, after obtaining the initial reference features of the object to be monitored, the server starts the continuous tracking module. This module predicts the target's position in the next frame of each video frame by combining the target detection results with a Kalman filter, and ensures the continuity of the tracking ID through feature similarity matching. The server monitors the trajectory of the bounding box center coordinates of the object in real time. When it detects that the center coordinates have moved from the entrance area to the on-duty monitoring area (i.e., the coordinate point falls within the polygonal area of ​​the on-duty monitoring area), the server immediately loads the environmental occlusion mask corresponding to that on-duty monitoring area from local storage. This mask file is stored in image format, and the server reads it into memory to form a pixel-level occlusion identification matrix, preparing for subsequent occlusion processing. The server also records the timestamp of the target entering the on-duty monitoring area and starts the on-duty status monitoring process.

[0045] Optionally, in some embodiments, the environmental occlusion mask can also be a foreground occlusion mask extracted from a background model constructed in real time using a Gaussian mixture model (GMM) or the Vibe algorithm. This means that if a temporary large object (such as a moving whiteboard) occludes the monitoring area, the server can also generate and subtract the mask of the object in real time, without limitation.

[0046] S103. Based on the overlap relationship between the real-time detection box of the object to be monitored and the environmental occlusion mask, the occluded part is deducted from the current video frame, and the remaining image area is extracted as the exposed area.

[0047] Among them, the real-time detection box represents the bounding rectangle of the object to be monitored detected by the server in the current video frame through the object detection algorithm, including the coordinates of the upper left corner, width and height information; the overlap relationship refers to the spatial intersection of the detection box area and the area with a pixel value of 1 in the environmental occlusion mask, and the specific pixel position of the occlusion is determined by geometric calculation; the subtracting of the occlusion part is used to represent the operation of removing or setting the pixel area that overlaps with the occlusion mask in the detection box from the original image; the exposed area represents the visible image area remaining after the occlusion part is subtracted, which usually includes the part of the human body such as the head, shoulders or upper body that is not blocked by static facilities.

[0048] Specifically, after obtaining the coordinates of the detection box of the object to be monitored in the current video frame, the server first calculates the spatial overlap area between the detection box and the preloaded environmental occlusion mask. Through pixel-by-pixel comparison, the server identifies which pixel positions within the detection box correspond to the occluded areas in the occlusion mask (i.e., pixels with a mask value of 1). Subsequently, the server crops the image region corresponding to the detection box from the current video frame and performs occlusion subtraction on this region: pixels overlapping with the occlusion mask are set to black (RGB value 0) or directly removed from the image, retaining the unoccluded pixel areas. After processing, the server obtains an image of an irregularly shaped exposed area, which may represent the upper half of a human body or the head and shoulders area. To facilitate subsequent feature extraction, the server performs geometric transformations and size normalization on the exposed area to ensure that the image format meets the input requirements of the feature extraction network.

[0049] S104. Calculate the local matching degree between the visual feature vector of the exposed area and the initial reference feature. When the local matching degree is greater than the preset discrimination threshold, determine that the object to be monitored is in the state of being on duty.

[0050] Among them, the local matching degree represents the similarity between the visual feature vector of the exposed area and the initial reference feature vector. It is calculated by mathematical methods such as cosine similarity or Euclidean distance, and the value range is usually between 0 and 1. The preset discrimination threshold is a critical value used to determine whether the feature matching is successful. The role of this threshold is to distinguish the local features of the same person from the features of different people or background noise. The setting scheme is usually determined based on the ROC curve analysis of a large number of test samples to determine the optimal value, which is generally set between 0.6 and 0.8. Too low a value will lead to false recognition, and too high a value will lead to false negatives. The personnel on-duty status is used to indicate the working status of the system confirming that there are authorized personnel on duty in the monitoring room performing their duties. This status is stored as a Boolean value in the server memory.

[0051] Specifically, after obtaining the image of the exposed area, the server immediately inputs it into the same feature extraction network as in step S101, and calculates a visual feature vector with the same dimension as the initial reference feature vector through forward propagation. The server then retrieves the initial reference feature vector corresponding to the current tracking ID from the feature library in memory, and uses the cosine similarity calculation formula Cosine(F_current, F_reference) = (F_current·F_reference) / (||F_current|| × ||F_reference||) to quantify the matching degree between the two feature vectors, where F_current represents the visual feature vector of the exposed area, F_reference represents the initial reference feature vector, "·" represents the vector dot product operation, and ||F_current|| and ||F_reference|| represent the magnitudes of the corresponding vectors, respectively. The server compares the calculated local matching degree value with a preset discrimination threshold. When the matching degree is greater than the threshold, the server updates the status identifier corresponding to the tracking ID to "personnel on duty" and records a confirmation timestamp in the system log. If the matching degree is less than or equal to the threshold, the server marks the detection as "suspected absence from duty" and starts a continuous detection counter. The absence from duty alarm will only be triggered when N consecutive frames (e.g., 5 frames) are below the threshold, thus avoiding false alarms caused by momentary occlusion or detection errors.

[0052] S105. After confirming that the personnel are on duty, calculate in real time the spatial overlap and texture similarity between the exposed area in the current video frame and the exposed area in the previous video frame.

[0053] Spatial overlap refers to the ratio of the intersection to the union of the current frame's exposed area and the previous frame's exposed area in pixel space. It is calculated using the IoU (Intersection over Union) algorithm and has a value range of 0 to 1. Texture similarity represents the degree of consistency between two exposed area images in terms of texture features. It is quantified by methods such as calculating the image's gradient histogram, LBP features, or structural similarity index (SSIM).

[0054] Specifically, after confirming personnel are on duty, the server activates the continuous monitoring module, which needs to maintain the exposed area image data of both the current and previous frames. The server first calculates spatial overlap, determining the intersection and union areas of two exposed areas through pixel-level comparison, and then uses the IoU formula to calculate the degree of overlap. Subsequently, the server calculates texture similarity. After converting the two exposed area images to grayscale, it extracts their respective Local Binarization Pattern (LBP) histograms or calculates the SSIM index, quantifying the texture similarity using the histogram correlation coefficient or SSIM value. The server stores these two calculation results in real-time in a monitoring buffer in memory, providing a basis for subsequent feature update decisions. Simultaneously, the server updates the frame buffer, storing the exposed area information of the current frame as "previous time" data, preparing for the comparison calculation in the next frame.

[0055] S106. When the spatial overlap is greater than the preset continuous threshold and the texture similarity is greater than the preset update safety threshold, the initial reference features are weighted and fused based on the visual feature vector of the exposed area in the current video frame to generate updated reference features.

[0056] Among them, the preset continuous threshold is a critical value for judging the spatial overlap of personnel positions when they are relatively stable. The purpose of this threshold is to ensure that feature updates are only performed when personnel are basically stationary, avoiding the pollution of the feature library by blurred images during rapid movement. The setting is usually between 0.7 and 0.9, and can be adjusted according to the size of the monitoring room and the range of personnel activity. The preset update safety threshold is a lower limit of texture similarity used to confirm the reliability of image quality. The purpose of this threshold is to prevent feature updates from being performed when image quality deteriorates due to sudden changes in lighting, camera shake, or obstruction by foreign objects. The setting is usually between 0.8 and 0.95. The updated reference feature is used to represent the new baseline feature vector generated after weighted fusion, which replaces the original initial reference feature for subsequent matching calculations.

[0057] Specifically, after obtaining the spatial overlap and texture similarity values ​​of the current frame, the server first compares the spatial overlap with a preset continuous threshold to confirm whether the person is in a relatively stationary state. Then, it compares the texture similarity with a preset update safety threshold to confirm whether the current image quality meets the feature update requirements. When both conditions are met, the server initiates the feature update process: extracting the initial reference feature vector corresponding to the current tracking ID from the memory feature library, and simultaneously obtaining the visual feature vector of the exposed area in the current frame. The calculation is performed using the weighted fusion formula F_updated=(1-α)×F_reference+α×F_current, where α is the update weight coefficient, typically set between 0.05 and 0.1 to ensure the impact of the new features is gradual. F_updated represents the generated updated reference feature, F_reference represents the initial reference feature, and F_current represents the visual feature vector of the exposed area in the current video frame. The server writes the calculated updated reference feature vector into the memory feature library, overwriting the original reference feature, and updates the relevant timestamp information. If either condition is not met, the server retains the original reference feature and continues to use the current feature for subsequent matching.

[0058] For example, suppose the monitored object "Zhang San" enters the entrance area, the system captures an unobstructed image frame of him wearing a dark blue jacket and extracts initial reference features. Later, after Zhang San enters the on-duty monitoring area, if he takes off his jacket because he feels hot, revealing his white shirt...

[0059] During the short period of removing the coat, the drastic change in the posture of the object to be monitored will cause the spatial overlap between the exposed area in the current video frame and the exposed area in the previous video frame to suddenly decrease, falling below the preset continuous threshold. At the same time, the removal of the coat will also inevitably cause the texture similarity to fall significantly below the preset update safety threshold. At this time, the server will not trigger weighted fusion and will continue to use the initial reference features reflecting the dark blue coat for matching.

[0060] However, once Zhang San takes off his coat and sits down stably again, the spatial overlap of his exposed area between consecutive frames will again exceed the preset continuity threshold. Furthermore, due to the stable appearance of the white shirt, the texture similarity will also exceed the preset update safety threshold. Under these updated conditions, the server will perform weighted fusion of the initial reference features based on the visual feature vector of the exposed area of ​​the white shirt in the current video frame. Through continuous iterative updates, the feature weight of the dark blue coat in the initial reference features is gradually diluted, while the feature weight of the white shirt is enhanced, ultimately generating updated reference features. Through steps S105 and S106, the video surveillance system can smoothly adapt to changes in the appearance of the monitored object, ensuring that even if the staff member changes their clothing, the monitoring of their on-duty status can be maintained continuously and accurately.

[0061] S107. Calculate the local matching degree of the objects to be monitored in subsequent video frames based on the updated reference features, so as to maintain the monitoring of personnel on-duty status.

[0062] Specifically, after generating updated reference features, the server immediately sets these features as the new matching benchmark for processing subsequent frames in the video stream. The server continues the cyclical process from steps S102 to S106: for each new video frame, target tracking and position monitoring are performed. While the monitored object is still within the on-duty monitoring area, the server repeatedly performs operations such as occlusion processing, exposed area extraction, feature extraction, and matching degree calculation. However, the reference features have now been updated to a new feature vector that incorporates historical information. The server uses the updated reference features to calculate the local matching degree and determines the on-duty status of personnel by comparing it with a preset discrimination threshold. Simultaneously, it continues to monitor spatial overlap and texture similarity, further optimizing the reference features when the update conditions are met. This cyclical monitoring mechanism allows the server to adapt to minor adjustments in personnel posture, gradual changes in lighting conditions, and slight changes in clothing, ensuring accurate on-duty status identification during long shifts until the monitored object leaves the on-duty monitoring area or the monitoring system is manually stopped.

[0063] In this embodiment, by employing the method of extracting initial reference features in response to the detection that the object to be monitored has completely entered the preset entrance area, extracting the exposed area and calculating the local matching degree based on the overlap relationship between the real-time detection box and the environmental occlusion mask, and generating updated reference features when the spatial overlap and texture similarity conditions are met, the video surveillance system server not only establishes a high-quality comparison benchmark without occlusion and realizes target locking using the unoccluded part in occluded environments, but also dynamically adapts to the gradual changes in the appearance of personnel. This effectively solves the problems of false alarms caused by the inability to identify incomplete human targets due to facility occlusion and the aging of long-term monitoring features in existing general target detection models, thereby achieving high-precision and long-term monitoring of personnel on-duty status in complex occlusion scenarios.

[0064] Based on the above embodiments, the method provided in this embodiment will be described in further detail below. Please refer to... Figure 2 This is another flowchart illustrating a method for monitoring the on-duty status of personnel in a fire control room, as described in this application.

[0065] S201. In response to detecting that the object to be monitored has completely entered the preset entrance area, capture an unobstructed image frame containing the object to be monitored, and extract the visual feature vector in the unobstructed image frame as the initial reference feature.

[0066] This step specifically includes:

[0067] Once the monitored object is detected to have entered the entrance area, multiple frames of candidate image sequences containing the monitored object are continuously acquired.

[0068] Foreground segmentation is performed on each frame of the candidate image sequence to generate a foreground mask that retains only the pixels of the object to be monitored;

[0069] Calculate the aspect ratio and effective pixel density of the effective region corresponding to the object to be monitored in the foreground mask image, and determine the candidate images whose aspect ratio is in the preset vertical range and whose effective pixel density is greater than the preset integrity threshold as reference sample images;

[0070] Visual feature vectors are extracted from all reference sample images, and all visual feature vectors are fused and calculated to serve as initial reference features.

[0071] The candidate image sequence represents a set of multiple frames captured by the server after detecting that the object to be monitored has entered the entrance area. This sequence typically contains 3-10 consecutive video frames to provide sufficient samples for subsequent quality screening. Foreground segmentation refers to the computational process by which the server uses image segmentation algorithms to separate the foreground target (i.e., the human body) from the background environment in the image. This is achieved through deep learning segmentation networks or traditional background modeling methods. The foreground mask image represents a binary image with the same size as the original image. Regions with a pixel value of 1 correspond to human pixels of the object to be monitored, and regions with a pixel value of 0 correspond to background pixels. This mask image is used to accurately identify the outline of the human body. The effective region refers to the connected regions with a pixel value of 1 in the foreground mask image, representing the actual space occupied by the object to be monitored in the current frame. The preset upright interval refers to the range of aspect ratio values ​​that conform to the natural standing posture of the human body. The purpose of this interval is to filter out image frames with non-standard postures such as bending over, squatting, or lying on one's side. The scheme is typically [1.5, 3.5], determined based on a large amount of anthropometric data. Effective pixel density represents the ratio of the number of foreground pixels to the total number of pixels in the effective area, used to measure the completeness of the human body contour. The calculation formula is the number of foreground pixels / the total number of pixels in the area. The preset completeness threshold is the pixel density critical value used to determine whether the human body is completely visible. The purpose of this threshold is to exclude human body missingness caused by partial occlusion or image quality issues. The scheme is generally set to 0.75-0.85 to ensure that at least 75% of the human body contour is clearly visible. Reference sample images are used to represent high-quality candidate images that simultaneously meet the aspect ratio and effective pixel density requirements. These images have complete human body features and standard poses. The arithmetic mean vector is the average feature vector obtained by summing the visual feature vectors extracted by the server from all reference sample images element by element and dividing by the number of images. This vector integrates the feature information of multiple frames of images, improving the robustness of the initial reference features.

[0072] Specifically, once the server detects an object entering a pre-defined entry area using an object detection algorithm, it immediately initiates a candidate image acquisition process. At preset time intervals (e.g., every 2-3 frames), it continuously captures video frames containing the object, forming a candidate image sequence and storing it in a memory buffer. Subsequently, the server performs foreground segmentation processing on each frame in the candidate image sequence, inputting the original RGB image into a pre-trained semantic segmentation network (e.g., DeepLabV3+ or U-Net). This network outputs a probability map of the same size as the input image. The server then sets a confidence threshold (e.g., 0.5) to assign the probability map to the target image. Figure 2 The image is first valued to generate a foreground mask, where pixel values ​​in the human body region are set to 1 and pixel values ​​in the background region are set to 0. Next, the server performs connected component analysis on each foreground mask, extracting the largest connected component as the effective region corresponding to the object to be monitored, and calculates the minimum bounding rectangle of this region. The aspect ratio is calculated using the width and height of the rectangle. Simultaneously, the server counts the number of foreground pixels with a value of 1 within the effective region, dividing this number by the total number of pixels in the region to obtain the effective pixel density. The server compares the aspect ratio of each frame with a preset upright interval and the effective pixel density with a preset integrity threshold. Only when both conditions are met is the candidate image marked as a reference sample image and added to the preferred set. After quality evaluation of all candidate images, the server crops each reference sample image in the preferred set into a standard-sized image patch containing a complete human body and inputs it into a pre-trained feature extraction network (such as a ReID network based on ResNet-50) to extract visual feature vectors of fixed dimensions (such as 512 or 1024 dimensions). Finally, the server performs element-wise arithmetic mean calculation on the feature vectors corresponding to all reference sample images to obtain an initial reference feature vector that integrates information from multiple frames. This vector is then L2 normalized and stored in the memory feature library. At the same time, an association is established with the tracking ID of the object to be monitored, providing reliable benchmark data for subsequent feature matching and identity recognition.

[0073] S202. Continuously track the coordinate position of the object to be monitored. When the coordinate position moves from the entrance area to the preset on-duty monitoring area, obtain the environmental occlusion mask that is pre-calibrated in the on-duty monitoring area.

[0074] This step and Figure 1 The description of step S102 in the embodiment is similar and will not be repeated here.

[0075] S203. Calculate the displacement of the center point of the real-time detection frame of the object to be monitored between the current video frame and the previous video frame.

[0076] Among them, the center point displacement represents the spatial movement distance of the geometric center of the real-time detection box of the object to be monitored between two adjacent frames. It is obtained by calculating the Euclidean distance between the center coordinates of the detection box in the current frame and the center coordinates of the detection box in the previous frame, and the unit is pixels. This value reflects the motion state and movement speed of the object to be monitored. The previous video frame refers to the image of the frame immediately preceding the current video frame in the time series. The server needs to maintain the detection results of this frame in memory for motion analysis.

[0077] Specifically, after the server continuously tracks the object to be monitored into the on-duty monitoring area, it begins to execute the motion state analysis process. This process requires simultaneously acquiring the detection box information of the object to be monitored in the current video frame and the previous video frame. The server first extracts the real-time detection box coordinates of the object to be monitored from the tracking cache in memory, including the coordinates of the top-left corner (x1_prev, y1_prev) and the bottom-right corner (x2_prev, y2_prev), and calculates the center point coordinates of the detection box as (center_x_prev, center_y_prev) = ((x1_prev + x2_prev) / 2, (y1_prev + y2_prev) / 2). Simultaneously, the server acquires the real-time detection box coordinates of the object to be monitored in the current video frame obtained through the object detection algorithm, and calculates the center point coordinates of the detection box in the current frame as (center_x_curr, center_y_curr). The server then uses the Euclidean distance formula to calculate the displacement of the center point: displacement = sqrt((center_x_curr - center_x_prev)). 2 +(center_y_curr-center_y_prev) 2 The system calculates the displacement in pixels, where (center_x_curr, center_y_curr) represents the center coordinates of the real-time detection box of the object to be monitored in the current video frame, and (center_x_prev, center_y_prev) represents the center coordinates of the real-time detection box of the object to be monitored in the previous video frame. The server stores the calculated center point displacement in the motion state cache and updates the tracking cache. It saves the detection box information of the current frame as the "previous moment" reference data for the next calculation, providing data support for subsequent motion state judgment and detection box expansion operations.

[0078] S204. When the displacement of the center point indicates that the object to be monitored is in a moving state, the boundary of the real-time detection box is expanded according to the direction of movement to obtain an expanded detection box.

[0079] Among them, the motion state refers to the significant spatial position change of the monitored object between adjacent frames. It is determined by comparing the displacement of the center point with a preset motion threshold. This state is used to distinguish between two different monitoring modes: stationary and moving. The expanded detection box is used to represent the new detection box after boundary expansion processing. This detection box has a larger coverage area than the original detection box and is used to compensate for the detection delay or edge omission problem that may be caused by the rapid movement of the target.

[0080] Specifically, after obtaining the displacement of the center point, the server first compares this value with a preset motion judgment threshold (e.g., 5 pixels). When the displacement is greater than the threshold, it confirms that the monitored object is in a moving state and then initiates a dynamic detection box expansion mechanism. The server determines the direction of movement by calculating the motion vector, specifically calculating the horizontal component dx = center_x_curr - center_x_prev and the vertical component dy = center_y_curr - center_y_prev. The direction of movement is determined by the positive and negative values ​​of dx and dy: dx > 0 indicates rightward movement, dx < 0 indicates leftward movement, dy > 0 indicates downward movement, and dy < 0 indicates upward movement. Subsequently, the server calculates expansion parameters based on the movement direction and speed, employing an adaptive expansion strategy: horizontal expansion amount expand_x = β × |dx|, vertical expansion amount expand_y = β × |dy|, where β is the expansion coefficient (usually set to 1.2-1.5), expand_x and expand_y represent the number of pixels to expand the real-time detection box in the horizontal and vertical directions, respectively, and dx and dy represent the horizontal and vertical components of the center point displacement, respectively. "||" indicates absolute value calculation. The server selectively expands the detection box boundary based on the movement direction: if the target moves to the right, the right boundary of the detection box is expanded to the right by expand_x pixels; if it moves to the left, the left boundary is expanded to the left; the vertical direction is handled similarly. Finally, the server generates new coordinates (x1_expanded, y1_expanded, x2_expanded, y2_expanded) for the expanded detection box, ensuring that the expanded detection box does not exceed the effective range of the video frame, and stores the expanded detection box data in a temporary cache to replace the original detection box for subsequent occlusion relationship calculations.

[0081] Since the bounding boxes output by general object detection models typically only enclose visible human body parts (such as the head and shoulders), these bounding boxes may lie entirely above an environmental occlusion mask (such as a desk) without overlapping. Therefore, this application first extends the bounding box downwards based on the proportions of the human head and shoulders to cover the inferred torso position, and then performs an intersection operation with the environmental occlusion mask to accurately locate which pixels should actually be visible but are occluded, and which are exposed.

[0082] S205. Use the extended detection box as a real-time detection box for calculating the overlap relationship with the environmental occlusion mask;

[0083] Specifically, after completing the detection box expansion operation, the server immediately executes the detection box replacement process, setting the expanded detection box, which has undergone boundary expansion processing, as the valid real-time detection box for the current frame, replacing the original target detection result. The server extracts the coordinate information of the expanded detection box from the temporary cache and writes it into the corresponding field of the detection box data structure, ensuring that all subsequent calculation operations based on the real-time detection box use the expanded boundary range. Simultaneously, the server updates the detection box record associated with the tracking ID of the monitored object, marking the expanded detection box as operating in "dynamic compensation mode" to distinguish it from the standard detection box in a static state. Subsequently, the server passes the updated real-time detection box data to the occlusion processing module, which performs geometric calculations of the spatial overlap relationship based on the expanded detection box and the pre-loaded environmental occlusion mask, including subsequent processing steps such as intersection region determination, occlusion pixel statistics, and exposed area extraction.

[0084] S206. Based on the overlap relationship between the real-time detection box of the object to be monitored and the environmental occlusion mask, the occluded part is deducted from the current video frame, and the remaining image area is extracted as the exposed area.

[0085] This step specifically includes:

[0086] Map the real-time detection bounding box of the object to be monitored to the coordinate system of the environmental occlusion mask, and determine the occlusion pixel area where the real-time detection bounding box and the environmental occlusion mask intersect;

[0087] Subtract the occluded pixel area from the pixel set of the real-time detection box to obtain the pixel difference set;

[0088] Perform connected component labeling on the pixel difference set to obtain several independent connected components;

[0089] Extract the color histograms of the reference sample images corresponding to the initial reference features and the corresponding regions of the connected components, and use them as reference histograms;

[0090] Calculate the correlation coefficient between the color histogram and the reference histogram for each connected component;

[0091] The connected pixel block with the highest correlation coefficient is selected as the exposed area.

[0092] Among them, the occluded pixel region refers to the set of pixels that overlap spatially with the real-time detection box and the environmental occlusion mask. The pixels in this region are completely blocked by static facilities and do not contain effective visual information of the object to be monitored. The pixel set represents the set of coordinates of all pixels within the boundary of the detection box, stored in the form of a two-dimensional coordinate array, containing the position information of each pixel in the region. The pixel difference set is used to represent the set of pixel coordinates remaining after removing the occluded pixel region from the pixel set of the detection box. This difference set contains all pixels that are not occluded by static facilities. The reference histogram refers to the color distribution statistics extracted from the reference sample image corresponding to the initial reference feature. It is usually in the form of a multi-dimensional histogram in RGB or HSV color space and is used to describe the typical color features of the object to be monitored. The correlation coefficient represents a numerical measure of the similarity between two color histograms. It is calculated by statistical methods such as Pearson correlation coefficient or Bach distance. The value range is usually from 0 to 1. The larger the value, the more similar the color distribution.

[0093] Specifically, after obtaining the extended detection box, the server immediately initiates the occlusion analysis and exposure region extraction process. This process requires handling the spatial relationship between the real-time detection box and the preloaded environmental occlusion mask. The server first performs a coordinate system mapping operation, transforming the coordinates of the real-time detection box in the current video frame (usually pixel coordinates relative to the frame image) to the standard coordinate system used by the environmental occlusion mask through an affine transformation matrix, ensuring that the two geometric objects are calculated in the same coordinate reference system. Subsequently, the server determines the overlapping area between the real-time detection box and the environmental occlusion mask using a geometric intersection algorithm, specifically employing a pixel-level point-by-point comparison method: traversing each pixel coordinate (x, y) within the detection box range, checking the pixel value at the corresponding position in the occlusion mask; if the mask value is 1 (indicating occlusion), then the pixel coordinate is added to the occlusion pixel region set. Next, the server performs set operations, subtracting the occlusion pixel region from the complete pixel set of the real-time detection box to obtain a pixel difference set, which contains the coordinates of all pixels not occluded by static facilities. The server then performs a connected component labeling algorithm on the pixel difference set, using 8-connectivity or 4-connectivity neighborhood rules. It clusters spatially adjacent pixels into independent connected components using depth-first search or breadth-first search methods, assigning a unique identifier ID to each component. After obtaining several independent connected components, the server retrieves the reference sample image data saved in step S201 from memory and calculates the color histograms of these images in the HSV color space as reference histograms. These histograms reflect the typical color distribution characteristics of the object to be monitored. The server then extracts the corresponding image region for each connected component, calculates its own color histogram, and compares its similarity with the reference histogram using a histogram correlation coefficient algorithm (such as the Bach coefficient or correlation coefficient). Finally, the server selects the connected pixel block with the highest correlation coefficient as the final exposed region. This region represents the image part most likely to contain the true features of the object to be monitored under the current occlusion conditions. The server stores the pixel data and boundary coordinate information of this exposed region in a memory cache, providing input data for subsequent feature extraction and matching operations.

[0094] It should be noted that, considering that the reference sample image corresponding to the initial reference feature is usually a complete image containing full-body features (including top and pants / skirt), while the exposed area in the current video frame is affected by occlusion and often only presents a part of the monitored object (such as only the upper body), directly comparing the two global histograms may result in a low correlation coefficient due to the lack of lower body color information.

[0095] Therefore, in some embodiments, the server can employ a locally weighted comparison strategy to calculate the correlation coefficient. Specifically, the server first divides the reference sample image corresponding to the initial reference features into vertical regions, cropping the upper half of the reference sample image according to a preset human body proportion (e.g., 60% upper and 40% lower), and calculates the color histogram of this upper half image as a "local reference histogram." Alternatively, the server introduces an adaptive weighting mechanism when calculating the correlation coefficient, identifying the missing hue intervals (i.e., colors specific to the lower body) in the exposed area of ​​the reference sample image, reducing the weight of these missing hue intervals in the correlation coefficient calculation formula, while increasing the weight of the hue interval containing the dominant color in the exposed area. Through the above-mentioned locally weighted or region segmentation methods, the server can avoid matching failures caused by inconsistent color distribution between the whole body and the lower body, accurately associating the exposed area with the object to be monitored.

[0096] In addition, before extracting the visual feature vectors of the exposed regions, the server needs to format the irregular exposed regions to ensure the consistency of the input tensor dimensions. Specific processing methods include, but are not limited to, the following two:

[0097] Method 1: Mask Filling. The original real-time detection bounding box containing the exposed area is cropped out, and a binary mask channel of the same size as the detection bounding box is generated. Positions corresponding to occluded areas are set to 0, and positions corresponding to exposed areas are set to 1. The cropped image and the mask channel are concatenated as input to the feature extraction network (e.g., a 4-channel input), allowing the network to focus only on features in non-zero regions.

[0098] Method 2: Background Zeroing and Size Normalization. Create a completely black background canvas (pixel value of 0), and fill the canvas with the extracted exposed area pixels according to their original relative positions. Then, use bilinear interpolation to resize the filled image to the standard input size required by the feature extraction network (e.g., 256x128).

[0099] S207. Calculate the local matching degree between the visual feature vector of the exposed area and the initial reference feature;

[0100] This step specifically includes:

[0101] Extract the luminance components of the exposed area in the preset color space and calculate the mean value of the luminance components;

[0102] Calculate the mean brightness reference value corresponding to the reference sample image, and generate the brightness compensation coefficient based on the ratio of the mean brightness component to the mean brightness reference value.

[0103] The pixel values ​​of the exposed area are adjusted using a brightness compensation coefficient to obtain the compensated exposed area.

[0104] Extract the visual feature vector of the compensated exposed area and calculate the cosine similarity between it and the vector elements of the corresponding dimension in the initial reference features, as the local matching degree.

[0105] The preset color space refers to the standard color representation model used for image brightness analysis, typically YUV, Lab, or HSV color spaces. The Y, L, or V components represent the image's brightness information, separate from the color information. The brightness component represents a numerical channel in the preset color space that specifically describes the brightness level of the image. This component reflects the light intensity of the pixel without being affected by hue or saturation. The average brightness reference value refers to the average value of the brightness components extracted from the reference sample image in step S201, representing the standard brightness characteristics of the object under ideal lighting conditions. The brightness compensation coefficient represents the proportional relationship between the current exposed area and the reference brightness. This coefficient is used to normalize the image's brightness to eliminate the influence of lighting changes. The calculation formula is the average reference brightness value divided by the average current brightness value. The vector elements of the corresponding dimension represent the numerical components with the same index position in the visual feature vector. Since the feature vector is usually a high-dimensional vector (e.g., 512-dimensional or 1024-dimensional), similarity calculation needs to be performed dimension-by-dimensional during local matching.

[0106] Specifically, after obtaining the exposed area, the server immediately initiates a lighting compensation and feature matching process, designed to eliminate the impact of ambient lighting changes on the accuracy of feature matching. The server first converts the RGB image of the exposed area to a preset color space (typically Lab color space for better perceptual uniformity), extracting the L component as the luminance component. This component represents the brightness distribution of the exposed area as a single-channel grayscale image. Subsequently, the server calculates the sum of the values ​​of all valid pixels (excluding background pixels) in the luminance component, divided by the number of pixels, to obtain the mean of the luminance component. Simultaneously, the server retrieves the reference sample image data saved in step S201 from memory, performs the same color space conversion and luminance extraction operations on these reference images, and calculates the reference luminance mean. The server then calculates the luminance compensation coefficient, calculated as compensation_factor = reference_brightness_mean / current_brightness_mean, where reference_brightness_mean represents the reference luminance mean calculated from the reference sample image, and current_brightness_mean represents the mean of the luminance component calculated for the current exposed area. The brightness compensation coefficient is then subject to reasonable constraints (typically limited to between 0.5 and 2.0) to prevent image distortion caused by overcorrection. Next, the server uses the brightness compensation coefficient to perform multiplication operations on each pixel value in the exposed area to adjust it, generating a compensated image of the exposed area. The overall brightness of this image is now consistent with the lighting conditions of the initial reference features. The server then inputs the compensated image of the exposed area into the same deep learning feature extraction network as in step S201, calculating a fixed-dimensional visual feature vector through forward propagation. Finally, the server retrieves the initial reference feature vector corresponding to the current tracking ID from the memory feature library and calculates the similarity between the two feature vectors dimension-by-dimensional using the cosine similarity calculation formula cosine_similarity=(A·B) / (||A||×||B||), where A is the visual feature vector of the compensated exposed area, B is the initial reference feature vector, "·" represents the vector dot product operation, and ||A|| and ||B|| represent the magnitudes of vectors A and B, respectively.

[0107] S208. When the local matching degree is greater than the preset discrimination threshold, it is determined that the monitored object is in the state of being on duty.

[0108] This step and Figure 1 The description of step S104 in the embodiment is similar and will not be repeated here.

[0109] S209. Calculate the sparse optical flow field between the exposed area in the previous video frame and the exposed area in the current video frame.

[0110] Among them, the sparse optical flow field represents the set of motion vectors calculated between specific feature points of two frames of images by optical flow algorithm. This optical flow field only contains the motion information of key points with significant texture features in the image, and has the advantages of low computational cost and strong noise resistance compared with dense optical flow.

[0111] Specifically, after the server completes the extraction of the exposed area in step S206, it immediately starts the motion analysis module to evaluate the stability of the exposed area. This analysis requires comparing the motion state of the exposed area between adjacent frames. The server first extracts the exposed area image data corresponding to the previous video frame from the memory cache. This data includes the visible human body portion obtained from the previous frame through the same occlusion processing procedure. Subsequently, the server converts the two exposed area images from the previous and current moments into grayscale images for optical flow calculation. The server uses the Shi-Tomasi corner detection algorithm to identify key points with strong texture features in the exposed area of ​​the previous moment. These key points are typically located at clothing edges, areas with significant texture changes, or human body contour boundaries. Next, the server calls the Lucas-Kanade optical flow algorithm, using the key points in the exposed area of ​​the previous moment as starting points, to search for the corresponding positions of these points in the exposed area of ​​the current moment, calculating the displacement vector of each key point. The algorithm uses a local window (usually 15×15 pixels) for template matching, determining the optimal matching position of the pixel by minimizing the brightness difference. The server combines the motion vectors of all successfully tracked key points to form a sparse optical flow field. This optical flow field is stored as a vector array, with each element containing the coordinates of the starting point, the coordinates of the ending point, and the corresponding motion vector (u, v). The server also performs quality assessment on the optical flow calculation results, removing unreliable optical flow vectors through reverse tracking verification and error threshold filtering to ensure the accuracy and stability of the optical flow field data.

[0112] The purpose of introducing sparse optical flow to determine the proportion of dynamic pixels is to compensate for the limitation of relying solely on "space overlap (IoU)" in detecting "in-situ deformation." For example, if a staff member is sitting in a chair and waving or turning around dramatically, the IoU of their bounding box may not change much (or even be very high), but in reality, the person's posture has changed drastically, resulting in severe motion blur. Forcibly updating features in this situation would lead to the blurring of features and contamination of the feature library. Therefore, combining optical flow with this method can more accurately capture such in-situ non-rigid deformations, ensuring the purity of the updated reference features.

[0113] S210. Statistically determine the proportion of dynamic pixels in a sparse optical flow field whose optical flow modulus is greater than a preset static threshold.

[0114] Among them, the optical flow modulus represents the Euclidean distance of the optical flow vector, reflecting the motion amplitude of the pixel; the preset static threshold is the critical value of the optical flow modulus used to distinguish between static and moving states. The function of this threshold is to filter out pseudo motion signals caused by image noise, minor camera jitter, or slight changes in illumination. The setting scheme is usually set to 1.0-2.0 pixels / frame based on the stability of the camera and the monitoring environment. For high-definition cameras, it can be appropriately increased to 2.5-3.0 pixels / frame; dynamic pixels refer to feature points whose optical flow modulus exceeds the preset static threshold. These points represent the pixel positions in the exposed area that have actually undergone significant displacement; the dynamic pixel ratio represents the ratio of the number of dynamic pixels to the total number of effective optical flow points in the sparse optical flow field. This ratio reflects the overall motion activity level of the exposed area.

[0115] Specifically, after obtaining the sparse optical flow field data, the server immediately initiates a motion statistical analysis process. This process is used to quantitatively assess the motion intensity of the exposed area and provide data support for subsequent stability assessment. The server first traverses each valid optical flow vector in the sparse optical flow field, extracting the horizontal component u and vertical component v for each vector, and calculating the magnitude of the optical flow vector using the Euclidean distance formula magnitude=sqrt(u²+v²). The server compares each calculated optical flow magnitude with a preset static threshold. When the magnitude is greater than the threshold, the corresponding feature point is marked as a dynamic pixel and the dynamic point counter is incremented. After completing the magnitude calculation and classification statistics for all optical flow vectors, the server calculates the dynamic pixel ratio using the formula dynamic_ratio=dynamic_point_count / total_valid_flow_count, where dynamic_point_count is the number of dynamic pixels obtained, and total_valid_flow_count is the total number of qualified optical flow vectors in the sparse optical flow field. The server stores the calculation results in the motion analysis cache as a percentage. This value directly reflects the level of motion activity in the currently exposed area: a percentage close to 0% indicates that the exposed area is basically still, while a higher percentage indicates that there is obvious human movement or posture change.

[0116] S211. When the proportion of dynamic pixels is less than the preset micro-motion judgment value, the exposed area is determined to meet the stability requirements, and the subsequent steps of calculating spatial overlap and texture similarity are performed.

[0117] Among them, the preset micro-motion judgment value refers to the dynamic pixel ratio threshold used to determine whether the exposed area is in a relatively static state. The function of this threshold is to distinguish normal micro-physiological movements (such as breathing fluctuations, slight swaying) from significant posture changes or rapid movement. The setting scheme is usually set to 15%-25% according to the actual working scenario of the fire monitoring room, taking into account the slight limb movements that the on-duty personnel may have under normal working conditions.

[0118] Specifically, after completing the dynamic pixel ratio statistics, the server immediately executes a stability assessment process, which is a crucial step in determining whether to continue feature updates. The server retrieves the calculated dynamic pixel ratio from the motion analysis cache and compares it with a pre-configured preset micro-motion judgment value. When the dynamic pixel ratio is less than the preset micro-motion judgment value, the server confirms that the currently displayed area is in a relatively stable state. At this point, although the personnel are on duty, there are no drastic posture changes or rapid movements, and the image quality is suitable for feature extraction and matching calculations. The server then updates the stability status flag of the displayed area, setting it to stable mode, and records the stability confirmation timestamp in the system log. After confirming stability, the server immediately initiates the subsequent feature update preparation process, including activating the spatial overlap calculation module and the texture similarity analysis module. These two modules perform in-depth similarity analysis on the displayed areas of the current frame and the previous frame. If the proportion of dynamic pixels is greater than or equal to the preset micro-motion judgment value, the server determines that the exposed area is currently in motion and does not meet the stability requirements. At this time, the server will skip the calculation steps related to feature update, directly maintain the current reference feature unchanged, and continue to monitor the stability of the exposed area in subsequent frames until the motion ends.

[0119] S212. After confirming that the personnel are on duty, calculate in real time the spatial overlap and texture similarity between the exposed area in the current video frame and the exposed area in the previous video frame.

[0120] This step and Figure 1 The description of step S105 in the embodiment is similar and will not be repeated here.

[0121] S213. When the spatial overlap is greater than the preset continuous threshold and the texture similarity is greater than the preset update safety threshold, the initial reference features are weighted and fused based on the visual feature vector of the exposed area in the current video frame to generate updated reference features.

[0122] This step specifically includes:

[0123] The visual feature vector of the exposed area in the current video frame is weighted and summed with the initial reference feature according to the first preset weight to obtain the temporary reference feature;

[0124] Calculate the vector Euclidean distance between the temporary reference feature and the initial reference feature;

[0125] If the Euclidean distance of the vector is less than the preset drift limit value, and the area ratio of the exposed region in the current video frame is greater than the preset update validity threshold, then the temporary reference feature is determined as the update reference feature.

[0126] If the Euclidean distance of the vector is greater than or equal to the preset drift limit, or the area ratio of the exposed region is less than the preset update validity threshold, then the update is abandoned and the initial reference features remain unchanged.

[0127] The first preset weight represents the weight coefficient assigned to the visual feature vector of the exposed area in the current frame during weighted fusion calculation. This weight controls the influence of new feature information on the initial reference feature. The setting scheme usually adopts a small value of 0.05-0.15 to ensure that the feature update is gradual. The temporary reference feature refers to the intermediate result feature vector obtained by weighted summation calculation. This vector needs to be verified for quality before it can be determined as the final update reference feature. The vector Euclidean distance represents the geometric distance between two high-dimensional feature vectors in Euclidean space. It is used to quantify the degree of deviation between the temporary reference feature and the initial reference feature. The preset drift threshold value is the distance critical value used to detect whether the feature vector has deviated excessively. The function of this threshold is to prevent the initial reference feature from gradually drifting to a state that is too different from the original target during continuous update. The setting scheme is usually set to 10%-20% of the original feature magnitude. The preset update effectiveness threshold is the minimum required value of the exposed area ratio. The function of this threshold is to ensure that feature updates are only performed when the exposed part of the human body is sufficiently large. The setting scheme is usually set to 0.35-0.50.

[0128] Specifically, after completing the similarity calculation and confirming that the dual conditions are met in step S212, the server immediately initiates the weighted fusion process. The server retrieves the initial reference feature vector corresponding to the current tracking ID from the memory feature library, and simultaneously obtains the visual feature vector of the exposed region in the current video frame. It then performs feature fusion calculation using the weighted summation formula temp_feature=(1-μ)×initial_reference+μ×current_feature, where μ is the first preset weight, temp_feature represents the calculated temporary reference feature, initial_reference represents the current initial reference feature, and current_feature represents the visual feature vector of the exposed region in the current video frame. After obtaining the temporary reference feature, the server calculates the vector Euclidean distance between the temporary reference feature and the initial reference feature using the Euclidean distance formula: D=sqrt(∑(t_i-s_i)^2), where D represents the vector Euclidean distance, i represents the dimension index of the feature vector, t_i represents the value of the i-th dimension in the temporary reference feature, and s_i represents the value of the i-th dimension in the initial reference feature. Then, the Euclidean distance of the vector is compared with a preset drift threshold. Simultaneously, the server calculates the area ratio of the visible region in the current video frame using the formula area_ratio = visible_area / total_detection_box_area, where visible_area represents the pixel area of ​​the visible region in the current video frame, and total_detection_box_area represents the total pixel area of ​​the complete expanded detection box used for calculation. This area ratio is then compared with a preset update validity threshold. If the Euclidean distance is less than the preset drift threshold and the area ratio of the visible region is greater than the preset update validity threshold, the server determines the temporary reference feature as the update reference feature and writes it into the memory feature library, overwriting the original initial reference feature data. If the Euclidean distance is greater than or equal to the preset drift threshold or the area ratio of the visible region is less than the preset update validity threshold, the server abandons the update operation, keeps the initial reference feature unchanged, records the reason for the update failure in the system log, and continues to use the current initial reference feature for subsequent matching operations.

[0129] S214. Calculate the local matching degree of the objects to be monitored in subsequent video frames based on the updated reference features, so as to maintain the monitoring of personnel on-duty status.

[0130] This step and Figure 1 The description of step S107 in the embodiment is similar and will not be repeated here.

[0131] In this embodiment, the video surveillance system server not only establishes a highly robust comparison benchmark, but also compensates for detection delays caused by target motion, eliminates interference from background debris and lighting differences, and prevents motion blur data from polluting the feature library. This effectively solves the problem that related technologies are susceptible to the decline in recognition accuracy due to non-standard postures of personnel, rapid movement, environmental clutter, and changes in lighting. As a result, it achieves high-stability monitoring of personnel on-duty status throughout the entire process in dynamic and complex scenarios.

[0132] The server of the video surveillance system in the embodiments of this invention is described below from the perspective of hardware processing. Please refer to [link / reference needed]. Figure 3 This is a schematic diagram of the physical device structure of a server for a video surveillance system in an embodiment of this application.

[0133] It should be noted that, Figure 3 The server structure of the video surveillance system shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0134] like Figure 3 As shown, the server of the video surveillance system includes a CPU 301, which can perform various appropriate actions and processes according to a program stored in the read-only memory ROM 302 or a program loaded from the storage section 308 into the random access memory RAM 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0135] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0136] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by CPU 301, it performs the various functions defined in the present invention.

[0137] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0138] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0139] Specifically, the server of the video surveillance system in this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the on-duty monitoring method for personnel in a fire monitoring room provided in the above embodiment.

[0140] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the server of the video surveillance system described in the above embodiments; or it may exist independently and not assembled into the server of the video surveillance system. The storage medium carries one or more computer programs that, when executed by a processor of a server of the video surveillance system, cause the server of the video surveillance system to implement the fire monitoring room personnel on-duty monitoring method provided in the above embodiments.

[0141] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0142] As used in the above embodiments, depending on the context, the term "when..." can be interpreted as meaning "if...", "after...", "in response to determining...", or "in response to detecting...". Similarly, depending on the context, the phrase "when determining..." or "if (the stated condition or event) is interpreted as meaning "if determining...", "in response to determining...", "when (the stated condition or event) is detected", or "in response to detecting (the stated condition or event)".

[0143] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method for monitoring personnel presence in a fire control room, applied to a server in a video surveillance system, characterized in that, include: In response to detecting that the object to be monitored has completely entered the preset entrance area, an unobstructed image frame containing the object to be monitored is captured, and the visual feature vector in the unobstructed image frame is extracted as the initial reference feature. The coordinate position of the object to be monitored is continuously tracked. When the coordinate position moves from the entrance area to the preset on-duty monitoring area, the environmental occlusion mask pre-calibrated in the on-duty monitoring area is obtained. Based on the overlap relationship between the real-time detection frame of the object to be monitored and the environmental occlusion mask, the occluded part is subtracted from the current video frame, and the remaining image area is extracted as the exposed area. Calculate the local matching degree between the visual feature vector of the exposed area and the initial reference feature. When the local matching degree is greater than a preset discrimination threshold, determine that the object to be monitored is in a state of personnel on duty. After confirming that the personnel are on duty, the spatial overlap and texture similarity between the exposed area in the current video frame and the exposed area in the previous video frame are calculated in real time. When the spatial overlap is greater than a preset continuous threshold and the texture similarity is greater than a preset update safety threshold, the initial reference features are weighted and fused based on the visual feature vector of the exposed area in the current video frame to generate updated reference features. The local matching degree is calculated for the monitored object in subsequent video frames based on the updated reference features, so as to maintain the monitoring of the personnel's on-duty status.

2. The method according to claim 1, characterized in that, In response to detecting that the object to be monitored has completely entered the preset entrance region, an unobstructed image frame containing the object to be monitored is captured, and the visual feature vector in the unobstructed image frame is extracted as the initial reference feature, specifically including: Once the monitored object is detected to have entered the entrance area, multiple frames of candidate image sequences containing the monitored object are continuously acquired. Foreground segmentation is performed on each frame of the candidate image sequence to generate a foreground mask image that retains only the pixels of the object to be monitored; Calculate the aspect ratio and effective pixel density of the effective region corresponding to the object to be monitored in the foreground mask image, and determine the candidate images whose aspect ratio is in a preset upright range and whose effective pixel density is greater than a preset integrity threshold as reference sample images; Visual feature vectors are extracted from all the reference sample images, and all visual feature vectors are fused and calculated to serve as the initial reference features.

3. The method according to claim 2, characterized in that, Based on the overlap between the real-time detection bounding box of the object to be monitored and the environmental occlusion mask, the occluded portion is subtracted from the current video frame, and the remaining image area is extracted as the exposed area, specifically including: Map the real-time detection box of the object to be monitored to the coordinate system where the environmental occlusion mask is located, and determine the occlusion pixel area where the real-time detection box and the environmental occlusion mask intersect; Subtract the occluded pixel region from the pixel set of the real-time detection box to obtain the pixel difference set; Perform connected component labeling on the pixel difference set to obtain several independent connected components; Extract the color histogram of the reference sample image corresponding to the initial reference feature and the corresponding region of the connected component, as the reference histogram; Calculate the correlation coefficient between the color histogram of each connected component and the reference histogram; The connected pixel block with the largest correlation coefficient is selected as the exposed area.

4. The method according to claim 3, characterized in that, Calculating the local matching degree between the visual feature vector of the exposed area and the initial reference feature specifically includes: Extract the luminance components of the exposed area in a preset color space, and calculate the mean value of the luminance components; Calculate the mean brightness reference value corresponding to the reference sample image, and generate a brightness compensation coefficient based on the ratio of the mean of the brightness component to the mean brightness reference value; The pixel values ​​of the exposed area are adjusted using the brightness compensation coefficient to obtain the compensated exposed area; Extract the visual feature vector of the compensated exposed area and calculate the cosine similarity between it and the vector elements of the corresponding dimension in the initial reference features, as the local matching degree.

5. The method according to claim 3, characterized in that, The step of weighted fusing the initial reference features based on the visual feature vectors of the exposed areas in the current video frame to generate updated reference features specifically includes: The visual feature vector of the exposed area in the current video frame is weighted and summed with the initial reference feature according to the first preset weight to obtain the temporary reference feature; Calculate the vector Euclidean distance between the temporary reference feature and the initial reference feature; If the Euclidean distance of the vector is less than a preset drift limit value, and the area ratio of the exposed region in the current video frame is greater than a preset update validity threshold, then the temporary reference feature is determined as the update reference feature. If the Euclidean distance of the vector is greater than or equal to a preset drift limit, or if the area ratio of the exposed region is less than a preset update validity threshold, then the update is abandoned and the initial reference feature remains unchanged.

6. The method according to claim 3, characterized in that, After selecting the connected pixel block with the largest correlation coefficient as the exposed area, the method further includes: Calculate the sparse optical flow field between the exposed area in the previous video frame and the exposed area in the current video frame; The proportion of dynamic pixels in the sparse optical flow field whose optical flow modulus is greater than a preset static threshold is statistically analyzed. When the proportion of dynamic pixels is less than the preset micro-motion judgment value, the exposed area is determined to meet the stability requirements, and subsequent steps of calculating spatial overlap and texture similarity are performed.

7. The method according to claim 6, characterized in that, Before extracting the remaining image region as the exposed region by subtracting the occluded portion from the current video frame based on the overlap relationship between the real-time detection bounding box of the object to be monitored and the environmental occlusion mask, the method further includes: Calculate the displacement of the center point of the real-time detection frame of the object to be monitored between the current video frame and the previous video frame; When the displacement of the center point indicates that the object to be monitored is in a moving state, the boundary of the real-time detection box is expanded according to the direction of movement to obtain an expanded detection box; The extended detection box is used as a real-time detection box for calculating the overlap relationship with the environmental occlusion mask.

8. A server for a video surveillance system, characterized in that, The server of the video surveillance system includes: one or more processors and a memory; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the server of the video surveillance system to perform the method as described in any one of claims 1-7.

9. A computer-readable storage medium comprising instructions, characterized in that, When the instruction is executed on the server of the video surveillance system, the server of the video surveillance system performs the method as described in any one of claims 1-7.

10. A computer program product, characterized in that, When the computer program product is run on the server of the video surveillance system, it causes the server of the video surveillance system to perform the method as described in any one of claims 1-7.

Images