Machine room anomaly identification method and device, electronic equipment and storage medium

Abstract

The application relates to a machine room anomaly recognition method and device, electronic equipment and a storage medium, and relates to the technical field of computer vision and artificial intelligence. The method comprises the following steps: collecting a continuous image frame sequence of a machine room monitoring area; constructing and maintaining a static background model and a dynamic background model; determining a total foreground area based on the difference between an image frame and a static reference background image, and filtering a dynamic foreground area from the total foreground area to obtain a static foreground area; under the spatiotemporal joint constraint, target tracking is performed on the static foreground area corresponding to each image frame to screen out candidate abnormal targets; the static reference background image and the image frame containing the candidate abnormal target are input into a multimodal large model to obtain an abnormal recognition result of whether the candidate abnormal target is an abnormal stacking object or causes a channel blockage. The application can realize high-precision, real-time automatic detection of abnormal stacking objects and channel blockages in a complex machine room monitoring environment, and significantly reduces the consumption of computing resources.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method, apparatus, electronic device, and storage medium for identifying anomalies in computer rooms. Background Technology

[0002] In the modern operation and maintenance management of data centers and critical infrastructure, the security and stable operation of the data center is the primary task. Utilizing video surveillance systems for 24 / 7 uninterrupted monitoring has become one of the core technical means to ensure data center security. In this scenario, abnormal object detection is of significant application value in promptly identifying safety hazards such as blocked passageways, improper equipment placement, and obstructed fire exits. However, the currently widely adopted data center monitoring models still mainly rely on manual periodic inspections or post-event review of recordings. This approach not only requires a large investment of manpower but also suffers from inherent shortcomings such as slow response times, personnel fatigue, and high false negative rates, making it difficult to meet the increasingly demanding requirements for intelligent and refined security management.

[0003] To improve the level of monitoring automation, the following three methods are usually used to achieve automatic detection.

[0004] 1) Background-modeling-based object detection methods employ Gaussian mixture models (GMM), kernel density estimation, or other background modeling techniques. First, morphological processing and clustering of foreground pixels are performed, and the static time of each foreground cluster is calculated. When the static time exceeds a threshold, it is identified as a residual object. However, in semi-controlled environments such as computer rooms, during long-term operation, static anomalous objects slowly absorb into the background model, making it impossible to continuously detect them. Simultaneously, external disturbances such as pedestrian traffic and changes in lighting are easily misidentified as anomalous objects, resulting in severe false positives and false negatives.

[0005] 2) Object stacking detection methods based on object detection networks (such as YOLO) use deep learning networks to detect object categories in surveillance videos in real time and combine them with rule or classification models to determine whether there are leftover objects. However, in data center scenarios, these methods suffer from insufficient model generalization ability due to a lack of a large amount of real labeled data, and can only identify the existence of objects, but cannot determine whether they actually cause passage blockage or safety hazards, thus failing to perform in-depth semantic risk assessment.

[0006] 3) Anomaly recognition methods based on multimodal large models: These methods use visual and linguistic multimodal large models to perform semantic understanding and scene description on monitoring images, thereby determining whether anomalies exist. Although these methods perform well in open scenarios, they consume huge amounts of computational resources and have high response latency in continuous 24 / 7 monitoring scenarios, making it difficult to meet the real-time and resource efficiency requirements of practical applications, thus limiting the actual deployment and application of the technology. Summary of the Invention

[0007] The technical problem to be solved by the present invention is to provide a method, device, electronic device and storage medium for identifying abnormalities in a computer room.

[0008] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: Firstly, a method for identifying anomalies in a computer room includes: Acquire a continuous sequence of image frames from the monitored area of ​​the computer room; Construct and maintain a static background model and a dynamic background model; wherein, the static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on a historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate; the dynamic background model is used to perform real-time adaptive background modeling on the continuous image frame sequence to generate a dynamic foreground region, and the update frequency of its model parameters is equal to or greater than the preset video frame rate. For each image frame in the continuous image frame sequence, a total foreground region is determined based on the difference between the image frame and the static reference background image, and the dynamic foreground region is filtered out from the total foreground region to obtain a static foreground region; Under spatiotemporal constraints, target tracking is performed on the static foreground regions corresponding to each of the image frames to filter out candidate abnormal targets; wherein, the spatiotemporal constraints include: spatially focusing the analysis range on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold. In response to the selection of the candidate anomalous targets, the static reference background image and the image frame containing the candidate anomalous targets to be verified are input into a multimodal large model for semantic analysis to obtain the anomalous identification result of whether the candidate anomalous targets are anomalous piles or whether they cause channel blockage.

[0009] The beneficial effects of this invention are as follows: By constructing static and dynamic dual-background models that work collaboratively to accurately separate dynamic interference from static targets in video scenes, this effectively solves the problems of static objects being absorbed by the background and dynamic interference causing false alarms in traditional background modeling. By introducing dual spatial and temporal constraints, the system focuses its analysis on key areas and filters based on the duration of target stillness, ensuring targeted detection. Finally, the system only triggers the multimodal large model for semantic-level verification when highly suspicious candidate targets are identified, realizing a collaborative mechanism of "real-time filtering by small models and on-demand judgment by large models." This not only significantly reduces the consumption of computing resources but also greatly shortens the anomaly response latency while maintaining high-precision identification capabilities for abnormal accumulations and channel blockages, significantly improving the automation level and reliability of data center security monitoring.

[0010] Based on the above technical solution, the present invention can be further improved as follows.

[0011] Furthermore, the dynamic background model is a Gaussian mixture model, which extracts and filters out dynamic foreground regions generated by dynamically changing backgrounds in the image frames by performing pixel-level real-time probabilistic statistical modeling on the continuous image frame sequence.

[0012] The beneficial effects of adopting the above-mentioned further scheme are as follows: using a Gaussian mixture model as a dynamic background model, this model can more accurately depict and distinguish the dynamic changes in the scene caused by light fluctuations, small disturbances, etc. by performing real-time and dynamic probabilistic statistical modeling on each pixel. This allows for more effective separation and filtering of dynamic foreground areas formed by transient interferences such as pedestrians passing by and light and shadow flickering from the total foreground area, thereby significantly improving the extraction purity of static foreground areas and enhancing the robustness of the entire system to dynamic interference in complex computer room environments.

[0013] Furthermore, the static background model is used to calculate the median gray value of each pixel in the historical image frame sequence to generate the static reference background image; and the static reference background image is periodically updated or triggered by semantic signals output by the multimodal large model that indicate significant changes in the computer room scene environment.

[0014] The beneficial effects of adopting the above-mentioned further scheme are as follows: generating a static background using the median method can ensure the purity of the static reference background image, and its periodic updates can ensure that the model can adapt to long-term slow changes in the environment; while introducing a specific update mechanism triggered by the semantic signals of the large model gives the system the ability to intelligently perceive and respond to significant changes in the environment, enabling the static background model to quickly complete self-calibration in scenarios such as all lights on and equipment layout adjustments, thereby maintaining high robustness and detection accuracy in complex and ever-changing computer room environments.

[0015] Furthermore, filtering out the dynamic foreground region from the total foreground region includes: A morphological dilation operation is performed on the dynamic foreground region to obtain the dilated dynamic foreground region; Perform a logical AND operation between the total foreground region and the complement of the expanded dynamic foreground region.

[0016] The beneficial effects of adopting the above-mentioned further solution are: by performing morphological dilation on the dynamic foreground region and then performing logical filtering, the sticky pixels and residual noise at the edges of dynamic objects can be removed more thoroughly, thereby significantly improving the purity and integrity of the static foreground region and effectively reducing the risk of false detection and false negative due to incomplete target outlines or residual dynamic shadows.

[0017] Furthermore, the step of performing target tracking on the static foreground regions corresponding to each of the image frames under spatiotemporal joint constraints to filter out candidate abnormal targets includes: The static foreground region is subjected to a bitwise AND operation with a predefined binary mask image representing the key areas of the computer room, so as to spatially focus the analysis scope on the computer room passage and entrance / exit areas. A state estimation based on a Kalman filter is used to track each quasi-stationary target in the computer room passage and entrance / exit area, and the number of consecutive frames of each quasi-stationary target is recorded. For each quasi-stationary target, the duration of continuous stillness is determined based on the number of consecutively occurring frames. When the duration of continuous stillness exceeds the preset threshold, the quasi-stationary target is selected as a candidate abnormal target.

[0018] The beneficial effects of adopting the above-mentioned further solution are as follows: By performing a bitwise AND operation between the static foreground area and the predefined key area mask of the computer room, precise spatial focusing is effectively achieved, completely eliminating interference from non-monitored areas such as walls and fixed equipment. This allows computing resources and analytical efforts to be concentrated on the channels and entrances / exits where blockages are truly likely to occur. Simultaneously, using state estimation based on a Kalman filter for target tracking can robustly record and accumulate the number of consecutive frames for each target, thereby accurately calculating its actual static duration. This spatiotemporal joint constraint mechanism ensures that only targets that remain static within the key area for more than a preset duration are identified as candidate anomalies, significantly improving the targeting and accuracy of the screening and effectively avoiding false alarms caused by temporarily placed items or brief visits by personnel.

[0019] Furthermore, the state estimation based on a Kalman filter for tracking quasi-stationary targets in the computer room passage and entrance / exit areas also includes anti-drift processing: In the process noise covariance matrix of the Kalman filter, the velocity component in the target state vector is assigned a noise weight lower than that of its position component to suppress detection frame drift caused by ambient light flicker.

[0020] The beneficial effects of adopting the above-mentioned further scheme are: by assigning a noise weight lower than that of the position component to the target velocity component in the Kalman filter, the slight drift of the detection frame caused by common environmental disturbances such as light flicker and equipment vibration in the computer room can be effectively suppressed, thereby ensuring the accuracy of the calculation of the continuous static duration of the tracked target and enhancing the temporal stability and robustness of abnormal target screening.

[0021] Furthermore, the state estimation based on a Kalman filter for tracking quasi-stationary targets in the computer room passage and entrance / exit areas also includes anti-occlusion fault-tolerant processing. When any of the quasi-stationary targets fails to match the current detection box within a consecutive preset number of frames, inertial position prediction is performed based on the state of the quasi-stationary target at the previous moment, and the tracking identifier of the quasi-stationary target and the accumulation of the number of consecutively appearing frames are maintained within a preset tolerance number of frames. If the quasi-stationary target reappears within the preset tolerance frame number and matches the predicted position, then continuous tracking of the quasi-stationary target is resumed.

[0022] The beneficial effects of adopting the above-mentioned further scheme are as follows: when the target is obscured due to reasons such as personnel briefly passing through the computer room, the system can predict the inertial position based on the target's motion state before obscuration and maintain continuous tracking and duration accumulation of the target within a set tolerance period. Once the target reappears within the tolerance period and matches the predicted position, continuous tracking can be resumed immediately. This effectively avoids problems such as tracking interruption, target ID switching, or reset of static duration accumulation caused by short-term obscuration, significantly enhancing the stability and robustness of the system in actual complex computer room environments and ensuring the continuity of abnormal object detection.

[0023] Further, the step of inputting the static reference background image and the image frame containing the candidate anomalous target to be verified into a multimodal large model for semantic analysis to obtain an anomaly identification result as to whether the candidate anomalous target is an anomalous accumulation or whether it causes channel blockage includes: Based on the current image frame, generate a labeled image in which all the candidate anomalous targets are marked with rectangular boxes, and combine the labeled image with the static reference background image to form an image pair; The image pairs and structured prompt text are input into the multimodal large model; the prompt text is used to guide the multimodal large model to answer questions such as whether the candidate abnormal target is an abnormal pile or whether it causes channel blockage, and whether the computer room scene environment has changed significantly by comparing the differences between the marked image and the static reference background image.

[0024] The beneficial effects of adopting the above-mentioned further approach are as follows: by constructing image pairs of labeled images and static background images, and supplementing them with structured prompt text to guide the multimodal large model in differential analysis, precise guidance for semantic-level verification is achieved. This method can fully leverage the visual understanding and logical reasoning capabilities of the multimodal large model to efficiently determine whether the labeled target is a real abnormal accumulation of objects or constitutes a channel blockage. At the same time, it intelligently assesses whether the overall scene environment has undergone significant changes, ensuring a high confidence level in the anomaly identification results.

[0025] Furthermore, the method also includes performing the following operations based on the semantic analysis results of the multimodal large model: If the semantic analysis result determines that the candidate abnormal target is an abnormal pile or causes channel blockage, an alarm event is generated and the current frame screenshot is recorded. If the semantic analysis results indicate that the computer room environment has changed significantly, then the static reference background image will be updated adaptively.

[0026] The beneficial effects of adopting the above-mentioned further solutions are as follows: When the multimodal large model confirms the presence of abnormal piles or channel blockages, it can immediately generate alarm events and retain evidence, realizing a complete link from perception and identification to security response, greatly shortening the response time from problem discovery to alarm triggering, and improving security operation and maintenance efficiency. Simultaneously, when the multimodal large model determines that the scene environment has undergone significant changes, it can intelligently trigger adaptive updates of the static reference background image, effectively solving the problem of background model failure and subsequent false detections caused by drastic environmental changes (such as long-term changes in lighting patterns or equipment layout adjustments). This semantic understanding-based closed-loop feedback mechanism can intelligently drive and update the static reference background image based on the multimodal large model's advanced understanding of scene changes, achieving self-calibration of the baseline background and long-term environmental adaptability.

[0027] Secondly, the present invention provides a computer room anomaly identification device, which adopts the following technical solution: A data center anomaly detection device, comprising: The image acquisition module is used to acquire continuous image frame sequences of the computer room monitoring area; The model maintenance module is used to construct and maintain a static background model and a dynamic background model. The static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on a historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate. The dynamic background model is used to perform real-time adaptive background modeling on the continuous image frame sequence to generate a dynamic foreground region, and the update frequency of its model parameters is equal to or greater than the preset video frame rate. The foreground determination module is used to determine the total foreground region for each image frame in the continuous image frame sequence based on the difference between the image frame and the static reference background image, and to filter out the dynamic foreground region from the total foreground region to obtain the static foreground region; The target screening module is used to perform target tracking on the static foreground regions corresponding to each of the image frames under spatiotemporal constraints, so as to screen out candidate abnormal targets; wherein, the spatiotemporal constraints include: spatially focusing the analysis range on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold. An anomaly identification module is used to respond to the selection of candidate anomaly targets by inputting the static reference background image and the image frame containing the candidate anomaly targets to be verified into a multimodal large model for semantic analysis, and to obtain an anomaly identification result as to whether the candidate anomaly target is an abnormal accumulation or whether it causes channel blockage.

[0028] Thirdly, the present invention provides an electronic device that adopts the following technical solution: An electronic device includes a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor, when executing the computer program, implements the data center anomaly identification method as described in any of the first aspects.

[0029] Fourthly, the present invention provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data center anomaly identification method as described in any of the first aspects.

[0030] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and will become apparent from the description or may be learned by practice of the invention. Attached Figure Description

[0031] Figure 1 This is a flowchart illustrating the data center anomaly identification method provided by the present invention. Figure 2 A schematic flowchart of the dynamic-static separation method for dual-background models provided by the present invention; Figure 3 A schematic diagram of the confirmation process for candidate abnormal targets provided by the present invention; Figure 4 This is a flowchart illustrating the asynchronous verification and closed-loop feedback process of the large model provided by the present invention. Figure 5 This is a schematic diagram demonstrating the anomaly detection effect in the computer room provided by the present invention; Figure 6 This is a screenshot of the actual alarm for the abnormal object provided by the present invention; Figure 7 This is a schematic diagram of the structure of the computer room anomaly identification device provided by the present invention; Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0032] The principles and features of the present invention are described below. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0033] Please refer to Figure 1, Figure 1 This is a flowchart illustrating the data center anomaly identification method provided by the present invention. The method may include the following steps S101-S105.

[0034] S101, Acquire a continuous sequence of image frames from the monitoring area of ​​the acquisition room.

[0035] Specifically, fixed-position surveillance cameras are deployed within the computer room to continuously monitor key areas such as walkways and equipment air vents. Real-time RTSP video streams are acquired from the surveillance cameras via the Real-time Streaming Protocol (RTSP), and the acquired RTSP video streams are decoded in real-time to obtain a continuous sequence of image frames. t represents the time sequence number.

[0036] Optionally, after real-time decoding, further processing is performed on consecutive image frame sequences. Each image frame in The following preprocessing operations are performed sequentially to improve the efficiency and robustness of subsequent processing: 1) Grayscale conversion: Converting the original three-channel (usually RGB) color image into a single-channel 8-bit grayscale image can reduce the complexity of subsequent calculations; and since background modeling, foreground extraction, etc. are usually based on brightness information, removing color information can significantly reduce the amount of data, improve processing speed, and meet real-time requirements. 2) Noise suppression: A 5x5 median filter is used to process the grayscale image, which can effectively remove isolated noise points introduced by the sensor or transmission process, while preserving the image edge information well. 3) Region of Interest (ROI) Loading: Loads pre-configured ROI mask images. The ROI mask image The operation and maintenance personnel manually draw the map through the management interface, marking only key areas such as the walkways in the computer room and the air outlets of the equipment as valid detection areas (pixel value marked as 1), while areas such as the walls and fixed parts of the cabinets are marked as invalid areas (pixel value marked as 0).

[0037] S102. Construct and maintain the static background model and the dynamic background model.

[0038] Among them, the static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on the historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate.

[0039] Historical image frame sequences refer to a segment of computer room monitoring video stream that is continuously captured and temporarily stored during system startup, initialization, or background reconstruction tasks. This video is broken down into a series of images arranged sequentially in time. For example, after system startup, the initial 200 consecutive images generated by the camera are captured.

[0040] The inherent static layout of a data center refers to the physical entities and structures within the data center whose positions and forms remain fixed under normal operation and maintenance conditions. These constitute the inherent background of the monitoring screen and serve as the benchmark reference for distinguishing normal backgrounds from abnormal foregrounds. Examples include: cement walls, anti-static floors, fixed server racks (non-movable), ceilings, cable trays, fixed air conditioning vents, and signs that do not move for extended periods.

[0041] A static reference background image refers to an idealized and pure representation of the inherent static layout of a computer room scene, without containing any temporarily appearing foreground objects such as people, mobile devices, or abandoned items. It serves as the baseline image for subsequent change detection.

[0042] The update frequency of a static reference background image refers to the frequency at which the static reference background image is recalculated and generated.

[0043] The preset video frame rate refers to the inherent frame rate of video captured by the surveillance camera or processed by the system.

[0044] In one embodiment, a static background model is used to calculate the median grayscale value of each pixel in a historical image frame sequence to generate a static reference background image; and the static reference background image is periodically updated or triggered by a semantic signal output by a multimodal large model that indicates a significant change in the computer room scene environment.

[0045] For example, after system startup, an initial, continuous surveillance video segment (e.g., the initial sequence of 200 historical image frames) is acquired. For each pixel location in the image, the grayscale values ​​of all historical frames at that location are statistically analyzed, and the median is calculated as the final, stable background value for that pixel. Through pixel-by-pixel calculation, a complete, high-definition, single-channel grayscale image of the same size as the video frame is finally synthesized. This serves as a static reference background image. This image represents the inherent, unchanging physical layout of the server room, such as walls, fixed server racks, and floors. To ensure its reliability as a stable reference, the background image is not updated in real-time. Its update frequency (e.g., once per hour or triggered by semantic signals indicating significant changes in the server room environment output by a multimodal large model) is much lower than the frame rate of the video stream (e.g., 25fps or 30fps), thus ensuring it provides a persistent and reliable benchmark for subsequent dynamic change detection.

[0046] The dynamic background model is used to perform real-time adaptive background modeling on a continuous sequence of image frames to generate dynamic foreground regions, and the update frequency of its model parameters is equal to or greater than the preset video frame rate.

[0047] In one embodiment, the dynamic background model is a Gaussian mixture model (GMM), which extracts and filters out dynamic foreground regions generated by dynamically changing backgrounds in image frames by performing pixel-level real-time probabilistic statistical modeling on a continuous sequence of image frames.

[0048] Specifically, the expression for the Gaussian mixture model is as follows: (1) in, Indicates in Time of the first The grayscale value of each pixel Indicates in Time of the first The probability density of the grayscale value of each pixel being identified as a background pixel by the Gaussian mixture model. This indicates the number of Gaussian distributions in the Gaussian mixture model. Indicates in Time of the first The weights of a Gaussian distribution Indicates in Time of the first The pixel number The mean of a Gaussian distribution, Indicates in Time of the first The pixel number The variance of a Gaussian distribution Indicated by and The parameter is the first The probability density function of a Gaussian distribution.

[0049] When the matching condition is met Update parameters as needed: (2) in, , Indicates the learning rate. This represents the adjustment coefficient. In this example, it is implemented using BackgroundSubtractorMOG2 from the OpenCV library. The model parameters can be set as follows: =5, historical frame count is 500, background threshold varThreshold=16, but this embodiment is not limited to these. The learning rate is automatically adjusted by the algorithm to adapt to normal dynamics such as slow changes in light and periodic air conditioning airflow.

[0050] Using GMM as the representation of each pixel in the real-time video stream, a hierarchy is maintained in memory in parallel. A probabilistic model composed of Gaussian distributions. This model is continuously updated in real time with each new image input frame, dynamically learning and characterizing the color (or grayscale) value changes of each pixel over a short period. For example, a pixel might mostly appear as the dark gray of a server rack (one Gaussian distribution), occasionally brighten due to light reflection (another Gaussian distribution), and sometimes change to the color of clothing when a pedestrian passes by (yet another Gaussian distribution). The GMM uses weights to represent the frequency of these states, with the distribution with the highest weights typically considered the current background.

[0051] When processing a new image frame, the current observation value of each pixel is matched against all Gaussian distributions established for that pixel by the Gaussian Model (GMM). If the pixel value does not conform to any distribution (or only conforms to a distribution with very low weights), it is identified as foreground, i.e., a dynamic foreground region. In the collaborative architecture of this embodiment, these dynamic regions identified by the GMM in real time are precisely the interference parts that need to be filtered out from the total foreground region. In this way, the GMM is specifically responsible for capturing and separating instantaneous changes in the scene (such as people moving around, light and shadow flickering).

[0052] S103. For each image frame in a continuous image frame sequence, determine the total foreground region based on the difference between the image frame and the static reference background image, and filter out the dynamic foreground region from the total foreground region to obtain the static foreground region.

[0053] In one embodiment, filtering out the dynamic foreground region from the total foreground region includes: performing a morphological dilation operation on the dynamic foreground region to obtain a dilated dynamic foreground region; and performing a logical AND operation between the complement of the total foreground region and the dilated dynamic foreground region.

[0054] For example, such as Figure 2 As shown, firstly, the current frame is calculated. With static reference background image The absolute difference between pixels. For each pixel, if the difference corresponding to that pixel is greater than a preset long-term background threshold ( If the value is 25, then this pixel is marked as the total foreground, and a total foreground mask is generated. That is, the total foreground area.

[0055] The total foreground mask can be generated using the following expression: (3) in, Indicates the long-term background threshold. This is an indicator function.

[0056] Current frame Feed in a dynamic background model (GMM) and output a dynamic foreground mask. That is, the current frame The dynamic foreground region generated by moving objects (such as pedestrians, flickering lights, etc.). Apply a radius of A circular structuring element with a resolution of 5 pixels is morphologically dilated to expand the area of ​​the dynamic object, resulting in a dilated dynamic foreground region. .in, Let this be the radius of the expansion kernel. Then, logical operations are performed. That is, from the overall foreground area Medium filter removes dynamic foreground region This yields the final static foreground mask. The mask theoretically only includes the area of ​​newly appearing, stationary objects in the scene, i.e., the static foreground area.

[0057] Based on the binary foreground markers output by the static background model and the dynamic background model respectively and Calculate the state of each pixel in the current frame. The pixel state codes and their meanings are shown in Table 1 below.

[0058] Table 1 Pixel Status Encoding Table

[0059] S104. Under the joint spatiotemporal constraints, target tracking is performed on the static foreground regions corresponding to each image frame to filter out candidate abnormal targets.

[0060] Among them, the spatiotemporal joint constraints include: spatially focusing the analysis scope on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold.

[0061] Specifically, under the joint spatiotemporal constraints, truly suspicious candidate anomalies requiring verification are further screened from the initially selected static foreground areas. The joint spatiotemporal constraints are manifested in the joint constraints of space and time: spatial constraints focus the analysis scope on the region of interest, such as key areas of the server room like passageways and entrances, thus physically eliminating irrelevant foreground noise in non-critical areas like walls and tops of fixed server racks, greatly improving the efficiency and targeting of subsequent processing; temporal constraints require the target to remain stationary for a duration exceeding a preset threshold, effectively filtering out interference from temporarily placed toolboxes, short-staying maintenance personnel, and other briefly stationary objects, ensuring that only objects that remain for extended periods and may pose a safety hazard trigger subsequent verification.

[0062] In one embodiment, step S104 includes the following sub-steps S1041-S1043.

[0063] S1041. Perform a bitwise AND operation between the static foreground region and a predefined binary mask image representing the key areas of the computer room, so as to spatially focus the analysis scope on the computer room passage and entrance / exit areas.

[0064] Specifically, such as Figure 3 As shown, the static foreground area Compared with the predefined binarized mask image representing the key areas of the computer room. Perform a bitwise AND operation to obtain the foreground mask only within the region of interest. Through the analysis of Perform connected component analysis: The process involves identifying all independent white pixel clusters. Then, area filtering is applied to each white pixel cluster, removing tiny regions smaller than 500 pixels to eliminate residual noise. Each remaining connected region is treated as a candidate object (i.e., a quasi-stationary object), and a bounding box is generated for it.

[0065] S1042. Using state estimation based on Kalman filter, track each quasi-stationary target in the computer room passage and entrance / exit area, and record the number of consecutive frames of each quasi-stationary target.

[0066] Specifically, the improved SORT (Simple Online and Realtime Tracking) algorithm can be used to perform spatiotemporal correlation tracking on the generated candidate pile targets and record the number of consecutive frames of each quasi-stationary target.

[0067] When a new candidate target appears for the first time, initialize its Kalman filter state, create a tracking trajectory, and start a stationary timer. Construct the target state space based on the Kalman filter and define the target state vector. ,in Indicates the coordinates of the target center. Indicates area, Indicates the aspect ratio. Represents the velocity component.

[0068] Optionally, step S1042 further includes anti-drift processing: in the process noise covariance matrix of the Kalman filter, the velocity component in the target state vector is assigned a noise weight lower than that of its position component to suppress detection box drift caused by ambient light flicker.

[0069] To accommodate the static characteristics of the equipment stored in the computer room, a process noise covariance matrix is ​​constructed during the prediction phase. Through non-uniform setting The process noise weights that suppress velocity components force the filter to predict that the target position remains stationary, thereby smoothing out the detection box drift caused by light flicker.

[0070] Optionally, step S1042 further includes anti-occlusion fault tolerance processing: when any quasi-stationary target does not match the current detection box within a preset number of consecutive frames, inertial position prediction is performed based on the state of the quasi-stationary target at the previous moment, and the tracking mark of the quasi-stationary target and the accumulation of the number of consecutive occurrences are maintained within a preset tolerance number of frames; if the quasi-stationary target reappears within the preset tolerance number of frames and matches the predicted position, then continuous tracking of the quasi-stationary target is resumed.

[0071] To address the issue of short-term occlusion caused by personnel passing through the server room, prediction-preservation logic is introduced. In subsequent frames, a Kalman filter is used to predict the target position, and a Hungarian algorithm is used for matching. A maximum number of frames tolerable is set. (Preferred 10 frames) When the target does not match a detection box in the current frame, inertial position prediction is performed based on the state of the previous frame, while maintaining the target's ID and cumulative hit count. If in The target reappears within the frame and meets the position matching condition (IOU with the predicted bounding box is greater than 1). (preferably 0.3) and shape consistency constraints (aspect ratio change rate) If the time limit is exceeded, then tracking will resume and temporal continuity will be maintained. If it exceeds... If no detection box is found in the frame, the target tracking is terminated.

[0072] S1043. For each quasi-stationary target, determine the duration of continuous stationary state based on the number of consecutively occurring frames. When the duration of continuous stationary state exceeds a preset threshold, the quasi-stationary target is selected as a candidate abnormal target.

[0073] Specifically, such as Figure 3 As shown, calculate each target Consecutive hits frames And convert it into static duration: .in, Indicates the video frame rate. When... (Preferably 30~60 seconds) mark the target as a candidate anomaly region that needs to be verified by a large model.

[0074] S105. In response to the selection of candidate anomalous targets, the static reference background image and the image frame containing the candidate anomalous targets to be reviewed are input into the multimodal large model for semantic analysis to obtain the anomalous identification result of whether the candidate anomalous target is an anomalous pile or whether it causes channel blockage.

[0075] In one embodiment, step S105 may include: generating a labeled image based on the current image frame, in which all candidate anomalous targets are marked with rectangular boxes, and forming an image pair together with a static reference background image; inputting the image pair and structured prompt text into a multimodal large model; the prompt text is used to guide the multimodal large model to answer questions such as whether the candidate anomalous targets are anomalous piles or whether they cause channel blockage, and whether the computer room scene environment has changed significantly, by comparing the differences between the labeled image and the static reference background image.

[0076] Specifically, such as Figure 4 As shown, when the static duration of a candidate target reaches a preset threshold, a "scene-benchmark" image pair is constructed, and semantic-level analysis is performed using a multimodal large language model. In the verification stage, a complete scene image containing all candidate target markers is extracted from the current video frame. (i.e., the marked image), and load the static reference background image saved during the initialization phase. .in, This represents a real-time acquisition frame, on which the spatial location of the candidate target is marked in the form of a rectangle; This represents a static reference background image without foreground objects. Then, the two images are... Submit asynchronously via API to a multimodal large language model (such as the Qwen2.5 VL-32B large model) deployed in the cloud. A structured text prompt is also attached: Task: Data Center Security Analysis. Input: [Image 1] is a real-time monitoring screen, where a red box marks an object that has been stationary for a long time; [Image 2] is a normal idle background for the same scene. Please return a JSON-formatted answer to the following questions: 1. 'is_blockage': Determine whether the object within the red box constitutes a blockage of the passage or poses a security risk (true / false). 2. 'suggest_bg_update': Determine whether the overall environment (e.g., lighting) of [Image 1] has undergone a significant and persistent change compared to [Image 2], which may cause the background model to fail (true / false). The multimodal large model combines visual understanding and text reasoning capabilities to analyze whether the labeled area contains newly added objects or changes in ambient lighting and shadows, and outputs JSON verification results. . Indicates whether there is a blockage. This indicates a suggestion to update the background. Among them, : Confirm the presence of actual debris or channel blockage; The cause is determined to be shadow, light reflection, or temporary interference. This indicates that the scene lighting, angle, or environment has changed significantly, and it is recommended to update the static reference background image.

[0077] In one embodiment, the method further includes: based on the semantic analysis results of the multimodal large model, performing the following operations: if the semantic analysis results determine that the candidate abnormal target is an abnormal pile or causes channel blockage, then an alarm event is generated and the current frame screenshot is recorded; if the semantic analysis results determine that the computer room scene environment has changed significantly, then an adaptive update of the static reference background image is triggered.

[0078] Specifically, wait for and parse the JSON result returned by the multimodal large model. For example... Figure 4 As shown, if the parsing yields If a real anomaly is confirmed, such as a pile of objects or a blocked passage, an alarm event is generated and a screenshot of the current frame is recorded; if the parsed data is... If it is determined to be a false detection, the candidate target is removed to avoid false alarms; if it is parsed... Trigger background adaptive update to the current frame Update static reference background image It also clears the candidate cache, enabling the system to adapt to significant changes in ambient lighting or layout.

[0079] Once a genuine anomaly, such as a physical object piled up or a channel blockage, is confirmed, a structured alarm record is generated in JSON format. The record includes the following content: {"timestamp":"2025-11-12T20:45:10Z","event_type":"Obstruction Detected", "snapshot_base64":"..."}. This JSON alarm record is then sent to the centralized monitoring and alarm platform in the data center via an HTTP POST request, along with a screenshot of the abnormal object.

[0080] Figure 5 This is an algorithm demonstration example, used to show the ability of the present invention to generate a detection box and trigger an alarm around a stationary pile of objects. Figure 5 A stationary mineral water bottle and a red sign were placed in the center of the passageway, while an inspector walked past them. The invention successfully generated a red detection frame around the stationary object, ultimately triggering an alarm. However, no detection frame was generated for the moving inspector. This clearly demonstrates that the invention can effectively distinguish between a truly stationary object and normal dynamic disturbances, exhibiting high accuracy and practicality.

[0081] Figure 6 The screenshots show actual alarm records generated by the system in a real environment, directly reflecting the alarm effectiveness of this invention on-site. After receiving the JSON alarm records, the monitoring and alarm platform will notify maintenance personnel via SMS and application for on-site handling.

[0082] Furthermore, please refer to Table 2, which compares the performance data of 20 computer room surveillance cameras over a week under the traditional large-model timed polling method and the method of this invention.

[0083] Table 2 Comparison of performance data between the two methods As shown in Table 2, under the same hardware resource constraints, traditional methods have to extend the detection interval to 20 minutes to control computing power costs, resulting in severe real-time lag and monitoring blind spots. In contrast, this invention utilizes a small model for real-time scanning throughout the entire period, triggering a large model for verification only when a suspected abnormal target is detected and the static duration condition is met. Experimental data shows that this invention reduces the anomaly response time from 20 minutes to 1 minute (a 20-fold improvement) while reducing the computing power consumption of the large model by approximately 86%. This verifies that the collaborative architecture of the small and large models in this invention can achieve efficient, real-time, and accurate identification of data center anomalies with extremely low computing resource consumption.

[0084] This invention provides a method for identifying anomalies in data centers. By constructing static and dynamic dual-background models that work collaboratively, the method accurately separates dynamic interference from static targets in video scenes, effectively solving the problems of static objects being absorbed by the background and dynamic interference causing false alarms in traditional background modeling. By introducing dual spatial and temporal constraints, the system focuses its analysis on key areas and filters based on the duration of target stillness, ensuring targeted detection. Finally, the system only triggers a multimodal large model for semantic-level verification when highly suspicious candidate targets are identified, achieving a collaborative mechanism of "real-time filtering by small models and on-demand judgment by large models." This not only significantly reduces computational resource consumption but also greatly shortens anomaly response latency while maintaining high-precision identification capabilities for abnormal object accumulation and channel blockages, significantly improving the automation level and reliability of data center security monitoring.

[0085] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of the computer room anomaly identification device provided by the present invention. Figure 7 As shown, the device may include: Image acquisition module 701 is used to acquire continuous image frame sequences of the computer room monitoring area; The model maintenance module 702 is used to construct and maintain a static background model and a dynamic background model. The static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on a historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate. The dynamic background model is used to perform real-time adaptive background modeling on the continuous image frame sequence to generate a dynamic foreground region, and the update frequency of its model parameters is equal to or greater than the preset video frame rate. Foreground determination module 703 is used to determine the total foreground region for each image frame in the continuous image frame sequence based on the difference between the image frame and the static reference background image, and filter out the dynamic foreground region from the total foreground region to obtain the static foreground region; The target screening module 704 is used to perform target tracking on the static foreground regions corresponding to each of the image frames under spatiotemporal joint constraints, so as to screen out candidate abnormal targets; wherein, the spatiotemporal joint constraints include: spatially focusing the analysis range on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold. Anomaly identification module 705 is used to, in response to the selection of the candidate anomaly targets, input the static reference background image and the image frame containing the candidate anomaly targets to be verified into a multimodal large model for semantic analysis, and obtain anomaly identification results for whether the candidate anomaly targets are abnormal piles or whether they cause channel blockage.

[0086] In some embodiments, the data center anomaly identification device of the present invention can be implemented in a combination of hardware and software. As an example, the data center anomaly identification device of the present invention can be a processor in the form of a hardware decoding processor, which is programmed to execute the data center anomaly identification method of the present invention. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0087] The modules described in the embodiments of this invention can be implemented in software or hardware. The names of the modules are not, in some cases, limiting the scope of the module itself.

[0088] An electronic device according to an embodiment of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the above-mentioned methods for identifying abnormal data centers. That is, an electronic device according to an embodiment of the present invention may include, but is not limited to: a processor and a memory; the memory is used to store the computer program; the processor is used to execute the method for identifying abnormal data centers shown in any embodiment of the present invention by calling the computer program.

[0089] In one alternative embodiment, an electronic device is provided, such as Figure 8 As shown, Figure 8 The illustrated electronic device 4000 includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, for example, via a bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one type, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present invention.

[0090] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this invention. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0091] Bus 4002 may include a path for transmitting information between the aforementioned components. Bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 4002 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus 4002 is represented by only one thick line, but this does not mean that there is only one bus or one type of bus.

[0092] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0093] The memory 4003 stores application code (computer program) for executing the present invention, and its execution is controlled by the processor 4001. The processor 4001 executes the application code stored in the memory 4003 to implement the content shown in the foregoing method embodiments.

[0094] Among them, electronic devices can also be terminal devices, which can be any device that can install applications, including at least one of smartphones, tablets, laptops, desktop computers, smart speakers, smartwatches, smart TVs, and smart in-vehicle devices.

[0095] It should be noted that, Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0096] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any of the above-described methods for identifying abnormalities in a computer room.

[0097] Alternatively, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, a floppy disk, and an optical data storage device, etc.

[0098] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the electronic device to perform the aforementioned computer room anomaly identification method.

[0099] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0100] It should be understood that the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of methods and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0101] The computer-readable storage medium provided in this invention can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EEPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0102] The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the method shown in the above embodiments.

[0103] The above description is merely a preferred embodiment of the present invention and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure in this invention is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions disclosed in this invention.

[0104] It should be noted that the terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and represent a limitation on a specific order or sequence. Where appropriate, the order of use for similar objects can be interchanged so that the embodiments of this application described herein can be implemented in an order other than that shown or described.

[0105] Those skilled in the art will recognize that this invention can be implemented as a system, method, or computer program product. Therefore, this invention can be specifically implemented in the following forms: it can be entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software, generally referred to herein as a "circuit," "module," or "system." Furthermore, in some embodiments, this invention can also be implemented as a computer program product contained in one or more computer-readable media, which includes computer-readable program code.

[0106] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method for identifying anomalies in a computer room, characterized in that, include: Acquire a continuous sequence of image frames from the monitored area of ​​the computer room; Construct and maintain a static background model and a dynamic background model; wherein, the static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on a historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate; the dynamic background model is used to perform real-time adaptive background modeling on the continuous image frame sequence to generate a dynamic foreground region, and the update frequency of its model parameters is equal to or greater than the preset video frame rate. For each image frame in the continuous image frame sequence, a total foreground region is determined based on the difference between the image frame and the static reference background image, and the dynamic foreground region is filtered out from the total foreground region to obtain a static foreground region; Under spatiotemporal constraints, target tracking is performed on the static foreground regions corresponding to each of the image frames to filter out candidate abnormal targets; wherein, the spatiotemporal constraints include: spatially focusing the analysis range on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold. In response to the selection of the candidate anomalous targets, the static reference background image and the image frame containing the candidate anomalous targets to be verified are input into a multimodal large model for semantic analysis to obtain the anomalous identification result of whether the candidate anomalous targets are anomalous piles or whether they cause channel blockage.

2. The method for identifying abnormalities in a computer room according to claim 1, characterized in that, The dynamic background model is a Gaussian mixture model, which extracts and filters out dynamic foreground regions generated by dynamically changing backgrounds in the image frames by performing pixel-level real-time probabilistic statistical modeling on the continuous image frame sequence.

3. The method for identifying abnormalities in a computer room according to claim 1, characterized in that, The static background model is used to calculate the median gray value of each pixel in the historical image frame sequence to generate the static reference background image; and the static reference background image is periodically updated or triggered by semantic signals output by the multimodal large model that indicate significant changes in the computer room scene environment.

4. The method for identifying abnormalities in a computer room according to claim 1, characterized in that, The step of filtering out the dynamic foreground region from the total foreground region includes: A morphological dilation operation is performed on the dynamic foreground region to obtain the dilated dynamic foreground region; Perform a logical AND operation between the total foreground region and the complement of the expanded dynamic foreground region.

5. The method for identifying abnormalities in a computer room according to claim 1, characterized in that, Under spatiotemporal joint constraints, target tracking is performed on the static foreground regions corresponding to each of the image frames to filter out candidate abnormal targets, including: The static foreground region is subjected to a bitwise AND operation with a predefined binary mask image representing the key areas of the computer room, so as to spatially focus the analysis scope on the computer room passage and entrance / exit areas. A state estimation based on a Kalman filter is used to track each quasi-stationary target in the computer room passage and entrance / exit area, and the number of consecutive frames of each quasi-stationary target is recorded. For each quasi-stationary target, the duration of continuous stillness is determined based on the number of consecutively occurring frames. When the duration of continuous stillness exceeds the preset threshold, the quasi-stationary target is selected as a candidate abnormal target.

6. The method for identifying abnormalities in a computer room according to claim 5, characterized in that, The state estimation based on a Kalman filter is used to track each quasi-stationary target in the computer room passage and entrance / exit area, and anti-drift processing is also included. In the process noise covariance matrix of the Kalman filter, the velocity component in the target state vector is assigned a noise weight lower than that of its position component to suppress detection frame drift caused by ambient light flicker.

7. The method for identifying abnormalities in a computer room according to claim 5, characterized in that, The state estimation based on a Kalman filter is used to track each quasi-stationary target in the computer room passage and entrance / exit areas, and the method also includes anti-occlusion fault-tolerant processing. When any of the quasi-stationary targets fails to match the current detection box within a consecutive preset number of frames, inertial position prediction is performed based on the state of the quasi-stationary target at the previous moment, and the tracking identifier of the quasi-stationary target and the accumulation of the number of consecutively appearing frames are maintained within a preset tolerance number of frames. If the quasi-stationary target reappears within the preset tolerance frame number and matches the predicted position, then continuous tracking of the quasi-stationary target is resumed.

8. The method for identifying abnormalities in a computer room according to claim 1, characterized in that, The step of inputting the static reference background image and the image frame containing the candidate anomalous target to be verified into a multimodal large model for semantic analysis to obtain anomaly identification results, including whether the candidate anomalous target is an anomalous pile or whether it causes channel blockage, includes: Based on the current image frame, generate a labeled image in which all the candidate anomalous targets are marked with rectangular boxes, and combine the labeled image with the static reference background image to form an image pair; The image pairs and structured prompt text are input into the multimodal large model; the prompt text is used to guide the multimodal large model to answer questions such as whether the candidate abnormal target is an abnormal pile or whether it causes channel blockage, and whether the computer room scene environment has changed significantly by comparing the differences between the marked image and the static reference background image.

9. The method for identifying abnormalities in a computer room according to claim 8, characterized in that, The method further includes: Based on the semantic analysis results of the aforementioned multimodal large model, the following operations are performed: If the semantic analysis result determines that the candidate abnormal target is an abnormal pile or causes channel blockage, an alarm event is generated and the current frame screenshot is recorded. If the semantic analysis results indicate that the computer room environment has changed significantly, then the static reference background image will be updated adaptively.

10. A computer room anomaly detection device, characterized in that, include: The image acquisition module is used to acquire continuous image frame sequences of the computer room monitoring area; The model maintenance module is used to construct and maintain a static background model and a dynamic background model. The static background model is used to generate a static reference background image representing the inherent static layout of the computer room scene based on a historical image frame sequence, and the update frequency of the static reference background image is lower than the preset video frame rate. The dynamic background model is used to perform real-time adaptive background modeling on the continuous image frame sequence to generate a dynamic foreground region, and the update frequency of its model parameters is equal to or greater than the preset video frame rate. The foreground determination module is used to determine the total foreground region for each image frame in the continuous image frame sequence based on the difference between the image frame and the static reference background image, and to filter out the dynamic foreground region from the total foreground region to obtain the static foreground region; The target screening module is used to perform target tracking on the static foreground regions corresponding to each of the image frames under spatiotemporal constraints, so as to screen out candidate abnormal targets; wherein, the spatiotemporal constraints include: spatially focusing the analysis range on the region of interest, and temporally requiring the target to remain stationary for a duration exceeding a preset threshold. An anomaly identification module is used to respond to the selection of candidate anomaly targets by inputting the static reference background image and the image frame containing the candidate anomaly targets to be verified into a multimodal large model for semantic analysis, and to obtain an anomaly identification result as to whether the candidate anomaly target is an abnormal accumulation or whether it causes channel blockage.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the computer room anomaly identification method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the computer room anomaly identification method as described in any one of claims 1 to 9.

Images