Machine room monitoring method and device based on electronic fence, terminal equipment and storage medium

Abstract

The invention discloses a machine room monitoring method and device based on an electronic fence, terminal equipment and a storage medium, and the method can continue to judge the on-off state of a target cabinet when a person is judged to be close to the target cabinet, thereby enabling the person to be in the on-off state when the person is close to the cabinet and the cabinet is in the on-off state. Recording that the target cabinet is operated and generating corresponding prompt information; if a person is close to the target cabinet and the on-off state of the target cabinet is an off state, when it is judged that the staying time of the person in the target cabinet exceeds a preset duration threshold value, an abnormal behavior can be warned, and corresponding first warning information is generated; according to the method, the personnel position, the cabinet position and the on-off state can be considered at the same time, the combination of the personnel position and the on-off state of the cabinet is realized so as to accurately identify a normal operation behavior or an abnormal stay behavior, and corresponding prompt information or alarm information is generated, so that the personnel behavior identification accuracy during machine room monitoring is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of computer room monitoring, and particularly to a computer room monitoring method, device, terminal device and storage medium based on an electronic fence. Background Art

[0002] In the power production site, it is often necessary to monitor the computer room, such as the process of real-time monitoring and management of various equipment, environmental parameters and personnel activities in the computer room to ensure the safety of the computer room operation. In the computer room, as a container for storing key infrastructure such as servers and network devices, the state of the cabinet and the operation behavior of the surrounding personnel need to be strictly monitored.

[0003] However, the traditional computer room monitoring means can only detect the position of the staff and the spatial distance between the staff and the cabinet through infrared sensors, and it cannot judge the switch state of the cabinet. When a person passes by the cabinet normally, the system may misjudge it as an operation or maintenance of the cabinet. Although the traditional monitoring technology can detect the movement and position of the person, it cannot accurately determine whether there is an interaction behavior between the person and the cabinet. Therefore, the traditional computer room monitoring means cannot combine the position of the person with the switch state of the cabinet, and thus cannot accurately identify the normal operation behavior or abnormal stay behavior of the person, resulting in a low accuracy rate of identifying the behavior of the person during the computer room monitoring. Summary of the Invention

[0004] Embodiments of the present invention provide a computer room monitoring method, device, terminal device and storage medium based on an electronic fence, which can simultaneously consider three key factors of the personnel position, the cabinet position and the switch state, and realize accurate judgment of the personnel operation behavior. Then, the position of the person can be combined with the switch state of the cabinet to accurately identify the normal operation behavior or abnormal stay behavior of the person, and generate corresponding prompt information or warning information, thereby improving the accuracy rate of identifying the behavior of the person during the computer room monitoring.

[0005] An embodiment of the present invention provides a computer room monitoring method based on an electronic fence, including:

[0006] Select a to-be-monitored area in the computer room monitoring image where there are a plurality of cabinets to generate an electronic fence; wherein, the electronic fence includes: a working area for monitoring the operation behavior of personnel.

[0007] Extract the personnel position of each person, the cabinet position of each cabinet and the switch state of each cabinet in the working area; wherein, the switch state includes: an open state or a closed state.

[0008] When it is determined that the distance difference between the personnel position of a person and the cabinet position of a cabinet is less than a preset distance threshold, mark the cabinet as a target cabinet, and then determine whether the switch state of the target cabinet is in the on state;

[0009] If so, generate a prompt message indicating that the target cabinet is being operated;

[0010] If not, when it is determined that the residence time of a person in the target cabinet exceeds a preset duration threshold, generate a first warning message indicating not to stay in the target cabinet for a long time.

[0011] Preferably, the electronic fence further includes: a normal area for monitoring the identity of personnel;

[0012] The selection of the area to be monitored in the computer room monitoring image with a number of cabinets to generate an electronic fence includes:

[0013] After graying and binarizing the computer room monitoring image, generate a computer room monitoring image to be recognized;

[0014] Input the computer room monitoring image to be recognized into a preset area division model, so that the area division model marks the area with a number of cabinets in the computer room monitoring image as the area to be monitored; extract the cabinet positions of each cabinet in the image in the area to be monitored; for each cabinet, use the cabinet position as the center and the preset distance threshold as the radius to generate a cabinet operation area corresponding to the cabinet; generate a working area for monitoring the operation behavior of personnel according to the cabinet operation areas corresponding to each cabinet; mark the area in the area to be monitored except for the working area as a normal area for monitoring the identity of personnel.

[0015] Preferably, it further includes:

[0016] Obtain the face image to be recognized of each person in the normal area;

[0017] For each person, compare the face image to be recognized of the person with each preset face image in the preset authorized list;

[0018] When it is determined that any face image to be recognized does not match the preset face image, generate a second warning message indicating that there is an unauthorized person entering the computer room.

[0019] Preferably, the working area corresponds to a monitoring time period; wherein, the monitoring time period is used to represent the cabinet maintenance time range during which personnel are required to maintain the cabinet;

[0020] The extraction of the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area includes:

[0021] When it is determined that the current moment belongs to the monitored time period, the personnel positions of each person, the cabinet positions of each cabinet, and the switch states of each cabinet in the working area are extracted.

[0022] Preferably, it further includes:

[0023] When it is determined that the current moment does not belong to the monitored time period and it is determined that there are people in the working area, a third warning message for indicating not to stay in the working area is generated.

[0024] Preferably, the switch states of each cabinet in the working area are extracted by the following method:

[0025] Each cabinet in the working area is identified through a preset cabinet recognition model, and a plurality of first bounding boxes corresponding to each cabinet are extracted; wherein, each first bounding box corresponds to a confidence level.

[0026] Non-maximum suppression operation is performed according to the confidence levels of each first bounding box, and a second bounding box corresponding to each cabinet is output.

[0027] For each cabinet, the cabinet door of the cabinet is identified according to the second bounding box corresponding to the cabinet to determine the switch state of the cabinet.

[0028] Preferably, it further includes:

[0029] During the training process of the preset cabinet recognition model, the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample is calculated according to the following formula:

[0030]

[0031] Wherein, GIOU is the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample, IOU is an index value for measuring the overlapping degree of the predicted bounding box and the actual bounding box, A is the area of the predicted bounding box corresponding to the cabinet sample, B is the area of the actual bounding box corresponding to the cabinet sample, C is the area of the smallest closed region that simultaneously includes the predicted bounding box and the actual bounding box, and A∪B represents the union area of the predicted bounding box and the actual bounding box.

[0032] Based on the above method embodiments, the present invention correspondingly provides apparatus embodiments.

[0033] An embodiment of the present invention provides a computer room monitoring device based on an electronic fence, including: an electronic fence generation module, a working area information extraction module, and a monitoring result generation module.

[0034] The electronic fence generation module is used to select a to-be-monitored area with a number of cabinets in the computer room monitoring image and generate an electronic fence; wherein, the electronic fence includes: a working area for monitoring the operation behavior of personnel.

[0035] The working area information extraction module is used to extract the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area; wherein, the switch state includes: the cabinet is in an open state or the cabinet is in a closed state.

[0036] The monitoring result generation module is used to mark the cabinet as a target cabinet when it is determined that the distance difference between the personnel position of a person and the cabinet position of a cabinet is less than a preset distance threshold, and then determine whether the switch state of the target cabinet is in an open state.

[0037] If so, generate a prompt message for indicating that the target cabinet is being operated.

[0038] If not, when it is determined that the staying time of a person at the target cabinet exceeds a preset duration threshold, generate a first warning message for indicating not to stay at the target cabinet for a long time.

[0039] Based on the above method embodiments, the present invention correspondingly provides terminal device embodiments.

[0040] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a computer room monitoring method based on an electronic fence as described in the above invention embodiments.

[0041] Based on the above method embodiments, the present invention correspondingly provides storage medium embodiments.

[0042] Another embodiment of the present invention provides a storage medium. The computer-readable storage medium includes a stored computer program. Wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute a computer room monitoring method based on an electronic fence as described in the above invention embodiments.

[0043] By implementing the present invention, the following beneficial effects are achieved:

[0044] An embodiment of the present invention provides a computer room monitoring method, device, terminal device and storage medium based on an electronic fence. After selecting a monitoring area in a computer room monitoring image, the present invention can generate a corresponding electronic fence, and the working area in the electronic fence can monitor the operation behavior of personnel, extract the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area. Thus, when it is determined that the distance difference between the personnel position and the cabinet position is less than a preset distance threshold, the cabinet is marked as a target cabinet, and then it is determined whether the switch state of the target cabinet is the on state. If so, a prompt message is generated; if not, an alarm message is generated when it is determined that the personnel stay time exceeds a preset duration threshold. Compared with the prior art, the present invention can mark a cabinet as a target cabinet when it is determined that a person is close to the cabinet, and continue to determine whether the switch state of the target cabinet is the on state. Thus, when a person is close to the cabinet and the cabinet is in the on state, it is recorded that the target cabinet is being operated and a corresponding prompt message is generated; if a person is close to the target cabinet and the switch state of the target cabinet is the off state, it is not recorded that the target cabinet is being operated, and when it is determined that the personnel stay time at the target cabinet exceeds a preset duration threshold, the abnormal behavior is warned and a corresponding first alarm message is generated. The present invention simultaneously considers three key factors: personnel position, cabinet position, and switch state, and realizes accurate judgment of personnel operation behavior. The present invention can combine the position of personnel with the switch state of the cabinet to accurately identify the normal operation behavior or abnormal stay behavior of personnel, and generate corresponding prompt messages or alarm messages, thereby improving the accuracy of personnel behavior recognition during computer room monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 FIG. is a schematic flowchart of a computer room monitoring method based on an electronic fence provided by an embodiment of the present invention.

[0046] Figure 2 FIG. is a schematic flowchart of a remote monitoring method provided by an embodiment of the present invention.

[0047] Figure 3 FIG. is a schematic structural diagram of a computer room monitoring device based on an electronic fence provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0049] AsFigure 1 As shown in the figure, it is a schematic flowchart of a computer room monitoring method based on an electronic fence provided by an embodiment of the present invention. The computer room monitoring method based on the electronic fence includes:

[0050] Step S1: Select a to-be-monitored area in the computer room monitoring image where there are several cabinets, and generate an electronic fence; wherein, the electronic fence includes: a working area for monitoring the operation behavior of personnel;

[0051] Step S2: Extract the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area; wherein, the switch state includes: an on state or an off state;

[0052] Step S3: When it is determined that the distance difference between the personnel position of a person and the cabinet position of a cabinet is less than a preset distance threshold, mark the cabinet as a target cabinet, and then determine whether the switch state of the target cabinet is the on state;

[0053] If so, generate a prompt message for indicating that the target cabinet is being operated;

[0054] If not, when it is determined that the staying time of a person at the target cabinet exceeds a preset duration threshold, generate a first warning message for indicating not to stay at the target cabinet for a long time.

[0055] Therefore, the present invention effectively solves the problem that the traditional technology cannot accurately identify operation behaviors through the dimensions of personnel position, cabinet position, cabinet state, and time. Since the prior art only relies on sensors such as infrared sensors to detect the spatial distance between personnel and cabinets, but cannot obtain the switch state of the cabinets. When a person passes by a closed cabinet normally, the system will misjudge it as an operation behavior; while when a person stays in front of a closed cabinet for a long time, the system cannot trigger an alarm. The present invention discriminates the distance difference between the personnel position of a person and the cabinet position of a cabinet, and can first identify the switch state of the cabinet when detecting that the person enters the warning distance. If the cabinet is on, it is directly determined as a normal operation; if it is off, the current staying time of the person can be calculated, such as starting to analyze the current staying time of the person through a time series based on LSTM. When the person stays for more than the preset duration, an alarm can be issued through the generated first warning message.

[0056] For step S1, in a preferred embodiment, the computer room monitoring image of the present invention can be collected by a camera deployed at the power production site, so as to monitor the computer room at the power production site to detect the normal operation behaviors and abnormal staying behaviors of personnel.

[0057] Schematically, in an embodiment of the present invention, a working area for monitoring the operation behavior of personnel may also be provided in the electronic fence, and a specified monitoring range may be delimited so as to effectively monitor and manage the interaction behavior between personnel and the cabinet within this range.

[0058] By setting and marking the working area in the electronic fence, the present invention can more specifically and accurately determine whether a person has entered to interact with the cabinet, so as to reduce false alarms and missed alarms and improve the recognition accuracy of personnel behavior.

[0059] For step S2, in a preferred embodiment, the working area corresponds to a monitoring time period; wherein, the monitoring time period is used to represent the cabinet maintenance time range during which personnel are required to maintain the cabinet;

[0060] Then, when the present invention determines that the current moment belongs to the monitoring time period, it can extract the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area, so that subsequent monitoring of the operation behavior of personnel can be carried out according to the personnel position, cabinet position, and switch state.

[0061] Furthermore, when the embodiment of the present invention determines that the current moment does not belong to the monitoring time period and determines that there are personnel in the working area, it generates a third warning message for indicating that personnel should not stay in the working area.

[0062] It can be understood that the present invention can start monitoring the operation behavior of personnel in the working area based on the personnel position, cabinet position, and switch state within the cabinet maintenance time range, and when it is not within the cabinet maintenance time range, it can warn the personnel who enter the working area at this time, realizing the monitoring of the entry and exit of personnel during non-maintenance time and preventing personnel from entering the working area during the non-cabinet maintenance time range to ensure the safety of the computer room.

[0063] Therefore, the present invention can not only monitor the personnel, cabinet position, and switch state in the working area in real time, but also introduce a monitoring time period, so as to realize the refined management of the computer room monitoring activities. Specifically, when it is determined that the current moment falls within the monitoring time period, the extraction and processing of various information in the working area will be immediately started, such as capturing the position of each person, the specific position of each cabinet, and their switch states, so as to accurately determine whether the personnel are performing normal maintenance operations or there are abnormal behaviors.

[0064] When it is determined that the current moment does not belong to the monitoring time period, that is, within the range of non-cabinet maintenance time, the monitoring strategy of the present invention will be adjusted. At this time, if a person is detected in the working area, the alarm mechanism will be immediately triggered to generate a third alarm message for characterizing "Do not stay in the working area", so as to strictly restrict the entry and exit of personnel during non-maintenance time, effectively avoid potential safety hazards, and ensure the safety of the computer room during non-working hours.

[0065] Therefore, the present invention can not only accurately monitor and record the operation behaviors of personnel within the cabinet maintenance time range, but also strictly monitor and manage the entry and exit of personnel during non-maintenance time. Through the dual-guarantee mechanism, the overall safety and operation and maintenance efficiency of the computer room can be improved.

[0066] For step S3, when the present invention detects that the distance between a certain person and a certain cabinet is less than a preset distance threshold, the cabinet will be marked as the target cabinet. Subsequently, by judging the switch state of the target cabinet, it is possible to accurately identify whether the personnel are performing normal operation behaviors.

[0067] If the target cabinet is in the open state, it is determined as a normal operation behavior, and a prompt message will be immediately generated so that the operation and maintenance personnel can understand the operation dynamics in the current computer room and respond in a timely manner to possible maintenance requirements.

[0068] If the target cabinet is in the closed state, the embodiment of the present invention will continue to monitor the staying time of the personnel in front of the target cabinet. Once the staying time exceeds the preset duration threshold, a corresponding first alarm message will be generated to prevent potential abnormal staying behaviors, such as unauthorized access or deliberate damage, etc.

[0069] Therefore, the present invention realizes the accurate judgment of the operation behaviors of personnel by automatically identifying and judging the operation behaviors of personnel and simultaneously considering three key factors: the position of personnel, the position of the cabinet, and the switch state, so as to accurately identify the normal operation behaviors or abnormal staying behaviors of personnel and generate corresponding prompt messages or alarm messages.

[0070] In a preferred embodiment, the electronic fence of the present invention further includes: a normal area for monitoring the identity of personnel;

[0071] When selecting a to-be-monitored area with several cabinets in the computer room monitoring image to generate an electronic fence, it specifically includes:

[0072] After performing gray-scale processing and binary processing on the computer room monitoring image, a to-be-recognized computer room monitoring image is generated;

[0073] Input the monitored computer room surveillance image to be recognized into a preset area division model, so that the area division model marks the area with several cabinets in the computer room surveillance image as the area to be monitored; extract the cabinet positions of each cabinet in the image in the area to be monitored; for each cabinet, generate a corresponding cabinet operation area with the cabinet position as the center and a preset distance threshold as the radius; generate a working area for monitoring the operation behavior of personnel according to the cabinet operation areas corresponding to each cabinet; mark the area other than the working area in the area to be monitored as the normal area for monitoring the identity of personnel.

[0074] Schematically, the present invention can extract the cabinet positions of each cabinet in the monitored computer room surveillance image through a trained area division model, and generate a working area according to the cabinet positions and a preset distance threshold, so as to demarcate the remaining area except the working area as the normal area.

[0075] Specifically, a circular area with the geometric center of each cabinet as the center and a preset distance threshold (such as 1.5 meters) as the radius is defined as the working area, which is the core activity range for personnel to operate equipment. And all the spaces in the area to be monitored except the working area are the main activity areas for non-operation behaviors such as personnel's daily passage and inspection, so these main activity areas can be marked as the normal area to realize the authorized identification of personnel's identity in the normal area.

[0076] Based on the above, the working area corresponds to a monitoring time period. It can be understood that the present invention can temporarily lift the access restriction to the working area within the planned monitoring time period. At this time, authorized personnel can enter the working area through the normal area (without triggering an alarm), but still monitor the behavior of authorized personnel in the working area by target tracking of personnel and identification of the cabinet status, so that authorized personnel can perform normal cabinet maintenance operations within the specified time, avoiding false alarms caused by continuous restriction of the electronic fence.

[0077] Schematically, assume that a certain computer room plans to perform maintenance from 9:00 to 11:00. Before 9:00, the electronic fence of the working area is in the "open" state, that is, personnel are prohibited from entering; during 9:00 - 11:00, the electronic fence of the working area is "closed", and after authorized personnel are authorized by face recognition in the normal area, they can enter the working area. At this time, it can be monitored whether personnel are operating at the specified cabinet to prevent entering other areas by mistake, or whether the cabinet is being operated normally. After 11:00: the electronic fence returns to the "open" state, and at this time, any personnel are prohibited from entering.

[0078] Therefore, through the above-mentioned monitoring time period setting and the monitoring operation of the working area, the present invention not only ensures the operation flexibility but also improves the safety through dynamic restriction and real-time monitoring.

[0079] In a preferred embodiment, the monitoring process for the normal area may be as follows:

[0080] Obtain the to-be-identified face images of each person in the normal area;

[0081] For each person, compare the to-be-identified face image of the person with each preset face image in the preset authorization list;

[0082] When it is determined that any to-be-identified face image does not match the preset face image, generate a second warning message for indicating that an unauthorized person has entered the computer room.

[0083] It can be understood that during the monitoring process of the normal area, by comparing the to-be-identified face images of the personnel with each preset face image in the preset authorization list, the entry of unauthorized personnel can be detected and warned in a timely manner, so that the security management of the computer room can be strengthened and unauthorized personnel can be prevented from entering the computer room for illegal operations.

[0084] Therefore, the embodiments of the present invention accurately divide the working area and the normal area. In the working area, the operation behavior is mainly monitored; in the normal area, the identity recognition is mainly carried out. Then, the present invention can more accurately identify the operation behavior of the personnel in the working area and the identity of the personnel in the normal area, reduce false alarms and missed detections, and improve the accuracy of monitoring.

[0085] In a preferred embodiment, the training process of the area division model is as follows: intercept and collect the video data of the cameras deployed on-site, and after preprocessing the images, divide them into a training set and a test set. The training set is used to construct the area division model and perform model training on the area division model to obtain model parameters; the test set is used to verify the obtained model parameters. Finally, the trained model can be input into the intelligent video monitoring system so that when performing video monitoring, the video monitoring screen can be divided into areas, thereby realizing the generation of an electronic fence and dividing the working area and the normal area in the electronic fence.

[0086] Furthermore, the embodiments of the present invention can also identify personnel information, cabinet nameplates and switch states based on the YOLOv5 algorithm, and introduce the DeepSORT algorithm to perform secondary verification on the work content and staff in the working area, making the detection results in the working area more accurate and reducing the missed detection situation when multiple staff are working in the same working area, ensuring the compliance operation of the on-site staff.

[0087] In a preferred embodiment, when grayscale processing and binarization processing are performed on the computer room monitoring image, the weighted average method can be selected to perform grayscale processing on the frame image, and the threshold method can be selected to perform binarization processing on the grayscale frame image, so as to achieve background separation, which is convenient for subsequent selection and marking of the area of the electronic fence.

[0088] And the above-mentioned weighted average method is specifically as follows:

[0089] Gray(x,y) = 0.3×R(x,y) + 0.59×G(x,y) + 0.11×B(x,y);

[0090] Where Gray(x,y) represents the image after grayscale processing, and R(x,y), G(x,y), and B(x,y) respectively represent the sensitivity weights of three different colors of red, blue, and green.

[0091] The above-mentioned threshold method is specifically as follows:

[0092]

[0093] Where f(x,y) represents the binarized image, g(x,y) represents the image after grayscale processing, and T is a preset threshold.

[0094] In a preferred embodiment, the switch state of each cabinet in the working area can be extracted in the following manner:

[0095] Through the trained cabinet recognition model, each cabinet in the working area is recognized, and a number of first bounding boxes corresponding to each cabinet are extracted; among them, each first bounding box corresponds to a confidence level;

[0096] Perform non-maximum suppression operation according to the confidence level of each first bounding box, and output the second bounding box corresponding to each cabinet;

[0097] For each cabinet, the cabinet door of the cabinet is recognized according to the second bounding box corresponding to the cabinet to determine the switch state of the cabinet.

[0098] Further, during the training process of the preset cabinet recognition model, the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample is calculated according to the following formula:

[0099]

[0100] Among them, GIOU is the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample. IOU is the index value used to measure the overlapping degree between the predicted bounding box and the actual bounding box. A is the area of the predicted bounding box corresponding to the cabinet sample. B is the area of the actual bounding box corresponding to the cabinet sample. C is the area of the smallest closed region that contains both the predicted bounding box and the actual bounding box. A∪B represents the union area of the predicted bounding box and the actual bounding box.

[0101] It can be understood that the present invention can use the trained cabinet recognition model to recognize the cabinets in the working area. During the recognition process, the model will output several first bounding boxes for each cabinet, and each bounding box corresponds to a confidence level. Schematically, this confidence level represents the probability that the bounding box accurately frames the cabinet. The higher the confidence level, the more likely it is that this bounding box can accurately frame the cabinet.

[0102] Furthermore, through non-maximum suppression, redundant bounding boxes can be removed, and only the bounding box that is most likely to represent the real target is retained. The non-maximum suppression operation is as follows: First, set a threshold, and label each predicted box with a confidence level. Based on the predicted box with the highest confidence level among all the predicted boxes, traverse the remaining candidate boxes, calculate the IOU value between the remaining candidate boxes and the selected predicted box. If it is greater than the initially set threshold, then it is removed. Then repeat the above operation for the predicted box with the second highest confidence level, and keep repeating the screening operation until there are no two predicted boxes that are highly similar. Output the final result, so that redundant predicted boxes can be removed.

[0103] Schematically, for the multiple first bounding boxes of each cabinet, they are sorted according to their confidence levels, and the bounding box with the highest confidence level is selected as the benchmark. Then calculate the overlapping degree between other bounding boxes and this benchmark bounding box. If the IOU of a certain bounding box and the benchmark bounding box exceeds the preset threshold, it is considered that these two bounding boxes are very likely to frame the same target. At this time, the bounding box with the lower confidence level will be removed. After the NMS operation, each cabinet will output a corresponding second bounding box, and this second bounding box is the one that can most accurately represent the position of this cabinet.

[0104] Finally, for each cabinet, use its corresponding second bounding box to recognize the cabinet door. Then, by further analyzing the image features within the second bounding box, such as the shape, color, texture, etc. of the cabinet door, it can be determined whether the cabinet door is in the open state or the closed state, and the switch state corresponding to each cabinet in the computer room can be obtained.

[0105] Furthermore, during the model training process, the loss function can be used to measure the difference between the model prediction result and the true result. Therefore, in the embodiments of the present invention, by minimizing the loss function, the model can continuously adjust its own parameters to make the prediction result closer and closer to the true result. In the training process of identifying the switch state of the cabinet in the present invention, by calculating the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample, the gap between the current prediction result and the true result can be clarified. Then, an optimization algorithm is used, such as adjusting the parameters of the model by stochastic gradient descent, to reduce the loss function value, thereby improving the performance of the model.

[0106] Furthermore, in order to improve the recognition accuracy of the cabinet recognition model of the present invention, the present invention can also solve the problem of insufficient training data and improve the adaptability of the model to complex lighting, occlusion and other scenarios in the following ways:

[0107] Collect four images of the monitoring area, adjust the brightness, contrast, and hue information of the images through data augmentation, and perform random shearing, scaling, and rotation operations on the images to increase the diversity of the training images; place the four adjusted images in the positions where the first one is placed in the upper left, the second one is placed in the upper right, the third one is placed in the lower left, and the fourth one is placed in the lower right, and splice them into a newly spliced image, and the newly spliced image still contains the marked box information; schematically, there is an obvious edge dividing line in the middle of the newly spliced image. If there is a situation where the marked box is covered during splicing, and when the marked box in a single image exceeds the edge dividing line between two adjacent spliced images, the part exceeding the edge dividing line can be edge-processed. Thus, the number of samples can be increased through splicing, so as to improve the detection accuracy of the cabinet recognition model for the target.

[0108] Train the adjusted new image with the cabinet recognition model, so that the predicted box can be obtained by using the adaptive anchor box, compare the predicted box with the true box of the object, and calculate the loss value of the cabinet recognition model.

[0109] Specifically, the working process of the adaptive anchor box adopted by the present invention is as follows: First, extract the width and height information of each target from the dataset, scale the targets in each image to a specified size in equal proportion to ensure that the width and height of the targets meet the specified size, then convert the bounding boxes of the targets from relative coordinates to absolute coordinates, multiply by the scaled width and height, and then retain the bounding boxes with both width and height greater than or equal to two pixels. Cluster the bounding boxes into n anchor boxes through the k-means clustering algorithm, and randomly mutate the width and height of the anchor boxes, and evaluate and select the optimal anchor box as the predicted box corresponding to the cabinet.

[0110] Therefore, in the embodiments of the present invention, the width and height information of the target is extracted from the dataset and scaled to a specified size, ensuring that the targets in different images have a unified scale, enabling the model to perform feature extraction and matching more accurately when processing cabinets of different sizes. And by converting the bounding box from relative coordinates to absolute coordinates and retaining the bounding boxes whose width and height are both greater than or equal to two pixels, some invalid or too-small target information can be filtered out, reducing the interference of noise on the model.

[0111] In a preferred embodiment, the present invention introduces the DeepSORT algorithm to achieve target tracking, so as to meet the safety management requirements of the production site, ensure the working positions of personnel, and prevent personnel from entering the working area during non-monitoring time periods.

[0112] The process of target tracking is as follows: Input the original frame image of the video, use the YOLOv5 target detection algorithm to obtain the prediction box of the object (such as personnel), and extract the features of the target corresponding to the prediction box; schematically, the features of the personnel in the prediction box can be extracted, for example, the appearance features such as the clothing and body shape of the personnel, or the dynamic features such as the movement speed and direction, so as to quickly locate the position of the personnel in each frame.

[0113] Calculate the matching degree between the targets in the previous and current frames through similarity calculation, that is, calculate the similarity of the images by measuring the distance between two frame images, and finally judge the similarity degree of the images according to the calculated value, and then use the Hungarian algorithm to associate the data and assign an IP to each target.

[0114] Schematically, in the embodiments of the present invention, the similarity is calculated by measuring the distance between the personnel in the previous and current frame images. If the feature similarity of the two frame targets is high, it indicates the same target; otherwise, it may be a new target, thus solving the problem of "whether the targets in the previous and current frames are the same" to ensure the continuity of tracking.

[0115] Based on the similarity calculation result and with the goal of optimal matching through the Hungarian algorithm, the current frame target is associated with the historical tracking target to avoid misbinding the tracking trajectories of different personnel. Then, the embodiments of the present invention can assign an independent ID (such as "Person A", "Person B") to each target and continuously track its movement in the video frame, so that in the production site, the action path of the personnel can be traced through the ID to determine whether they enter the working area during non-monitoring time periods.

[0116] In a preferred embodiment, as Figure 2 shown in the monitoring flow chart, the embodiments of the present invention may further include the following steps:

[0117] Step 1. Personnel information entry: Collect personal information of all on-site workers, including face information, identity information, and affiliated departments, establish an information database for employees, classify department and personnel information, and store it in the database of the intelligent video monitoring system;

[0118] Step 2. Model input into the monitoring system: Intercept and collect video data of cameras deployed on-site. After preprocessing the images, divide them into a training set and a test set. The training set is used to construct an electronic fence model for the electronic cabinet area in the substation and train the model to obtain model parameters. The test set is used to verify the obtained model parameters. Input the model into the intelligent video monitoring system, divide the monitored video images, and further achieve the division of the working area;

[0119] Step 3. Electronic fence restriction: Generate an electronic fence for the electronic cabinets in the substation, and divide the electronic fence into a working area and a normal area. The electronic fence formed by the normal area is used to continuously monitor the entry of personnel. Once a person entering the area without permission is detected, an alarm is immediately triggered. The electronic fence of the working area will be briefly closed according to the working time arrangement. After the staff passes through the normal area normally, they enter the permitted working area to carry out operations, and the electronic fence of the working area starts to open;

[0120] Step 4. Status recognition: Use the YOLOv5 algorithm to identify the opening and closing status of the cabinet doors in the correct working area, improve the accuracy of positioning the cabinets that personnel can operate, and prevent misjudgment caused by multiple people operating in the working area at the same time;

[0121] Step 5. Target tracking: Introduce the DeepSORT algorithm to achieve target tracking, so as to meet the safety management requirements of the production site, ensure the working positions of personnel, and prevent personnel from entering areas outside the working area.

[0122] Therefore, in the embodiment of the present invention, by establishing a multi-dimensional work detection area based on video monitoring and electronic fences, the detection results of the working area are more accurate. By dividing the video area collected by the video monitoring system, setting corresponding working areas, verifying the work permits of personnel entering the area, detecting unauthorized personnel entering the area and triggering an alarm in time to remind on-site workers to operate on the correct cabinets, and by reading information such as work content, personnel, and location in the system, it can be synchronized to the video monitoring system, realizing real-time change of the virtual electronic fence according to on-site production work and meeting the actual need for accurate alarm.

[0123] As Figure 3 shown, based on the above embodiments of various electronic fence-based computer room monitoring methods, the present invention correspondingly provides device embodiments;

[0124] An embodiment of the present invention provides a computer room monitoring device based on an electronic fence, including: an electronic fence generation module, a working area information extraction module, and a monitoring result generation module;

[0125] The electronic fence generation module is used to select a to-be-monitored area with a plurality of cabinets in the computer room monitoring image and generate an electronic fence; wherein, the electronic fence includes: a working area for monitoring the operation behavior of personnel.

[0126] The working area information extraction module is used to extract the personnel position of each person, the cabinet position of each cabinet, and the switch state of each cabinet in the working area; wherein, the switch state includes: the cabinet is in an open state or the cabinet is in a closed state.

[0127] The monitoring result generation module is used to mark the cabinet as a target cabinet when it is determined that the distance difference between the personnel position of a person and the cabinet position of a cabinet is less than a preset distance threshold, and then determine whether the switch state of the target cabinet is in an open state;

[0128] If so, generate a prompt message for indicating that the target cabinet is being operated;

[0129] If not, when it is determined that the staying time of a person at the target cabinet exceeds a preset duration threshold, generate a first warning message for indicating not to stay at the target cabinet for a long time.

[0130] It should be noted that the device embodiments described above are only illustrative. The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical modules. They may be located in one place or distributed to multiple network modules. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement without creative efforts.

[0131] Those skilled in the art can clearly understand that for the convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the foregoing method embodiments and will not be described in detail here.

[0132] Based on the above embodiments of various computer room monitoring methods based on an electronic fence, the present invention correspondingly provides an embodiment of a terminal device.

[0133] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for monitoring a computer room based on an electronic fence described in any method item embodiment of the present invention.

[0134] The terminal device may be a computing terminal device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0135] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor, or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device, connecting various parts of the entire terminal device through various interfaces and lines.

[0136] The memory may be used to store the computer program. The processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0137] Based on the above embodiments of various methods for monitoring a computer room based on an electronic fence, the present invention correspondingly provides an embodiment of a storage medium item.

[0138] An embodiment of the present invention provides a storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute a method for monitoring a computer room based on an electronic fence according to any method embodiment of the present invention.

[0139] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0140] The above is the preferred embodiment of the present invention. It should be pointed out that for those of ordinary skill in the art in the technical field of the present invention, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A computer room monitoring method based on electronic fence, characterized in that: include: An area to be monitored with several cabinets in the monitoring image of the computer room is selected to generate an electronic fence; wherein the electronic fence includes: a working area for monitoring the operation behavior of personnel; Extracting the personnel position of each person in the work area, the cabinet position of each cabinet, and the switch state of each cabinet; wherein the switch state includes: an open state or a closed state; When it is determined that the distance difference between a person's position and a cabinet position of a cabinet is less than a preset distance threshold, marking the cabinet as a target cabinet, and then determining whether the switch state of the target cabinet is an on state; If yes, a prompt message is generated to indicate that the target cabinet is being operated. If not, when it is determined that the stay time of a person in the target cabinet exceeds the preset time threshold, a first warning message is generated to indicate that the person should not stay in the target cabinet for a long time.

2. A computer room monitoring method based on electronic fence as claimed in claim 1, characterized in that: The electronic fence also includes: a normal area for monitoring the identity of personnel; The step of selecting a monitored area with a plurality of cabinets in the monitoring image of the computer room and generating an electronic fence includes: After grayscale processing and binarization processing are performed on the computer room monitoring image, a computer room monitoring image to be identified is generated; The computer room monitoring image to be identified is input into a preset area division model, so that the area division model marks the area with several cabinets in the computer room monitoring image as the area to be monitored; the cabinet position of each cabinet in the image is extracted from the area to be monitored; for each cabinet, a cabinet operation area corresponding to the cabinet is generated with the cabinet position as the center and a preset distance threshold as the radius; based on the cabinet operation area corresponding to each cabinet, a working area for monitoring personnel operation behavior is generated; and the area except the working area in the area to be monitored is marked as a normal area for monitoring personnel identity.

3. A computer room monitoring method based on electronic fence as claimed in claim 2, characterized in that: Also includes: Obtaining the face image to be identified of each person in the normal area; For each person, the facial image to be identified is compared with each preset facial image in the preset authorization list; When it is determined that any face image to be identified does not match the preset face image, a second alarm information is generated to indicate that an unauthorized person has entered the computer room.

4. A computer room monitoring method based on electronic fence as claimed in claim 1, characterized in that: The working area corresponds to a monitoring time period; wherein the monitoring time period is used to represent the cabinet maintenance time range during which personnel are required to perform maintenance on the cabinet; The extracting the personnel position of each person in the working area, the cabinet position of each cabinet, and the switch status of each cabinet includes: When it is determined that the current time is within the monitoring time period, the personnel position of each person in the working area, the cabinet position of each cabinet, and the switch status of each cabinet are extracted.

5. A computer room monitoring method based on electronic fence as claimed in claim 4, characterized in that: Also includes: When it is determined that the current time is not within the monitoring time period and when it is determined that there are people in the working area, a third warning message is generated to indicate that one should not stay in the working area.

6. A computer room monitoring method based on electronic fence as claimed in claim 5, characterized in that: The switch status of each cabinet in the working area is extracted by: Using a preset cabinet recognition model, each cabinet in the working area is identified, and a plurality of first bounding boxes corresponding to each cabinet are extracted; wherein each first bounding box corresponds to a confidence level; Perform a non-maximum suppression operation according to the confidence of each first bounding box, and output a second bounding box corresponding to each cabinet; For each cabinet, the cabinet door is identified according to the second boundary frame corresponding to the cabinet, and the switch state of the cabinet is determined.

7. A computer room monitoring method based on electronic fence as claimed in claim 6, characterized in that: Also includes: During the training process of the preset cabinet recognition model, the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample is calculated according to the following formula: Among them, GIOU is the loss function value between the predicted bounding box corresponding to the cabinet sample in the training sample and the actual bounding box corresponding to the cabinet sample, IOU is the indicator value used to measure the overlap between the predicted bounding box and the actual bounding box, A is the area of ​​the predicted bounding box corresponding to the cabinet sample, B is the area of ​​the actual bounding box corresponding to the cabinet sample, C is the area of ​​the minimum closure area that contains both the predicted bounding box and the actual bounding box, and A∪B represents the union area of ​​the predicted bounding box and the actual bounding box.

8. A computer room monitoring device based on electronic fence, characterized in that: include: Electronic fence generation module, work area information extraction module and monitoring result generation module; The electronic fence generation module is used to select a to-be-monitored area with several cabinets in the monitoring image of the computer room, and generate an electronic fence; wherein the electronic fence includes: a working area for monitoring the operation behavior of personnel; The work area information extraction module is used to extract the personnel position of each person in the work area, the cabinet position of each cabinet, and the switch state of each cabinet; wherein the switch state includes: the cabinet is in the open state or the cabinet is in the closed state; The monitoring result generating module is used to mark a cabinet as a target cabinet when it is determined that the distance difference between a personnel position of a person and a cabinet position of a cabinet is less than a preset distance threshold, and then determine whether the switch state of the target cabinet is an open state; If yes, a prompt message is generated to indicate that the target cabinet is being operated. If not, when it is determined that the stay time of a person in the target cabinet exceeds the preset time threshold, a first warning message is generated to indicate that the person should not stay in the target cabinet for a long time.

9. A terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, it implements a computer room monitoring method based on electronic fence as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the computer room monitoring method based on electronic fence as described in any one of claims 1 to 7.

Images