A cabinet U position detection method based on two-stage visual recognition and perspective correction

Abstract

The application discloses a cabinet U position detection method based on two-stage visual identification and perspective correction, and belongs to the technical field of computer vision and artificial intelligence. The method comprises the following steps: acquiring cabinet image data through a camera arranged in a computer room; adopting a first-stage visual identification model to detect irregular cabinets from the image and output four vertex coordinates of the cabinet; performing perspective correction on the image according to the vertex coordinates to obtain a standard rectangular front image of the cabinet; inputting the corrected image into a second-stage visual identification model to identify the U position occupation state and construct a U position state matrix; calculating the cabinet vacancy rate based on the state matrix and displaying the detection result in the form of a heat map; and submitting the detection result with a confidence degree lower than a preset threshold to manual review and using the detection result for incremental training and optimization of the model. The application has the advantages of high precision, low cost, strong generalization, strong real-time performance and sustainable optimization, and can be widely applied to intelligent monitoring of cabinet U positions in data centers under complex scenes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and artificial intelligence, and in particular to a cabinet U position detection method based on two-stage visual recognition and perspective correction. BACKGROUND

[0002] With the continuous expansion of data center room construction scale, the utilization efficiency of cabinet U position resources has become an important factor affecting the operation and deployment efficiency of IT assets. Currently, the collection of cabinet U position occupation information mainly relies on manual inspection or the deployment of infrared, pressure, current and other sensors to realize state detection. However, these methods generally have the following shortcomings:

[0003] Dependence on manual: traditional manual inventory method is low in efficiency and prone to errors, and it is difficult to realize dynamic and high-frequency update; high deployment cost: using physical sensor scheme requires installing equipment for each U position, which is complicated, high in cost and difficult to maintain; poor robustness: due to the actual scenarios such as inclined installation, shielding and shooting angle deviation of the cabinet, existing image recognition methods often cannot stably output standardized images, resulting in insufficient U position detection accuracy; weak model generalization: existing image algorithms mostly use single-stage detection or end-to-end scheme, which is difficult to balance the dual tasks of "overall cabinet recognition" and "fine-grained U position positioning"; lack of closed-loop optimization mechanism: once the traditional visual model is trained, it is fixed, lacks manual participation and self-learning mechanism for low confidence results, and cannot adapt to data drift and complex scene expansion in long-term operation.

[0004] Therefore, how to design a cabinet U position detection method with high precision, low deployment cost, adaptability to complex visual angle environment, and online learning and continuous optimization ability has become a core problem that needs to be solved in the current technical field. SUMMARY

[0005] The present application aims to overcome the problems of low accuracy, poor environmental adaptability and high deployment and maintenance cost in the existing technology in cabinet U position recognition and vacancy rate evaluation, and proposes a cabinet U position detection method based on two-stage visual recognition and perspective correction.

[0006] In order to achieve the above-mentioned purpose, the present application realizes the technical scheme as follows:

[0007] A cabinet U position detection method based on two-stage visual recognition and perspective correction, comprising the following steps:

[0008] I. Obtain cabinet image data

[0009] Obtain cabinet image data, which is collected by a camera deployed at the top or middle of the computer room, or uploaded by a manual inspection terminal;

[0010] Preferably, training data preparation, labeling and training

[0011] Data acquisition and preprocessing are performed on the cabinets in the machine room by artificial or machine room inherent cameras, and two models are trained in stages: an irregular cabinet target detection model (hereinafter referred to as model one) and a rectified cabinet empty U position target detection model (hereinafter referred to as model two).

[0012] The original data is labeled.

[0013] For model one, the coordinates of the four corners of the cabinet outline quadrilateral (x1, y1, x2, y2, x3, y3, x4, y4) are labeled. When the cabinet is complete and visible, the actual four corners are labeled. When partially obscured, the complete outline is inferred from the visible part.

[0014] For model two, the vacant U position and non-vacant U position regions are labeled, and the labels are divided into two types: vacant and non-vacant. Among them, vacant is a continuous 2U and above uncovered area (such as U5-U6), and non-vacant is a device covered area, with a strict boundary alignment to the grid line.

[0015] Next, the training data is preprocessed, including geometric distortion, color disturbance, and occlusion simulation, to expand and enhance the data set.

[0016] For model two, Mosaic enhancement is performed, and at least one vacant sample is forced to be included when 4 images are spliced to alleviate the class imbalance problem.

[0017] The preprocessed data is used for training.

[0018] For model one, the rotation algorithm is used, and the IoU calculation is based on the minimum circumscribed rotating rectangle. In the inference part, the output inference frame is an irregular quadrilateral, and the coordinates of the four corners of each frame (x1, y1, x2, y2, x3, y3, x4, y4) are regressed.

[0019] The loss function is composed of traditional classification loss, coordinate loss and confidence loss. The improved version calculates the regression error of each vertex, the rotation angle error and the scale loss separately.

[0020] For model two, CoordinateAttention (CA) is inserted into the Feature Pyramid Network (FPN) layer to enhance the positioning ability of long and narrow devices (such as 1U servers)

[0021] The regression loss uses EIoULoss to optimize the consistency of the boundary frame length and width.

[0022] The model after training is stored locally for subsequent use; after deployment in the computer room, incremental training is performed every certain period of time (for example, one week) according to the artificial review results to improve the model accuracy and adapt to the actual situation of the computer room.

[0023] II. First-stage visual recognition model: cabinet detection

[0024] The first-stage visual recognition model is used to detect irregular cabinets in image data, and the four vertex coordinates of the cabinet are output.

[0025] Specifically, an improved target detection algorithm is used to detect the cabinet region for the first time and locate the irregularly distributed cabinet target.

[0026] The photos taken by the camera or manually are obtained, the photo data is sorted, and the sorted data is input into the queue for detection.

[0027] The four vertices of the picture are predicted, and the results with a confidence greater than a threshold are retained, and the prediction results are saved locally.

[0028] III. Image perspective correction

[0029] Based on the four vertex coordinates, image perspective correction is performed to generate standard rectangular cabinet image data.

[0030] Specifically, the detected cabinet region is geometrically corrected to eliminate perspective distortion caused by the shooting angle.

[0031] The four vertices of the irregular target detection are obtained, and the maximum width and height of the recognized irregular quadrilateral are calculated as the total width and height of the target picture.

[0032] The perspective transformation is performed according to the size of the target picture using cv2.getPerspectiveTransform and cv2.warpPerspective, and the new picture is stored as wraped_img.

[0033] After perspective transformation and cropping, all pictures are stored locally for the second target detection.

[0034] IV. Second-stage visual recognition model: U-bit state detection

[0035] The corrected image data is input into the second-stage visual recognition model to recognize the occupancy state of the U-bit and construct a U-bit state matrix.

[0036] Specifically, based on the target detection model, the corrected cabinet image is subjected to fine target detection to count the U-bit occupancy state.

[0037] Determine the detection grid size, because of the vacancy 2U and above is calculated as vacancy, need to define a grid to detect all vacant target is false detection in a sliding manner.

[0038] For each picture, target detection inference will get the results labeled as vacant and non-vacant, and return the confidence information of each picture.

[0039] Filter the detection results, keep the prediction box with confidence greater than the threshold (for example, 0.6), balance the recall rate and false positive rate; Weighted NMS (weight = confidence x IoU) is performed on the overlapping box to suppress repeated detection.

[0040] Build U-bit state matrix, initialize all vacant matrix , 1 indicates vacant, 0 indicates occupied. For each non-vacant detection box, calculate the covered U-bit range, and mark 0 at the corresponding U-bit position. If the device crosses the U-bit, round up.

[0041] For each vacant detection box, verify whether the covered U-bit is greater than the sliding grid size, and whether it is all 1 in the matrix S. If it passes, the vacant interval is retained.

[0042] Five, vacancy rate calculation

[0043] Based on the U-bit state matrix, the vacancy rate of the cabinet is calculated;

[0044] Six, detection result output and visualization

[0045] The detection results are output to the computer room management platform and visualized in the form of a heat map.

[0046] Seven, model optimization and active learning mechanism

[0047] The detection results with confidence lower than the preset threshold are manually reviewed, and the results are used for incremental training and continuous optimization of the model.

[0048] Preferably, the first stage visual recognition model adopts a structure supporting irregular target detection, and outputs four vertex coordinates of an irregular quadrilateral. In the training process, the following combined weighted loss function is used for optimization, the expression is:

[0049] ;

[0050] Wherein: 、 、 、 、 is the weight coefficient of each loss term, used to balance the magnitude of different losses. is a classification error; is a vertex regression error; is a rotation angle error; is a scale error; is a confidence error;

[0051] wherein, The expression is:

[0052] ;

[0053] wherein: represents the number of samples in a batch, i.e. the number of images input at one time for training; represents the i-th sample being calculated; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box; ; represents the i-th vertex of the bounding box.

[0054] The expression is:

[0055] ;

[0056] wherein: represents the number of samples in a batch, i.e. the number of images input at one time for training; represents the i-th sample being calculated; represents the i-th target of the model prediction; represents the i-th target of the model prediction; represents the i-th target of the model prediction; represents the i-th target of the model prediction;

[0057] The expression is:

[0058] ;

[0059] wherein: represents the dimension of the scale, i.e. the width and height of the target; represents the predicted scale; represents the real scale.

[0060] In particular, the vertex position loss is to calculate the regression error of each vertex of the irregular quadrilateral box; the angle loss takes into account the influence of the rotation angle of the target, and increases the calculation of the angle error of each vertex relative to the center point; the scale loss considers the difference between the actual area of the polygon and the area of the circumscribed rectangle of the box.

[0061] For model two, CoordinateAttention (CA) is inserted into the Feature Pyramid Network (FPN) layer to enhance the positioning ability of slender devices such as 1U servers:

[0062]

[0063] wherein GAP is a global average pooling, GMP is a global maximum pooling, is an element-wise multiplication.

[0064] The classification loss adopts Focalloss to alleviate the class imbalance problem:

[0065]

[0066] wherein, is a class balance coefficient to adjust the loss weight of positive and negative samples. is a focus parameter, the greater the loss attenuation of easy samples is more obvious, and the model pays more attention to difficult samples; is a direction to adjust the prediction probability, which always represents the prediction probability of the correct class by the model.

[0067] Preferably, the regression part in the combined weighted loss function further adopts an extended intersection over union loss function, and the calculation formula is as follows:

[0068] ;

[0069] wherein, is an intersection over union, which measures the overlap degree of two boxes; represents a center point distance penalty term, that is, the squared Euclidean distance between the center point of the predicted box and the center point of the real box; represents the squared length of the diagonal of the minimum closed region; and represent the squared width and height of the minimum closed region.

[0070] Through artificial inspection and photographing, the image is uploaded to the computer room management platform for analysis, or the camera deployed in the computer room is used to obtain the cabinet image in real time, and the image is preprocessed. Further, the image preprocessing technical points include Gaussian filter denoising, adaptive histogram equalization, etc.

[0071] In particular, Gaussian filter denoising is to smooth the original image by using a Gaussian kernel function, and eliminate the interference caused by uneven illumination or sensor noise. The kernel function formula is:

[0072] ;

[0073] wherein, is the weight value of the Gaussian filter at , represents the position coordinate of the current pixel relative to the center of the filter, is the standard deviation of the Gaussian distribution, which controls the smoothing degree of the filter, is an exponential function used to generate the bell-shaped curve of the Gaussian distribution, is a normalization coefficient to ensure that the sum of the Gaussian kernel is 1.

[0074] In particular, the adaptive histogram equalization enhances the contrast of the image by the CLAHE algorithm, and improves the detail definition of the cabinet edge and the U-position equipment. By dividing the image into local sub-blocks, the contrast enhancement amplitude of each sub-block is limited to avoid noise amplification, and the transition between blocks is smoothed by bilinear interpolation, thereby effectively suppressing the generation of over-enhancement and artifacts while significantly improving the local details and contrast of the image.

[0075] Preferably, the acquisition device for acquiring cabinet image data includes cameras deployed on the top and middle of the machine room to obtain overhead and horizontal dual-view images, and an artificial inspection terminal equipped with a wide-angle anti-distortion lens; the acquisition device supports image resolution that meets the whole-cabinet imaging requirements, and has the data synchronization capability of timed acquisition and automatic uploading to the platform.

[0076] Preferably, the image perspective correction includes:

[0077] For the irregular quadrilateral formed by the four vertices obtained by the first-stage target detection, the width and height mapped to the standard rectangle are calculated according to the vertices, which are used to determine the target image size;

[0078] The getPerspectiveTransform and warpPerspective functions of OpenCV are used to perform geometric transformation to eliminate the perspective distortion caused by the shooting angle;

[0079] The transformation result is cropped into a standard rectangular cabinet image warped_img, which is stored locally for target detection processing by the second-stage model.

[0080] Preferably, based on the target detection model, fine target detection is performed on the corrected cabinet image to count the U-position occupancy state.

[0081] The detection grid size is determined, and only the vacancy of 2U or more is considered as vacancy. A grid is defined to detect whether all vacancy targets are false detections in a sliding manner.

[0082] According to the actual height (unit: pixels) of the corrected cabinet image and the standard U-position height (44.45mm), the pixel height corresponding to each U is calculated:

[0083] ;

[0084] wherein: represents each pixel occupied by a rack unit; is a standard height of a rack unit; is the height of the entire rack, and only when the length of the continuous uncovered area is not less than 2U, the area is considered as empty.

[0085] Further, the prediction confidence output by the second-stage visual recognition model is calculated, and if the confidence is higher than 0.6, the confidence is directly adopted;

[0086] If the confidence is between 0.4 and 0.6, the corresponding detection result is submitted for manual review;

[0087] The result after review is recorded and added to the training sample set for subsequent incremental training of the model;

[0088] The empty state is determined by a sliding window operation, the window covers not less than 2U positions, and if the prediction results in the window range are all empty, the whole area is marked as empty.

[0089] Preferably, based on the U position occupancy state recognition result output by the second-stage visual recognition model, the empty state determination by the sliding window operation comprises:

[0090] a state vector with a length of is constructed, wherein 0 represents a rack unit is occupied, and 1 represents empty;

[0091] When the device detection area spans multiple rack units, the actual occupied U number is rounded up, and the corresponding position in the state vector is assigned a value of 0;

[0092] The state vector is subjected to sliding window processing, and if all state values in the sliding area are 1, it is determined that the window covered area is an effective empty section;

[0093] The empty rate is calculated according to the following formula:

[0094] .

[0095] Preferably, the following image preprocessing operations are performed on the image data before training the second-stage visual recognition model, comprising:

[0096] Gaussian filtering denoising and CLAHE enhancement are performed on the original image to improve the local contrast and clarity of the image;

[0097] Randomly perform geometric perturbation operations, including perspective transformation, rotation, scaling;

[0098] Color perturbation is based on HSV color space transformation;

[0099] Occlusion simulation generates occlusion samples by superimposing irregular black blocks;

[0100] The second stage model further adopts a Mosaic image splicing enhancement strategy during training, and the improvement points include:

[0101] a. Ensure that each set of spliced images contains at least one image sample containing an empty U bit to balance the class ratio;

[0102] b. When constructing the candidate sample set, give higher selection probability to samples containing empty U bits;

[0103] c. Based on the detection accuracy of each class in the current training period, dynamically optimize the scaling ratio and spatial arrangement of the spliced images;

[0104] Embed the CA module in the Feature Pyramid Network (FPN) of the second stage visual recognition model to fuse channel and spatial position features, thereby enhancing the model's recognition ability for slender targets.

[0105] And, embed the CoordinateAttention (CA) module in the Feature Pyramid Network (FPN) of the second stage visual recognition model to fuse channel and spatial position features, thereby enhancing the model's recognition ability for slender targets (such as 1U servers).

[0106] Preferably, the following post-processing operations are performed on the U bit detection results output by the second stage visual recognition model:

[0107] Use the WeightedNMS algorithm to merge duplicate detection boxes, and the weight is determined by the IoU and the detection confidence;

[0108] Generate a cabinet U bit vacancy rate heat map, which supports multi-dimensional vacancy rate data visualization and provides historical trend comparison function;

[0109] If the vacancy rate fluctuation of a cabinet within a continuous monitoring period exceeds a certain threshold, an abnormal space alarm is triggered;

[0110] The artificial audit results will be automatically added to the training set by the system to realize online continuous optimization of the model structure through active learning mechanism;

[0111] The active learning mechanism comprises: pushing a detection result with a detection confidence between 0.4 and 0.6 to an artificial auditing system, automatically generating a labeling file and incorporating the labeling file into a training data set after auditing and confirmation, and triggering an incremental training task and a model parameter updating process every set period (for example, every week).

[0112] The application effectively solves the perspective distortion problem caused by shooting angle, cabinet tilt or partial occlusion by adopting a two-stage visual recognition structure, i.e., the first stage uses irregular target detection to accurately position the cabinet area, significantly improving the accuracy and robustness of cabinet U position detection; the second stage further detects the fine U position state of the corrected image, introduces a feature pyramid network (FPN) and a CoordinateAttention (CA) module, and effectively improves the recognition accuracy of slender targets (such as 1U servers). At the same time, the application adopts a pure visual detection scheme to replace the traditional sensor detection and artificial inspection, avoiding the high cost, complex maintenance and low efficiency of artificial inspection caused by the sensor scheme. Only a small number of cameras are needed to realize one-to-many real-time dynamic monitoring, greatly reducing the deployment cost and maintenance difficulty, and improving the automation level. In addition, the detection method proposed by the application has strong adaptability and can be compatible with cabinets of various brands and specifications, and the vacancy rate determination threshold can be flexibly customized, significantly reducing the system expansion cost. The application also introduces an active learning and artificial review mechanism, which realizes online continuous optimization of the model through artificial review and incremental training of low confidence results, ensures long-term stability and detection accuracy, and adapts to data drift and complex environment changes in long-term operation. Finally, the application also provides intuitive U position state heat map and historical trend analysis tools, which can quickly find the imbalance and abnormal fluctuations of cabinet resource distribution, and trigger an automatic alarm mechanism, greatly improving the response speed and overall operation and maintenance efficiency of the computer room management personnel. Therefore, the application is significantly superior to the prior art in terms of detection accuracy, economy, automation level, compatibility, long-term stability and operation and maintenance efficiency, and is particularly suitable for intelligent management of modern data centers. BRIEF DESCRIPTION OF DRAWINGS

[0113] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0114] Figure 1 The method flowchart of the application;

[0115] Figure 2 The overall structure schematic diagram of the embodiment of the application;

[0116] Figure 3 Figure 1 is an example of a cabinet original image of an embodiment of the present application;

[0117] Figure 4 Figure 2 is an example of a cabinet perspective transformation of an embodiment of the present application;

[0118] Figure 5 Figure 3 is an example of an irregular target detection result of an embodiment of the present application;

[0119] Figure 6 Figure 4 is an example of an empty target detection result of an embodiment of the present application. DETAILED DESCRIPTION

[0120] In order to make the objectives, technical solutions and advantages of embodiments of the present application clearer, the technical solutions of embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0121] As shown in Figure 1 Figure 1, an embodiment of the present application provides a cabinet U position detection method based on two-stage visual recognition and perspective correction, comprising the following steps:

[0122] Step 1: Obtain cabinet image data, which is collected by a camera deployed on the top or middle of a machine room, or uploaded by a manual inspection terminal;

[0123] Step 2: Detect irregular cabinets by a first-stage visual recognition model on the image data, and output four vertex coordinates of the cabinet;

[0124] Step 3: Perform image perspective correction based on the four vertex coordinates to generate standard rectangular cabinet image data;

[0125] Step 4: Input the corrected image data into a second-stage visual recognition model to identify the occupancy state of the U position and construct a U position state matrix;

[0126] Step 5: Calculate the vacancy rate of the cabinet based on the U position state matrix;

[0127] Step 6: Output the detection result to a machine room management platform and visualize it in the form of a heat map;

[0128] Step 7: Manually review the detection results with a confidence level lower than a preset threshold, and use the results for incremental training and continuous optimization of the model.

[0129] Embodiment 1: Image data acquisition method

[0130] Acquire rack image data, which is collected by cameras deployed on the top or middle of the server room, or captured and uploaded by manual inspection terminals;

[0131] In this embodiment, the data for the front image of the server rack is obtained through a combination of manual inspection photography and fixed camera capture. For example... Figure 2 As shown, the original images of the server racks can be acquired by cameras deployed within the server room. The cameras are preferably installed at the top or middle of the server room aisles, providing both overhead and forward (horizontal) view coverage, respectively. For example, a Hikvision camera can be deployed directly above and in front of the server rack to obtain an overhead view of the rack's front; the camera parameters should be set to high-definition mode to ensure sufficient resolution for recognition. Simultaneously, manual inspection personnel can use a mobile terminal equipped with a distortion-corrected wide-angle lens (e.g., Canon) to photograph each server rack individually. During photography, personnel should maintain a horizontal distance of approximately 1.5 meters from the front of the rack and control the camera's vertical angle within ±30° to ensure the acquired image covers the entire rack with minimal distortion.

[0132] Through the two methods described above, the acquired image data covers a variety of typical scenarios, including different shooting angles (overhead view, horizontal view, etc.) and different lighting conditions (normal lighting, emergency lighting, etc.). To ensure the robustness of the algorithm, the acquired raw images must be clear and contain the complete rack target (the complete visible area of ​​the rack must reach more than 90% of the image). In actual deployment, fixed cameras can be set to periodically acquire images, for example, automatically taking a picture of the front of the rack every 30 minutes and transmitting the image to the server via a network (such as the RTSP protocol); images taken during manual inspections can be uploaded via mobile applications (JPEG format is recommended, and the image quality compression should not be less than 90%). If necessary, preprocessing operations can be performed on the acquired raw images, such as Gaussian filtering for noise reduction and adaptive histogram equalization, to improve image quality and the accuracy of subsequent recognition. The acquisition of the above image data provides a reliable data foundation for subsequent model training and real-time detection.

[0133] Example 2: Model Training and Annotated Data Construction

[0134] In this embodiment, the collected rack images are used to train two visual recognition models: Model 1 is used for irregular target detection in the rack area, and Model 2 is used for rack U-position occupancy detection. To construct the training dataset, the images need to be labeled and data augmented. The specific steps are as follows:

[0135] Data annotation and label definition: First, use professional annotation tools (such as LabelImg, etc.) to annotate the original image, and generate annotation files (such as JSON or YOLO format) containing target position and category. For model one, the outline of the cabinet in each image is annotated with an irregular quadrilateral, i.e. the pixel coordinates of the four corners of the cabinet are marked. If the cabinet is completely visible, its actual four corner positions are annotated; if the cabinet is partially obscured, the complete rectangular outline is inferred from the visible part and the top point position is annotated. For model two, the occupancy status of each U position on the front of the cabinet needs to be annotated, and the label is divided into "vacant" and "non-vacant" two categories. Specifically, if a cabinet has no equipment obstruction for more than 2U height, the continuous region is labeled as vacant (vacant), such as the vacant U5-U6 region; any region covered by equipment is labeled as non-vacant (non-vacant), and all marked boundaries are strictly aligned with the horizontal separation line of the cabinet U position. If a device is found to occupy more than 1.5U height, it is considered to occupy two adjacent U positions (round up the number of occupied U positions); small objects that cover less than half a U height can not be considered as occupancy labels. Through polygon or rectangular annotation tools, cabinet four corner coordinate information and U position region state labels can be obtained at the same time, providing training samples for two-stage models.

[0136] Data set augmentation and enhancement: To improve the robustness and generalization ability of the model, various data enhancement operations are performed on the training data after annotation. First, geometric distortion enhancement is introduced: random perspective transformation, rotation, scaling and other transformations are performed on the image to simulate different angles of shooting and cabinet tilt, so that the model can adapt to the change of view angle. Second, color disturbance: randomly adjust the hue, saturation, and brightness of the image in the HSV color space to enhance the model's adaptability to different lighting and color temperature conditions. Third, occlusion simulation: overlay black rectangular blocks or other occlusions at random positions in the image to simulate temporary occlusions in front of the cabinet (such as people or debris passing by) to improve the model's detection performance in the presence of partial occlusions. In addition, for model two data, we specially use the Mosaic data enhancement strategy: four different cabinet front images are spliced into one image to expand the data samples. In each Mosaic spliced image, at least one vacant area sample is forced to be included to alleviate the imbalance between vacant (vacant) and non-vacant (non-vacant) categories. After the above processing, the final training data set size is ensured to be no less than 2000 diversified image samples.

[0137] Model training: The enhanced data is used to train model one and model two respectively. Model one can use an improved YOLO architecture or other target detection networks that can output irregular quadrilateral detection boxes. During training, a combined loss function is used to optimize classification accuracy and vertex position accuracy and other indicators. Model two can use existing lightweight target detection networks (such as YOLOv5 or YOLOv8, etc.) to identify U-bit states. During training, the labeled cabinet vertex coordinates and U-bit states are used as supervision signals to continuously adjust the model parameters to make them converge. When the training reaches the expected accuracy, the model weight files (such as.pt format or.onnx format) are exported for inference deployment. At this point, the offline training and data construction of the two-stage model are completed.

[0138] Example 3: First-stage irregular target detection implementation and confidence mechanism

[0139] This example is for the detection of the entire cabinet. The trained first-stage model is used to detect irregular targets on the collected images to accurately locate the position and shape of the cabinet. For example, Figure 4As shown, the model one performs positioning detection on the cabinet in the original image, and outputs four vertex coordinates of the cabinet outer contour. The specific implementation process is as follows: first, the input original image is scaled and normalized according to the model requirements (for example, scaled to 640*640 resolution, and the pixel value is normalized to the interval [0, 1]), and then sent to the neural network of the model one for inference. The model one adopts an improved YOLO target detection architecture, and the special part is that the detection head supports predicting irregular quadrilateral boxes instead of the traditional axis-aligned rectangular boxes. That is, the model directly regresses the image coordinate positions of the four corners of the cabinet, so as to accurately depict the inclined rectangular contour of the cabinet in the image. In order to improve the detection accuracy, a composite loss function is introduced during the model training, and the classification error, vertex position error, rotation angle error, scale error and confidence error are respectively given weights for joint optimization, so that the model can better learn the inclined angle and shape features of the cabinet target. After the model infers the cabinet candidate region, the confidence mechanism is applied to filter the effective detection results. Each prediction output has a confidence score indicating the reliability of the detection region as a cabinet. In this embodiment, the confidence threshold is set to 0.6, that is, when the confidence of the cabinet candidate region output by the model is lower than 0.6, it is considered as unreliable detection and discarded; the result higher than the threshold is considered as reliable cabinet recognition. If there are multiple candidate detection boxes (for example, when there are multiple cabinets in the image), the non-maximum suppression (NMS) algorithm can be used to optimize the overlapping detection boxes, and only the detection box with the highest confidence or the boxes with large overlap are combined and processed. Thus, only one accurate positioning polygon detection box is obtained for each cabinet in each image. The four vertex coordinates obtained by detection are sorted and numbered in clockwise or counterclockwise order for subsequent steps. Through the above irregular target detection mechanism, the position and shape of the cabinet in the image can be accurately extracted, and reliable basis can be provided for perspective correction.

[0140] Specifically, the first stage visual recognition model adopts a structure supporting irregular target detection, and outputs four vertex coordinates of an irregular quadrilateral. During the training process, a combined weighted loss function is used for optimization, and the expression is as follows:

[0141] ;

[0142] Wherein: , , , , is the weight coefficient of each loss term, used to balance the magnitude of different losses; is the classification error; is the vertex regression error; is the rotation angle error; is the scale error; is the confidence error;

[0143] wherein, is the expression:

[0144] ;

[0145] wherein: represents the number of samples in a batch, i.e. the number of images input at once for training; represents the i-th sample being calculated; represents the i-th vertex of the bounding box; represents the i-th vertex of the bounding box predicted by the model; represents the i-th vertex of the i-th sample predicted by the model; represents the i-th vertex of the i-th sample predicted by the model; ; represents the i-th vertex of the i-th sample predicted by the model; ; represents the i-th vertex of the i-th sample predicted by the model;

[0146] is the expression:

[0147] ;

[0148] wherein: represents the number of samples in a batch, i.e. the number of images input at once for training; represents the i-th sample being calculated; represents the rotation angle of the i-th target predicted by the model; represents the real rotation angle;

[0149] is the expression:

[0150] ;

[0151] wherein: represents the dimension of the scale, i.e. the width and height of the target; represents the predicted scale; represents the real scale.

[0152] In particular, the vertex position loss is to calculate the regression error of each vertex of the irregular quadrilateral box; the angle loss takes into account the influence of the rotation angle of the target, and increases the calculation of the angle error of each vertex relative to the center point; the scale loss considers the difference between the actual area of the polygon and the area of the circumscribed rectangle of the box.

[0153] For Model Two, CoordinateAttention (CA) is inserted into the Feature Pyramid Network (FPN) layer to enhance the positioning ability of slender devices (such as 1U servers):​​

[0154]

[0155] wherein GAP is global average pooling, GMP is global max pooling, is element-wise multiplication.

[0156] The classification loss adopts Focalloss to alleviate the class imbalance problem:

[0157]

[0158] wherein, is a class balance coefficient, adjusting the loss weight of positive and negative samples. is a focus parameter, the greater the loss attenuation of easy samples is more obvious, and the model pays more attention to difficult samples; is the direction of adjusting the prediction probability, which always represents the prediction probability of the correct class by the model.

[0159] The regression part in the combined weighted loss function further adopts an extended intersection over union loss function, and the calculation formula is as follows:

[0160] ;

[0161] wherein, is the intersection over union, which measures the overlap degree of two boxes; represents the center point distance penalty term, that is, the squared Euclidean distance between the center point of the predicted box and the center point of the real box; represents the squared length of the diagonal of the minimum closed region; and represent the squared width and height of the minimum closed region.

[0162] Embodiment 4: Image perspective correction method and size calculation method

[0163] This embodiment is based on the cabinet four corner coordinates obtained above, and performs perspective distortion correction on the cabinet image to restore the standard rectangular view of the front of the cabinet and establish the corresponding relationship between the pixel size and the actual size. As shown in Figure 3 , the front view of the cabinet can be obtained after perspective transformation on the original inclined image. Figure 2 The specific implementation steps are as follows:

[0164] First, according to the four vertex coordinates of the cabinet obtained in Embodiment 3, the width and height of the corresponding irregular quadrilateral of the cabinet in the original image are calculated. When calculating, the distance between opposite sides of the quadrilateral is measured respectively: on the one hand, the length of the upper edge and the lower edge (i.e. the distance between the two corners at the top and the distance between the two corners at the bottom of the cabinet) is calculated, and the maximum value is taken as the target width of the cabinet after correction; on the other hand, the length of the left side edge and the right side edge is calculated, and the maximum value is taken as the target height. Thus the pixel size (target width x target height) of the target image after perspective transformation is determined. Next, based on the corresponding relationship between the four vertex coordinates of the original image and the four vertex coordinates of the rectangular image after correction, the perspective transformation matrix is calculated. The 3x3 perspective transformation matrix can be obtained by using the cv2.getPerspectiveTransform function of the OpenCV library, and then the cv2.warpPerspective function is used to project and transform the original image, so as to map the cabinet region to a regular rectangular shape. The transformed image is cropped according to the calculated target width and height, so as to obtain a rectangular image (i.e. a perspective correction image, as shown in FIG. 8) containing only the front face of the cabinet. In this image, the upper and lower edges and the left and right edges of the cabinet are parallel to the image frame, and there is no shrinkage and deformation caused by the original perspective. Figure 3

[0165] After perspective correction, the relationship between the image pixel size and the actual physical size of the cabinet needs to be established in order to determine the U-bit unit scale. Specifically, the total pixel height of the corrected cabinet image can be compared with the actual total U height of the cabinet. Assuming that the detected cabinet is a standard 42U cabinet, the actual height of each U is 44.45mm, so the total height of the cabinet is about 42x44.45=1866.9mm. The pixel height of the cabinet from top to bottom in the perspective-corrected image (denoted as H_px) is measured, and the pixel ratio per millimeter is H_px divided by 1866.9. Further, the pixel height of each U in the image is h_px≈(H_px / 1866.9)x44.45. In a simple case, the total pixel height of the corrected image can also be directly divided into several equal parts, with each part corresponding to the pixel value of the height of one U. After determining the pixel size corresponding to one U by the above method, a vertical grid division is established on the corrected image, and the height of each grid is one U. In this way, the pixel ruler of the cabinet U-bit is obtained, which provides a basis for subsequent U-bit detection and vacancy rate calculation. It should be noted that if the cabinet is not of standard height or the number of Us is known, the relationship between the pixel and the actual size can also be calculated by pre-measuring the calibration points on the cabinet. The above calculation method is not limited to specific cabinet models. The key is that through perspective correction and size calculation, the cabinet image can be standardized, so that the subsequent algorithm can accurately analyze by U-bit unit.

[0166] Specifically, the image perspective correction includes: ​

[0167] For the irregular quadrilateral composed of the four vertices obtained by the first stage target detection, the width and height mapped to the standard rectangle are calculated according to the vertices, for determining the target image size;

[0168] The geometric transformation is performed using the getPerspectiveTransform and warpPerspective functions of OpenCV, to eliminate the perspective distortion caused by the shooting angle;

[0169] The transformation result is cropped into a standard rectangular cabinet image warped_img, which is stored locally for the target detection processing by the second stage model.

[0170] Further, in the standard rectangular image obtained after the perspective transformation, the pixel height of each U position is converted by the following formula:

[0171] ;

[0172] Wherein: represents the pixel occupied by each position; is the height of the standard position; is the height of the entire real cabinet, and only when the length of the continuous uncovered area is not less than 2U, the area is considered as vacant state.

[0173] Embodiment 5: Second stage U position occupancy state detection and state matrix construction

[0174] After the perspective correction of the cabinet image, this embodiment uses the second stage model to detect the U position occupancy state of the corrected cabinet front image, to identify the vacant or occupied state of each U position, and to construct the corresponding state matrix. As shown in Figure 5 , model two is used to detect the U position occupancy state of the cabinet image, and the corresponding state matrix is constructed. Figure 3After the obtained cabinet front image is detected, the vacant area and the equipment occupied area can be marked. The specific implementation steps are as follows: first, input the cabinet front correction image obtained in embodiment 4 into model two for target detection. Model two has learned the features of different U height areas in the cabinet during training, and can identify which areas are vacant (vacant) and which areas have equipment (non-vacant). The model outputs several detection boxes, each of which is attached with a category label (vacant or non-vacant) and a confidence score. Usually, each cabinet equipment corresponds to a non-vacant detection box (may be 1U or several U high), and continuous vacant space (at least 2U high) will correspond to a vacant detection box. Then, the detection results output by the model are filtered and optimized. First, a confidence threshold is used for screening, and detection boxes with a confidence lower than a preset threshold (such as 0.6) are discarded, so as to balance the recall rate and the accuracy rate and reduce the influence of false positives. Then, the remaining detection boxes are merged by using a weighted non-maximum suppression (NMS) algorithm. Specifically, according to the overlap degree (IoU) between the detection boxes and the confidence of each detection box, a weighted score (for example, the confidence x IoU as a weight factor) is calculated, and the detection boxes with high overlap and the same category are fused into a single box. Through this strategy, repeated detection that the model may output can be eliminated, and only one detection result is retained for the same equipment or the same vacant area, so as to improve the accuracy of the state judgment. After obtaining stable detection results, a cabinet U position state matrix is constructed. Assuming that the cabinet has N U positions in total (for example, N = 42), a one-dimensional array or a corresponding matrix is established, where each element represents the state of the corresponding U position. Initially, all elements of the matrix are assigned a value of 1, indicating that it is assumed to be vacant. Then, the matrix is updated according to the detection results: for each detection box with a category of non-vacant, calculate its vertical coverage range in the cabinet, and change the corresponding position in the matrix to 0 to represent occupation. In specific implementation, the pixel to U height mapping relationship calculated in embodiment 4 can be used to correspond the top and bottom pixel positions of the non-vacant detection box to the specific U number range. For example, a detection box covers about 1.5U from the 10th pixel row to the 30th pixel row in the correction image, so the starting U position and the adjacent 2U positions below it in the matrix are marked as 0 (occupied). In this way, each detected equipment area will make the U grid positions covered by it in the matrix 0. If the equipment coverage range is not an integer U position, the upward rounding method is used to ensure that as long as more than half of the U is occupied, it is counted as one U position. For each detection box of the vacant category, it is further verified whether the corresponding vacant area in the matrix is indeed empty and meets the condition of being continuous for more than 2U.Specifically, the height span of the vacant detection frame in the corrected image is calculated, corresponding to the number of U, for example, a certain vacant frame covers U5 to U7, and the span is 3U. Only when the span is not less than 2U and the matrix in the range is previously all 1 (i.e. not marked as occupied by any device), can the detection frame finally be confirmed as a real vacant area. If the height covered by the vacant frame is less than 2U (such as only 1U high), it is considered as invalid detection or temporary gap according to the pre-set rules, and is not counted separately as vacant. Through this sliding window judgment mechanism, it is ensured that only the vacant section of two or more U positions is counted as valid vacancy, and the possible false detection or meaningless sporadic gap is filtered out. After the above process, the cabinet U position state matrix S is finally obtained, wherein the elements with a value of 1 correspond to vacant U positions, and the elements with a value of 0 correspond to U positions occupied by devices. The matrix comprehensively describes the current occupancy of each U position in the cabinet.

[0175] Specifically, the prediction confidence output by the second-stage visual recognition model is calculated, and if the confidence is higher than 0.6, it is directly accepted;

[0176] If the confidence is between 0.4 and 0.6, the corresponding detection result will be submitted for manual review;

[0177] The result after review is recorded and added to the training sample set for subsequent incremental training of the model;

[0178] The vacant state is determined by a sliding window operation, and the window covers no less than 2U positions. If the prediction results in the window range are all vacant, the whole area is marked as vacant.

[0179] Embodiment 6: Vacancy rate calculation method and sliding window judgment mechanism

[0180] After obtaining the cabinet U position state matrix S, this embodiment calculates the vacancy rate of the cabinet, and further ensures the stability of the vacancy judgment through the sliding window mechanism. As Figure 6As shown, the rack vacancy rate is defined as the proportion of vacant unit slots (U-slots) in the rack, and can be calculated as follows: the number of U-slots with a value of 1 (vacant) in the statistical matrix S is divided by the total number of U-slots N in the rack, and then multiplied by 100% to obtain the vacancy rate percentage. For example, if 10 slots in a 42U rack are vacant, the vacancy rate is approximately 10 / 42 ≈ 23.8%. In actual calculation, the number of vacant U-slots can be directly obtained from the matrix constructed in the previous embodiment, thus quickly obtaining the vacancy rate indicator. It is worth noting that this embodiment incorporates a sliding window determination mechanism into the identification process of vacant areas to ensure that the calculated vacancy rate is reliable and effective. Specifically, in embodiment 5, only areas with 2 or more consecutive vacant units are counted as truly vacant. This rule can be understood as applying a sliding window with a height of 2U in the vertical direction of the rack to scan the U-slot sequence: only when all consecutive U-slots covered by the sliding window are empty (no equipment, occupancy mark is 0) is a valid vacancy segment determined. If any space within the sliding window is occupied, that window is not counted as vacant. By moving the sliding window, all vacant intervals that meet the specified length can be detected, and the count of vacant U-positions can be adjusted accordingly. This mechanism effectively avoids including isolated vacant positions in the vacancy rate, thereby improving the accuracy of the statistical results in depicting the actual available space. The final output vacancy rate considers both the total amount of vacant space in the rack and the continuity of vacant space, making the vacancy rate calculation more in line with the needs of data center operations and maintenance for assessing available space.

[0181] Specifically, based on the U-position occupancy status recognition result output by the second-stage visual recognition model, the determination of vacancy status through sliding window operation includes:

[0182] Build length is state vector , where 0 represents The bit is occupied; 1 indicates that it is empty.

[0183] When the equipment detection area spans multiple When the bit is in the state vector, round up the actual number of bits in U and assign the corresponding position a value of 0.

[0184] A sliding window process is performed on the state vector. If all state values ​​within the sliding area are 1, the area covered by the window is determined to be a valid empty segment.

[0185] The vacancy rate is calculated according to the following formula:

[0186] .

[0187] Example 7: Display method of test results (heat map)

[0188] The cabinet vacancy rate and the U-bit state matrix obtained by the above detection are connected to the machine room management platform, and the cabinet vacancy is displayed in a heat map mode, so as to facilitate the checking and analysis of the operation and maintenance personnel. The system associates the vacancy rate of each cabinet with its spatial position, and presents it in a color-coded matrix or graph on the monitoring interface. For example, the platform can draw a top-down layout diagram of the machine room or a cabinet list, and mark the vacancy rates of each cabinet in the form of a heat map: cabinets with high vacancy rates are displayed in cool tones (such as blue or green), and cabinets with low vacancy rates and high equipment occupancy rates are displayed in warm tones (such as red), and the color depth reflects the specific vacancy percentage. It can be seen at a glance which cabinets are almost empty (color is blue / shallow), and which cabinets are saturated (color is red / deep). This multi-dimensional data display method is like a heat distribution map, which can help administrators quickly locate the uneven distribution of cabinet resources. In addition, the platform interface can also provide historical data trend analysis function. By recording and drawing the vacancy rate curve of each cabinet over time, a time series graph or dynamic heat map is formed, and the operation and maintenance personnel can observe the change trend of the cabinet vacancy over a period of time. For example, when the vacancy rate of a cabinet continues to decrease, it may indicate that new equipment is being installed; on the contrary, if the vacancy rate increases, it may indicate that the equipment is being moved out or removed. If the system detects that the vacancy rate of a cabinet has abnormally fluctuated in a short period of time (for example, fluctuation of more than 20% within 24 hours, or continuous fluctuation), the system can trigger an alarm to indicate that there may be unstable equipment removal / insertion or abnormal data. This combination of visual display and early warning mechanism greatly improves the intuitiveness and real-time performance of cabinet resource monitoring, and provides strong support for data center operation and maintenance.

[0189] Specifically, Gaussian filtering and CLAHE enhancement are performed on the original image to improve the local contrast and clarity of the image.

[0190] Randomly perform geometric perturbation operations, including perspective transformation, rotation, and scaling.

[0191] Color perturbation is based on HSV color space transformation.

[0192] The occlusion simulation generates occlusion samples by superimposing irregular black blocks.

[0193] The second stage model training further adopts a Mosaic image splicing enhancement strategy to ensure that at least one image sample containing a vacant U-bit is included in each spliced image to balance the class ratio.

[0194] Furthermore, a CoordinateAttention (CA) module is embedded in the feature pyramid network (FPN) of the second stage visual recognition model to fuse channel and spatial position features, thereby enhancing the recognition ability of the model for slender targets such as 1U servers.

[0195] Example 8: Continuous optimization strategy of the model (manual review, active learning, incremental training)

[0196] This embodiment introduces the continuous optimization mechanism of the detection model after deployment to ensure that the model maintains high accuracy and adapts to new environments in the long run. This optimization strategy mainly realizes through the introduction of manual review feedback and active learning, that is, on the basis of automatic detection, the cycle of human correction and incremental training is added to continuously improve the performance of the model. Specifically, the system initiates a manual review process for detection results with low confidence. Low confidence usually refers to the confidence score given by the model falling within an intermediate low interval, for example, between 0.4 and 0.6, indicating that the model is not sure about the judgment. For such results, the system sends the corresponding cabinet image segment and detection label to the manual labeling platform or the operation and maintenance personnel interface for manual verification and correction. The manual review personnel confirm whether the U area is empty or occupied by equipment according to the actual situation, or mark the equipment missed by the model, and then submit the audit results. After receiving the manual labeling feedback, the system adds these corrected samples to the model training set and automatically generates new labeled data in the training format. When a certain number of manually corrected samples are accumulated or the optimization interval is reached periodically (for example, every week), the incremental training process of the model is triggered. Incremental training uses the previously trained model weights as the initial value to continue training the model on the expanded training set. On the one hand, it continues to learn new data features so that the model can adapt to the gradual changes in the computer room environment (such as changes in lighting and replacement of equipment types); on the other hand, fine-tuning by reducing the learning rate (for example, set to 1 / 10 of the initial learning rate) can avoid forgetting existing knowledge and overfitting to new data. After a complete incremental training, the updated model parameters are obtained, and the new model is deployed to replace the old model for subsequent image inference. In this way, the model can be continuously iteratively optimized. It should be noted that this active learning continuous optimization strategy can effectively reduce the workload of comprehensive manual labeling, focusing human resources on a small number of difficult samples with model uncertainty, thereby efficiently improving the model accuracy. In the method of the present application, by automatically extracting a certain number (such as 100) of the latest cabinet images from the computer room management platform every week for manual verification and incremental training, the model can continuously adapt to changes in cabinet layout or equipment installation, and still maintain high recognition accuracy and robustness for the state of the cabinet U position after long-term operation. This human-machine collaborative optimization method ensures the practicality and reliability of the system after deployment, further improving the engineering feasibility of the cabinet U position detection method in actual data center operation.

[0197] Specifically, the weighted non-maximum suppression (WeightedNMS) algorithm is used to merge the repeated detection boxes, and the weight is determined by the IoU and the detection confidence;

[0198] A cabinet U position vacancy rate thermal map is generated, which supports the visual display of multi-dimensional vacancy rate data and provides a historical trend comparison function.

[0199] If the vacancy rate fluctuation of a certain cabinet exceeds the set threshold in the continuous monitoring period, a space anomaly alarm is triggered.

[0200] The artificial review result will be automatically added to the training set by the system, and the model structure is continuously optimized online through the active learning mechanism.

[0201] The active learning mechanism includes: pushing the detection results with a detection confidence between 0.4 and 0.6 to the artificial review system, automatically generating a labeled file after verification and incorporating it into the training data set, and triggering an incremental training task and model parameter update process every set period (e.g., every week).

[0202] In summary, the present application improves the accuracy, automation and scalability of the cabinet U position state detection in the machine room by constructing a "two-stage visual recognition + perspective correction + active learning optimization" overall solution. The first stage uses an improved YOLO structure to accurately detect the boundaries of irregular cabinets, and through geometric transformation, it completes the correction processing of standard rectangular images, solving the U position misrecognition problem caused by angle deviation, tilt, shielding and other factors in traditional images. The second stage integrates the Mosaic enhancement strategy and the CoordinateAttention attention mechanism, focusing on the occupation judgment of 1U and other slender devices, and combining the sliding window mechanism and state matrix construction method, effectively avoiding false positives and false negatives in vacancy judgment.

[0203] At the same time, the present application innovatively introduces a man-machine cooperation mechanism based on confidence interval: when the model output confidence is between 0.4 and 0.6, the system automatically submits artificial review, and the review result is used as training data for iteration, combined with the periodic incremental training mechanism (such as automatic optimization every week), to ensure that the model always has continuous adaptability and high-precision prediction ability to changes in the field. In addition, the detection result can be displayed in real time in the machine room management platform through methods such as thermal map and historical trend chart, significantly improving the resource visualization level and operation response efficiency.

[0204] The overall solution takes into account the advantages of "easy deployment, lightweight model, accurate recognition, data closed loop, and continuous self-evolution of algorithm", which is significantly superior to traditional manual inspection and physical sensor solutions, and has strong practicality in cost control, maintenance efficiency, engineering versatility and platform integration capability, especially suitable for U position resource intelligent management application scenarios in modern data centers, edge machine rooms and mixed deployment environments of multiple brands of cabinets.

[0205] In the description of the application, reference can be made to terms such as "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" etc. It is intended that there are at least one embodiment or example described in connection with the particular feature, structure, material or characteristic described in connection with the embodiment or example. Moreover, the described particular feature, structure, material or characteristic can be combined in any suitable manner in one or more embodiments or examples. Furthermore, different embodiments or examples described in the specification and different features, structures, materials or characteristics of different embodiments or examples can be combined and combined in any suitable manner without departing from the scope of the present application.

[0206] Any process or method descriptions or blocks in flow charts described herein and elsewhere in this specification can be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logic functions or steps in the process. As well, preferred embodiments of this application can be facilitated by one or more modules, segments or portions of code that implement pre-defined functions or steps in the processes described herein.

[0207] The above description is only specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art can easily think of various changes or replacements within the technical scope disclosed in the present application, and these should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for detecting rack U-positions based on dual-stage visual recognition and perspective correction, characterized in that, Includes the following steps: Acquire rack image data, which is collected by cameras deployed on the top or middle of the server room, or captured and uploaded by manual inspection terminals; The first-stage visual recognition model is used to detect irregular cabinets in the image data and outputs the coordinates of the four vertices of the cabinet. Based on the coordinates of the four vertices, perform image perspective correction to generate standard rectangular cabinet image data; The corrected image data is input into the second-stage visual recognition model to identify the occupancy status of the U-position and construct the U-position state matrix; Based on the U-position state matrix, the vacancy rate of the computer cabinet; Determining the idle state through sliding window operations includes: Build length is U-position state matrix , where 0 represents The bit is occupied, 1 indicates it is empty, N u Indicates the total number of server racks Number of digits; When the equipment detection area spans multiple When the length of the continuous uncovered area is not less than 2U, the area is determined to be in an empty state. The actual number of U occupies is rounded up, and the corresponding position in the state vector is assigned a value of 0. A sliding window process is performed on the state vector. If all state values ​​within the sliding area are 1, the area covered by the sliding window is determined to be a valid empty segment. The vacancy rate is calculated according to the following formula: ； The test results are output to the data center management platform and displayed visually in the form of a heat map; Detection results with confidence levels below a preset threshold are manually reviewed, and the results are used for incremental training and continuous optimization of the model.

2. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 1, characterized in that, The first-stage visual recognition model adopts a structure that supports the detection of irregular objects, and outputs the coordinates of the four vertices of an irregular quadrilateral. During training, it is optimized using the following combined weighted loss function, expressed as: ； in: , , , , These are the weighting coefficients for each loss term, used to balance the magnitudes of different losses; It is classification error; It is the vertex regression error; It is the rotation angle error; It is a scale error; It is the confidence error; in, The expression is: ； in: This represents the number of samples in a batch, i.e., the number of images input for one training session. Represents the number being calculated One sample; The first one representing the bounding box One vertex; The representative model predicted the first The first sample The coordinates of each vertex ; Represents the actual vertex coordinates of the labeled area; The expression is: ； in: This represents the number of samples in a batch, i.e., the number of images input for one training session. Represents the number being calculated One sample; The model predicts the first The rotation angle of each target; Represents the actual angle of rotation; The expression is: ； in: The dimensions representing scale, namely the width and height of the target; Indicates the scale of the prediction; It represents the true scale.

3. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 2, characterized in that, The regression component of the combined weighted loss function further employs the extended cross-union loss function, calculated as follows: ； in: It is the intersection-union ratio, which measures the degree of overlap between two boxes; This represents the center point distance penalty, which is the squared Euclidean distance between the center point of the predicted box and the center point of the ground truth box. This represents the square of the diagonal length of the smallest closed region; and This represents the square of the width and height of the smallest closed region.

4. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 1, characterized in that, The data acquisition device for acquiring rack image data includes cameras deployed at the top and middle of the computer room to acquire top-down and horizontal dual-view images, and includes a manual inspection terminal equipped with a wide-angle anti-distortion lens. The acquisition device supports image resolution that meets the requirements for full-cabinet imaging and has the ability to acquire data at regular intervals and automatically upload it to the platform for data synchronization.

5. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 1, characterized in that, The image perspective correction includes: For the irregular quadrilateral formed by the four vertices obtained from the first stage of target detection, the width and height of the standard rectangle mapped to it are calculated based on its vertices to determine the size of the target image. The projection transformation matrix from the image platform to the target plane is calculated, and the geometric transformation is performed by applying the matrix; the view transformation algorithm is implemented by solving the function algorithm of the OpenCV library. The transformation result is cropped into a standard rectangular cabinet image (warped_img), which is stored locally for target detection processing in the second-stage model.

6. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 5, characterized in that, In the standard rectangular image obtained after perspective transformation, the pixel height per U bit is calculated using the following formula: ； in: Represent each The number of pixels occupied by a bit; It is the standard The height of the position; It is the actual height of the entire rack, and the area is considered vacant only when the length of the continuous uncovered area is not less than 2U.

7. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 1, characterized in that, The method further includes: Calculate the prediction confidence of the second-stage visual recognition model output. If the confidence is higher than 0.6, it is directly accepted. If the confidence level is between 0.4 and 0.6, the corresponding test results will be submitted for manual review. The results of the review are recorded and added to the training sample set for subsequent incremental training of the model; The vacancy status is determined by a sliding window operation, where the window covers a position of not less than 2U. If the prediction results within the window range are all vacant, the entire area is marked as vacant.

8. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 1, characterized in that, Before training the visual recognition model in the second stage, the following preprocessing operations are performed on the image data: Gaussian filtering and CLAHE enhancement are applied to the original image; Perform geometric perturbations, including perspective transformations, rotations, and scaling; Color perturbation is performed in the HSV color space; Occlusion is simulated by overlaying irregular black blocks; During the second stage of model training, a Mosaic image stitching enhancement strategy is employed, specifically including: Ensure that the stitched image includes samples with empty U-positions; Increase the selection probability of vacant U-position samples; Dynamically optimize the scaling and arrangement of stitched images based on category accuracy; The second-stage model embeds a channel attention (CA) module into the feature pyramid network (FPN) to enhance the recognition ability of slender targets.

9. The cabinet U-position detection method based on dual-stage visual recognition and perspective correction according to claim 8, characterized in that, Perform the following post-processing operations on the U-position detection results output by the second-stage visual recognition model: The weighted nonmaximum suppression algorithm is used to merge duplicate detection boxes, with the weights determined by both IoU and detection confidence. Generate a heat map of rack U-position vacancy rate. The heat map supports the visualization of multi-dimensional vacancy rate data and provides a historical trend comparison function. If the vacancy rate of a certain cabinet fluctuates beyond the set threshold during a continuous monitoring period, a space anomaly alarm will be triggered. The results of manual review will be automatically added to the training set by the system to achieve online continuous optimization of the model structure through an active learning mechanism; The active learning mechanism includes: pushing detection results with a detection confidence level between 0.4 and 0.6 to a manual review system; automatically generating labeled files and including them in the training dataset after review and confirmation; and triggering incremental training tasks and model parameter update processes every set period.

Images