Data center equipment checking method and system

Abstract

The invention provides a data center equipment checking method and system, and belongs to the technical field of artificial intelligence, and the method comprises the steps: obtaining an image frame and three-dimensional point cloud data of a data center; inputting the image frame and the three-dimensional point cloud data into a device identification model to obtain actual information of each device in the data center output by the device identification model; wherein the equipment identification model is obtained by training according to an image frame sample and a three-dimensional point cloud data sample; and comparing the actual information of each device with preset information stored in a preset device database to obtain device inventory difference data. According to the invention, automatic identification and actual information extraction of data center assets are realized through the multi-modal equipment identification model fusing the image frames and the three-dimensional point cloud data, and the data center assets are compared with the preset database to generate the inventory difference data, so that the efficiency and accuracy of equipment inventory are improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method and system for inventorying data center equipment. Background Technology

[0002] With the deepening of digital transformation, data centers have become a critical information infrastructure supporting the operation of society. Data centers deploy a large number of devices such as servers, switches, and storage; efficient and precise management of these devices is fundamental to ensuring the stable operation of data centers.

[0003] Existing equipment inventory methods involve attaching Radio Frequency Identification (RFID) tags to each device, using readers to read them in batches, and then comparing them with records in the configuration management database. However, this method is inefficient when dealing with a large number of devices. Furthermore, RFID signals are easily shielded and interfered with in metal cabinets and high-density equipment environments, leading to frequent missed reads and misreads, thus compromising the accuracy of the inventory results.

[0004] Therefore, improving the efficiency and accuracy of equipment inventory has become an urgent technical problem to be solved. Summary of the Invention

[0005] This invention provides a data center equipment inventory method, system, electronic device, storage medium, and computer program product to overcome the deficiencies in the prior art and improve the efficiency and accuracy of equipment inventory.

[0006] This invention provides a method for inventorying data center equipment, comprising the following steps: Acquire image frames and 3D point cloud data from the data center; The image frames and the 3D point cloud data are input into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples; The actual information of each device is compared with the preset information stored in the preset device database to obtain the device inventory difference data.

[0007] According to a data center equipment inventory method provided by the present invention, the equipment identification model includes a point cloud feature encoder, an image feature extraction network, a feature fusion module, an occlusion inference module, and a multi-task prediction head; the step of inputting the image frame and the three-dimensional point cloud data into the equipment identification model to obtain the actual information of each equipment in the data center output by the equipment identification model includes: Point cloud features are extracted from the 3D point cloud data using the point cloud feature encoder. The image feature extraction network extracts image appearance features from the image frame; The feature fusion module fuses the image appearance features and the point cloud features to generate fused features. The occlusion inference module identifies the occlusion feature regions in the fused features that represent the occlusion of the device, and infers and corrects the occlusion feature regions based on the adjacent device information of the occlusion feature regions to generate a corrected feature map. The corrected feature map is processed by the multi-task prediction head, and the actual information of each device in the data center is output in parallel.

[0008] According to a data center equipment inventory method provided by the present invention, the step of identifying occlusion feature regions representing occlusion of equipment in the fused features through the occlusion inference module, and inferring and correcting the occlusion feature regions based on the adjacent equipment information of the occlusion feature regions to generate a corrected feature map, specifically includes: Calculate the depth confidence of each feature position on the fused feature; The region formed by feature locations whose depth confidence is lower than a preset threshold is defined as the occluded feature region. A graph neural network structure is constructed, wherein the occlusion feature region is defined as the target node to be inferred, and the device that is physically adjacent to the occlusion feature region and has been identified is defined as the neighbor node; The graph neural network process fuses the initial fusion features corresponding to the target node with the known features of the neighboring nodes to update the features of the target node and generate a corrected feature map.

[0009] According to a data center equipment inventory method provided by the present invention, the method further includes a training method for the equipment identification model: Acquire image frame samples and 3D point cloud data samples of the data center under any environmental conditions, and acquire annotation information corresponding to all device samples in the data center; The image frame samples and the three-dimensional point cloud data samples under any of the environmental conditions are input into the initial model to obtain the prediction information corresponding to each of the device samples output by the initial model. A comprehensive loss function is constructed based on the annotation information and prediction information corresponding to each of the device samples. The initial model is trained with the goal of minimizing the comprehensive loss function to obtain the device recognition model.

[0010] According to a data center equipment inventory method provided by the present invention, the labeling information includes equipment type labels, three-dimensional bounding box labels, and U-position occupancy index labels; the prediction information includes equipment type prediction probability, predicted three-dimensional bounding box, and predicted U-position occupancy index.

[0011] According to a data center equipment inventory method provided by the present invention, the step of constructing a comprehensive loss function based on the annotation information and the prediction information corresponding to each of the equipment samples includes: A classification loss is constructed based on the device type label of each device sample and the predicted probability of all device types; Based on the intersection-union ratio, center point distance, and aspect ratio consistency between the center point of the 3D bounding box label of each device sample and the predicted 3D bounding box, a 3D localization loss is constructed. Map the U-bit occupancy index label of each of the device samples to a continuous value label, and map the predicted U-bit occupancy index of each of the device samples to a predicted continuous value; Based on the difference between the predicted continuous value and the continuous value label of each of the device samples, a U-position detection loss is constructed; The comprehensive loss function is constructed based on the weighted sum of the classification loss, the 3D localization loss, the U-position detection loss, and the regularization term.

[0012] According to a data center equipment inventory method provided by the present invention, the environmental conditions include lighting conditions, acquisition angle, and equipment obstruction.

[0013] According to a data center equipment inventory method provided by the present invention, the step of acquiring image frames and three-dimensional point cloud data of the data center includes: The video stream from the data center is acquired using calibrated acquisition equipment. Extract at least one of the image frames from the video stream; The video stream is processed using synchronous positioning and mapping technology to generate the 3D point cloud data.

[0014] The present invention also provides a data center equipment inventory system, comprising the following modules: The acquisition module is used to acquire image frames and 3D point cloud data from the data center; The processing module is used to input the image frame and the 3D point cloud data into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples; The comparison module is used to compare the actual information of each device with the preset information stored in the preset device database to obtain device inventory difference data.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the data center equipment inventory method described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the data center equipment inventory method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the data center equipment inventory method as described above.

[0018] In summary, one or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages: By acquiring image frames and 3D point cloud data from the data center, a comprehensive data foundation containing rich visual features and precise spatial information is provided for subsequent equipment identification. Based on this, the image frames and 3D point cloud data are input into a trained equipment identification model for automated processing. This efficiently transforms the raw pixel and point cloud data into structured actual equipment information, enabling intelligent identification of physical assets and significantly improving inventory efficiency and accuracy. Finally, by comparing the actual information of each device output by the model with preset information stored in a pre-defined equipment database, inconsistencies between physical assets and system records can be quickly and accurately identified, generating equipment inventory discrepancy data. This enables dynamic updates of data center equipment, further improving the efficiency and accuracy of equipment inventory. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the data center equipment inventory method provided by the present invention.

[0021] Figure 2 This is the second flowchart of the data center equipment inventory method provided by the present invention.

[0022] Figure 3 This is the third flowchart of the data center equipment inventory method provided by the present invention.

[0023] Figure 4 This is the fourth flowchart of the data center equipment inventory method provided by the present invention.

[0024] Figure 5 This is the fifth flowchart illustrating the data center equipment inventory method provided by the present invention.

[0025] Figure 6 This is the sixth flowchart of the data center equipment inventory method provided by the present invention.

[0026] Figure 7 This is a schematic diagram of the data center equipment inventory system provided by the present invention.

[0027] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] It should be noted that in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terms "upper," "lower," etc., indicating orientation or positional relationships according to the accompanying drawings, are only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the system or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0030] The terms "first," "second," etc., used in this invention are used to distinguish similar objects, not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] The following is combined with Figures 1-8 This invention describes the data center equipment inventory method, system, electronic device, storage medium, and computer program product provided by the present invention.

[0032] Reference Figure 1 , Figure 1 This is one of the flowcharts illustrating the data center equipment inventory method provided by the present invention, such as... Figure 1 As shown, steps 101 to 103 are included: Step 101: Acquire image frames and 3D point cloud data from the data center.

[0033] This invention provides an asset inventory method. The first step of this method is to acquire image frames and 3D point cloud data from a data center.

[0034] An image frame is a two-dimensional image containing color and texture information of the data center environment. Image frames can clearly record visual features such as the appearance of equipment panels, model identification, and indicator light status. Three-dimensional point cloud data is a dataset composed of a large number of three-dimensional coordinate points. This dataset can geometrically describe the physical dimensions, spatial location, and layout relationships of racks, equipment, and other facilities within the data center, accurately reconstructing the spatial structure of the physical environment.

[0035] In one specific implementation, an operator can use data acquisition equipment to scan the target data center environment. This scanning process generates raw data describing the physical state of the data center. This raw data is further processed to output image frames and 3D point cloud data that can be used in subsequent steps.

[0036] Step 102: Input the image frames and 3D point cloud data into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples.

[0037] After acquiring data from the data center, image frames and 3D point cloud data are input into the device recognition model to obtain the actual information of each device within the data center, output by the device recognition model. This step is performed because the raw multimodal data itself cannot directly provide the results required for asset inventory. An intelligent processing model is necessary to analyze and interpret the data, transforming the raw pixel and point coordinates into meaningful asset information.

[0038] In this embodiment, the device identification model is a deep learning model trained based on image frame samples and 3D point cloud data samples. This model has the ability to automatically identify and locate physical devices from complex multimodal inputs. The actual information of each device refers to a series of structured data output by the device identification model. This data describes in detail the current state of each physical device within the data center, such as its specific type, 3D spatial location, and the rack space it occupies.

[0039] In one specific implementation, the image frames and 3D point cloud data acquired in the previous step are input as a whole into a pre-trained device recognition model. Upon receiving this input, the device recognition model performs comprehensive analysis and processing. After processing, the device recognition model outputs the actual information of all identified devices within the data center.

[0040] In subsequent embodiments, the specific implementation method of the actual information of each device in the data center output by the device identification model, as well as the specific training method of the device identification model, will be described in detail.

[0041] Step 103: Compare the actual information of each device with the preset information stored in the preset device database to obtain the device inventory difference data.

[0042] After obtaining the actual information for each device, this information is compared with the preset information stored in the pre-defined device database to obtain device inventory discrepancy data. This step is performed because the core purpose of asset inventory is to identify inconsistencies between physical reality and recorded information. Simply obtaining the current actual information of the equipment is insufficient; it must be compared with a benchmark to identify changes, additions, or relocations of assets.

[0043] In this embodiment, the preset device database is a database used to store official asset records, such as a configuration management database (CMDB). This database stores preset information. Preset information refers to the expected state record of each asset within the data center, which may include the device's model, configuration attributes, IP address, and expected physical location and topology. Device inventory discrepancy data refers to a structured report generated after comparison. This report clearly indicates all differences between the actual information and the preset information.

[0044] In one specific implementation, a comparison processing program performs this task. The program first accesses a pre-defined equipment database to obtain pre-defined information for all assets. Then, the program compares the actual information for each device output from the previous step with this pre-defined information item by item. This comparison process not only checks the matching degree of device quantity and attributes but also analyzes whether the spatial location and topological relationship of the devices are consistent with the records. Finally, the program integrates all identified inconsistencies to generate equipment inventory discrepancy data.

[0045] The above steps enable automated detection of discrepancies between asset records and physical reality. This method can quickly generate an accurate anomaly report, clearly listing newly added, removed, or relocated devices. This allows managers to respond promptly to asset changes, ensuring the accuracy and real-time performance of the configuration management database, thereby improving the efficiency and reliability of asset management.

[0046] In one possible implementation, refer to Figure 2 . Figure 2 This is the second flowchart illustrating the data center equipment inventory method provided by the present invention. Step 101 specifically includes the following steps: Step 201: Acquire the video stream from the data center using the calibrated acquisition equipment.

[0047] Step 202: Extract at least one image frame from the video stream.

[0048] Step 203: Process the video stream using synchronous positioning and mapping technology to generate 3D point cloud data.

[0049] Specifically, the video stream from the data center is first acquired using calibrated acquisition equipment. This acquisition equipment can be a panoramic camera or other device capable of recording video. Before use, the equipment undergoes calibration, correcting its intrinsic and extrinsic parameters and lens distortion to ensure the original accuracy of the data acquisition. The video stream is a series of dynamic images continuously captured by the acquisition equipment, comprehensively recording the scene as the equipment moves within the data center.

[0050] After acquiring the video stream, the next step is to extract at least one image frame from it. Since the video stream contains a large amount of image information, extracting one or more clear image frames is sufficient to meet the subsequent needs of analyzing the device's visual features. Simultaneously, the entire video stream is used for more complex spatial calculations.

[0051] Finally, the video stream is processed using Simultaneous Localization and Mapping (SLAM) technology to generate 3D point cloud data. SLAM is an algorithm that can estimate its own position in real time and simultaneously build an environmental map in an unknown environment. Here, this technology processes the entire video stream data, tracking the motion trajectory of the acquisition device by analyzing visual changes between frames. While tracking the trajectory, the technology projects feature points from the image back into 3D space, thereby gradually constructing dense 3D point cloud data describing the entire data center environment.

[0052] Through the above steps, all two modalities of data required for subsequent processing can be obtained in a single acquisition task. Using calibrated equipment and advanced simultaneous localization and mapping (SMR) technology ensures high accuracy in both the generated image frames and 3D point cloud data. Furthermore, since both data sources originate from the same video stream, they are naturally aligned spatially and temporally, avoiding complex subsequent data registration work and significantly improving the efficiency and quality of data preparation.

[0053] In one possible implementation, the present invention provides a specific implementation of a device recognition model. The device recognition model includes a point cloud feature encoder, an image feature extraction network, a feature fusion module, an occlusion inference module, and a multi-task prediction head. Therefore, referring to... Figure 3 , Figure 3 This is the third flowchart of the data center equipment inventory method provided by the present invention. Step 102 is specifically implemented through the following steps: Step 301: Extract point cloud features from 3D point cloud data using a point cloud feature encoder.

[0054] Step 302: Extract image appearance features from image frames using an image feature extraction network.

[0055] Step 303: The image appearance features and point cloud features are fused using the feature fusion module to generate fused features.

[0056] Step 304: Identify the occlusion feature regions in the fused features that represent the occlusion of the device through the occlusion inference module, and infer and correct the occlusion feature regions based on the neighboring device information of the occlusion feature regions to generate a corrected feature map.

[0057] Step 305: Process the corrected feature map using a multi-task prediction head and output the actual information of each device in the data center in parallel.

[0058] In this embodiment, point cloud features refer to information extracted from 3D point cloud data that characterizes the device's geometry and spatial location. Image appearance features refer to information extracted from image frames that characterizes the device's visual appearance, such as color, texture, and model identification. Fusion features are a unified data representation combining point cloud features and image appearance features. The occlusion inference module is a specially designed network structure for handling complex situations where the device is partially occluded. The occlusion feature region refers to the portion of the fused features identified as corresponding to the occluded device in the physical world. The corrected feature map is feature data that has been processed by the occlusion inference module, with the occlusion region information supplemented or optimized. The multi-task prediction head is the output of the device recognition model; it can simultaneously perform multiple different prediction tasks.

[0059] In one specific implementation, the device recognition model processes the data as follows: First, a point cloud feature encoder, such as a sparse convolutional network, extracts point cloud features from the input 3D point cloud data. ,Right now Where N is the number of point clouds. This network can efficiently process sparse point clouds and generate feature vectors describing the device's geometry. Simultaneously, an image feature extraction network, such as ResNet-50, extracts image appearance features from the input image frames. ,Right now These features contain rich color and texture details of the device panel. Next, the feature fusion module fuses these two types of features. Specifically, this module projects the point cloud features onto the image coordinate system through an attention alignment layer, and then combines it with the image appearance features to generate a fused feature that contains both spatial and visual information. ,Right now .

[0060] Subsequently, the fused feature is passed to the occlusion inference module. This module first employs a depth-aware attention mechanism to calculate the depth confidence at each location in the fused feature. Regions with a depth confidence below a preset threshold are identified as occluded feature regions. For these occluded feature regions, a graph neural network structure is used for inference and correction. The graph neural network defines the occluded region as the target node to be processed and uses the information of its physically adjacent and already identified devices as known features to update the feature representation of the target node, ultimately generating a corrected feature map. Finally, the multi-task prediction head receives this corrected feature map. This prediction head contains multiple parallel branches, such as a classification branch, a localization branch, and a U-position detection branch. These branches simultaneously parse the feature map and output, in parallel, actual information such as the device type, 3D bounding box, and U-position occupancy index of each device in the data center.

[0061] The technical effect of this implementation is a significant improvement in the accuracy and robustness of device identification. By fusing features from both visual and spatial modalities, the identification results are more reliable than single-modal methods. Its core advantage lies in its dedicated occlusion inference module, which effectively solves the common device occlusion problem in data center environments, enabling accurate inference of device information even with incomplete field of view. Finally, the multi-task prediction head design allows the model to output all the key information needed for inventory checks at once, improving processing efficiency.

[0062] In one possible implementation, embodiments of the present invention provide a specific method for occlusion reasoning. (Refer to...) Figure 4 , Figure 4 This is the fourth flowchart of the data center equipment inventory method provided by the present invention. Step 304 specifically includes the following steps: Step 401: Calculate the depth confidence of each feature location on the fused feature.

[0063] Step 402: The region consisting of feature locations with depth confidence below a preset threshold is designated as the occluded feature region.

[0064] Step 403: Construct a graph neural network structure, wherein the occluded feature region is defined as the target node to be inferred, and the device that is physically adjacent to the occluded feature region and has been identified is defined as the neighboring node.

[0065] Step 404: Through graph neural network processing, the initial fused features corresponding to the target node are fused with the known features of neighboring nodes to update the features of the target node and generate the corrected feature map.

[0066] In this embodiment, depth confidence is a numerical value. This value represents the model's confidence in the accuracy of the physical spatial depth information corresponding to a certain location in the fused features. A graph neural network (GNN) is a neural network specifically designed to handle the relationships between nodes. The target node in the GNN represents the occluded feature region to be inferred. Neighboring nodes represent devices that are physically adjacent to the occluded feature region and have been successfully identified.

[0067] In one specific implementation, the occlusion inference module first performs the first step. The occlusion inference module calculates the depth confidence of each feature location on the input fused features. This can be achieved through a depth-aware attention mechanism, such as using a 3D convolutional network. Process the fusion features and use an activation function Map the results to confidence scores between 0 and 1, i.e. Next, the occlusion inference module compares the depth confidence score of each location with a preset threshold, preferably 0.5. All feature locations with depth confidence scores below the preset threshold are combined to form an occlusion feature region.

[0068] Subsequently, a graph neural network structure is constructed. In this structure, the entire occluded feature region is defined as the target node to be inferred. Simultaneously, based on spatial topology, this module defines physically adjacent and identified devices as neighboring nodes. Finally, the structure is processed by the graph neural network. This network fuses the initial fused features corresponding to the target node with the known features of the neighboring nodes. Specifically, it extracts valid feature information from the neighboring nodes and uses this information to update and complete the feature representation of the target node. After this process, the features of the target node are updated, ultimately generating a corrected feature map.

[0069] Through the steps described above, the device identification model acquires the ability to perform logical reasoning even when some information is missing. It doesn't simply ignore obscured areas; instead, it actively recovers the key features of obscured devices using contextual information from the surrounding environment. This allows it to accurately identify partially obscured devices and estimate their parameters, significantly improving the final identification accuracy in crowded and complex data center environments.

[0070] In one possible implementation, embodiments of the present invention provide a method for training a device recognition model. (Refer to...) Figure 5 , Figure 5 This is the fifth flowchart of the data center equipment inventory method provided by the present invention. The equipment identification model is specifically trained through the following steps: Step 501: Obtain image frame samples and 3D point cloud data samples of the data center under any environmental conditions, and obtain the annotation information corresponding to all device samples in the data center.

[0071] Step 502: Input the image frame samples and 3D point cloud data samples under any environmental condition into the initial model to obtain the prediction information corresponding to each device sample output by the initial model.

[0072] Step 503: Construct a comprehensive loss function based on the annotation and prediction information corresponding to each device sample.

[0073] Step 504: Train the initial model with the goal of minimizing the comprehensive loss function to obtain the device recognition model.

[0074] In this embodiment, image frame samples and 3D point cloud data samples are paired data used for training. Labeling information is the true answer or label associated with each sample, providing a supervisory signal for model training. Environmental conditions refer to the specific physical circumstances when the samples were collected. The initial model refers to a model with a preset network structure but whose parameters have not yet been optimized before training begins. Prediction information is the preliminary result output by the initial model for the input samples. The comprehensive loss function is a quantitative metric used to calculate the gap between the predicted information and the labeled information.

[0075] In one specific implementation, the training process begins with step 501. The operator acquires image frame samples and 3D point cloud data samples from the data center under any environmental conditions. These environmental conditions specifically refer to different lighting conditions, acquisition angles, and device occlusion to ensure the diversity of the training data. Simultaneously, the annotation information corresponding to all device samples within the data center is acquired. Next, in step 502, the image frame samples and 3D point cloud data samples under any environmental condition are input into the initial model. After processing the input, the initial model outputs the prediction information corresponding to each device sample.

[0076] Then, in step 503, a comprehensive loss function is constructed based on the annotation and prediction information corresponding to each device sample. This function is designed to comprehensively evaluate the model's performance; subsequent embodiments will detail the method for constructing the comprehensive loss function. Finally, in step 504, the initial model is trained with the goal of minimizing the comprehensive loss function. This training process uses optimization algorithms, such as backpropagation and gradient descent, to repeatedly adjust the internal parameters of the initial model. Each adjustment aims to reduce the value of the comprehensive loss function. This process continues until the function value converges or a preset number of training epochs is reached. The initial model after training becomes the final device recognition model.

[0077] The technical advantage of this implementation is that it yields a high-performance device recognition model. Because the training data covers various lighting conditions, angles, and occlusion scenarios, the model learns more robust features, significantly improving its generalization ability. This means the model not only performs well on the training data but also maintains high accuracy when facing real-world deployment environments it has never encountered before. By optimizing with a comprehensive loss function as the objective, the model achieves balanced and excellent performance across multiple key tasks, including device classification, localization, and U-position detection.

[0078] In one possible implementation, the annotation information includes a device type label, a 3D bounding box label, and a U-position occupancy index label; the prediction information includes a device type prediction probability, a predicted 3D bounding box, and a predicted U-position occupancy index. Therefore, referring to... Figure 6 , Figure 6This is a flowchart of the data center equipment inventory method provided by the present invention, step 503 is specifically implemented through the following steps: Step 601: Construct a classification loss based on the device type label of each device sample and the predicted probability of all device types.

[0079] Step 602: Construct the 3D localization loss based on the intersection-union ratio, center point distance, and aspect ratio consistency between the center point of the 3D bounding box label of each device sample and the predicted 3D bounding box.

[0080] Step 603: Map the U-position occupancy index label of each device sample to a continuous value label, and map the predicted U-position occupancy index of each device sample to a predicted continuous value.

[0081] Step 604: Construct the U-position detection loss based on the difference between the predicted continuous values ​​and the continuous value labels of each device sample.

[0082] Step 605: Construct a comprehensive loss function based on the weighted sum of classification loss, 3D localization loss, U-position detection loss, and regularization term.

[0083] Specifically, in step 601, the device type label is a pre-defined one-hot encoding representing the true category of each device sample. The device type prediction probability is the distribution of the model's confidence in predicting that a device sample belongs to each possible category. The classification loss is a calculated value that reflects the difference between the probability distribution predicted by the model and the true device type label.

[0084] Furthermore, classification loss The Focal Loss function is used for calculation. The calculation method is as follows: In this formula, The label represents the actual device type of the device sample. If the sample belongs to category i, its value is 1, otherwise it is 0. The representative model predicts the probability that the device sample belongs to device type i. This formula includes a modulation factor. This modulation factor dynamically adjusts the weight of each sample in the total loss. When a sample is easily and correctly classified by the model, its... The value is close to 1, and the modulation factor is close to 0, thus reducing the contribution of the device's sample to the total loss.

[0085] In step 602, the 3D bounding box label refers to the six-parameter data describing the true spatial location and dimensions of the device sample. The predicted 3D bounding box is the six-parameter data of the predicted spatial location and dimensions output by the model for the device sample. The 3D localization loss is a numerical value used to measure the geometric difference between the predicted 3D bounding box and the 3D bounding box label.

[0086] Furthermore, 3D positioning loss The calculation is performed using CIoU Loss, and the formula is as follows: The above formula comprehensively considers three key geometric factors. The first factor is the Intersection over Union (IoU), which represents the degree of volume overlap between the predicted 3D bounding box and its label. The second factor is the center point distance, determined by the formula... The item indicates that, among which, It predicts the 3D bounding box. It is a 3D bounding box tag. express and The square of the Euclidean distance between the centers of the two points. c is the value that can be simultaneously covered. and The diagonal length of the minimum bounding box is used as a normalization factor. This term penalizes the deviation between the center point of the predicted box and the ground truth box. The third factor is aspect ratio consistency, determined by the formula... The term is represented as follows, where v is a metric parameter used to measure... and The consistency of aspect ratio. α is a positive weighting coefficient used to adjust the importance of aspect ratio consistency in the total positioning loss. The final 3D positioning loss is a value calculated by combining these three factors.

[0087] In step 603, continuous value labels refer to floating-point values ​​generated by mapping the actual U-bit occupancy index labels. Predicted continuous values ​​refer to floating-point values ​​generated by mapping the U-bit occupancy indexes predicted by the model. This step first processes the U-bit occupancy index labels for each device sample. A predefined mapping function... This is applied to the U-bit occupancy index label. The mapping function converts U-bit indices, such as U1, U2, etc., into corresponding continuous values, such as 1.0, 2.0, etc. For example, if the U-bit occupancy index label for a device sample is U5, the resulting continuous value label after mapping is 5.0. Then, this step performs the same operation on the predicted U-bit occupancy index for that device sample. If the model's predicted U-bit occupancy index for that device sample is U6, then after mapping... After processing, the predicted continuous value is 6.0. The output of step 603 is a pair of values ​​generated for each device sample: a continuous value label and a predicted continuous value.

[0088] In step 604, the U-position detection loss is a numerical value. This value represents the average deviation between the U-position predicted by the model and the actual U-position in a batch of device samples.

[0089] Furthermore, step 604 constructs the U-position detection loss based on the difference between the predicted continuous values ​​and the continuous value labels of each device sample. Specifically, the U-position detection loss is calculated using ordinal regression loss, and its calculation formula is as follows: In this formula, N is the total number of device samples in the current calculation batch. This means averaging the loss across all samples. The core of the formula is... .in, It is the continuous value label of the m-th device sample, which represents the true U-bit of that device sample. It is the continuous predicted value of the model output for the m-th device sample. This is the absolute value operator, used to calculate the absolute magnitude of the difference between the predicted and actual values. The entire calculation process is as follows: for each sample in the batch, calculate the absolute difference between its predicted continuous value and the continuous value label, then add all these differences together, and finally divide by the total number of samples N to obtain the average U-bit loss.

[0090] In step 605, the regularization term is an additional loss term. Its main function is to constrain the complexity of the model and prevent overfitting on the training data. Specifically, in this scheme, the regularization term also applies constraints to the parameters of the occlusion inference module to improve its generalization ability.

[0091] The comprehensive loss function. Its specific calculation formula is as follows: In this formula, It is the final value of the overall loss function. , and These are the classification loss, 3D localization loss, and U-position detection loss that were constructed in the previous steps. It is a regularization term, which can be the L2 norm of the graph neural network parameters in the occlusion inference module, i.e. α, β, γ, and λ are used to penalize the parameter size of the occlusion inference module. α, β, γ, and λ are pre-defined weighting coefficients. These coefficients are used to adjust the proportion of each sub-loss in the overall loss function.

[0092] Through the steps described above, a unified and adjustable comprehensive loss function was used to effectively train the multi-task model. By adjusting the weight coefficients α, β, γ, and λ, operators can flexibly control the learning priority of the model on different tasks according to specific needs. This approach ensures that the model does not neglect the learning of other tasks due to the excessive value of one sub-loss, enabling the model to achieve a better performance balance across multiple dimensions such as classification, localization, and U-position detection, thereby improving the overall performance of the device recognition model and the accuracy of the final inventory count.

[0093] Reference Figure 7 , Figure 7 This is a schematic diagram of the data center equipment inventory system provided by the present invention. The system includes: The acquisition module is used to acquire image frames and 3D point cloud data from the data center; The processing module is used to input image frames and 3D point cloud data into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples; The comparison module is used to compare the actual information of each device with the preset information stored in the preset device database to obtain the device inventory difference data.

[0094] In one possible implementation, the processing module is further configured to: Point cloud features are extracted from 3D point cloud data using a point cloud feature encoder. Image appearance features are extracted from image frames using an image feature extraction network; The feature fusion module fuses image appearance features and point cloud features to generate fused features. The occlusion inference module identifies the occlusion feature regions in the fused features that represent the occlusion of the device, and infers and corrects the occlusion feature regions based on the neighboring device information of the occlusion feature regions to generate a corrected feature map. The corrected feature map is processed by a multi-task prediction head, and the actual information of each device in the data center is output in parallel.

[0095] In one possible implementation, the processing module is further configured to: Calculate the depth confidence of each feature location on the fused feature; The region consisting of feature locations with depth confidence below a preset threshold is designated as the occluded feature region. Construct a graph neural network structure, in which occlusion feature regions are defined as target nodes to be inferred, and devices that are physically adjacent to the occlusion feature regions and have been identified are defined as neighboring nodes. By using graph neural network processing, the initial fusion features corresponding to the target node are fused with the known features of neighboring nodes to update the features of the target node and generate a corrected feature map.

[0096] In one possible implementation, the system further includes a model training module for: Acquire image frame samples and 3D point cloud data samples from the data center under any environmental conditions, and obtain the annotation information corresponding to all device samples in the data center; Input image frame samples and 3D point cloud data samples under any environmental condition into the initial model to obtain the prediction information corresponding to each device sample output by the initial model; A comprehensive loss function is constructed based on the annotation and prediction information corresponding to each device sample. The initial model is trained with the goal of minimizing the comprehensive loss function to obtain the device recognition model.

[0097] In one possible implementation, the model training module is further used for: A classification loss is constructed based on the device type label of each device sample and the predicted probability of all device types. Based on the intersection-union ratio, center point distance, and aspect ratio consistency between the center point of the 3D bounding box label of each device sample and the predicted 3D bounding box, a 3D localization loss is constructed. Map the U-position occupancy index label of each device sample to a continuous value label, and map the predicted U-position occupancy index of each device sample to a predicted continuous value; Based on the difference between the predicted continuous values ​​and the continuous value labels of each device sample, a U-position detection loss is constructed; A comprehensive loss function is constructed based on the weighted sum of classification loss, 3D localization loss, U-position detection loss, and regularization term.

[0098] In one possible implementation, the acquisition module is further configured to: The video stream from the data center is acquired using calibrated acquisition equipment. Extract at least one image frame from the video stream; The video stream is processed using simultaneous localization and mapping (SLAM) technology to generate 3D point cloud data.

[0099] It should be noted that the data center equipment inventory system provided by the present invention can execute the data center equipment inventory method of any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0100] Figure 8 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 8As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830, and a communication bus 840. The processor 810, communication interface 820, and memory 830 communicate with each other via the communication bus 840. The processor 810 can call logical instructions in the memory 830 to execute a data center equipment inventory method. This method includes: acquiring image frames and 3D point cloud data of the data center; inputting the image frames and 3D point cloud data into a device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein the device recognition model is trained based on image frame samples and 3D point cloud data samples; and comparing the actual information of each device with preset information stored in a preset device database to obtain equipment inventory discrepancy data.

[0101] Furthermore, the logical instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0102] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions, and when the program instructions are executed by a computer, the computer is able to execute the data center equipment inventory method provided in the above embodiments.

[0103] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the data center equipment inventory method provided in the above embodiments.

[0104] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0105] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or some parts of embodiments.

[0106] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for inventorying data center equipment, characterized in that, include: Acquire image frames and 3D point cloud data from the data center; The image frames and the 3D point cloud data are input into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples; The actual information of each device is compared with the preset information stored in the preset device database to obtain the device inventory difference data.

2. The data center equipment inventory method according to claim 1, characterized in that, The device recognition model includes a point cloud feature encoder, an image feature extraction network, a feature fusion module, an occlusion inference module, and a multi-task prediction head; the step of inputting the image frame and the 3D point cloud data into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model includes: Point cloud features are extracted from the 3D point cloud data using the point cloud feature encoder. The image feature extraction network extracts image appearance features from the image frame; The feature fusion module fuses the image appearance features and the point cloud features to generate fused features. The occlusion inference module identifies the occlusion feature regions in the fused features that represent the occlusion of the device, and infers and corrects the occlusion feature regions based on the adjacent device information of the occlusion feature regions to generate a corrected feature map. The corrected feature map is processed by the multi-task prediction head, and the actual information of each device in the data center is output in parallel.

3. The data center equipment inventory method according to claim 2, characterized in that, The step of identifying occlusion feature regions representing device occlusion in the fused features through the occlusion inference module, and inferring and correcting the occlusion feature regions based on the adjacent device information of the occlusion feature regions to generate a corrected feature map, specifically includes: Calculate the depth confidence of each feature position on the fused feature; The region formed by feature locations whose depth confidence is lower than a preset threshold is defined as the occluded feature region. A graph neural network structure is constructed, wherein the occlusion feature region is defined as the target node to be inferred, and the device that is physically adjacent to the occlusion feature region and has been identified is defined as the neighbor node; The graph neural network process fuses the initial fusion features corresponding to the target node with the known features of the neighboring nodes to update the features of the target node and generate a corrected feature map.

4. The data center equipment inventory method according to claim 1, characterized in that, It also includes the training method for the device recognition model: Acquire image frame samples and 3D point cloud data samples of the data center under any environmental conditions, and acquire annotation information corresponding to all device samples in the data center; The image frame samples and the three-dimensional point cloud data samples under any of the environmental conditions are input into the initial model to obtain the prediction information corresponding to each of the device samples output by the initial model. A comprehensive loss function is constructed based on the annotation information and prediction information corresponding to each of the device samples. The initial model is trained with the goal of minimizing the comprehensive loss function to obtain the device recognition model.

5. The data center equipment inventory method according to claim 4, characterized in that, The annotation information includes device type label, 3D bounding box label, and U-position occupancy index label; the prediction information includes device type prediction probability, predicted 3D bounding box, and predicted U-position occupancy index.

6. The data center equipment inventory method according to claim 5, characterized in that, The step of constructing a comprehensive loss function based on the annotation information and prediction information corresponding to each of the device samples includes: A classification loss is constructed based on the device type label of each device sample and the predicted probability of all device types; Based on the intersection-union ratio, center point distance, and aspect ratio consistency between the center point of the 3D bounding box label of each device sample and the predicted 3D bounding box, a 3D localization loss is constructed. Map the U-bit occupancy index label of each of the device samples to a continuous value label, and map the predicted U-bit occupancy index of each of the device samples to a predicted continuous value; Based on the difference between the predicted continuous value and the continuous value label of each of the device samples, a U-position detection loss is constructed; The comprehensive loss function is constructed based on the weighted sum of the classification loss, the 3D localization loss, the U-position detection loss, and the regularization term.

7. The data center equipment inventory method according to claim 4, characterized in that, The environmental conditions include lighting conditions, acquisition angle, and equipment obstruction.

8. The data center equipment inventory method according to claim 1, characterized in that, The acquisition of image frames and 3D point cloud data from the data center includes: The video stream from the data center is acquired using calibrated acquisition equipment. Extract at least one of the image frames from the video stream; The video stream is processed using synchronous positioning and mapping technology to generate the 3D point cloud data.

9. A data center equipment inventory system, characterized in that, include: The acquisition module is used to acquire image frames and 3D point cloud data from the data center; The processing module is used to input the image frame and the 3D point cloud data into the device recognition model to obtain the actual information of each device in the data center output by the device recognition model; wherein, the device recognition model is trained based on image frame samples and 3D point cloud data samples; The comparison module is used to compare the actual information of each device with the preset information stored in the preset device database to obtain device inventory difference data.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data center equipment inventory method as described in any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data center equipment inventory method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the data center equipment inventory method as described in any one of claims 1 to 8.

Images