A method and system for detecting cabinet U-position anomalies based on multimodal feature fusion

Abstract

This invention provides a method and system for detecting U-position anomalies in server racks based on multimodal feature fusion. The method includes: acquiring a reference image and a detection image of the target server rack; extracting multidimensional features of the target server rack from the reference image and the detection image; and outputting the U-position anomaly detection result of the target server rack based on the multidimensional features of the target server rack in the reference image and the detection image. The method in this application, based on the multidimensional features of the target server rack in the reference image and the detection image, achieves accurate identification of the actual installation status of the U-position equipment in the server rack, significantly improving the accuracy and comprehensiveness of server rack U-position anomaly detection.

Description

Technical Field

[0001] This invention relates to the field of communication technology, and in particular to a method and system for detecting cabinet U-position anomalies based on multimodal feature fusion. Background Technology

[0002] With the development and construction of data centers, increasingly higher demands are being placed on the real-time monitoring and timely, accurate, and comprehensive inspection of various IT equipment within the data center. Among related technologies, RFID-based solutions essentially rely on online status detection of USB storage modules, which cannot guarantee the true status of the corresponding IT equipment. Summary of the Invention

[0003] This invention provides a method and system for detecting anomalies in rack U-positions based on multimodal feature fusion. By combining the multidimensional features of the target rack in the reference image and the multidimensional features of the target rack in the detection image, it achieves accurate identification of the actual installation status of the U-position equipment in the rack, significantly improving the accuracy and comprehensiveness of rack U-position anomaly detection. It effectively solves the shortcomings of traditional RFID detection, which can only monitor the online status of U-position modules and cannot truly reflect the actual on-site installation status of IT equipment. It effectively meets the actual business needs of data centers for real-time monitoring, high-precision detection, and refined inventory of IT equipment.

[0004] This invention provides a method for detecting cabinet U-position anomalies based on multimodal feature fusion, comprising the following steps: Acquire baseline and inspection images of the target cabinet; Extract the multidimensional features of the target cabinet from the reference image and the multidimensional features of the target cabinet from the detection image; the multidimensional features include the structural features, appearance features and regional statistical features of the target cabinet; Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, the U-position anomaly detection result of the target cabinet is output.

[0005] According to the present invention, a method for detecting anomalies in the U-position of a server rack based on multimodal feature fusion is provided, wherein the structural features of the target server rack are extracted in the following manner: The reference image and detection image of the target cabinet are preprocessed, including grayscale conversion, contrast enhancement and Gaussian noise reduction. The preprocessed reference image and the preprocessed detection image are used as inputs to the SIFT algorithm to obtain the structural features of the target cabinet in the reference image and the structural features of the target cabinet in the detection image.

[0006] According to the present invention, a method for detecting cabinet U-position anomalies based on multimodal feature fusion is provided, wherein the appearance features of the target cabinet include: color features, texture features, and gradient statistical features.

[0007] According to the present invention, a method for detecting cabinet U-position anomalies based on multimodal feature fusion is provided, wherein the regional statistical features of the target cabinet include: shape moment features, area statistical features, geometric features, and spatial density features.

[0008] According to the present invention, a method for detecting U-position anomalies in server racks based on multimodal feature fusion is provided, wherein the step of outputting the U-position anomaly detection result of the target server rack based on the multidimensional features of the target server rack in the reference image and the multidimensional features of the target server rack in the detection image includes: Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, structural confidence, appearance confidence, and regional statistical confidence are determined. The structural confidence indicates the degree of matching of structural features between the reference image and the detection image; the appearance confidence indicates the degree of difference in appearance features between the reference image and the detection image; and the regional statistical confidence indicates the degree of matching of statistical features of each region between the reference image and the detection image. Based on the structural confidence level, appearance confidence level, and regional statistical confidence level, the U-position anomaly detection result of the target cabinet is determined.

[0009] According to the present invention, a method for detecting U-position anomalies in a server rack based on multimodal feature fusion is provided. The step of determining the U-position anomaly detection result of the target server rack based on the structural confidence level, appearance confidence level, and regional statistical confidence level includes: The structural confidence, appearance confidence, and regional statistical confidence are weighted and fused to obtain the fused confidence. The target cabinet is determined to have a U-position anomaly based on the fusion confidence level.

[0010] According to the present invention, a method for detecting cabinet U-position anomalies based on multimodal feature fusion, after acquiring the reference image and the detection image of the target cabinet, the method further includes: The cabinet border and cabinet vertices in the reference image and the detection image are identified respectively; Based on the identified rack frame and rack vertices, the rack areas in the reference image and the detection image are determined respectively; The cabinet area in the reference image and the cabinet area in the detection image are corrected respectively to obtain the corrected reference image and the corrected detection image.

[0011] This invention also provides a cabinet U-position anomaly detection system based on multimodal feature fusion, comprising the following modules: The acquisition module is used to acquire the baseline image and inspection image of the target cabinet; The extraction module is used to extract multidimensional features of the target cabinet in the reference image and multidimensional features of the target cabinet in the detection image; the multidimensional features include structural features, appearance features and regional statistical features of the target cabinet; The detection module is used to output the U-position anomaly detection result of the target cabinet based on the multi-dimensional features of the target cabinet in the reference image and the multi-dimensional features of the target cabinet in the detection image.

[0012] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the cabinet U-position anomaly detection method based on multimodal feature fusion as described above.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cabinet U-position anomaly detection method based on multimodal feature fusion as described above.

[0014] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cabinet U-position anomaly detection method based on multimodal feature fusion as described above.

[0015] This invention provides a cabinet U-position anomaly detection method and system based on multimodal feature fusion. By combining the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, it can accurately determine the actual installation status of the cabinet U-position equipment, significantly improving the accuracy and comprehensiveness of cabinet U-position anomaly detection. It effectively solves the shortcomings of traditional RFID detection, which can only monitor the online status of U-position modules and cannot truly reflect the actual on-site installation status of IT equipment. It effectively meets the actual business needs of data centers for real-time monitoring, high-precision detection, and refined inventory of IT equipment. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1This is one of the flowcharts of the cabinet U-position anomaly detection method based on multimodal feature fusion provided by the present invention.

[0018] Figure 2 is a schematic diagram of the cabinet U-position anomaly detection effect provided by the present invention.

[0019] Figure 3 is a schematic diagram of the cabinet image acquisition provided by the present invention.

[0020] Figure 4 is a schematic diagram of the cabinet image preprocessing and correction effect provided by the present invention.

[0021] Figure 5 This is the second flowchart of the cabinet U-position anomaly detection method based on multimodal feature fusion provided by the present invention.

[0022] Figure 6 This is a schematic diagram of the cabinet U-position anomaly detection system based on multimodal feature fusion provided by the present invention.

[0023] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] The following is combined Figures 1 to 7 This invention describes a method and system for detecting cabinet U-position anomalies based on multimodal feature fusion.

[0026] To facilitate a clearer understanding of the technical solutions of the various embodiments of this application, some technical content related to the various embodiments of this application will be introduced first.

[0027] With the development and construction of data centers, increasingly higher demands are being placed on the real-time monitoring and timely, accurate, and comprehensive inspection of various IT equipment within the server room. Currently, RFID-based or improved sensor + RFID technology is widely used for remote monitoring of rack-level U-position equipment, achieving good results. However, this type of solution monitors the online status of the U-position module, not the equipment itself, which leads to the following problems: First, when the backend system detects an anomaly in the U-position, it cannot determine whether the problem is due to poor communication, misaligned connections, or detached tags in the U-position module, or whether the equipment has actually been illegally removed / moved. Second, although the online monitoring of the U-position module shows normal operation, if the actual system service is interrupted, it is difficult to distinguish whether it is due to the illegal removal / moving of the relevant equipment or other reasons. In summary, existing RFID-based or sensor + improved RFID technology solutions essentially detect the online status of the U-position module and cannot ensure the true status of the corresponding IT equipment. When an alarm for an abnormal U-position status occurs, it is impossible to determine the cause and the risk level of the fault, which is not conducive to quickly generating a response plan.

[0028] Figure 1 This is one of the flowcharts of the cabinet U-position anomaly detection method based on multimodal feature fusion provided by the present invention, such as... Figure 1 As shown, the method includes the following: Step 101: Obtain the baseline image and inspection image of the target cabinet.

[0029] Specifically, in this embodiment, an image of the server rack under normal installation conditions is used as the reference image, and an image of the server rack at the moment of inspection is used as the inspection image. The reference image includes the standard rack U-position layout and the compliant installation status of the equipment, providing a unified reference standard for identifying rack U-position anomalies, thereby effectively ensuring the accuracy of rack U-position detection.

[0030] Step 102: Extract the multidimensional features of the target cabinet from the reference image and the multidimensional features of the target cabinet from the detection image; the multidimensional features include the structural features, appearance features and regional statistical features of the target cabinet.

[0031] Specifically, after acquiring the reference image and the detection image of the target cabinet, this application can extract features from the reference image and the detection image respectively to obtain the structural features, appearance features, and regional statistical features corresponding to the target cabinet. Among them, the structural features represent the frame structure information of the cabinet frame and the U-position card slots; the appearance features represent the appearance details of the cabinet equipment such as color, texture, and gradient; and the regional statistical features represent the shape, area, spatial density, and other statistical information of local areas of the cabinet. This enables multi-dimensional and comprehensive feature acquisition of the cabinet image, thereby effectively identifying various abnormal scenarios such as structural changes, appearance modifications, and additions or removals of equipment in the cabinet's U-positions, significantly improving the accuracy and comprehensiveness of cabinet U-position detection.

[0032] Step 103: Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, output the U-position anomaly detection result of the target cabinet.

[0033] Specifically, after extracting the multidimensional features of the target cabinet from the reference image and the detection image, this application compares the corresponding multidimensional features of the reference image and the detection image one by one. This accurately quantifies the degree of structural matching, appearance difference, and regional statistical feature matching between the two images, comprehensively determining whether the target cabinet has any abnormalities such as misinstallation, omission, over-installation, or offset of U-position equipment, and outputs the corresponding U-position anomaly detection results. In other words, this application overcomes the shortcomings of traditional RFID, which can only detect the online status of modules and cannot identify the actual installation status of equipment. By comparing multidimensional image features, it accurately identifies the actual installation status of U-position equipment in the cabinet, effectively improving the accuracy and comprehensiveness of cabinet U-position anomaly detection, and meeting the business needs of high-precision monitoring and inventory of data center IT equipment.

[0034] In other words, this application utilizes a multimodal feature fusion-based U-position anomaly detection technology. It compares images acquired under abnormal conditions with previously stored images, analyzing and processing multi-dimensional features including structural, appearance, and statistical characteristics to determine the location and category of image differences. This assists managers in quickly locating, analyzing, and diagnosing problems related to anomalies. It should be noted that this application's technical solution combines three complementary features—structural, appearance, and statistical—overcoming the limitations of single-feature detection. It exhibits good adaptability to changes in lighting, viewing angle differences, and partial occlusion, demonstrating strong robustness. Specifically designed for the unique needs of cabinet inspection, it incorporates advanced feature selection and parameter optimization. Through multimodal fusion and intelligent decision-making mechanisms, it significantly improves the system's reliability and practicality while maintaining high detection accuracy.

[0035] The method described in the above embodiments, based on the multi-dimensional features of the target cabinet in the reference image and the multi-dimensional features of the target cabinet in the detection image, achieves accurate identification of the actual installation status of the U-position equipment in the cabinet, significantly improving the accuracy and comprehensiveness of cabinet U-position anomaly detection. It effectively solves the shortcomings of traditional RFID detection, which can only monitor the online status of U-position modules and cannot truly reflect the actual on-site installation status of IT equipment. It effectively meets the actual business needs of data centers for real-time monitoring, high-precision detection, and refined inventory of IT equipment.

[0036] In some embodiments, the structural features of the target cabinet are extracted based on the following method: The baseline and detection images of the target cabinet are preprocessed, including grayscale conversion, contrast enhancement, and Gaussian denoising. The preprocessed reference image and the preprocessed detection image are used as inputs to the SIFT algorithm to obtain the structural features of the target cabinet in the reference image and the structural features of the target cabinet in the detection image.

[0037] Specifically, before extracting structural features, this application first preprocesses the reference image and the detection image to reduce interference from lighting, shooting noise, and insufficient image contrast, thereby optimizing image quality and making the cabinet structure outline features clearer. Optionally, the preprocessing includes grayscale conversion, contrast enhancement, and Gaussian denoising preprocessing on the reference image and the detection image of the target cabinet; wherein, grayscale conversion can simplify image channel information and reduce the computational load of subsequent feature extraction; contrast enhancement can highlight the cabinet frame, U-slot slots, and equipment edge details; Gaussian denoising can filter out interference information such as salt-and-pepper noise and texture noise caused by the on-site shooting environment.

[0038] Optionally, after preprocessing the reference image and the detection image of the target cabinet, the reference image and the detection image after grayscale conversion, contrast enhancement and Gaussian denoising can be input into the SIFT algorithm respectively. By utilizing the scale invariance and rotation invariance properties of the SIFT algorithm, the structural features of the target cabinet in the reference image and the structural features of the target cabinet in the detection image are extracted respectively, thereby improving the accuracy of cabinet U-position structural anomaly detection.

[0039] For example, in this embodiment, structural feature extraction is mainly performed using the SIFT (Scale-Invariant Feature Transform) algorithm. Before calling the SIFT algorithm, the image is preprocessed first. First, grayscale conversion is performed, transforming the 3-channel (RGB) image into a 1-channel (grayscale) image, allowing the focus to be on extracting structural shape-related features. Second, contrast enhancement is performed using histogram equalization to stretch the image's grayscale distribution, improving local contrast and making features such as edges and corners more prominent, while also reducing the impact of illumination changes on detection. Third, Gaussian denoising is performed to reduce the influence of sensor noise or random noise caused by environmental interference during image acquisition. The preprocessed image can then be used as input to the SIFT algorithm to extract 128-dimensional structural features.

[0040] The method described in the above embodiments, by preprocessing the reference image and the detection image, can effectively suppress interference caused by changes in lighting, noise, and shooting conditions, simplify redundant image information, enhance the edge details of the cabinet structure, and reveal the structural features of the target cabinet.

[0041] The method described in the above embodiments, by preprocessing the reference image and the detection image, can effectively suppress interference caused by changes in illumination, image noise, and differences in shooting environment, simplify redundant image information, and enhance the edge details of the cabinet structure. This allows for the accurate extraction of the structural features of the target cabinet and improves the accuracy of cabinet U-position anomaly detection.

[0042] In some embodiments, the appearance features of the target cabinet include: color features, texture features, and gradient statistical features.

[0043] Specifically, in this embodiment, the appearance features of the target cabinet include color features, texture features, and gradient statistical features. Color features characterize the color distribution of equipment and panels within the cabinet's U-position area, effectively identifying color anomalies caused by equipment replacement, missing parts, or obstruction by foreign objects. Texture features characterize the texture coarseness and distribution information of the cabinet surface and equipment panels, distinguishing between different models and installation states of IT equipment, and accurately identifying subtle anomalies such as equipment misalignment or improper installation. Gradient statistical features characterize the degree of grayscale change at the edges of the cabinet image, focusing on the gradient information of the cabinet border, U-position interfaces, and equipment edges, effectively capturing subtle changes in appearance structure such as equipment edge offset, equipment protrusion, and abnormal installation gaps. In other words, this application, through multi-dimensional appearance features, can accurately capture various subtle appearance changes in the cabinet's U-position area, effectively identifying appearance-related U-position anomalies such as equipment misinstallation, omissions, offsets, equipment replacement, and surface obstruction, significantly improving the accuracy and reliability of cabinet U-position anomaly detection.

[0044] For example, the extraction and processing of appearance features in this application mainly considers color and texture features. First, multi-color space conversion analysis is performed based on the original color image, converting the image from the BGR color space to HSV and Lab spaces respectively. Then, a 64-bin normalized histogram is calculated on the three channels of each color space to form a color distribution feature vector. Next, texture features are extracted. After converting the image to grayscale, a Local Binary Pattern (LBP) texture histogram is calculated, and the gradient magnitude is calculated and its statistical features (mean, standard deviation, maximum, and minimum) are extracted. Then, color moment features are calculated, including the mean, standard deviation, and third moment of each color channel. Finally, all extracted color histograms, texture features, gradient statistics, and color moments are combined into a complete appearance feature description, providing rich visual feature information for subsequent multimodal fusion. Optionally, the total dimension of appearance features = 3 (color space) × 3 (channels) × 64 (bin) = 576-dimensional color features.

[0045] The method described in the above embodiments can accurately capture various subtle appearance changes in the rack U-position area through multi-dimensional appearance features such as color, texture, and gradient statistics. It can effectively make up for the shortcomings of structural features that can only identify frame deformation, effectively identify subtle anomalies in the rack U-position, and improve the comprehensiveness and accuracy of U-position anomaly detection.

[0046] In some embodiments, the regional statistical characteristics of the target cabinet include: shape moment characteristics, area statistical characteristics, geometric characteristics, and spatial density characteristics.

[0047] Specifically, in this embodiment, the regional statistical features of the target cabinet include shape moment features, area statistical features, geometric features, and spatial density features. Shape moment features characterize the device outline within the U-position area of ​​the cabinet, effectively identifying anomalies such as device deformation and inconsistent regional outlines. Area statistical features are used to statistically analyze the pixel area of ​​the effective U-position area and the area occupied by the devices, accurately identifying area-related anomalies such as missing devices, insufficient or excessive installation area occupancy. Geometric features characterize geometric parameters such as the aspect ratio, area angle, and boundary distance of the U-position area, effectively identifying geometric installation anomalies such as device tilt, installation offset, and area deformation. Spatial density features are used to statistically analyze the pixel distribution and device density of the local U-position area, identifying subtle spatial distribution anomalies such as device stacking and local gaps. In other words, this application, through four types of regional statistical features, can accurately and comprehensively determine the shape, size, geometric relationship, and spatial distribution of the local U-position area of ​​the cabinet, achieving precise identification of device installation anomalies.

[0048] For example, in this embodiment, the regional statistical features of the target cabinet are based on the detected differential region contours. First, the shape moment features of each contour are calculated to obtain the geometric moments of the contours and derive 7 Hu invariant moments, which are then logarithmically transformed to enhance numerical stability. Next, spatial distribution statistical analysis is performed, calculating the area statistics of all contours, including total area, average area, standard deviation, and maximum area, while also counting the number of regions. Then, geometric features are extracted, calculating geometric attributes such as roundness, compactness, aspect ratio, and fill degree for each contour. Finally, spatial density analysis is performed, calculating the ratio of the total area of ​​the differential regions to the total number of pixels in the entire image. The shape moments, area statistics, geometric features, and spatial density are combined to form a complete statistical feature descriptor, providing regional-level statistical characteristic information for multimodal fusion. Optionally, the total dimension of the regional statistical features of the target cabinet = 7 (shape moments) + 5 (area statistics) + 4 (geometric features) + 1 (spatial density) = 17-dimensional statistical features.

[0049] The method described above, through multi-dimensional regional statistical features such as shape moment features, area statistical features, geometric features, and spatial density features, can accurately capture regional anomalies such as equipment gaps, area deviations, geometric deformations, and abnormal spatial arrangements in the U-position area, significantly improving the comprehensiveness and accuracy of cabinet U-position anomaly detection.

[0050] In some embodiments, based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, the U-position anomaly detection result of the target cabinet is output, including: Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, the structural confidence, appearance confidence, and regional statistical confidence are determined. The structural confidence indicates the degree of matching of structural features between the reference image and the detection image; the appearance confidence indicates the degree of difference in appearance features between the reference image and the detection image; and the regional statistical confidence indicates the degree of matching of statistical features of each region between the reference image and the detection image. Based on structural confidence, appearance confidence, and regional statistical confidence, the U-position anomaly detection results of the target cabinet are determined.

[0051] Specifically, in this embodiment, after extracting the multidimensional features of the target cabinet in the reference image and the detection image, the multidimensional features corresponding to the reference image and the detection image can be compared one by one to determine the structural confidence, appearance confidence, and regional statistical confidence. The structural confidence quantifies the similarity between the two cabinet images in structural dimensions such as frame outline and U-slot slots; the appearance confidence quantifies the degree of difference between the two images in appearance dimensions such as color, texture, and gradient; and the regional statistical confidence quantifies the overall matching degree between the two images in terms of the shape, area, geometric form, and spatial distribution of the U-slot area. Optionally, after obtaining the structural confidence, appearance confidence, and regional statistical confidence, the magnitude and deviation of the three confidence scores can be comprehensively evaluated to determine whether the target cabinet has abnormal conditions such as misinstallation, omission, over-installation, offset, replacement, or obstruction of U-slot equipment, ultimately outputting the U-slot anomaly detection result corresponding to the target cabinet.

[0052] For example, in this embodiment, the quantified structural confidence provides a basis for evaluating the structural feature dimension for multimodal fusion decision-making. It can be based on SIFT feature matching results for quantitative analysis, converting the SIFT matching results into a 0-1 confidence score, comprehensively considering the two core factors: the number of matching points and the reprojection error. Optionally, the formula for calculating the structural confidence C_struct is as follows: C_struct = α×(Nmatch / Nmax) + β×exp(-E_reproj / σ) Where Nmatch represents the number of matching points, Nmax is the maximum number of reference matching points (set to 50), which is normalized by dividing by 50, E_reproj is the reprojection error, which uses an exponential function to ensure that the smaller the error, the higher the confidence level, σ is the error scaling factor (set to 5), and the influence weights of the two factors are adjusted by α and β, which can be set to 0.6 and 0.4 respectively.

[0053] Optionally, appearance confidence is achieved by calculating the differences in multi-dimensional visual features. Based on the feature information extracted from appearance features, the difference values ​​of each feature are calculated using correlation coefficients and Euclidean distance.

[0054] (1) Color histogram differences: D_color = 1 - ρ(H1, H2) Where ρ represents the histogram correlation coefficient, and H1 and H2 represent the color histograms of the two images.

[0055] (2) Texture / gradient difference: D_feature = ||F1- F2||2 Where F1 and F2 represent the texture feature vectors or gradient feature vectors of the two images, and ||…||2 represents the Euclidean norm.

[0056] (3) Calculate the weighted average: Where N=3 represents three appearance features: color, texture, and gradient; Di is the difference value among the three features; and Davg is the calculated comprehensive appearance difference score. Based on this score, the appearance confidence level is mapped, and the suggested value range is shown in Table 1. This appearance confidence level will also participate in the multimodal fusion decision.

[0057] Table 1 Difference range (D) Confidence level (C_appearance) illustrate D<0.1 0.9 The differences are minimal, the appearances are highly consistent, and the confidence level is the highest. 0.1 ≤ D<0.3 0.7 The differences are minor, the appearances are relatively consistent, and the confidence level is high. 0.3 ≤ D<0.5 0.5 The difference is moderate, and there are some changes in appearance. The confidence level is moderate. Optionally, the regional statistical confidence score is achieved by comprehensively evaluating the reasonableness of regional quantity and area differences to determine statistical quality. First, the regional quantity quality score and area difference quality score are calculated, then weighted and fused according to certain weights to generate the final statistical confidence score.

[0058] Region quantity and quality: C_region = min(N_regions / 10, 1) Mass of area difference: C_area = 1 - min(Δ_area / 10000, 1) Statistical confidence level: C_statistical = 0.6 × C_area + 0.4 × C_region Where N_regions is the number of detected difference regions, and the number of regions is set to 10; Δ_area is the total area difference between the difference regions between the two images, and the area difference threshold is set to 10000, which is equivalent to a 100×100 pixel area.

[0059] The method described in the above embodiments, by constructing three types of confidence indicators—structure, appearance, and regional statistics—can effectively identify whether there are abnormal states such as misinstallation, omission, over-installation, offset, replacement, and obstruction of U-position equipment in the target cabinet. This significantly improves the ability to identify various abnormalities in the cabinet's U-positions, effectively ensuring the accuracy and comprehensiveness of the cabinet's U-position anomaly detection results, and meeting the business needs of refined supervision and accurate inventory of data center cabinet equipment.

[0060] In some embodiments, the U-position anomaly detection result of the target cabinet is determined based on structural confidence, appearance confidence, and regional statistical confidence, including: The structural confidence score, appearance confidence score, and regional statistical confidence score are weighted and fused to obtain the fused confidence score. The fusion confidence level is used to determine whether there are any abnormal U-positions in the target cabinet.

[0061] Specifically, in this embodiment, after obtaining the structural confidence level, appearance confidence level, and regional statistical confidence level corresponding to the cabinet, appropriate weights can be assigned to the three types of confidence levels based on the contribution of each dimension's features to the cabinet's U-position anomaly detection. The multi-dimensional confidence levels are then weighted and fused to obtain a fused confidence level that comprehensively characterizes the cabinet's U-position installation status. The weights can be flexibly configured according to the actual environment of the data center and the equipment testing scenario to adapt to testing scenarios of varying complexity. Optionally, the calculated fused confidence level can be compared with a preset threshold to accurately determine whether the target cabinet has an U-position anomaly.

[0062] For example, in this embodiment, feature fusion processing can be performed based on the quantification results of each feature. First, the confidence scores of the three features—structure, appearance, and statistics—are weighted and fused according to preset weights: C_fusion = α×C_struct + β×C_appearance + γ×C_statistical. Then, a multi-expert voting mechanism is employed. When the confidence scores of at least two features exceed their independent thresholds (e.g., structure > 0.6, appearance > 0.5, statistics > 0.4), the system ultimately determines that there is a significant difference. This fusion process comprehensively evaluates evidence from various dimensions to arrive at a reliable conclusion through a dual mechanism of weighted integration and voting decision-making. Optionally, based on the above output conclusions, abnormal structure locations can be labeled, abnormal ranges can be marked on the detection image, and an analysis report can be generated. For example, the actual detection effect is as follows: Figure 2 As shown.

[0063] The method described in the above embodiments integrates multi-dimensional detection information, including the cabinet structure outline, appearance details, and regional spatial distribution, through a multi-confidence weighted fusion decision-making approach. This effectively avoids the defects of single-dimensional feature detection, which is easily affected by changes in lighting, environmental noise, and shooting angle deviations, as well as the defects of single-index judgment, which are prone to missed detections and false detections. It accurately identifies subtle and complex abnormal scenarios such as misinstallation, omission, over-installation, offset, replacement, and obstruction of cabinet U-position equipment, significantly improving the accuracy of cabinet U-position anomaly identification.

[0064] In some embodiments, after acquiring the baseline image and the detection image of the target cabinet, the method further includes: Identify the rack frame and rack vertices in the baseline image and the detection image respectively; Based on the identified rack borders and rack vertices, the rack areas in the reference image and the detection image are determined respectively; The cabinet area in the reference image and the cabinet area in the detection image are corrected respectively to obtain the corrected reference image and the corrected detection image.

[0065] Specifically, in this embodiment, considering the influence of factors such as the shooting angle and shooting distance in the server room, the reference image and the detection image acquired on-site may have problems such as rack tilt and image offset, which can easily introduce detection errors. Therefore, after completing image acquisition, this application can perform edge detection and feature point recognition on the reference image and the detection image to accurately locate the rack frame and the rack vertex position in the image. Based on the recognized rack frame and vertex coordinates, the rack area in the image is selected, and invalid interference areas such as background clutter, walls, and equipment cables outside the rack are removed. On this basis, perspective transformation and image correction are performed on the rack area according to the standard rack rectangle parameters to correct the rack area with tilt and deformation into a standard regular rectangle area, and finally obtain a corrected reference image and detection image with consistent specifications.

[0066] For example, such as Figure 3 and Figure 4 As shown, after the equipment is properly installed and in place, this application acquires an image of the current server rack as a reference image for subsequent anomaly detection and judgment. Optionally, if an anomaly occurs during monitoring, such as a U-position being offline, the on-site monitoring camera is activated to take a picture, or an inspection image is obtained through inspection equipment or other means. The DeepLab-v3+ object detection algorithm is used to perform edge recognition and corner vertex coordinate extraction on the rack edge points. The segmented and extracted image is then transformed using OpenCV to convert the quadrilateral region into a rectangle.

[0067] The method described in the above embodiments accurately determines the effective detection area of ​​the cabinet by identifying the cabinet frame and vertices, and effectively overcomes the shooting angle deviation during image acquisition by correcting the cabinet area, so that the reference image and the detection image are consistent in terms of viewpoint, scale and posture, effectively reducing the risk of false detection and missed detection caused by image distortion, and improving the accuracy of cabinet U position anomaly detection results.

[0068] For example, such as Figure 5 As shown in the embodiments of this application, a method for detecting abnormal U-positions in server racks based on multimodal feature fusion is provided. When the background monitoring detects abnormalities in the U-position module or system service, rack images are collected in a timely manner through video surveillance or inspection. Using U-position detection technology based on multimodal feature fusion, the image is compared and analyzed with the stored images from multiple dimensions such as structural features, appearance features, and statistical features to determine the location and category of differences. For abnormal situations, the problem can be quickly located, analyzed, and diagnosed, improving the accuracy and timeliness of daily monitoring and inspection management of IT equipment in data centers and other computer rooms. This helps remote management personnel to quickly discover on-site problems and make judgments and handle them, improving the problem response speed and enhancing the service guarantee capabilities and intelligent management level of the data center.

[0069] The following describes the rack U-position anomaly detection system based on multimodal feature fusion provided by this invention. The rack U-position anomaly detection system described below can be referred to in conjunction with the rack U-position anomaly detection method based on multimodal feature fusion described above. For example, the rack U-position anomaly detection system based on multimodal feature fusion is as follows: Figure 6 As shown, it includes: The acquisition module 610 is used to acquire the baseline image and the detection image of the target cabinet; The extraction module 620 is used to extract multidimensional features of the target cabinet in the reference image and the target cabinet in the detection image; the multidimensional features include the structural features, appearance features and regional statistical features of the target cabinet; The detection module 630 is used to output the U-position anomaly detection result of the target cabinet based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image.

[0070] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740. The processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute a cabinet U-position anomaly detection method based on multimodal feature fusion. This method includes: acquiring a reference image and a detection image of the target cabinet; extracting multidimensional features of the target cabinet from the reference image and the detection image; the multidimensional features include structural features, appearance features, and regional statistical features of the target cabinet; and outputting the U-position anomaly detection result of the target cabinet based on the multidimensional features of the target cabinet in the reference image and the detection image.

[0071] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0072] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the cabinet U-position anomaly detection method based on multimodal feature fusion provided by the above methods. The method includes: acquiring a reference image and a detection image of the target cabinet; extracting multidimensional features of the target cabinet from the reference image and the detection image; the multidimensional features include structural features, appearance features, and regional statistical features of the target cabinet; and outputting the U-position anomaly detection result of the target cabinet based on the multidimensional features of the target cabinet in the reference image and the detection image.

[0073] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the rack U-position anomaly detection method based on multimodal feature fusion provided by the above methods. The method includes: acquiring a reference image and a detection image of the target rack; extracting multidimensional features of the target rack from the reference image and the detection image; the multidimensional features include structural features, appearance features, and regional statistical features of the target rack; and outputting the U-position anomaly detection result of the target rack based on the multidimensional features of the target rack in the reference image and the detection image.

[0074] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0075] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting anomalies in rack U-positions based on multimodal feature fusion, characterized in that, include: Acquire baseline and inspection images of the target cabinet; Extract the multidimensional features of the target cabinet from the reference image and the multidimensional features of the target cabinet from the detection image; The multidimensional features include the structural features, appearance features, and regional statistical features of the target cabinet; Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, the U-position anomaly detection result of the target cabinet is output.

2. The cabinet U-position anomaly detection method based on multimodal feature fusion according to claim 1, characterized in that, The structural features of the target cabinet were extracted based on the following method: The reference image and detection image of the target cabinet are preprocessed, including grayscale conversion, contrast enhancement and Gaussian noise reduction. The preprocessed reference image and the preprocessed detection image are used as inputs to the SIFT algorithm to obtain the structural features of the target cabinet in the reference image and the structural features of the target cabinet in the detection image.

3. The cabinet U-position anomaly detection method based on multimodal feature fusion according to claim 1, characterized in that, The appearance features of the target cabinet include: color features, texture features, and gradient statistical features.

4. The cabinet U-position anomaly detection method based on multimodal feature fusion according to claim 1, characterized in that, The regional statistical characteristics of the target cabinet include: shape moment characteristics, area statistical characteristics, geometric characteristics, and spatial density characteristics.

5. The cabinet U-position anomaly detection method based on multimodal feature fusion according to any one of claims 1-4, characterized in that, The step of outputting the U-position anomaly detection result of the target cabinet based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image includes: Based on the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image, structural confidence, appearance confidence, and regional statistical confidence are determined. The structural confidence indicates the degree of matching of structural features between the reference image and the detection image; the appearance confidence indicates the degree of difference in appearance features between the reference image and the detection image; and the regional statistical confidence indicates the degree of matching of statistical features of each region between the reference image and the detection image. Based on the structural confidence level, appearance confidence level, and regional statistical confidence level, the U-position anomaly detection result of the target cabinet is determined.

6. The cabinet U-position anomaly detection method based on multimodal feature fusion according to claim 5, characterized in that, The step of determining the U-position anomaly detection result of the target cabinet based on the structural confidence level, appearance confidence level, and regional statistical confidence level includes: The structural confidence, appearance confidence, and regional statistical confidence are weighted and fused to obtain the fused confidence. The target cabinet is determined to have a U-position anomaly based on the fusion confidence level.

7. The cabinet U-position anomaly detection method based on multimodal feature fusion according to any one of claims 1-4, characterized in that, After acquiring the reference image and detection image of the target cabinet, the method further includes: The cabinet border and cabinet vertices in the reference image and the detection image are identified respectively; Based on the identified rack frame and rack vertices, the rack areas in the reference image and the detection image are determined respectively; The cabinet area in the reference image and the cabinet area in the detection image are corrected respectively to obtain the corrected reference image and the corrected detection image.

8. A cabinet U-position anomaly detection system based on multimodal feature fusion, characterized in that, include: The acquisition module is used to acquire the baseline image and inspection image of the target cabinet; The extraction module is used to extract the multidimensional features of the target cabinet in the reference image and the multidimensional features of the target cabinet in the detection image; The multidimensional features include the structural features, appearance features, and regional statistical features of the target cabinet; The detection module is used to output the U-position anomaly detection result of the target cabinet based on the multi-dimensional features of the target cabinet in the reference image and the multi-dimensional features of the target cabinet in the detection image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the cabinet U-position anomaly detection method based on multimodal feature fusion as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cabinet U-position anomaly detection method based on multimodal feature fusion as described in any one of claims 1 to 7.

Images