Intelligent machine room panoramic image fusion method and system based on multi-dimensional vision

Abstract

The invention belongs to the technical field of computer vision and image processing, and particularly relates to an intelligent machine room panoramic image fusion method and system based on multi-dimensional vision, and the method comprises the following specific steps: completing the initial calibration of a collection device, building a machine room initial world coordinate system, selecting a fixed feature reference point, and recording reference information; through time alignment of hardware synchronous acquisition and timestamp interpolation and space alignment of dynamic calibration parameters and a unified coordinate system, mapping distortion caused by sampling rates and coordinate system differences of visible light, infrared and 3D point clouds is thoroughly eliminated, accurate complementation of multi-dimensional information is realized, and semantic conflicts and information loss during fusion are avoided; according to the method, a dynamic real-time calibration mechanism based on the fixed characteristics of the machine room is established, iterative updating of fusion parameters is combined, a traditional static calibration method is replaced, parameter deviation caused by vibration and temperature change can be corrected in real time, error accumulation is avoided fundamentally, and long-term high-precision operation of a fusion system is guaranteed.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image processing technology, specifically to a method and system for intelligent panoramic image fusion of computer rooms based on multidimensional vision. Background Technology

[0002] With the deepening of digital transformation, data center server rooms, as the core hubs for computing power and data flow, face increasingly higher requirements for equipment density and operational stability. Comprehensive operation and maintenance monitoring covering servers, switches, power distribution equipment, and pipelines has become crucial for ensuring continuous business operations. Traditional server room operation and maintenance relies on manual inspections, which suffers from low efficiency, numerous blind spots, and delayed fault response, making it unsuitable for the needs of large-scale, intelligent server room management. Therefore, automated monitoring and operation and maintenance solutions based on vision technology are widely used.

[0003] Multidimensional vision technology, combining visible light, infrared, and 3D point cloud data, enables comprehensive perception of the appearance, operating temperature, and spatial layout of equipment in a data center. Compared to a single visual dimension, it can capture more complete operational characteristics of the data center. Visible light images can identify visible defects such as equipment damage and loose wiring, while infrared images can detect hidden faults such as abnormal equipment temperatures. 3D point clouds can reconstruct the spatial structure of the data center and the relationship between equipment positions, providing rich data support for subsequent intelligent analysis. Panoramic image fusion technology, as a core component of multidimensional vision perception, aligns, fuses, and stitches multi-view, multi-modal image data to generate a complete and coherent panoramic image of the data center. This solves the problems of limited monitoring range from a single perspective and fragmented data from multiple perspectives, achieving "what you see is what you get" panoramic monitoring and management of the data center. It has become one of the core technical supports for intelligent data center operation and maintenance.

[0004] However, existing panoramic image fusion methods for intelligent data centers still have the following technical problems:

[0005] Multimodal data exhibits significant spatiotemporal misalignment. There are fundamental differences in the sampling rates and coordinate systems of visible light, infrared, and 3D point clouds, which can easily lead to timestamp deviations and spatial mapping distortions. This results in the inability of multidimensional information to be accurately complementary, leading to semantic conflicts and information loss during fusion. Vibrations from the operation of equipment in the data center and changes in ambient temperature can cause camera calibration parameter shifts. Traditional methods lack dynamic calibration mechanisms, and alignment errors accumulate continuously after long-term operation, further reducing fusion accuracy.

[0006] To address these issues, a method and system for fusion of panoramic images of intelligent data centers based on multidimensional vision are proposed. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for fusion of panoramic images of intelligent computer rooms based on multi-dimensional vision, so as to solve the problems mentioned in the background art.

[0008] To achieve the above objectives, the present invention provides the following technical solution: a panoramic image fusion method for intelligent data centers based on multi-dimensional vision, the specific steps of which are as follows:

[0009] Step 1: System Initialization and Data Acquisition: Complete the initial calibration of the acquisition equipment, establish the initial world coordinate system of the computer room, select fixed feature reference points and record the reference information; synchronously acquire visible light, infrared, and 3D point cloud data and mark them with timestamps, preprocess to remove invalid data and standardize the format;

[0010] Step 2: Dynamic calibration of calibration parameters: Extract the real-time coordinates of the reference point, compare them with the benchmark to calculate and correct the parameter offset, and interpolate to compensate for any deficiencies to avoid error accumulation;

[0011] Step 3, Multimodal Spatiotemporal Alignment: Using 3D point cloud as a reference, synchronize the temporal dimension of multimodal data; unify the spatial coordinate system, complete the mapping between 2D image and 3D point cloud, verify semantic consistency, and fill in missing information;

[0012] Step 4: Feature Layered Fusion: Extract the low-level features and high-level semantic features of each modality, dynamically allocate fusion weights according to the data center scenario, and generate a fused feature set through a multi-scale network;

[0013] Step 5: Panoramic stitching: Cross-view matching and fusion features, seamless stitching, brightness equalization and geometric correction are performed to generate a preliminary panoramic image;

[0014] Step 6, Post-processing and quality verification: Denoise and equalize image parameters, and verify by alignment accuracy and information integrity indicators. If the standards are not met, return to the corresponding step for correction.

[0015] Step 7: Parameter Iteration and Optimization: Store key parameters and update them iteratively, optimize the algorithm strategy, and update the reference point benchmark regularly to ensure long-term accuracy;

[0016] Step 8, Output and Interface Opening: Output standardized panoramic images and open interfaces to provide multi-dimensional visual data support for data center operation and maintenance systems.

[0017] Preferably, the detailed steps of step one are as follows: initialize the fusion system, complete the initial intrinsic and extrinsic parameter calibration of each acquisition device, establish an initial world coordinate system based on the physical space of the computer room, and select several fixed feature reference points in the computer room to record their three-dimensional coordinates and multimodal visual features in the initial coordinate system as the reference for subsequent dynamic calibration.

[0018] The multimodal acquisition device is activated, and through hardware synchronous triggering and precise timestamp labeling mechanism, multi-view and multimodal data acquisition is carried out on the entire data room to obtain visible light images, infrared thermal images and 3D point cloud data. At the same time, a unified timestamp is added to each frame of data to ensure full coverage of the acquisition view of the device during the acquisition process and avoid blind spots in data room monitoring.

[0019] The collected raw data undergoes preliminary preprocessing to remove obviously invalid data and to standardize the data format, laying the foundation for subsequent processing.

[0020] Preferably, the detailed steps of step two are as follows: extract fixed feature reference points of the computer room from each viewpoint from the multimodal data collected in step one, and obtain the pixel coordinates and point cloud coordinates of the reference points in the real-time collected data through feature matching algorithm;

[0021] The coordinates of the reference points extracted in real time are compared with the baseline coordinates recorded in step one to calculate the offset of the camera's intrinsic and extrinsic parameters. Based on the spatial geometric model, the calibration parameters of the offset are corrected in real time to generate a dynamically calibrated calibration parameter set. If the number of reference points extracted at a certain viewpoint is insufficient or the matching accuracy is not up to standard, the calibration parameters of adjacent viewpoints are called for interpolation correction to ensure that the calibration parameters of all acquisition devices are in an accurate state, thereby avoiding the accumulation of alignment errors in long-term operation from the root.

[0022] Preferably, the detailed steps of step three are as follows: Based on the timestamps of each modal data, and using the sampling frequency of the 3D point cloud as a benchmark, time interpolation and downsampling are performed on visible light images and infrared thermal images with different sampling rates: interpolation is performed to generate frames for intervals with missing timestamps, and downsampling is performed on modal data with excessively high sampling frequencies, so that the multimodal data are completely synchronized in the time dimension, eliminating the spatiotemporal misalignment caused by timestamp deviations, and ensuring that the information of the computer room scene at the same time node can be accurately matched; using the spatial coordinate system of the 3D point cloud as a benchmark, the external parameters of the dynamically calibrated visible light camera and infrared thermal imager are mapped to this coordinate system to achieve spatial coordinate unification of multimodal devices; based on the unified coordinate system and dynamic calibration parameters, the visible light images and infrared thermal images are mapped to the spatial structure of the 3D point cloud through projection transformation and feature point refinement to achieve spatial mapping matching between two-dimensional images and three-dimensional point clouds;

[0023] Semantic consistency verification is performed on the spatial mapping results of multimodal data to remove semantic conflict information caused by occlusion and interference, and missing information regions are filled in to ensure accurate complementarity of multi-dimensional information and avoid information loss and semantic conflict during fusion.

[0024] Preferably, the detailed steps of step four are as follows: Based on precise spatiotemporal alignment, and considering the characteristics of dense equipment and diverse features in the data center scenario, a hierarchical feature extraction strategy is adopted to retain the core information of each modality. Then, an adaptive fusion algorithm is used to achieve deep fusion of multimodal features. Texture, edge, and equipment contour features of visible light images, temperature gradient and thermal anomaly region features of infrared thermal images, spatial structure of 3D point clouds, three-dimensional morphology of equipment, and distance features are extracted to form a multimodal low-level feature set. Based on prior knowledge of data center equipment, semantic parsing is performed on the low-level feature set to extract high-level semantic features such as data center area attributes, equipment type, and operating status, forming a multimodal high-level semantic feature set.

[0025] Based on the scene characteristics of different areas in the data center, dynamic fusion weights are assigned to each modality feature. A multi-scale fusion network is adopted to deeply fuse the low-level features with the high-level semantic features to generate a multi-modal fusion feature set, which not only retains the key information of each modality, but also avoids feature redundancy and semantic conflicts.

[0026] Preferably, the detailed steps of step five are as follows: Based on the fused multimodal feature set, feature matching and panoramic stitching of multi-view data are performed to avoid gaps and misalignments caused by alignment errors in traditional stitching; cross-view feature matching is performed on the multimodal fused feature set of different views in the computer room, and an improved feature point matching algorithm is used to accurately identify the overlapping areas and feature correspondences between each view; using the unified world coordinate system of the computer room as a reference, the fused features of the multi-view data are seamlessly stitched, and the stitched area is processed for brightness equalization and texture smoothing to eliminate the problems of sudden brightness changes and texture breaks at the stitching lines caused by uneven lighting and equipment reflection, generating a preliminary multi-dimensional visual panoramic fused image of the computer room; geometric correction is performed on the preliminary panoramic image to ensure that the spatial proportion of the image is consistent with the actual physical space of the computer room, avoiding spatial distortion after stitching.

[0027] Preferably, the detailed steps of step six are as follows: Targeted optimization of the preliminary panoramic image is performed, and a quality verification mechanism is established to ensure the accuracy and integrity of the fused image, meeting the actual needs of data center operation and maintenance; For image noise and artifacts caused by the complex environment of the data center, an adaptive denoising algorithm is used for denoising processing, while the brightness, contrast, and color of the image are globally balanced to improve the visual coherence and clarity of the image; Two core verification indicators are set: alignment accuracy indicator and information integrity indicator. The panoramic fused image is compared with the benchmark information of fixed feature reference points in the data center to verify whether it reaches the preset accuracy threshold; If the verification meets the standard, proceed to the next step; If it does not meet the standard, it is determined to be an alignment or fusion error, and the process is returned to the corresponding step for reprocessing, forming a closed-loop quality control.

[0028] Preferably, the detailed steps of step seven are as follows: Key data such as dynamic calibration parameters, spatiotemporal alignment parameters, and feature fusion weights from this fusion process are stored to form a data center fusion parameter database; based on the quality verification results of multiple fusions, the parameter database is iteratively updated, and the deviation calculation model for dynamic calibration, the interpolation strategy for spatiotemporal alignment, and the weight allocation rules for feature fusion are optimized through machine learning algorithms, enabling the fusion system to adapt to long-term changes in the data center environment; the fixed feature reference points in the data center are periodically recalibrated, and the benchmark information is updated to ensure the effectiveness of the dynamic calibration benchmark and achieve long-term high-precision operation of the fusion system.

[0029] Preferably, the detailed steps of step eight are as follows: the multi-dimensional visual panoramic fusion image of the data center that has been quality verified and optimized is output in a standardized manner, and at the same time, the data application interface is opened to the data center operation and maintenance system. The visible light, infrared and 3D spatial information in the panoramic fusion image is analyzed in a structured manner to provide high-precision visual data support for subsequent operation and maintenance work such as equipment defect identification, temperature anomaly warning, and spatial layout management.

[0030] The intelligent data center panoramic image fusion system based on multi-dimensional vision includes a multimodal data acquisition module, a dynamic calibration module, a spatiotemporal alignment module, a hierarchical adaptive fusion module, a panoramic stitching module, a quality verification and closed-loop correction module, a parameter iterative optimization module, and a data output and interface module.

[0031] The multimodal data acquisition module integrates a visible light camera, an infrared thermal imager, and a 3D lidar. Through hardware synchronous triggering and a precise timestamp labeling mechanism, it achieves synchronous acquisition of multi-source data; completes raw data preprocessing, removes invalid data, and standardizes the format to provide high-quality data support for subsequent processing;

[0032] The core of the dynamic calibration module is to solve the problems of calibration parameter offset and error accumulation. It extracts fixed feature reference points in the computer room, compares them with the reference coordinates to calculate the parameter offset and corrects it in real time. If the reference point matching is not up to standard, it calls the adjacent view parameters for interpolation correction to ensure the long-term accuracy of the calibration parameters of the acquisition equipment.

[0033] The spatiotemporal alignment module solves the problem of spatiotemporal misalignment of multimodal data. In the time dimension, it uses the sampling frequency of 3D point cloud as a benchmark and achieves multimodal data synchronization through interpolation and downsampling. In the spatial dimension, it unifies the coordinate system, completes the projection mapping between 2D image and 3D point cloud, verifies semantic consistency, and fills in missing information.

[0034] The hierarchical adaptive fusion module extracts the low-level features and high-level semantic features of each modality, dynamically allocates fusion weights according to the data center scenario, and achieves deep fusion through a multi-scale fusion network to avoid semantic conflicts and information loss, while retaining the core information of each modality.

[0035] The panoramic stitching module completes multi-view fusion feature matching, achieves seamless stitching based on a unified coordinate system, performs brightness equalization, texture smoothing and geometric correction on the stitched area, and generates a distortion-free and seamless panoramic image of the computer room.

[0036] The quality verification and closed-loop correction module verifies the fusion quality through two major indicators: alignment accuracy and information integrity. If the standard is not met, the module returns to the corresponding module for correction, forming a closed-loop control to ensure fusion accuracy.

[0037] The parameter iteration optimization module stores core parameters and establishes a database. It iteratively optimizes the algorithm strategy based on multiple verification results, and regularly updates the reference point benchmark to adapt to long-term changes in the data center environment and avoid error accumulation.

[0038] The data output and interface module outputs standardized panoramic fusion images and provides open application interfaces to provide structured, multi-dimensional visual data support for data center operation and maintenance systems.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] 1) This application completely eliminates the mapping distortion caused by the sampling rate and coordinate system differences of visible light, infrared, and 3D point clouds through hardware synchronous acquisition and time alignment of timestamp interpolation, dynamic calibration parameters and spatial alignment of unified coordinate system, so as to achieve accurate complementarity of multi-dimensional information and avoid semantic conflicts and information loss during fusion.

[0041] 2) This application establishes a dynamic real-time calibration mechanism based on the fixed characteristics of the computer room. Combined with the iterative update of the fusion parameters, it replaces the traditional static calibration method. It can correct parameter deviations caused by vibration and temperature changes in real time, avoid error accumulation from the root, and ensure the long-term high-precision operation of the fusion system. Attached Figure Description

[0042] Figure 1 This is a flowchart of the fusion method steps in this invention;

[0043] Figure 2 This is a diagram showing the modular composition of the fusion system in this invention;

[0044] Figure 3 This is a diagram showing the unit composition of the multimodal data acquisition module;

[0045] Figure 4 This is a diagram showing the unit composition of the dynamic calibration module;

[0046] Figure 5 This is a diagram showing the unit composition of the spatiotemporal alignment module;

[0047] Figure 6 This is a diagram showing the unit composition of the hierarchical adaptive fusion module.

[0048] Figure 7 This is a diagram showing the unit composition of the panoramic stitching module;

[0049] Figure 8 This is a unit composition diagram of the quality verification and closed-loop correction module;

[0050] Figure 9 A diagram showing the unit composition of the parameter iterative optimization module;

[0051] Figure 10 This is a diagram showing the unit composition of the data output and interface module. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example:

[0054] Please see Figure 1-10 The present invention provides a technical solution:

[0055] A multi-dimensional vision-based panoramic image fusion method for intelligent data centers, the specific steps of which are as follows:

[0056] Step 1: System Initialization and Data Acquisition: Complete the initial calibration of the acquisition equipment, establish the initial world coordinate system of the computer room, select fixed feature reference points and record the reference information; synchronously acquire visible light, infrared, and 3D point cloud data and mark them with timestamps, preprocess to remove invalid data and standardize the format;

[0057] Step 2: Dynamic calibration of calibration parameters: Extract the real-time coordinates of the reference point, compare them with the benchmark to calculate and correct the parameter offset, and interpolate to compensate for any deficiencies to avoid error accumulation;

[0058] Step 3, Multimodal Spatiotemporal Alignment: Using 3D point cloud as a reference, synchronize the temporal dimension of multimodal data; unify the spatial coordinate system, complete the mapping between 2D image and 3D point cloud, verify semantic consistency, and fill in missing information;

[0059] Step 4: Feature Layered Fusion: Extract the low-level features and high-level semantic features of each modality, dynamically allocate fusion weights according to the data center scenario, and generate a fused feature set through a multi-scale network;

[0060] Step 5: Panoramic stitching: Cross-view matching and fusion features, seamless stitching, brightness equalization and geometric correction are performed to generate a preliminary panoramic image;

[0061] Step 6, Post-processing and quality verification: Denoise and equalize image parameters, and verify by alignment accuracy and information integrity indicators. If the standards are not met, return to the corresponding step for correction.

[0062] Step 7: Parameter Iteration and Optimization: Store key parameters and update them iteratively, optimize the algorithm strategy, and update the reference point benchmark regularly to ensure long-term accuracy;

[0063] Step 8, Output and Interface Opening: Output standardized panoramic images and open interfaces to provide multi-dimensional visual data support for data center operation and maintenance systems.

[0064] The detailed steps of step one are as follows: Initialize the fusion system, complete the initial intrinsic and extrinsic parameter calibration of each acquisition device (visible light camera, infrared thermal imager, 3D lidar), establish an initial world coordinate system based on the physical space of the computer room, and select several fixed feature reference points in the computer room, such as fixed cabinet markings, metal positioning targets, wall reference scales, etc., distributed in different areas of the computer room, with high recognition, non-moving, and anti-interference characteristics, and record their three-dimensional coordinates and multimodal visual features in the initial coordinate system as the reference for subsequent dynamic calibration;

[0065] The multimodal acquisition device is activated, and through hardware synchronous triggering and precise timestamp labeling mechanism, multi-view and multimodal data acquisition is carried out on the entire data room to obtain visible light images, infrared thermal images and 3D point cloud data. At the same time, a unified timestamp is added to each frame of data to ensure full coverage of the acquisition view of the device during the acquisition process and avoid blind spots in data room monitoring.

[0066] The collected raw data is pre-processed to remove obvious invalid data, such as completely black or white images and point cloud flying points caused by electromagnetic interference. The data is also standardized and converted to lay the foundation for subsequent processing.

[0067] The detailed steps of step two are as follows: Extract fixed feature reference points of the computer room from the multimodal data collected in step one, and obtain the pixel coordinates and point cloud coordinates of the reference points in the real-time collected data through feature matching algorithms (such as SIFT+FLANN);

[0068] The coordinates of the reference points extracted in real time are compared with the baseline coordinates recorded in step one to calculate the offset of the camera's intrinsic and extrinsic parameters, including the deviation values ​​of focal length, principal point, rotation matrix, and translation vector. Based on the spatial geometric model, the calibration parameters of the offset are corrected in real time to generate a dynamically calibrated calibration parameter set. If the number of reference points extracted at a certain viewpoint is insufficient or the matching accuracy is not up to standard, the calibration parameters of adjacent viewpoints are called for interpolation correction to ensure that the calibration parameters of all acquisition devices are in an accurate state, thereby avoiding the accumulation of alignment errors in long-term operation from the root cause.

[0069] The detailed steps of step three are as follows: Based on the timestamps of each modality data, and using the sampling frequency of the 3D point cloud as a benchmark, time interpolation and downsampling are performed on visible light images and infrared thermal images with different sampling rates: interpolation is performed to generate frames for intervals with missing timestamps, and downsampling is performed on modality data with excessively high sampling frequencies, so that the multimodal data are completely synchronized in the time dimension, eliminating the spatiotemporal misalignment caused by timestamp deviations, and ensuring that the information of the computer room scene at the same time node can be accurately matched; using the spatial coordinate system of the 3D point cloud as a benchmark, the external parameters of the dynamically calibrated visible light camera and infrared thermal imager are mapped to this coordinate system to achieve spatial coordinate unification of multimodal devices; based on the unified coordinate system and dynamic calibration parameters, the visible light images and infrared thermal images are mapped to the spatial structure of the 3D point cloud through projection transformation and feature point refinement to achieve spatial mapping matching between two-dimensional images and three-dimensional point clouds;

[0070] Semantic consistency verification is performed on the spatial mapping results of multimodal data to remove semantic conflict information caused by occlusion and interference, such as areas where the device outline and point cloud structure are inconsistent at the same spatial location. Missing information areas are filled in to ensure accurate complementarity of multi-dimensional information and avoid information loss and semantic conflict during fusion.

[0071] The detailed steps of step four are as follows: Based on precise spatiotemporal alignment, and considering the characteristics of dense equipment and diverse features in the data center scenario, a hierarchical feature extraction strategy is adopted to retain the core information of each modality. Then, an adaptive fusion algorithm is used to achieve deep fusion of multimodal features. Texture, edge, and equipment contour features of visible light images, temperature gradient and thermal anomaly region features of infrared thermal images, spatial structure of 3D point clouds, three-dimensional morphology of equipment, and distance features are extracted to form a multimodal low-level feature set. Based on prior knowledge of data center equipment, such as the appearance, space, and temperature features of servers, switches, and power distribution cabinets, semantic parsing is performed on the low-level feature set to extract high-level semantic features such as data center area attributes, equipment types, and operating status, forming a multimodal high-level semantic feature set.

[0072] Based on the scene characteristics of different areas in the computer room, such as dense equipment areas, open passage areas, and distribution station areas, dynamic fusion weights are assigned to each modality feature. For example, the weight of 3D point cloud spatial features is increased in dense equipment areas, and the weight of infrared thermal image features is increased in temperature monitoring areas. A multi-scale fusion network is adopted to deeply fuse low-level features with high-level semantic features to generate a multi-modal fusion feature set, which not only retains the key information of each modality, but also avoids feature redundancy and semantic conflicts.

[0073] The detailed steps of step five are as follows: Based on the fused multimodal feature set, feature matching and panoramic stitching of multi-view data are performed to avoid gaps and misalignments caused by alignment errors in traditional stitching; cross-view feature matching is performed on the multimodal fused feature set from different perspectives of the computer room, and an improved feature point matching algorithm is adopted. To address the issue of high texture similarity in the computer room, spatial structural feature constraints are added to accurately identify the overlapping areas and feature correspondences between different perspectives; using the unified world coordinate system of the computer room as a reference, the fused features of the multi-view data are seamlessly stitched, and the stitched area is processed for brightness equalization and texture smoothing to eliminate the problems of sudden brightness changes and texture breaks at the stitching lines caused by uneven lighting and equipment reflection, generating a preliminary multi-dimensional visual panoramic fused image of the computer room; geometric correction is performed on the preliminary panoramic image to ensure that the spatial proportion of the image is consistent with the actual physical space of the computer room, avoiding spatial distortion after stitching.

[0074] The detailed steps of step six are as follows: Targeted optimization of the initial panoramic image is performed, and a quality verification mechanism is established to ensure the accuracy and integrity of the fused image, meeting the actual needs of data center operation and maintenance. For image noise and artifacts caused by the complex environment of the data center, an adaptive denoising algorithm is used for denoising processing. Simultaneously, the brightness, contrast, and color of the image are globally balanced to improve the visual coherence and clarity of the image. Two core verification indicators are set: alignment accuracy indicators (spatial mapping error of multimodal information, feature point matching error of panoramic stitching) and information integrity indicators (retention rate of visible light texture, infrared temperature, and 3D spatial information). The panoramic fused image is compared with the benchmark information of fixed feature reference points in the data center to verify whether it reaches the preset accuracy threshold. If the verification meets the standard, proceed to the next step; if it does not meet the standard, it is determined to be an alignment or fusion error, and the corresponding step is returned (if the calibration parameter is offset, return to S2; if there is a spatiotemporal alignment error, return to step three) for reprocessing, forming a closed-loop quality control.

[0075] The detailed steps of step seven are as follows: Store key data such as dynamic calibration parameters, spatiotemporal alignment parameters, and feature fusion weights from this fusion process to form a data center fusion parameter database; based on the quality verification results of multiple fusions, iteratively update the parameter database, and optimize the deviation calculation model of dynamic calibration, the interpolation strategy of spatiotemporal alignment, and the weight allocation rules of feature fusion through machine learning algorithms, so that the fusion system can adapt to long-term changes in the data center environment (such as fine-tuning of equipment layout and long-term fluctuations in ambient temperature); periodically recalibrate the fixed feature reference points in the data center, update the benchmark information, ensure the effectiveness of the dynamic calibration benchmark, and achieve long-term high-precision operation of the fusion system.

[0076] The detailed steps of step eight are as follows: Standardize the multi-dimensional visual panoramic fusion image of the data center that has been verified and optimized for quality, and open the data application interface for the data center operation and maintenance system (such as fault diagnosis system, intelligent inspection system, equipment management system). Perform structured analysis on the visible light, infrared, and 3D spatial information in the panoramic fusion image to provide high-precision visual data support for subsequent operation and maintenance work such as equipment defect identification, temperature anomaly warning, and spatial layout management.

[0077] The intelligent data center panoramic image fusion system based on multi-dimensional vision includes a multimodal data acquisition module, a dynamic calibration module, a spatiotemporal alignment module, a hierarchical adaptive fusion module, a panoramic stitching module, a quality verification and closed-loop correction module, a parameter iterative optimization module, and a data output and interface module.

[0078] The multimodal data acquisition module includes a multi-device integration unit, a synchronization triggering and timestamp unit, and a data preprocessing unit;

[0079] The dynamic calibration module includes a reference point extraction and matching unit, a parameter offset calculation and correction unit, and an interpolation correction unit;

[0080] The spatiotemporal alignment module includes a time alignment unit, a spatial alignment unit, and a semantic consistency verification and information completion unit;

[0081] The hierarchical adaptive fusion module includes a low-level feature extraction unit, a high-level semantic extraction unit, a dynamic weight allocation unit, and a multi-scale fusion unit;

[0082] The panoramic stitching module includes a cross-view feature matching unit, a panoramic seamless stitching unit, and a stitching optimization unit;

[0083] The quality verification and closed-loop correction module includes a fusion quality detection unit and a closed-loop scheduling correction unit;

[0084] The parameter iteration optimization module includes a parameter storage and database management unit, an algorithm strategy iteration optimization unit, and a baseline update unit;

[0085] The data output and interface module includes a panoramic image standardized output unit, an application interface open unit, and a structured data parsing and adaptation unit.

[0086] The multimodal data acquisition module integrates a visible light camera, an infrared thermal imager, and a 3D lidar. Through hardware synchronous triggering and a precise timestamp labeling mechanism, it achieves synchronous acquisition of multi-source data; completes raw data preprocessing, removes invalid data, and standardizes the format to provide high-quality data support for subsequent processing;

[0087] The core of the dynamic calibration module is to solve the problems of calibration parameter offset and error accumulation. It extracts fixed feature reference points in the computer room, compares them with the reference coordinates to calculate the parameter offset and corrects it in real time. If the reference point matching is not up to standard, it calls the adjacent view parameters for interpolation correction to ensure the long-term accuracy of the calibration parameters of the acquisition equipment.

[0088] The spatiotemporal alignment module solves the problem of spatiotemporal misalignment of multimodal data. In the time dimension, it uses the sampling frequency of 3D point cloud as a benchmark and achieves multimodal data synchronization through interpolation and downsampling. In the spatial dimension, it unifies the coordinate system, completes the projection mapping between 2D image and 3D point cloud, verifies semantic consistency, and fills in missing information.

[0089] The hierarchical adaptive fusion module extracts the low-level features and high-level semantic features of each modality, dynamically allocates fusion weights according to the data center scenario, and achieves deep fusion through a multi-scale fusion network to avoid semantic conflicts and information loss, while retaining the core information of each modality.

[0090] The panoramic stitching module completes multi-view fusion feature matching, achieves seamless stitching based on a unified coordinate system, performs brightness equalization, texture smoothing and geometric correction on the stitched area, and generates a distortion-free and seamless panoramic image of the computer room.

[0091] The quality verification and closed-loop correction module verifies the fusion quality through two major indicators: alignment accuracy and information integrity. If the standard is not met, the module returns to the corresponding module for correction, forming a closed-loop control to ensure fusion accuracy.

[0092] The parameter iteration optimization module stores core parameters and establishes a database. It iteratively optimizes the algorithm strategy based on multiple verification results, and regularly updates the reference point benchmark to adapt to long-term changes in the data center environment and avoid error accumulation.

[0093] The data output and interface module outputs standardized panoramic fusion images and provides open application interfaces to provide structured, multi-dimensional visual data support for data center operation and maintenance systems.

[0094] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or basic characteristics. Therefore, the embodiments should be considered exemplary and non-limiting in all respects. The scope of the invention is defined by the appended claims rather than the foregoing description. Therefore, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention, and no reference numerals in the claims should be construed as limiting the scope of the claims.

[0095] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for fusion of panoramic images of intelligent data centers based on multi-dimensional vision, characterized in that, The specific steps of this method are as follows: Step 1: System Initialization and Data Acquisition: Complete the initial calibration of the acquisition equipment, establish the initial world coordinate system of the computer room, select fixed feature reference points and record the reference information; synchronously acquire visible light, infrared, and 3D point cloud data and mark them with timestamps, preprocess to remove invalid data and standardize the format; Step 2: Dynamic calibration of calibration parameters: Extract the real-time coordinates of the reference point, compare them with the benchmark to calculate and correct the parameter offset, and interpolate to compensate for any deficiencies to avoid error accumulation; Step 3, Multimodal Spatiotemporal Alignment: Using 3D point cloud as a benchmark, synchronize the temporal dimension of multimodal data; A unified spatial coordinate system is used to complete the mapping between 2D images and 3D point clouds, verify semantic consistency, and fill in missing information. Step 4: Feature Layered Fusion: Extract the low-level features and high-level semantic features of each modality, dynamically allocate fusion weights according to the data center scenario, and generate a fused feature set through a multi-scale network; Step 5: Panoramic stitching: Cross-view matching and fusion features, seamless stitching, brightness equalization and geometric correction are performed to generate a preliminary panoramic image; Step 6, Post-processing and quality verification: Denoise and equalize image parameters, and verify by alignment accuracy and information integrity indicators. If the standards are not met, return to the corresponding step for correction. Step 7: Parameter Iteration and Optimization: Store key parameters and update them iteratively, optimize the algorithm strategy, and update the reference point benchmark regularly to ensure long-term accuracy; Step 8: Output and Interface Opening: Output standardized panoramic images and open interfaces to provide multi-dimensional visual data support for data center operation and maintenance systems.

2. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step one are as follows: Initialize the fusion system, complete the initial intrinsic and extrinsic parameter calibration of each acquisition device, establish an initial world coordinate system based on the physical space of the computer room, and select several fixed feature reference points in the computer room to record their three-dimensional coordinates and multimodal visual features in the initial coordinate system as the reference for subsequent dynamic calibration. The multimodal acquisition device is activated, and through hardware synchronous triggering and precise timestamp labeling mechanism, multi-view and multimodal data acquisition is carried out on the entire data room to obtain visible light images, infrared thermal images and 3D point cloud data. At the same time, a unified timestamp is added to each frame of data to ensure full coverage of the acquisition view of the device during the acquisition process and avoid blind spots in data room monitoring. The collected raw data undergoes preliminary preprocessing to remove obviously invalid data and to standardize the data format, laying the foundation for subsequent processing.

3. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step two are as follows: Extract fixed feature reference points of the computer room from the multimodal data collected in step one, and obtain the pixel coordinates and point cloud coordinates of the reference points in the real-time collected data through feature matching algorithm; The coordinates of the reference points extracted in real time are compared with the baseline coordinates recorded in step one to calculate the offset of the camera's intrinsic and extrinsic parameters. Based on the spatial geometric model, the calibration parameters of the offset are corrected in real time to generate a dynamically calibrated calibration parameter set. If the number of reference points extracted at a certain viewpoint is insufficient or the matching accuracy is not up to standard, the calibration parameters of adjacent viewpoints are called for interpolation correction to ensure that the calibration parameters of all acquisition devices are in an accurate state, thereby avoiding the accumulation of alignment errors in long-term operation from the root.

4. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step three are as follows: Based on the timestamps of each modality data, and using the sampling frequency of the 3D point cloud as a benchmark, time interpolation and downsampling are performed on visible light images and infrared thermal images with different sampling rates: interpolation is performed to generate frames for intervals with missing timestamps, and downsampling is performed on modality data with excessively high sampling frequencies, so that the multimodal data are completely synchronized in the time dimension, eliminating the spatiotemporal misalignment caused by timestamp deviations, and ensuring that the information of the computer room scene at the same time node can be accurately matched; using the spatial coordinate system of the 3D point cloud as a benchmark, the external parameters of the dynamically calibrated visible light camera and infrared thermal imager are mapped to this coordinate system to achieve spatial coordinate unification of multimodal devices; based on the unified coordinate system and dynamic calibration parameters, the visible light images and infrared thermal images are mapped to the spatial structure of the 3D point cloud through projection transformation and feature point refinement to achieve spatial mapping matching between two-dimensional images and three-dimensional point clouds; Semantic consistency verification is performed on the spatial mapping results of multimodal data to remove semantic conflict information caused by occlusion and interference, and missing information regions are filled in to ensure accurate complementarity of multi-dimensional information and avoid information loss and semantic conflict during fusion.

5. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step four are as follows: Based on precise spatiotemporal alignment, and considering the characteristics of dense equipment and diverse features in the data center scenario, a hierarchical feature extraction strategy is adopted to retain the core information of each modality. Then, an adaptive fusion algorithm is used to achieve deep fusion of multimodal features. Texture, edge, and equipment contour features of visible light images, temperature gradient and thermal anomaly region features of infrared thermal images, spatial structure of 3D point clouds, three-dimensional morphology of equipment, and distance features are extracted to form a multimodal low-level feature set. Based on prior knowledge of data center equipment, semantic parsing is performed on the low-level feature set to extract high-level semantic features of data center area attributes, equipment type, and operating status, forming a multimodal high-level semantic feature set. Based on the scene characteristics of different areas in the data center, dynamic fusion weights are assigned to each modality feature. A multi-scale fusion network is adopted to deeply fuse the low-level features with the high-level semantic features to generate a multi-modal fusion feature set, which not only retains the key information of each modality, but also avoids feature redundancy and semantic conflicts.

6. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step five are as follows: Based on the fused multimodal feature set, feature matching and panoramic stitching of multi-view data are performed to avoid gaps and misalignments caused by alignment errors in traditional stitching; cross-view feature matching is performed on the multimodal fused feature set of different views of the computer room, and an improved feature point matching algorithm is used to accurately identify the overlapping areas and feature correspondences between each view. Based on the unified world coordinate system of the computer room, the fusion features of multiple perspectives are seamlessly stitched together, and the stitched area is processed for brightness equalization and texture smoothing to eliminate the problems of sudden brightness changes and texture breaks at the stitching lines caused by uneven lighting and equipment reflection, thus generating a preliminary multi-dimensional visual panoramic fusion image of the computer room. Geometric correction is performed on the initial panoramic image to ensure that the spatial proportion of the image is consistent with the actual physical space of the computer room, thus avoiding spatial distortion after stitching.

7. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step six are as follows: Targeted optimization of the preliminary panoramic image is carried out, and a quality verification mechanism is established to ensure the accuracy and integrity of the fused image and meet the actual needs of data center operation and maintenance; Adaptive denoising algorithm is used to denoise the image noise and artifacts caused by the complex environment of the data center, and the brightness, contrast and color of the image are globally balanced to improve the visual coherence and clarity of the image. Two core verification metrics are set: alignment accuracy and information integrity. The panoramic fused image is compared with the benchmark information of fixed feature reference points in the computer room to verify whether the preset accuracy threshold has been reached. If the verification is successful, proceed to the next step; If the standard is not met, it is judged as an alignment or fusion error, and the corresponding step is returned for reprocessing to form a closed-loop quality control.

8. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step seven are as follows: Store the key data of dynamic calibration parameters, spatiotemporal alignment parameters, and feature fusion weights from this fusion process to form a data center fusion parameter database; based on the quality verification results of multiple fusions, iteratively update the parameter database, and optimize the deviation calculation model of dynamic calibration, the interpolation strategy of spatiotemporal alignment, and the weight allocation rules of feature fusion through machine learning algorithms, so that the fusion system can adapt to long-term changes in the data center environment; periodically recalibrate the fixed feature reference points in the data center, update the benchmark information, ensure the effectiveness of the dynamic calibration benchmark, and achieve long-term high-precision operation of the fusion system.

9. The intelligent data center panoramic image fusion method and system based on multi-dimensional vision according to claim 1, characterized in that: The detailed steps of step eight are as follows: Standardize and output the multi-dimensional visual panoramic fusion image of the data center that has been verified and optimized for quality, and open the data application interface for the data center operation and maintenance system. Perform structured analysis on the visible light, infrared, and 3D spatial information in the panoramic fusion image to provide high-precision visual data support for subsequent equipment defect identification, temperature anomaly warning, and spatial layout management and maintenance work.

10. A multi-dimensional vision-based intelligent data center panoramic image fusion system, characterized in that: It includes a multimodal data acquisition module, a dynamic calibration module, a spatiotemporal alignment module, a hierarchical adaptive fusion module, a panoramic stitching module, a quality verification and closed-loop correction module, a parameter iterative optimization module, and a data output and interface module; The multimodal data acquisition module integrates a visible light camera, an infrared thermal imager, and a 3D lidar. Through hardware synchronous triggering and a precise timestamp labeling mechanism, it achieves synchronous acquisition of multi-source data; completes raw data preprocessing, removes invalid data, and standardizes the format to provide high-quality data support for subsequent processing; The core of the dynamic calibration module is to solve the problems of calibration parameter offset and error accumulation. It extracts fixed feature reference points in the computer room, compares them with the reference coordinates to calculate the parameter offset and corrects it in real time. If the reference point matching is not up to standard, it calls the adjacent view parameters for interpolation correction to ensure the long-term accuracy of the calibration parameters of the acquisition equipment. The spatiotemporal alignment module solves the problem of spatiotemporal misalignment of multimodal data. In the time dimension, it uses the sampling frequency of 3D point cloud as a benchmark and achieves multimodal data synchronization through interpolation and downsampling. In the spatial dimension, it unifies the coordinate system, completes the projection mapping between 2D image and 3D point cloud, verifies semantic consistency, and fills in missing information. The hierarchical adaptive fusion module extracts the low-level features and high-level semantic features of each modality, dynamically allocates fusion weights according to the data center scenario, and achieves deep fusion through a multi-scale fusion network to avoid semantic conflicts and information loss, while retaining the core information of each modality. The panoramic stitching module completes multi-view fusion feature matching, achieves seamless stitching based on a unified coordinate system, performs brightness equalization, texture smoothing and geometric correction on the stitched area, and generates a distortion-free and seamless panoramic image of the computer room. The quality verification and closed-loop correction module verifies the fusion quality through two major indicators: alignment accuracy and information integrity. If the standard is not met, the module returns to the corresponding module for correction, forming a closed-loop control to ensure fusion accuracy. The parameter iteration optimization module stores core parameters and establishes a database. It iteratively optimizes the algorithm strategy based on multiple verification results, and regularly updates the reference point benchmark to adapt to long-term changes in the data center environment and avoid error accumulation. The data output and interface module outputs standardized panoramic fusion images and provides open application interfaces to provide structured, multi-dimensional visual data support for data center operation and maintenance systems.

Images