A heat failure early warning method of a multi-modal perception machine room inspection robot

Abstract

The present application relates to the field of heat failure early warning of machine room inspection robots, in particular to a heat failure early warning method of a multi-modal perception machine room inspection robot, which comprises: collecting infrared thermal images, acoustic, vibration and gas concentration data through a multi-modal sensor array to generate voxelized temperature distribution; extracting temperature features through wavelet transform and fractional Brownian motion algorithm, and fusing into a device global temperature feature set; constructing noise, vibration and gas concentration towers to extract auxiliary features, and combining non-extensive entropy and a graph attention model to fuse into a high-dimensional fault feature pool; constructing a fiber bundle model based on the topology of the machine room, and positioning the heat failure after optimizing the parameters through a simulated annealing algorithm; and finally generating a dynamic heat map and the like through digital twinning to achieve accurate early warning.

Description

Technical Field

[0001] This invention relates to the field of thermal fault early warning for data center inspection robots, specifically a thermal fault early warning method for data center inspection robots with multimodal perception. Background Technology

[0002] With the acceleration of digital transformation, data center equipment is trending towards higher density and higher power. The centralized deployment of servers, switches, and storage devices makes the thermal environment of data centers increasingly complex. Thermal failures, such as localized overheating and cooling system malfunctions, have become major causes of equipment downtime or data loss, posing a serious threat to the stable operation of data centers. Traditional methods for early warning of thermal failures in data centers have significant limitations.

[0003] Relying primarily on single-point temperature sensors or infrared thermal imagers for localized temperature measurement only captures surface temperature changes and cannot correlate with multi-source fault precursors such as abnormal vibrations, sudden noise changes, and gas leaks that accompany equipment operation. This leads to early, minor faults being easily overlooked, resulting in high false alarm and false negative rates. Furthermore, it cannot extract deep features reflecting the essence of thermal faults, such as transient temperature fluctuations of tiny hot spots at the chip level or complex thermal gradients formed by airflow turbulence inside the cabinet, making early warning difficult. The impact of the topology of the data center equipment on heat propagation is not considered, making it impossible to model the propagation path of thermal faults between devices, resulting in ambiguous fault location and difficulty in accurately pinpointing the root cause component.

[0004] Therefore, a thermal fault early warning method for a multimodal sensing data center inspection robot is needed to solve the above problems. Summary of the Invention

[0005] To address the technical problems mentioned in the background, this invention provides a scene adaptive adjustment method based on an intelligent AI terminal.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] This invention provides a thermal fault early warning method for a multimodal sensing data center inspection robot, the specific steps of which are as follows:

[0008] Step 1: Data Acquisition in the Computer Room: The robot uses a multimodal sensor array to collect infrared thermal images, acoustic data, vibration data, and gas concentration data of the computer room. A three-dimensional temperature field reconstruction algorithm is then used to generate a voxelized temperature distribution.

[0009] The inspection robot moves along a preset inspection path. Upon reaching a monitoring point, it triggers a multimodal sensor array to start collecting data. The multimodal sensor array includes an infrared thermal imager, an acoustic sensor, a vibration sensor, and a gas sensor. It acquires real-time data sets of equipment in the computer room, including infrared images, equipment operating noise, equipment vibration signals, and gas concentrations around the equipment. The acquired multiple frames of infrared images are then arranged to obtain an infrared image sequence.

[0010] The solution adopts a binocular structured light camera, which consists of two infrared cameras and one infrared projector. The infrared projector emits infrared light with preset patterns, such as stripes and dot patterns, to enhance the surface texture features of the equipment in the computer room, making it easier for the camera to recognize and match. The camera and projector are installed at the front end of the robot.

[0011] An infrared projector projects a pre-encoded structured light pattern onto the surface of the cabinet. The structured light pattern resembles a sequence of binary code stripes. The light is distorted by the shape or distance of the equipment surface. Two infrared cameras synchronously acquire images with the distorted pattern. Each acquisition generates an image from each camera, forming a binocular image pair. The acquired binocular images undergo epipolar correction processing. By using camera calibration parameters, the image coordinates are transformed to the epipolar constraint plane, ensuring that the search for corresponding points is performed only in the same row, improving matching efficiency and accuracy. A block-based matching algorithm is used to search for matching points in the epipolar-corrected binocular images. Centered on a pixel in the left image, a certain size image block is selected. The image block with the highest similarity is searched in the corresponding row of the right image to determine the matching point pair and generate a disparity image. Based on the principle of triangulation, combined with the baseline distance obtained from camera calibration, the camera focal length, and the disparity image, the depth value of each point on the computer cabinet surface is calculated pixel by pixel, finally generating a structured light depth image. Each pixel value in the image represents the actual distance of the corresponding point from the camera, providing accurate spatial location information for the reconstruction of the three-dimensional temperature field.

[0012] The structured light depth image and infrared image sequence are input into the U-Net-LSTM network. The encoder extracts multi-scale temperature features through convolution and outputs a temperature feature map. The LSTM layer captures the temperature change line in the time dimension through bidirectional LSTM. The decoder outputs a voxelized temperature distribution. ,in The dimensions are horizontal, vertical, and depth, with t representing a time point, covering the three-dimensional space inside the device.

[0013] It should be noted that the voxelized temperature distribution is achieved by fusing two-dimensional temperature information from infrared thermal imaging with three-dimensional spatial information from structured light depth maps via a network, and then reconstructing the data through an encoder-temporal modeling-decoder process. The final output is the temperature value of each tiny region in the three-dimensional space inside the device.

[0014] Step 2, Cooperative Temperature Features: The voxelized temperature distribution is decomposed into multiple scales by wavelet transform, and the Hurst exponent features at each scale are extracted by fractional Brownian motion algorithm. Then, the wavelet energy features are fused to output a set of global temperature features of the device that includes the surface and internal heat distribution of the device.

[0015] The voxelized temperature distribution is decomposed into multiple layers using a wavelet algorithm with a low-pass filter (HY) and a high-pass filter (HT). The decomposition scales for each filter are set to micro, meso, and macro scales. The micro scale captures localized, minute temperature abrupt changes, such as chip-level hotspots and localized heat dissipation anomalies. The meso scale reflects regional temperature gradient changes, such as heat transfer between server modules. The macro scale extracts global temperature trends, such as the overall heat dissipation status of the server rack and the influence of ambient temperature. The decomposition calculation logic using the recursive rules of the low-pass filter is as follows: ,in Let k be the low-frequency quantity at the k-th decomposition scale, where k is the decomposition scale number. For convolution operations, Represents the tensor product. , and For low-pass filters in different directions; the decomposition calculation logic using the recursive rules of high-pass filters is as follows: Where T is an abbreviation for voxelization temperature distribution. , and These are high-pass filters in different directions. , and These are high-frequency components in different directions, which, when superimposed, constitute a high-frequency quantity. ;

[0016] The surface and internal regions of the cabinet are labeled as region GS and region FI, respectively. Hurst exponents are calculated at k scales of wavelet decomposition to quantify the self-similarity of temperature distributions at different scales. The voxelized temperature distribution is then distributed across a spatial grid at scale k, and for any spatial location... , or Extract the time series of this spatial location , where N is the total number of time points; the mean temperature at this spatial location over time is calculated accordingly. Then, the cumulative deviation of the temperature series is obtained based on the mean. Its calculation logic is as follows: , The cumulative deviation over the first n time steps is calculated. These n time steps are then combined into a subsequence length, where n < N. The difference between the maximum and minimum subsequence lengths is taken to obtain the range. The standard deviation of the subsequence length is then calculated. The rescaled range is obtained by dividing the range by the standard deviation. Fitting by least squares method The slope is the Hurst exponent H, and its calculation logic is as follows: Where C is the intercept of the linear fit, Let H be the natural logarithm function, and argmin be the value of the parameter H that minimizes the expression within the parentheses; repeat the above steps for each scale to obtain the Hurst exponent for each scale. The transient thermal mutation characteristics at each scale are extracted by high-frequency quantities, and the energy of high-frequency quantities at each scale is calculated. Its calculation logic is as follows: The modulus maxima in high-frequency quantities are located, and the hot spot coordinates (r,t) and the corresponding temperature peak values ​​are extracted. The voxelized temperature distribution, fractional Brownian motion features and wavelet features are fused across scales to construct a global temperature feature set for the device.

[0017] Step 3: Multi-feature extraction: The inspection robot constructs a noise tower, an equipment vibration tower, and a gas concentration tower, outputs the corresponding features, and integrates them into an auxiliary judgment feature set. The auxiliary judgment features are then fused with the equipment global temperature feature set through a non-widespread entropy algorithm and a graph attention model to construct a high-dimensional temperature fault feature pool.

[0018] Noise towers are identified using power spectrum analysis and Empirical Mode Decomposition (EMD). Equipment operating noise is segmented into frames and windowed using a Hanning window. A Fast Fourier Transform (FFT) is performed on each frame using the Welch method to calculate the corresponding power spectrum. The power spectrum distribution curve is obtained by averaging the power spectra across multiple frames. EMD decomposition of the equipment operating noise yields several intrinsic mode functions (CLs). The noise energy ratio and instantaneous frequency of each CL are calculated. The noise energy calculation logic for a single CL is as follows: ,in and The start and end times of the calculation process are given, and dt is the time increment. The total noise energy of all intrinsic mode functions is obtained from this; the noise energy percentage is obtained by dividing the individual noise energy by the total noise energy; the instantaneous noise frequency is given. The calculation logic is as follows: ,in The sampling interval is... The instantaneous phase is given by `unwrap`, which is the phase unwrapping function.

[0019] The vibration tower of the equipment is identified using differential geometric manifold algorithms and quantum resonance analysis. The specific identification steps are as follows:

[0020] S001: Acquire the equipment vibration signal xc, select the embedding dimension mc using the spurious nearest neighbor method, set the time delay rk, and construct the phase space matrix using the embedding dimension and time delay. The construction logic is as follows: Where K represents the total sampling time points of the vibration equipment, and each row of the matrix corresponds to a phase space point;

[0021] S002: Obtain the neighborhood points corresponding to each phase space point through the k-nearest neighbor algorithm, and perform PCA dimensionality reduction on the neighborhood points to obtain the local tangent space;

[0022] S003: The quantum resonance analysis algorithm performs a Fourier transform on the equipment vibration signal to obtain the vibration spectrum. ,Will The embedded potential function is calculated as follows: ,in It is a small change;

[0023] S004: Substitute the potential energy function into the Schrödinger equation to obtain the vibrational Hamiltonian. Its calculation logic is as follows: Where hr is the reduced Planck constant, and mc is the equivalent mass, which is obtained by extracting the mass of equipment components from the database. The Laplace operator is used; the vibration characteristic equation is constructed and solved based on the Hamiltonian. The equation formula is as follows: The vibration energy EC is obtained. The vibration energy is divided by the reduced Planck constant to obtain the eigenfrequency. The normal eigenfrequency is obtained from historical data. The eigenfrequency and the normal eigenfrequency are combined and their absolute values ​​are taken to obtain the eigenfrequency deviation.

[0024] Gas concentration towers extract gas characteristics through fractional-order diffusion field reconstruction, specifically:

[0025] G001: Acquire data on the gas concentration in the computer room (UE), construct a fractional-order diffusion equation based on the gas concentration data, and the calculation logic is as follows: Where is the air diffusion coefficient of the computer room. If the source term is... A value of 0 indicates no leakage, while a value greater than 0 indicates the presence of leakage. For time t fractional derivative;

[0026] G002: Based on the fractional diffusion equation, the fractional diffusion field under normal conditions is obtained. It can be trained using historical fault-free data. The deviation between the real-time field and the normal field is marked as the diffusion anomaly index.

[0027] The power spectrum distribution curve, noise energy ratio, instantaneous frequency, local tangent space, intrinsic frequency deviation, and diffusion anomaly index are integrated into an auxiliary judgment feature set;

[0028] Calculate the joint non-extensive entropy of the feature xm of the device's global temperature feature set and the feature xv of the auxiliary decision feature set. Its calculation logic is as follows: , of which Summing the square of the joint probability distribution, where q is a non-extensive parameter; embedding the joint non-extensive entropy into the mutual information formula to obtain the correlation coefficient between any feature in the auxiliary judgment feature set and the features in the device's global temperature feature set. The logic for calculating its correlation coefficient is as follows: , The larger the value, the stronger the nonlinear correlation between the two features; the auxiliary features are screened by arranging the features of the auxiliary judgment feature set according to the magnitude of the correlation coefficient, and retaining the first m auxiliary features to obtain the screened auxiliary feature set;

[0029] Using features from the auxiliary feature set and the device's global temperature feature set as nodes, a feature association graph is constructed with the association coefficient as the initial edge weight. This feature association graph is then input into a graph attention model, which outputs attention coefficients. Specifically, the graph attention model works by applying attention coefficients to each temperature node. Calculate its connection with all connected auxiliary nodes. Attention coefficient Its calculation logic is as follows: ,in and Here, m1 represents the original feature vectors of the auxiliary nodes and temperature nodes, m2 and m3 are the indices of the auxiliary nodes and temperature nodes, and WT is the shared weight matrix, which is obtained by mapping the original features to a high-dimensional space. This is the attention parameter vector, used to learn the importance of feature associations. For activation functions; The larger the value, the more attention coefficients are obtained and two layers of GAT are stacked to obtain global correlation features. The global correlation features are then max-pooled to obtain the pooling result. Finally, the pooling result and feature nodes are concatenated to obtain a high-dimensional fault feature pool.

[0030] Step 4: Fiber Bundle Model Prediction: Construct a fiber bundle model with the topology of the computer room equipment as the base space and the high-dimensional fault feature pool as the fiber vectors. Then, use the simulated annealing algorithm to optimize the model parameters. With the F1 score as the target, iteratively update the parameters through the Metropolis criterion. Finally, calculate the feature propagation anomaly degree based on the optimized model to determine and locate thermal faults.

[0031] Using the physical topology of the data center equipment as the base space B (e.g., the location distribution of servers, switches, and cooling systems), it is discretized into grid-like nodes b. Each grid node corresponds to a device component or region. The topological relationships of the base space are represented by an adjacency matrix. If two grid points are physically connected, the corresponding adjacency matrix is ​​1; otherwise, it is 0. Fiber vectors are set. For any grid node b, the high-dimensional fault feature pool is defined. The fiber vector is obtained by mapping the high-dimensional fault feature pool to the base space.

[0032] The connection coefficients of the base space are obtained, and their calculation logic is as follows: ,in and Let be the numbers of any two nodes. For the connection coefficient, dm is the topological constraint coefficient, which controls the correlation between propagation intensity and physical distance, and dm is the physical distance. The fiber bundle model is constructed in this way.

[0033] The simulated annealing algorithm optimizes fiber bundle parameters. Initial topological constraint coefficients, provided by historical data, are marked as parameters to be optimized. The thermal fault prediction performance of the fiber bundle model is marked as the thermal energy value HM. Random perturbations are applied to the parameters to be optimized to generate new parameters, thus obtaining new thermal energy values ​​for these new parameters. The difference between the new thermal energy value and the current thermal energy value is calculated. If the thermal energy difference is less than 0, the new parameters to be optimized are directly accepted, and the current parameters are updated to the new parameters. If the thermal energy difference is greater than 0, random perturbations continue. When the number of iterations reaches a preset value, iteration stops, and the optimal topological constraint coefficients are output and sent to the fiber bundle model.

[0034] The connection coefficients of all neighboring nodes of each node in the base space are extracted to form several local connection coefficient sets. The mean of the connection coefficients of each node is calculated, and the maximum deviation between the connection coefficient and the mean of the node connection coefficients is obtained. The maximum deviation is marked as the thermal fault anomaly, and the thermal fault anomaly of each node in the base space is obtained. A preset anomaly threshold is extracted from the database, and nodes whose thermal fault anomalies exceed the preset anomaly threshold are identified as potential fault points. Spatial clustering is performed on the potential fault points, and neighboring fault points with a physical distance < preset distance are aggregated into predicted fault regions.

[0035] Step 5: Digital Twin Fault Scene Mapping: Align the physical coordinates of the base space with the three-dimensional coordinates of the digital twin, and import the optimized fiber bundle feature data; simulate future heat diffusion trends to generate dynamic thermal maps and vibration cloud maps;

[0036] The physical coordinates of the base space are precisely aligned with the three-dimensional coordinates of the digital twin model, and optimized fiber bundle feature data, such as anomaly degree and connection coefficient, are imported. The heat diffusion trend in future time periods is simulated through digital twin, and a visualized dynamic heat map and vibration cloud map are generated. The inspection robot sends the predicted fault area, dynamic heat map and vibration cloud map to the client.

[0037] Compared with the prior art, the beneficial effects of the present invention are:

[0038] By fusing multimodal features with non-extensive entropy and graph attention models, a high-dimensional fault feature pool is constructed to effectively cope with interference in the complex environment of the data center, improve the robustness of the model, and adapt to the general early warning needs of different devices.

[0039] By combining wavelet transform, fractional Brownian motion algorithm and other methods, nonlinear features and micro-mutation signals of equipment thermal distribution can be extracted, which can identify early and weak faults that are difficult to detect by traditional methods and achieve early warning.

[0040] Based on the fiber bundle model, the device topology base space and feature propagation path are constructed. Combined with parameter optimization, the diffusion law of thermal faults between devices is accurately characterized, and the accurate mapping from feature anomalies to physical locations is achieved. Attached Figure Description

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. The following drawings are not drawn to scale according to the actual size, but are intended to illustrate the main idea of ​​the present invention.

[0042] Figure 1 This is a flowchart of the method steps of the present invention;

[0043] Figure 2 The algorithm framework of this invention Figure 1 ;

[0044] Figure 3 The algorithm framework of this invention Figure 2 . Detailed Implementation

[0045] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are also within the scope of protection of the present invention.

[0046] Please refer to Figures 1-3 As shown, this invention provides a thermal fault early warning method for a multimodal sensing data center inspection robot, the specific steps of which are as follows:

[0047] Step 1: Data Acquisition in the Computer Room: The robot uses a multimodal sensor array to collect infrared thermal images, acoustic data, vibration data, and gas concentration data of the computer room. A three-dimensional temperature field reconstruction algorithm is then used to generate a voxelized temperature distribution.

[0048] The inspection robot moves along a preset inspection path. Upon reaching a monitoring point, it triggers a multimodal sensor array to start collecting data. The multimodal sensor array includes an infrared thermal imager, an acoustic sensor, a vibration sensor, and a gas sensor. It acquires real-time data sets of equipment in the computer room, including infrared images, equipment operating noise, equipment vibration signals, and gas concentrations around the equipment. The acquired multiple frames of infrared images are then arranged to obtain an infrared image sequence.

[0049] The solution adopts a binocular structured light camera, which consists of two infrared cameras and one infrared projector. The infrared projector emits infrared light with preset patterns, such as stripes and dot patterns, to enhance the surface texture features of the equipment in the computer room, making it easier for the camera to recognize and match. The camera and projector are installed at the front end of the robot.

[0050] An infrared projector projects a pre-encoded structured light pattern onto the cabinet surface. The structured light pattern resembles a sequence of binary code stripes. The light is distorted by the shape or distance of the equipment surface. Two infrared cameras simultaneously acquire images with the distorted pattern. Each acquisition generates one image from each camera, forming a binocular image pair. Epipolar correction is performed on the acquired binocular images. Using camera calibration parameters, the image coordinates are transformed to the epipolar constraint plane, ensuring that the search for corresponding points occurs only in the same row, improving matching efficiency and accuracy. A block-based matching algorithm is used to search for matching points in the epipolar-corrected binocular images. Centered on a pixel in the left image, a certain size image block is selected. The image block with the highest similarity in the corresponding row of the right image is searched to determine the matching point pair and generate a disparity image DT. Disparity is the difference in pixel coordinates of corresponding points in the horizontal direction. Based on the principle of triangulation, combined with the baseline distance BC obtained from camera calibration, the camera focal length FG, and the disparity image, a formula is used to calculate... The depth values ​​of each point on the surface of the computer cabinet are calculated pixel by pixel to generate a structured light depth image ZR. Each pixel value in the image represents the actual distance of the corresponding point from the camera, providing accurate spatial location information for the reconstruction of the three-dimensional temperature field.

[0051] The structured light depth image and infrared image sequence are input into the U-Net-LSTM network. The encoder extracts multi-scale temperature features through convolution and outputs a temperature feature map. The LSTM layer captures the temperature change line in the time dimension through bidirectional LSTM. The decoder outputs a voxelized temperature distribution. ,in The dimensions are horizontal, vertical, and depth, with t representing a time point, covering the three-dimensional space inside the device.

[0052] It should be noted that the voxelized temperature distribution is achieved by fusing two-dimensional temperature information from infrared thermal imaging with three-dimensional spatial information from structured light depth maps via a network, and then reconstructing the data through an encoder-temporal modeling-decoder process. The final output is the temperature value of each tiny region in the three-dimensional space inside the device.

[0053] Step 2, Cooperative Temperature Features: The voxelized temperature distribution is decomposed into multiple scales by wavelet transform, and the Hurst exponent features at each scale are extracted by fractional Brownian motion algorithm. Then, the wavelet energy features are fused to output a set of global temperature features of the device that includes the surface and internal heat distribution of the device.

[0054] The voxelized temperature distribution is decomposed into multiple layers using a wavelet algorithm with a low-pass filter (HY) and a high-pass filter (HT). The decomposition scales for each filter are set to micro, meso, and macro scales. The micro scale captures localized, minute temperature abrupt changes, such as chip-level hotspots and localized heat dissipation anomalies. The meso scale reflects regional temperature gradient changes, such as heat transfer between server modules. The macro scale extracts global temperature trends, such as the overall heat dissipation status of the server rack and the influence of ambient temperature. The decomposition calculation logic using the recursive rules of the low-pass filter is as follows: ,in Let k be the low-frequency quantity at the k-th decomposition scale, where k is the decomposition scale number. For convolution operations, Represents the tensor product. , and For low-pass filters in different directions; the decomposition calculation logic using the recursive rules of high-pass filters is as follows: Where T is an abbreviation for voxelization temperature distribution. , and These are high-pass filters in different directions. , and These are high-frequency components in different directions, which, when superimposed, constitute a high-frequency quantity. ;

[0055] The surface and internal regions of the cabinet are labeled as region GS and region FI, respectively. Hurst exponents are calculated at k scales of wavelet decomposition to quantify the self-similarity of temperature distributions at different scales. The voxelized temperature distribution is then distributed across a spatial grid at scale k, and for any spatial location... , or Extract the time series of this spatial location , where N is the total number of time points; the mean temperature at this spatial location over time is calculated accordingly. Then, the cumulative deviation of the temperature series is obtained based on the mean. Its calculation logic is as follows: , The cumulative deviation over the first n time steps is used to construct a subsequence length, where n < N. It should be noted that the cumulative deviation reflects the temperature's deviation from the mean. The range is obtained by subtracting the maximum and minimum values ​​of the subsequence length; the range reflects the fluctuation range of the cumulative deviation sequence. The standard deviation of the subsequence length is then calculated, and the rescaled range is obtained by dividing the range by the standard deviation. Fitting by least squares method The slope is the Hurst exponent H, and its calculation logic is as follows: Where C is the intercept of the linear fit, Let H be the natural logarithm function, and argmin be the value of the parameter H that minimizes the expression within the parentheses. It should be noted that H = 1 indicates strong self-similarity, and H = 0 indicates weak self-similarity. Repeating the above steps for each scale yields the Hurst exponent for each scale. ;

[0056] The transient thermal mutation characteristics at each scale are extracted by high-frequency quantities, and the energy of high-frequency quantities at each scale is calculated. This reflects the intensity of temperature fluctuations at this scale, and its calculation logic is as follows: The system locates the modulus maxima in high-frequency quantities, extracts the hotspot coordinates (r,t) and the corresponding temperature peak values, and fuses the voxelized temperature distribution, fractional Brownian motion features and wavelet features across scales to construct a global temperature feature set for the equipment. This global temperature feature set covers the temperature conditions of the surface and interior of the cabinet equipment, reflecting both the spatial coverage from the outside to the inside and highlighting the integrated description of the overall temperature state of the cabinet equipment, accurately reflecting the comprehensive temperature situation of each part of the cabinet equipment.

[0057] Step 3: Multi-feature extraction: The inspection robot constructs a noise tower, an equipment vibration tower, and a gas concentration tower, outputs the corresponding features, and integrates them into an auxiliary judgment feature set. The auxiliary judgment features are then fused with the equipment global temperature feature set through a non-widespread entropy algorithm and a graph attention model to construct a high-dimensional temperature fault feature pool.

[0058] Noise towers are identified using power spectrum analysis and Empirical Mode Decomposition (EMD). Equipment operating noise is segmented into frames and windowed using a Hanning window. A Fast Fourier Transform (FFT) is performed on each frame using the Welch method to calculate the corresponding power spectrum. The power spectrum distribution curve is obtained by averaging the power spectra across multiple frames. EMD decomposition of the equipment operating noise yields several intrinsic mode functions (CLs), each corresponding to a vibration mode at a different frequency scale. This is suitable for capturing transient noise during equipment start-up, shutdown, and sudden faults. The noise energy ratio and instantaneous frequency of each CL are calculated. Specifically, the noise energy calculation logic for a single CL is as follows: ,in and Let dt be the start and end times of the calculation process, and dt be the time increment. The total noise energy of all intrinsic mode functions is obtained by dividing the individual noise energy by the total noise energy. It should be noted that the role of Empirical Mode Decomposition (EMD) is to adaptively decompose equipment operating noise, breaking down complex noise signals into several intrinsic mode functions. These intrinsic mode functions correspond to vibration modes at different frequency scales, accurately capturing transient characteristics in the noise, such as abnormal noise during equipment start-up, shutdown, and sudden failures. Through EMD decomposition, the noise energy ratio and instantaneous frequency of each intrinsic mode function can be further extracted. These characteristics effectively reflect subtle changes in the equipment's operating state, such as abnormal noise energy at specific frequencies generated by faulty components, providing crucial auxiliary judgment criteria for thermal fault early warning and enhancing sensitivity to early, subtle faults.

[0059] Noise energy ratio; instantaneous noise frequency The calculation logic is as follows: ,in The sampling interval is... The instantaneous phase is represented by `unwrap`, which is the phase unwrapping function used to eliminate phase abrupt changes.

[0060] The vibration tower of the equipment is identified using differential geometric manifold algorithms and quantum resonance analysis. The specific identification steps are as follows:

[0061] S001: Acquire the equipment vibration signal xc, select the embedding dimension mc using the spurious nearest neighbor method, set the time delay rk, and construct the phase space matrix using the embedding dimension and time delay. The construction logic is as follows: Where K represents the total sampling time points of the vibration equipment, and each row of the matrix corresponds to a phase space point;

[0062] S002: The k-nearest neighbor algorithm is used to obtain the neighborhood points corresponding to each phase space point. PCA is then used to reduce the dimensionality of these neighborhood points to obtain the local tangent space. It should be noted that the core significance of the local tangent space lies in capturing the low-dimensional intrinsic structure of high-dimensional vibration data, providing key geometric features to support the identification of minor equipment faults and nonlinear vibration modes. PCA (Principal Component Analysis) is a commonly used dimensionality reduction algorithm that maps high-dimensional data to a low-dimensional space through linear transformation, reducing the data dimensionality while retaining the main features of the data. The role of PCA is to reduce the dimensionality of the neighborhood points in the phase space obtained by the k-nearest neighbor algorithm to obtain the local tangent space. This process can effectively extract key information from high-dimensional vibration data, remove redundant noise, and accurately capture the low-dimensional intrinsic structure of the data, thereby identifying minor equipment faults, such as early bearing wear and component loosening.

[0063] S003: The quantum resonance analysis algorithm performs a Fourier transform on the equipment vibration signal to obtain the vibration spectrum. Embedding the vibration spectrum into the potential energy function Its calculation logic is as follows: ,in For small changes, it is used to represent the integration range, the spatial distribution of vibration energy, and abnormal vibration can cause local peaks in the potential energy function;

[0064] S004: Substitute the potential energy function into the Schrödinger equation to obtain the vibrational Hamiltonian. Its calculation logic is as follows: Where hr is the reduced Planck constant, and mc is the equivalent mass, which is obtained by extracting the mass of equipment components from the database. The Laplace operator is used; the vibration characteristic equation is constructed and solved based on the Hamiltonian. The equation formula is as follows: The vibration energy EC is obtained. The vibration energy is divided by the reduced Planck constant to obtain the eigenfrequency. The normal eigenfrequency is obtained from historical data. The absolute value of the eigenfrequency and the normal eigenfrequency is taken to obtain the eigenfrequency deviation. The eigenfrequency deviation is a key indicator reflecting the structural state and fault characteristics of the equipment. This deviation often precedes changes in macroscopic characteristics such as vibration amplitude and temperature. It can be used as an auxiliary signal for equipment temperature faults to achieve early warning of temperature faults.

[0065] Gas concentration towers extract gas characteristics through fractional-order diffusion field reconstruction, specifically:

[0066] G001: UE that acquires gas concentration data in the computer room. Its sampling points cover key areas of the computer room, such as gaps between server racks, air conditioning vents, and the bottom of equipment. A fractional-order diffusion equation is constructed based on the gas concentration data. The calculation logic is as follows: Where is the air diffusion coefficient of the computer room. If the source term is... A value of 0 indicates no leakage, while a value greater than 0 indicates the presence of leakage. For time t fractional derivative;

[0067] G002: Based on the fractional diffusion equation, the fractional diffusion field under normal conditions is obtained. It can be trained using historical fault-free data. The deviation between the real-time field and the normal field is marked as the diffusion anomaly index.

[0068] The power spectrum distribution curve, noise energy ratio, instantaneous frequency, local tangent space, intrinsic frequency deviation, and diffusion anomaly index are integrated into an auxiliary judgment feature set;

[0069] Calculate the joint non-extensive entropy of the feature xm of the device's global temperature feature set and the feature xv of the auxiliary decision feature set. Its calculation logic is as follows: , of which Summing the square of the joint probability distribution, where q is a non-extensive parameter used to describe non-equilibrium states and relationships with long-range correlations or nonlinearities, such as the coupling relationship between temperature and vibration in a computer room; embedding the joint non-extensive entropy into the mutual information formula to obtain the correlation coefficient between any feature in the auxiliary judgment feature set and the features of the equipment's global temperature feature set. The logic for calculating its correlation coefficient is as follows: , The larger the value, the stronger the nonlinear correlation between the two features; the auxiliary features are screened by arranging the features of the auxiliary judgment feature set according to the magnitude of the correlation coefficient, and retaining the first m auxiliary features to obtain the screened auxiliary feature set;

[0070] Using features from the auxiliary feature set and the device's global temperature feature set as nodes, a feature association graph is constructed with the association coefficient as the initial edge weight. This feature association graph is then input into a graph attention model, which outputs attention coefficients. Specifically, the graph attention model works by applying attention coefficients to each temperature node. Calculate its connection with all connected auxiliary nodes. Attention coefficient Its calculation logic is as follows: ,in and Here, m1 represents the original feature vectors of the auxiliary nodes and temperature nodes, m2 and m3 are the indices of the auxiliary nodes and temperature nodes, and WT is the shared weight matrix, which is obtained by mapping the original features to a high-dimensional space. This is the attention parameter vector, used to learn the importance of feature associations. For activation functions; The larger the value, the stronger the explanatory power of the auxiliary features for the temperature features. Thus, all attention coefficients are obtained and two layers of GAT are stacked to obtain global correlation features. The global correlation features are then max-pooled to obtain the pooling result. Finally, the pooling result and feature nodes are concatenated to obtain a high-dimensional fault feature pool.

[0071] Step 4: Fiber Bundle Model Prediction: Construct a fiber bundle model with the topology of the computer room equipment as the base space and the high-dimensional fault feature pool as the fiber vectors. Then, use the simulated annealing algorithm to optimize the model parameters. With the F1 score as the target, iteratively update the parameters through the Metropolis criterion. Finally, calculate the feature propagation anomaly degree based on the optimized model to determine and locate thermal faults.

[0072] Using the physical topology of the data center equipment as the base space B (e.g., the location distribution of servers, switches, and cooling systems), it is discretized into grid-like nodes b. Each grid node corresponds to a device component or region. The topological relationships of the base space are represented by an adjacency matrix. If two grid points are physically connected, the corresponding adjacency matrix is ​​1; otherwise, it is 0. Fiber vectors are set. For any grid node b, the high-dimensional fault feature pool is defined. The fiber vector is obtained by mapping the high-dimensional fault feature pool to the base space.

[0073] The connection coefficient is a key component of the fiber bundle feature propagation algorithm, used to quantify the propagation pattern of high-dimensional fault feature pools in the device topology space. Its core idea is that the propagation strength of features between adjacent nodes is positively correlated with feature differences and negatively correlated with physical distance. The calculation logic of the connection coefficient is as follows: ,in and Let be the numbers of any two nodes. For the connection coefficient, dm is the topological constraint coefficient, which controls the correlation between propagation intensity and physical distance, and dm is the physical distance. The fiber bundle model is constructed in this way.

[0074] The simulated annealing algorithm optimizes fiber bundle parameters. Initial topological constraint coefficients, provided by historical data, are marked as parameters to be optimized. The thermal fault prediction performance of the fiber bundle model is marked as a thermal energy value (HM), where a lower thermal energy value indicates a better parameter. Random perturbations are applied to the parameters to be optimized to generate new parameters, thus obtaining new thermal energy values ​​for these new parameters. The difference between the new thermal energy value and the current thermal energy value is calculated. If the thermal energy difference is less than 0, the new parameters to be optimized are directly accepted, and the current parameters are updated to the new parameters. If the thermal energy difference is greater than 0, random perturbations continue. When the number of iterations reaches a preset value, iteration stops, and the optimal topological constraint coefficients are output and sent to the fiber bundle model.

[0075] The connection coefficients of all neighboring nodes of each node in the base space are extracted to form several local connection coefficient sets. The mean of the connection coefficients of each node is calculated, and the maximum deviation between the connection coefficient and the mean of the node connection coefficients is obtained. The maximum deviation is marked as the thermal fault anomaly, and the thermal fault anomaly of each node in the base space is obtained. A preset anomaly threshold is extracted from the database, and nodes whose thermal fault anomalies exceed the preset anomaly threshold are identified as potential fault points. Spatial clustering is performed on the potential fault points, and neighboring fault points with a physical distance < preset distance are aggregated into predicted fault regions.

[0076] Step 5: Digital Twin Fault Scene Mapping: Align the physical coordinates of the base space with the three-dimensional coordinates of the digital twin, and import the optimized fiber bundle feature data; simulate future heat diffusion trends to generate dynamic thermal maps and vibration cloud maps;

[0077] The physical coordinates of the base space are precisely aligned with the three-dimensional coordinates of the digital twin model, and optimized fiber bundle feature data, such as anomaly degree and connection coefficient, are imported. The heat diffusion trend in future time periods is simulated through digital twin, and a visualized dynamic heat map and vibration cloud map are generated. The inspection robot sends the predicted fault area, dynamic heat map and vibration cloud map to the client.

[0078] The above formulas are all dimensionless numerical calculations. Dimensionlessness can be achieved through standardization or other methods, which will not be elaborated here. The formulas are obtained through software simulation by collecting a large amount of data on the operation of computer room equipment (such as infrared thermal imaging, vibration, gas concentration, etc.), which closely matches the real situation of thermal fault early warning scenarios. The preset parameters (such as wavelet decomposition scale, anomaly threshold, etc.) are set by those skilled in the art based on the actual situation of the computer room equipment type, layout, etc.

[0079] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any other combination. If implemented in software, it can be presented in whole or in part as a computer program product, which includes one or more computer instructions or programs. Loading or executing these instructions or programs on a computer will produce, in whole or in part, the processes or functions described in the embodiments of this method (such as voxelized temperature distribution generation, feature correlation graph construction, fiber bundle model parameter optimization, etc.). The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one medium to another, for example, from one website, computer, server, or data center to another via wired (such as cable) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that is accessible to a computer, or a data storage device such as a server or data center that includes one or more sets of available media. Available media include magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), semiconductor media (such as solid-state drives), etc.

[0080] It should be understood that the sequence number of each step in each embodiment of this method does not represent the execution order. The execution order is determined by its function and internal logic (e.g., data needs to be collected through a multimodal sensor first, then feature extraction is performed, and finally fault warning is achieved), and does not constitute a limitation on the implementation process.

[0081] Those skilled in the art will recognize that the example units and algorithm steps described in conjunction with the embodiments disclosed herein (such as power spectrum calculation of noise towers, phase space matrix construction of vibration towers, and connection coefficient solution of fiber bundle models) can be implemented through electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can implement the functions using different methods for each specific application, but this should not be considered beyond the scope of this method.

[0082] In the several embodiments provided in this method, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the above apparatus embodiments are merely illustrative, and the unit division is only a logical functional division (such as dividing feature extraction into modules such as temperature features and auxiliary features). In practice, there may be other division methods, such as multiple units or components being combined or integrated into another system, or certain features being ignored or not executed. Furthermore, the mutual coupling, direct coupling, or communication connection shown or discussed can be implemented through an interface, and the indirect coupling or communication connection of the apparatus or unit can be electrical, mechanical, or other forms (such as the transmission of equipment vibration signals through a mechanical structure).

[0083] The units described as separate components may be physically separated or not. The components shown as units may be physically units or distributed across multiple network units. Some or all units may be selected to achieve the purpose of this embodiment according to actual needs (such as selecting only the features of key equipment areas for fault warning).

[0084] In addition, in each embodiment of this method, each functional unit can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit (such as integrating digital twin mapping and early warning output into the same module).

[0085] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this, the contribution of this method's technical solution to the prior art, or a portion of the technical solution, can be embodied as a software product. This computer software product is stored in a storage medium and contains several instructions that cause a computer device (such as a personal computer, server, network device, etc.) to execute all or part of the steps described in the various embodiments of this method (such as data acquisition, feature fusion, fault location, digital twin mapping, etc.). The aforementioned storage medium includes media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, and optical disks.

[0086] The above description is merely a specific embodiment of this method, but its protection scope is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this method should be included within the protection scope. Therefore, the protection scope of this method should be determined by the protection scope of the claims.

Claims

1. A thermal fault early warning method for a multimodal sensing data center inspection robot, characterized in that, Includes the following steps: The robot collects infrared thermal images, acoustic data, vibration data, and gas concentration data of the computer room using a multimodal sensor array, and generates a voxelized temperature distribution using a three-dimensional temperature field reconstruction algorithm. The voxelized temperature distribution is decomposed into multiple scales by wavelet transform, and the Hurst exponent features at each scale are extracted by fractional Brownian motion algorithm. Then, the wavelet energy features are fused to output a set of global temperature features of the device that includes the surface and internal heat distribution of the device. The inspection robot constructs a noise tower, an equipment vibration tower, and a gas concentration tower, outputs corresponding features, and integrates them into an auxiliary judgment feature set. The auxiliary judgment features are then fused with the equipment global temperature feature set through a non-extensive entropy algorithm and a graph attention model to construct a high-dimensional temperature fault feature pool. A fiber bundle model is constructed with the topology of the computer room equipment as the base space and the high-dimensional fault feature pool as the fiber vector. The model parameters are then optimized using the simulated annealing algorithm. With the F1 score as the objective, the parameters are iteratively updated using the Metropolis criterion. Finally, the feature propagation anomaly degree is calculated based on the optimized model to determine and locate thermal faults. Based on the alignment of spatial physical coordinates and digital twin three-dimensional coordinates, the optimized fiber bundle feature data is imported to simulate future heat diffusion trends and generate dynamic thermal maps and vibration cloud maps. A fiber bundle model is constructed, with the topology of the data center equipment as the base space and a high-dimensional fault feature pool as the fiber vectors. Specifically: Using the physical topology of the equipment in the data center as the base space, it is discretized into grid nodes, with each grid node corresponding to a device component or region. The topological relationships in the base space are represented by an adjacency matrix. If two grid points are physically connected, the corresponding adjacency matrix is ​​1, and otherwise 0. The fiber vector is set as a high-dimensional fault feature pool for any grid node. Then, the connection coefficients of the base space are obtained to construct the fiber bundle model. The fiber vector is obtained by mapping the high-dimensional fault feature pool to the base space. The model parameters are optimized using a simulated annealing algorithm, and then the feature propagation anomaly degree is calculated based on the optimized model to determine and locate thermal faults. Specifically: The simulated annealing algorithm optimizes fiber bundle parameters. The initial topological constraint coefficients are marked as parameters to be optimized. The initial topological constraint coefficients are provided by historical data. The thermal fault prediction performance of the fiber bundle model is marked as thermal energy value. The parameters to be optimized are randomly perturbed to generate new parameters to be optimized. The new thermal energy value of the new parameters to be optimized is obtained. The difference between the new thermal energy value and the current thermal energy value is obtained. If the thermal energy difference is less than zero, the new parameters to be optimized are directly accepted and the current parameters are updated to the new parameters. If the thermal energy difference is greater than zero, random perturbation continues. When the number of iterations reaches the preset value, the iteration stops and the optimal topological constraint coefficients are output and sent to the fiber bundle model. Extract the connection coefficients of all neighboring nodes of each node in the base space to form several local connection coefficient sets. Calculate the mean of the connection coefficients of each node and obtain the maximum deviation between the connection coefficient and the mean of the node connection coefficients. Mark the maximum deviation as the thermal fault anomaly degree to obtain the thermal fault anomaly degree of each node in the base space. Extract the preset anomaly degree threshold in the database and identify nodes whose thermal fault anomaly degree exceeds the preset anomaly degree threshold as potential fault points. Spatial clustering is performed on potential fault points, and adjacent fault points with a physical distance of less than a preset distance are aggregated into predicted fault areas.

2. The thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 1, characterized in that, The inspection robot constructs a noise tower, an equipment vibration tower, and a gas concentration tower, outputs corresponding features, and integrates them into an auxiliary judgment feature set. The specific steps are as follows: Noise towers are identified by power spectrum analysis and empirical mode decomposition. The equipment operating noise is divided into frames and Hanning windows are applied. The Welch method is used to perform fast Fourier transform on each frame of signal to calculate the corresponding power spectrum. The power spectrum distribution curve is obtained by averaging the power spectra of multiple frames. Empirical mode decomposition is performed on the equipment operating noise to obtain several intrinsic mode functions. The noise energy ratio and instantaneous frequency of each intrinsic mode function are obtained. The vibration tower of the equipment identifies vibration characteristics and acquires equipment vibration signals through differential geometric manifold algorithm and quantum resonance analysis. The embedding dimension is selected using the spurious nearest neighbor method, and a time delay is set. A phase space matrix is ​​constructed using the embedding dimension and time delay, with each row of the matrix corresponding to a phase space point. The k-nearest neighbor algorithm is used to obtain the neighborhood points corresponding to each phase space point, and PCA is applied to the neighborhood points to reduce the dimensionality and obtain the local tangent space. The quantum resonance analysis algorithm performs a Fourier transform on the equipment vibration signal to obtain the vibration spectrum, which is then embedded into a potential energy function. The potential energy function is then substituted into the Schrödinger equation to obtain the vibration Hamiltonian. Based on the Hamiltonian, a vibration characteristic equation is constructed and solved to obtain the vibration energy. The vibration energy is divided by the reduced Planck constant to obtain the eigenfrequency. The normal eigenfrequency is obtained from historical data. The absolute value of the eigenfrequency and the normal eigenfrequency is taken to obtain the eigenfrequency deviation. The gas concentration tower extracts gas characteristics by reconstructing a fractional-order diffusion field, obtains gas concentration data for the computer room, and constructs a fractional-order diffusion equation from the gas concentration data; Based on the fractional diffusion equation, the fractional diffusion field under normal conditions is obtained by training with historical fault-free data. The deviation between the real-time field and the normal field is marked as the diffusion anomaly index. Then, the power spectrum distribution curve, noise energy ratio, instantaneous frequency, local tangent space, intrinsic frequency deviation and diffusion anomaly index are integrated into an auxiliary judgment feature set.

3. The thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 2, characterized in that, The auxiliary feature set is selected using a non-extensive entropy algorithm, specifically as follows: For the features of the equipment global temperature feature set and the features of the auxiliary judgment feature set, calculate their joint non-extensive entropy; embed the joint non-extensive entropy into the mutual information formula to obtain the correlation coefficient of any feature in the auxiliary judgment feature set with the features of the equipment global temperature feature set; filter the auxiliary features; arrange the features of the auxiliary judgment feature set according to the size of the correlation coefficient; and retain the first m auxiliary features to obtain the filtered auxiliary feature set.

4. The thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 3, characterized in that, The specific steps for outputting a high-dimensional fault feature pool using the graph attention algorithm are as follows: Using the features of the auxiliary feature set and the device global temperature feature set as nodes, a feature association graph is constructed with the association coefficient as the initial edge weight. The feature association graph is input into the graph attention model and the attention coefficient is output. Specifically, the graph attention model calculates the attention coefficient of each temperature node with all connected auxiliary nodes, thereby obtaining all attention coefficients and performing two layers of GAT stacking to obtain global association features. The global association features are max pooled to obtain the pooling result, and then the pooling result and feature nodes are concatenated to obtain a high-dimensional fault feature pool.

5. A thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 1, characterized in that, The physical coordinates of the base space are aligned with the three-dimensional coordinates of the digital twin to simulate future heat diffusion trends, generating dynamic thermal maps and vibration cloud maps, specifically: The physical coordinates of the base space are precisely aligned with the three-dimensional coordinates of the digital twin model, and the optimized fiber bundle feature data, including anomaly degree and connection coefficient, are imported. The heat diffusion trend in future time periods is simulated through digital twin, and a visualized dynamic heat map and vibration cloud map are generated. The inspection robot sends the predicted fault area, dynamic heat map and vibration cloud map to the client.

6. A thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 1, characterized in that, The robot uses a multimodal sensor array to collect infrared thermal images, acoustic data, vibration data, and gas concentration data from the computer room. Specifically: The inspection robot moves along a preset inspection path. Upon reaching a monitoring point, it triggers a multimodal sensor array to start collecting data. The multimodal sensor array includes an infrared thermal imager, an acoustic sensor, a vibration sensor, and a gas sensor. It acquires real-time data sets of equipment in the computer room, including infrared images, equipment operating noise, equipment vibration signals, and gas concentrations around the equipment. The acquired multiple frames of infrared images are then arranged to obtain an infrared image sequence.

7. A thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 6, characterized in that, The specific steps for obtaining a structured light depth map and generating a voxelized temperature distribution using a 3D temperature field reconstruction algorithm are as follows: The system employs a binocular structured light camera solution, consisting of two infrared cameras and one infrared projector. The infrared projector emits infrared light with a preset pattern, and the cameras and projector are mounted on the front end of the robot. An infrared projector projects a pre-encoded structured light pattern onto the surface of the cabinet. The light is distorted by the shape or distance of the equipment surface. Two infrared cameras simultaneously acquire images with the distorted pattern. Each time an image is acquired, the left and right cameras each generate an image, forming a binocular image pair. The acquired binocular images are subjected to epipolar correction processing. Through camera calibration parameters, the image coordinates are transformed to the epipolar constraint plane to ensure that the search for corresponding points is only performed in the same row. A block-based matching algorithm is used to search for matching points in the epipolar-corrected binocular images. Taking a pixel in the left image as the center, a block of a certain size is taken. The block with the highest similarity is searched in the corresponding row of the right image to determine the matching point pair and generate a disparity image. Based on the principle of triangulation, combined with the baseline distance obtained by camera calibration, the camera focal length, and the disparity image, the depth value of each point on the computer cabinet surface is calculated pixel by pixel to finally generate a structured light depth image. The structured light depth image and infrared image sequence are input into the U-Net-LSTM network. The encoder extracts multi-scale temperature features through convolution and outputs a temperature feature map. The LSTM layer captures the temperature change line in the time dimension through bidirectional LSTM. The decoder outputs a voxelized temperature distribution.

8. A thermal fault early warning method for a multimodal sensing data center inspection robot according to claim 1, characterized in that, The process of outputting a global temperature feature set of the device, including the surface and internal heat distribution, is as follows: The voxelized temperature distribution is decomposed into multiple layers using wavelet algorithm with low-pass and high-pass filters. The decomposition scale of each filter is set to microscale, mesoscale, and macroscale. The low-frequency quantities of the decomposition scale are obtained through the decomposition calculation logic of the recursive rule of the low-pass filter. High-frequency quantities are obtained through the decomposition and calculation logic of the recursive rules of the high-pass filter; The surface and internal areas of the cabinet are labeled as regions and regions, respectively. The Hurst exponent is calculated at k scales using wavelet decomposition. The voxelized temperature distribution is distributed in a spatial grid at scale k. For any spatial location, the time series of that location is extracted. The mean temperature at that location in the time dimension is calculated, and the cumulative deviation of the temperature series is obtained based on the mean. The range is obtained by subtracting the maximum and minimum values ​​of the subsequence length, and then the standard deviation of the subsequence length is calculated. The rescaled range is obtained by dividing the range by the standard deviation. The data is fitted using the least squares method. The above steps are repeated for each scale to obtain the Hurst exponent at each scale. By extracting instantaneous thermal mutation features at various scales using high-frequency quantities, calculating the energy of high-frequency quantities at each scale, locating the modulus maxima in the high-frequency quantities, and extracting the hot spot coordinates and corresponding temperature peak values, the voxelized temperature distribution, fractional Brownian motion features, and wavelet features are fused across scales to construct a global temperature feature set for the device.

Images