Multi-modal intelligent linkage 3D visual data center operation and maintenance system and method

Abstract

The application discloses a multimodal intelligent linkage 3D visual data center operation and maintenance system and method, and relates to the technical field of data center operation and maintenance.The system comprises: a multimodal data fusion module for mapping physical sensor data, video monitoring and operation and maintenance logs to a unified three-dimensional coordinate system to construct a multimodal fusion feature tensor; a three-dimensional particle modeling module for constructing a particle space structure based on a Voronoi diagram and Delaunay triangulation and adaptively adjusting particle resolution; a multimodal scoring analysis module for calculating particle multidimensional scores and generating a semantic heat map to identify abnormal areas; a structured noise prediction module for simulating future responses under different instructions based on a diffusion model; an instruction generation and regulation module for realizing causal analysis and instruction optimization in combination with a knowledge graph; and a three-dimensional visual interactive module for supporting real-time rendering and interactive operation on a client.The application can improve the intelligence, real-time performance and visualization level of data center operation and maintenance.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data center operation and maintenance, in particular to a multi-modal intelligent linkage 3D visual data center operation and maintenance system and method. BACKGROUND

[0002] With the continuous expansion of the scale of data centers and the complexity of the operating environment, traditional operation and maintenance monitoring methods are difficult to meet the requirements of rapid perception of multi-source information, spatial positioning and trend prediction. The current common data center monitoring system mainly uses two-dimensional panels, tabular data or local video display, which has the significant shortcomings of information fragmentation, delayed abnormal response, lack of causal support for control operation, etc. Especially in the face of complex events such as sudden temperature rise and equipment failure spread, there is a lack of unified modeling and semantic linkage mechanism for multi-modal data, which leads to insufficient intelligent analysis and prediction ability, affecting system stability and operation efficiency.

[0003] Some existing research has proposed a scheme combining sensor data and image information for state monitoring, such as combining camera images and thermal sensor data for anomaly detection, but it is mostly limited to local areas, single modal features or static analysis, lacking multi-modal fusion, interactive visualization and control decision support capabilities based on a unified spatial model. In the field of image processing, although multi-modal visual models such as CLIP and ViT have been widely used in image-text semantic understanding, there is still a lack of mature solutions on how to collaboratively build a system-level analysis and control framework with heterogeneous information such as physical data and operation logs.

[0004] Therefore, there is an urgent need for a data center operation and maintenance system that can integrate multi-modal heterogeneous data, support semantic analysis and three-dimensional visual presentation, and have prediction simulation and linkage control capabilities, to solve the problems of perception delay, analysis fragmentation and control lag in existing technologies, and to improve the intelligence, real-time and visual interaction level of the system. SUMMARY

[0005] The present application proposes a multi-modal intelligent linkage 3D visual data center operation and maintenance system and method, which introduces a one-stop architecture of multi-modal feature tensor construction, three-dimensional particle modeling, semantic scoring analysis, structured prediction reasoning and visual linkage control, to solve the problems of multi-source data fragmentation, lack of real-time spatial correlation and prediction control capability in existing technologies.

[0006] The present application achieves the above-mentioned purposes through the following technical solutions:

[0007] The multi-modal intelligent linkage 3D visual data center operation and maintenance system comprises:

[0008] A multi-modal data fusion module is configured to collect physical sensor data, video monitoring data and operation and maintenance log data of a data center, map different modal data to a same three-dimensional space coordinate system through time synchronization and space alignment processing, and construct a multi-modal fusion feature tensor;

[0009] A three-dimensional particle modeling module is configured to construct a particle space structure of the data center based on a Voronoi diagram and a Delaunay triangulation method, establish a mapping relationship between particles and data items in the multi-modal fusion feature tensor, dynamically adjust a visualization resolution level of the particles according to a distance between the particles and a current perspective, and output a current particle state set;

[0010] A multi-modal scoring analysis module is configured to calculate a local concept score, a global concept score, a local visual score and a global visual score of each particle in the current particle state set, generate a semantic heat map according to a weighted scoring result, and identify an abnormal hotspot region meeting an abnormal prediction condition;

[0011] A structured noise prediction module is configured to construct a future particle state sketch for prediction, and generate a predicted particle state based on a structured noise reverse initialization and diffusion reasoning method, to simulate a system response possibly caused by different control instructions at a future time;

[0012] An instruction generation and regulation module is configured to combine the abnormal hotspot region, the predicted particle state and a knowledge graph rule, perform causal relationship analysis and control instruction optimization, and output a device control instruction set;

[0013] A three-dimensional visualization interaction module is configured to realize three-dimensional visualization interaction display on a client side through GPU acceleration and WebGL rendering technology, and at least includes real-time particle distribution rendering, abnormal hotspot region highlight display, trend evolution trajectory animation and device control instruction interactive operation.

[0014] Further improvements of the application are that the multi-modal data fusion module comprises:

[0015] A sensor data unit is configured to acquire physical sensor data of various devices in the data center, at least including temperature, humidity and power consumption, synchronize the physical sensor data according to a time stamp, and map the physical sensor data to a three-dimensional space coordinate system based on spatial installation positions of various sensors, to form physical features;

[0016] A video monitoring data unit is configured to acquire video monitoring data of the data center and perform image frame level decoding, extract visual features of a target region in an image, map the visual features to a three-dimensional space in combination with internal and external parameter information of a corresponding camera;

[0017] An operation and maintenance log data unit is configured to perform semantic analysis on operation and maintenance log texts and structure the operation and maintenance log texts into event triples, and map the event triples into a three-dimensional space coordinate system according to a preset space semantic mapping rule;

[0018] A data fusion unit is configured to fuse the physical features, the visual features and the semantic features in a unified three-dimensional space coordinate system and a time dimension to generate a multi-modal fusion feature tensor Wherein, x, y and z represent three-dimensional space coordinates, and t represents time.

[0019] Further improvement of the application is that the construction method of the particle space structure in the three-dimensional particle modeling module comprises:

[0020] The physical sensor nodes deployed in the data center space are taken as space seed points, a weighted Voronoi diagram is used for three-dimensional space region division, wherein the weight value of each physical sensor node is related to the sensor perception coverage, the historical abnormal record frequency or the importance of the region where the sensor node is located;

[0021] A centroid set is extracted in the weighted Voronoi region, and Delaunay triangulation is performed on the centroid set to construct a three-dimensional particle network structure;

[0022] After the construction is completed, a one-to-one mapping relationship is established according to the tensor values in the multi-modal fusion feature tensor in the region where each particle is located;

[0023] The three-dimensional particle modeling module monitors the local change gradient of the multi-modal fusion feature tensor during system operation, and dynamically adjusts the particle distribution structure based on a preset reconstruction threshold, refines the particles in a high abnormal region, and performs particle aggregation reconstruction in a stable region.

[0024] Further improvement of the application is that the three-dimensional particle modeling module determines the visualization resolution level of each particle according to the spatial distance between the particle and the current perspective center point and the temperature change gradient of the region where the particle is located, and the resolution level scheduling mechanism specifically comprises:

[0025] For each particle p i , the Euclidean distance d i between the particle and the perspective center point is calculated, and the temperature change gradient amplitude

[0026] A particle rendering weight function is constructed:

[0027]

[0028] In the formula, W i is the particle rendering weight, sigma is a Sigmoid function, beta and gamma are system preset adjustment coefficients.

[0029] According to the particle rendering weight W i A numerical value in the range of [0, 1], corresponding to a different visualization resolution level, and determining whether to retain, aggregate or refine the rendering granularity of the particle in the current frame;

[0030] In the system operation, the visualization resolution level of each particle can be dynamically updated, supporting visual continuous transition and smooth presentation, ensuring that the rendering of abnormal hotspot area particles is delicate, and the area far away from the view angle and slowly changing is dynamically simplified.

[0031] A further improvement of the present application is that the current particle state set at least includes:

[0032] The three-dimensional coordinate position (x i ,y i ,z i ) of the particle for spatial positioning;

[0033] The real-time monitored temperature value T i , humidity value H i and power consumption value P i , for characterizing the physical environment state of the space region corresponding to the particle;

[0034] The fusion feature vector v i , which is composed of the visual features and semantic features spliced by the visual features and semantic features extracted from the multi-modal fusion feature tensor of the region where the particle is located;

[0035] The visualization level label L i , for indicating the visualization resolution level of the particle under the current view angle;

[0036] The state entropy value for measuring the normalized uncertainty of the temperature, humidity and power consumption changes of the particle in the recent continuous multiple time points, and the calculation formula is:

[0037]

[0038] In the formula, p k is the distribution probability of any physical attribute k of the particle in a time window;

[0039] The life cycle counter C i , for recording the number of state updates since the generation of the particle, and triggering the particle resampling process when the upper limit of the example update life cycle set by the system is exceeded;

[0040] The abnormal cluster label A i , for identifying whether the particle is clustered and attributed to any abnormal hotspot area or potential fault domain in the abnormal pattern recognition, for supporting the priority calculation and abnormal tracking of the device control instructions.

[0041] Further improvement of the present application is that in the multi-modal scoring analysis module, the local concept score LC is based on the weighted average method of fusing local semantic heat and mask area proportion, used to measure the semantic concept saliency of the local area where the particle is located;

[0042] The global concept score GC is calculated by the image-text embedding similarity of the Alpha-CLIP model, used to evaluate the global semantic consistency of the particle image area and the target semantic text;

[0043] The local visual score LV is obtained by calculating the feature similarity of the support image and the current image in the local area of the particle, and the local average aggregation of the spatial matching matrix is included in the calculation process;

[0044] The global visual score GV is obtained by the overall embedding distribution between the support image and the current image extracted by the ViT or DINO model, and the EMD is used to measure the similarity of the particle region feature distribution between different images.

[0045] Further improvement of the present application is that in the multi-modal scoring analysis module, the context semantic consistency factor γ(p i ) is introduced, which is scored by the multi-modal large model on the consistency of the particle semantics and the global description of the current operation and maintenance scene, the value range is [0, 1], used to adjust the local concept score and the global concept score, and form the revised score item:

[0046] LC'(p i )=LC(p i )·γ(p i ), GC'(p i )=GC(p i )·γ(p i );

[0047] In the formula, p i represents the i-th particle, LC(p i ) is the local concept score of the particle p i , GC(p i ) is the global concept score of the particle p i ; LC'(p i ) is the revised local concept score of the particle p i , GC'(p i ) is the revised global concept score of the particle p i ;

[0048] The semantic-behavior association vector a(p i ) is introduced, which is obtained by text and video modal collaborative reasoning, used to predict the potential causal relationship between the area where the particle is located and the historical operation and maintenance event, and calculate the behavior enhancement weight item R(pi ):

[0049] R(p i )= <a(p i Emb event >;

[0050] In the formula, Emb event The semantic embedding vectors for known historical failure events; <·,·> represent vector dot products;

[0051] Calculate the particle information entropy based on the multimodal score distribution of the local region where the particle is located:

[0052]

[0053] In the formula, H local (p i ) is particle p i Information entropy of the surrounding local rating distribution; j is the score type index, j∈{1,2,3,4}; S j (p i ) is particle p i The score value of the j-th item; H global The mean entropy of the score distribution for all particles; i This indicates that the average value is taken over all particle numbers i.

[0054] Define the local-global entropy difference ΔH(p) i )=H local (p i )-H global The weights of each weighting term are adjusted as follows:

[0055] β′ j (p i )=β j ·(1+λ·△H(p i ));

[0056] In the formula, △H(p) i ) is particle p i The local-global entropy difference; λ is the entropy difference adjustment coefficient; β j β′ represents the base weight of the score for the j-th item. j (p i For particle p i The dynamically adjusted weighting coefficient of the j-th term;

[0057] Calculate the final weighted score S of the particle:

[0058] S(p i )=β′1(p i )·LC′(p i )+β′2(pi )·GC'(p i )+β'3(p i )·LV(p i )+β'4(p i )·GV(p i );

[0059] In the formula, S(p i ) is the final weighted score of the particle p i ; LV(p i ) is the local visual score of the particle p i ; GV(p i ) is the global visual score of the particle p i ;

[0060] The abnormal hotspot area is determined, and the abnormal prediction condition is as follows:

[0061]

[0062] In the formula, τ1 is an abnormal score threshold value; is a temperature change gradient at the position of the particle p i ; θ1 is a temperature change gradient threshold value; M heat (p i ) is a historical fault mask; θ2 is a temperature mask heat intensity threshold value; and ρ is a behavior correlation determination threshold value.

[0063] Further improvement of the present application is that the structured noise prediction module comprises:

[0064] A particle state sketch generator is configured to construct a future particle state sketch in a prediction period according to the current particle state set, the abnormal hotspot area and the semantic heat map, including particle position, attribute trend and semantic label distribution, and support to generate particle state evolution profiles at multiple time granularities respectively;

[0065] A structured noise initializer is configured to perform structured noise reverse initialization on the future particle state sketch coding to generate an initial state The calculation formula is as follows:

[0066]

[0067] In the formula, z0 is a latent variable representation of the future particle state sketch; α t is a noise scheduling coefficient of the diffusion time step t; and ∈ is a standard Gaussian structured noise.

[0068] A reverse diffusion reasoner is configured to perform a reverse diffusion process to generate a prediction particle state sequence P starting from the initial state (t+△t)And respectively simulate the evolution trajectory of system response under different control instructions;

[0069] An abnormal enhancement mechanism is used for adding an abnormal weight coefficient to the particle located in the abnormal hotspot area in the particle prediction process, so that the prediction process enhances the focusing property in the high-temperature and high-entropy area.

[0070] A closed-loop feedback updating unit is used for comparing the actual feedback state of the equipment with the predicted particle state, and constructing a prediction error function:

[0071]

[0072] In the formula, E(t) is the prediction error function value at time t; N is the total number of particles; is the predicted state vector of the i th particle; p i real is the actual feedback state vector of the i th particle;

[0073] When the prediction error function value E(t) exceeds the set threshold value, the structured noise generation parameter and the future particle state sketch are automatically updated.

[0074] Further improvement of the application is that the instruction generation and regulation module comprises:

[0075] A causal path modeler is used for combining the weighted score result of the abnormal hotspot area, the semantic heat map text and the device behavior rules defined in the knowledge graph, and constructing a causal path graph of event-causality-controllable variable;

[0076] A control instruction candidate generator is used for identifying the controllable variable from the causal path graph, calling the candidate device control instruction a corresponding to the controllable variable from the historical operation and maintenance instruction knowledge base i , and simulating the system response of each candidate device control instruction under different time granularity in combination with the predicted particle state;

[0077] A semantic response evaluation module is used for calling a multi-modal language model to generate a natural language summary of the device response of each candidate control instruction in the simulated predicted state, and performing semantic consistency comparison with the predicted particle scene, and calculating a response credibility score R(a i );

[0078] An instruction conflict detection module is used for detecting the combination of operation mutual exclusion, resource occupation conflict or device state limitation in the candidate device control instruction set, and eliminating the instruction groups that cannot be executed in parallel;

[0079] An execution risk evaluation module is used for combining the local temperature gradient, power consumption fluctuation amplitude and historical abnormal frequency in the predicted particle state, and constructing an instruction execution risk score p(a i), as a penalty factor of the control instruction utility function;

[0080] a fusion module for integrating the final weighted score S, the response credibility score R(a i ) and the execution risk score p(a i ), to calculate the control instruction utility function U(a i ):

[0081] U(a i ) = λ1·S + λ2·R(a i ) - λ3·p(a i );

[0082] wherein λ1, λ2, λ3 are weighting coefficients set by the system;

[0083] an instruction output selector for sorting the candidate device control instruction set according to the priority of the control instruction utility function U(a i ), and outputting the optimal device control instruction set executable by the current system to the device control end.

[0084] The multi-modal intelligent linkage 3D visual data center operation method is based on the multi-modal intelligent linkage 3D visual data center operation system as described above, and the method comprises:

[0085] Collecting physical sensor data, video monitoring data and operation log data of the data center, and through time synchronization and space alignment processing, uniformly mapping different modal data to the same three-dimensional space coordinate system, and constructing a multi-modal fusion feature tensor;

[0086] Based on the Voronoi diagram and Delaunay triangulation method, a particle space structure of the data center is constructed, a mapping relationship between the particles and the data items in the multi-modal fusion feature tensor is established, and the visualization resolution level of the particles is dynamically adjusted according to the distance between the particles and the current view angle, and a current particle state set is outputted;

[0087] Calculating the local concept score, global concept score, local visual score and global visual score of each particle in the current particle state set, generating a semantic heat map according to the weighted score result, and identifying an abnormal hotspot area meeting the abnormal prediction condition;

[0088] Constructing a future particle state sketch for prediction, and generating a predicted particle state based on a structured noise reverse initialization and diffusion reasoning method, for simulating the system response that may be triggered by different control instructions at a future time;

[0089] Combining the abnormal hotspot area, the predicted particle state and the knowledge graph rule, performing causal relationship analysis and control instruction optimization, and outputting a device control instruction set;

[0090] Three-dimensional visual interactive display is realized on the client side through GPU acceleration and WebGL rendering technology.

[0091] The beneficial effects of the present application are: by constructing a three-dimensional multi-modal fusion feature tensor, the representation forms of different types of data are unified, and the information integrability and spatiotemporal traceability are enhanced; by using the Voronoi diagram and Delaunay triangulation method to construct the particle space structure, the particle density and visualization resolution are adaptively adjusted in combination with the local gradient and perspective factors, the information presentation accuracy of key areas and the global rendering efficiency are improved; based on the local and global concept scores, visual similarity analysis and semantic causal reinforcement, multi-modal scores are calculated to form a semantic heat map, supporting rapid identification and spatial focusing of potential fault areas; based on structured noise modeling and diffusion reasoning mechanism, the future particle state change trajectory is predicted, the system evolution process under the action of different control instructions can be simulated, and the fault prevention and operation planning capability is enhanced; in combination with the predicted state, knowledge graph and language model, the causal path is constructed, the semantic consistency and risk score of the candidate control instruction are generated and evaluated, the intelligent optimization and execution conflict avoidance of the device control instruction are realized; based on GPU acceleration and WebGL rendering, the dynamic display and operation feedback of particle state, abnormal area and control process are realized, and the spatial perception and operation decision efficiency of the operation and maintenance personnel is enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0092] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0093] Among them:

[0094] Figure 1 The system structure block diagram of the present application;

[0095] Figure 2 The structure block diagram of the multi-modal data fusion module in the embodiment of the present application;

[0096] Figure 3 The implementation flowchart of the three-dimensional particle modeling module in the embodiment of the present application;

[0097] Figure 4 The implementation flowchart of the structured noise prediction module in the embodiment of the present application;

[0098] Figure 5 The method flowchart of the present application. DETAILED DESCRIPTION

[0099] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the described embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art belong to the scope of protection of the present application.

[0100] As shown in Figure 1 , for an embodiment of the present application, the embodiment provides a multi-modal intelligent linkage 3D visualization data center operation and maintenance system, at least including a multi-modal data fusion module, a three-dimensional particle modeling module, a multi-modal scoring analysis module, a structured noise prediction module, an instruction generation and regulation module, and a three-dimensional visualization interaction module.

[0101] (1) Multi-modal data fusion module

[0102] The multi-modal data fusion module is used for collecting physical sensor data, video monitoring data and operation and maintenance log data of the data center, and through time synchronization and space alignment processing, uniformly mapping different modal data to the same three-dimensional space coordinate system, and constructing a multi-modal fusion feature tensor.

[0103] As shown in Figure 2 , in one of the embodiments, the multi-modal data fusion module includes:

[0104] The sensor data unit is used for acquiring physical sensor data of various devices in the data center, at least including temperature, humidity and power consumption, synchronizing the physical sensor data according to the time stamp (such as NTP-based time stamp alignment), and mapping the physical sensor data to the three-dimensional space coordinate system based on the spatial installation position of each sensor, to form physical features;

[0105] The video monitoring data unit is used for acquiring video monitoring data of the data center and performing image frame level decoding, extracting visual features of a target region in the image, and mapping the visual features to the three-dimensional space in combination with the internal and external parameter information of the corresponding camera;

[0106] The operation and maintenance log data unit is used for performing semantic analysis (such as named entity recognition and event extraction technology) on the operation and maintenance log text and structuring it into event triples (for example: “server X”, “temperature anomaly”, “2024-10-01 14:22”), and mapping it to the three-dimensional space coordinate system according to the preset spatial semantic mapping rules (for example, a device number and position mapping table, and a fault event and facility area corresponding rule);

[0107] The data fusion unit is used for fusing the physical features, visual features and semantic features in the unified three-dimensional space coordinate system and time dimension, to generate a multi-modal fusion feature tensor wherein x, y, z represent three-dimensional space coordinates, and t represents time.

[0108] (2) Three-dimensional particle modeling module

[0109] The three-dimensional particle modeling module is used to construct a particle space structure of the data center based on a Voronoi diagram and a Delaunay triangulation method, to establish a mapping relationship between the particles and data items in the multi-modal fusion feature tensor, and to dynamically adjust the visualization resolution level of the particles according to the distance between the particles and the current view angle, and to output the current particle state set.

[0110] Specifically, as shown in Figure 3 the construction method of the particle space structure includes:

[0111] The physical sensor nodes (such as temperature and humidity sensors, power consumption meters, smoke sensors, etc.) deployed in the space of the data center are taken as space seed points, and a weighted Voronoi diagram is used for three-dimensional space region division, wherein the weight value of each physical sensor node is related to the sensor perception coverage, the historical abnormal record frequency, or the importance of the region (such as the core server area being more important than the peripheral area); based on the above weighted results, the three-dimensional space sub-region dominated by each sensor node is demarcated, ensuring that the space division is reasonable and has fault sensitivity;

[0112] The centroid set (i.e. the geometric center or feature point of the region) is extracted in the weighted Voronoi region, and Delaunay triangulation is performed on the centroid set to construct a three-dimensional particle network structure; the network has good spatial coverage and topological analyzability, which is beneficial to subsequent information propagation, reasoning and visualization operation;

[0113] After the construction, a one-to-one mapping relationship is established according to the tensor values in the multi-modal fusion feature tensor in the region where each particle is located, so that each particle can carry the fusion perception information of the region where it is located; the mapping data includes but is not limited to: current values of physical sensors (temperature and humidity, power consumption), video visual features, semantic event labels, etc.; the mapping relationship gives the particles multiple source attributes, making the particles become the carrying units of spatial data.

[0114] The three-dimensional particle modeling module monitors the local variation gradient of the multi-modal fusion feature tensor during system operation, and dynamically adjusts the particle distribution structure based on a preset reconstruction threshold, refines the particles in high heterogeneity regions (such as rapid temperature changes), increases the particle density to improve the monitoring accuracy, and performs particle aggregation reconstruction in stable regions (slow changes, low risk), reduces the particle density to reduce the computational burden.

[0115] The three-dimensional particle modeling method described in this embodiment enables the system to construct an efficient and adjustable spatial particle model based on the actual deployment environment, significantly improving the visualization and spatial analysis capabilities of multimodal data.

[0116] Furthermore, the 3D particle modeling module determines the visualization resolution level of each particle based on the spatial distance between each particle and the current viewpoint center point and the temperature gradient of the region where the particle is located. The resolution level scheduling mechanism specifically includes:

[0117] For each particle p i Calculate the Euclidean distance d between the particle and the center of the viewpoint. i Simultaneously calculate the magnitude of the temperature gradient in the local region where the particle is located.

[0118] d i =‖X i -C view ||;

[0119]

[0120] In the formula, X i For particle p i The corresponding three-dimensional coordinates; C view During the client rendering process, the system obtains the three-dimensional spatial coordinates of the current user's view center point in real time; For particle p i The set of spatially neighboring particles, b is any particle; T i T b These represent the real-time temperature values ​​of the particles;

[0121] Constructing particle rendering weight functions:

[0122]

[0123] In the formula, W i σ represents the particle rendering weights; σ is the Sigmoid function; β and γ are system-preset adjustment coefficients.

[0124] Based on particle rendering weight W i Values ​​in the range [0,1] correspond to different visualization resolution levels, and determine whether to retain, aggregate, or refine the rendering granularity of the particle in the current frame;

[0125] During system operation, the visualization resolution level of each particle can be dynamically updated, supporting continuous visual transitions and smooth presentation, ensuring that particles in abnormal hot spots are rendered delicately while areas far from the viewpoint and areas with slow changes are dynamically simplified.

[0126] In one preferred embodiment, the current particle state set at least includes:

[0127] a three-dimensional coordinate position (x i ,y i ,z i ) of the particle for spatial positioning;

[0128] a real-time monitored temperature value T i , a humidity value H i , and a power consumption value P i for characterizing the physical environment state of the space region corresponding to the particle;

[0129] a fusion feature vector v i composed of spliced visual features and semantic features extracted from the region where the particle is located in the multi-modal fusion feature tensor;

[0130] a visualization level label L i for indicating the visualization resolution level of the particle under the current perspective;

[0131] a state entropy value for measuring the normalized uncertainty of temperature, humidity, and power consumption changes of the particle in the recent continuous multiple time instants, and the calculation formula is:

[0132]

[0133] where p k is the distribution probability of any physical attribute k of the particle within a time window;

[0134] a life cycle counter C i for recording the number of state updates since the particle is generated, and triggering the particle resampling process when the upper limit of the system set example update life cycle is exceeded;

[0135] an anomaly clustering label A i for identifying whether the particle is clustered and attributed to any abnormal hotspot region or potential fault domain in the anomaly pattern recognition, for supporting the priority calculation and anomaly tracking of device regulation instructions.

[0136] (3) Multi-modal scoring analysis module

[0137] The multi-modal scoring analysis module is configured to receive the current particle state set and the multi-modal fusion feature tensor, calculate the local concept score, the global concept score, the local visual score, and the global visual score of each particle in the current particle state set, generate a semantic heat map according to the weighted scoring results, and identify an abnormal hotspot region that meets the abnormal pre-judgment condition.

[0138] Specifically, the local conceptual score LC is based on a weighted average method of fusing local semantic heat and mask area ratio, used to measure the semantic concept saliency of the local area where the particle is located.

[0139] The calculation formula of the local conceptual score LC is as follows:

[0140]

[0141] In the formula, p i represents the i-th particle, LC(p i ) is the local conceptual score of the particle p i ; α is a local saliency and heat weighting factor, 0 < α < 1; m i is the semantic mask area corresponding to the particle p i ; RTA(x, y) represents the semantic heat map intensity at the two-dimensional coordinate (x, y) (Refined Text Alignment); Area(m i ) is the area of the area corresponding to the mask m i ; Area(M) is the total area of all masks.

[0142] The global conceptual score GC is calculated by the image-text embedding similarity of the Alpha-CLIP model, used to evaluate the global semantic consistency between the particle image area and the target semantic text.

[0143] The global conceptual score GC is calculated by the Alpha-CLIP model as follows:

[0144]

[0145] In the formula, GC(p i ) is the global conceptual score of the particle p i ; is the normalized semantic embedding vector corresponding to the target text (such as the alarm description); is the normalized visual embedding vector corresponding to the image area where the particle p i is located.

[0146] The local visual score LV is obtained by calculating the feature similarity of the support image and the current image in the local area of the particle, and the local average aggregation of the spatial matching matrix is included in the calculation process.

[0147] The local visual score is used to evaluate the visual consistency between the area where the current particle is located and the support image and the current image. The score measures the visual semantic similarity between the known device or state area in the support sample and the corresponding area in the current operation scene, and is one of the important indicators for realizing spatial anomaly recognition and anomaly tracking.

[0148] In the specific implementation process, the calculation of the local visual score LV includes the following steps:

[0149] The support image I s and the current image I q are input into a pre-trained visual encoder (such as DINO v2, CLIP or Vision Transformer) at the same time.

[0150] For the three-dimensional spatial coordinates of each particle p i , the projection area R i of the particle in the image plane is located, and the corresponding set of visual embedding vectors is extracted:

[0151] F s = {f s (x, y)}, F q = {f q (x', y')}, (x, y), (x', y') ∈ R i ;

[0152] The cosine similarity calculation is performed on the embedding features of the support image and the current image in the local area R i of the particle, and a spatial matching matrix is obtained:

[0153]

[0154] On the basis of the matching matrix M, the local average aggregation is performed on the maximum matching value, and the local visual score LV of the particle is calculated: On the basis of the matching matrix M, the local average aggregation is performed on the maximum matching value, and the local visual score LV of the particle is calculated:

[0155]

[0156] The score reflects whether the local area of the current image has a similar semantic area in the support image, so as to be used for judging whether the device state is consistent, whether the structure is changed or whether the anomaly exists.

[0157] The visual encoder used in the application can be selected from the current mainstream image representation learning network, including but not limited to:

[0158] CLIP (Contrastive Language-Image Pretraining) model;

[0159] DINO or DINO v2;

[0160] LoFTR (Detector-Free Local Feature Matching);

[0161] Swin Transformer, ViT-B, and other lightweight self-attention structures.

[0162] The global visual score GV is the overall embedding distribution between the support image and the current image extracted by the ViT or DINO model, and the EMD (Earth Mover's Distance) is used to measure the similarity of the particle region feature distribution between different images.

[0163] The calculation formula of the global visual score GV is as follows:

[0164] GV(p i )=1-EMD(F s (p i ),F q (p i ));

[0165] In the formula, GV(p i ) is the global visual score of particle p i , the value closer to 1 indicates closer; F s (p i ) and F q (p i ) are the feature distributions of the support image and the current image in the region of particle p i .

[0166] Further, a context semantic consistency factor γ(p i ) is introduced in the multi-modal scoring analysis module, which gives the model a modeling structure for understanding the context, and scores the consistency of particle semantics and the global description of the current operation and maintenance scene by a multi-modal large model (such as LLaVA or GPT-4o), with a value range of [0, 1], which is used to adjust the local concept score and the global concept score to form the revised score item:

[0167] LC′(p i )=LC(p i )·γ(p i ), GC′(p i )=GC(p i )·γ(p i );

[0168] In the formula, LC′(p i ) is the revised particle p iLocal concept score; GC′(p i ) represents the corrected particle p i Global concept score;

[0169] Introducing semantic-behavior association vector a(p) i ), obtained through collaborative reasoning of text and video modalities, is used to predict the potential causal relationship between the region where the particle is located and historical operation and maintenance events, and to calculate the behavior enhancement weight term R(p i This is used to assess the degree of matching between particles and expected risk events.

[0170] R(p i )= <a(p i Emb event >;

[0171] In the formula, Emb event is the semantic embedding vector of a known historical fault event; <·,·> represents the vector dot product, which characterizes the potential causal relationship between the particle and the fault semantics;

[0172] Calculate the particle information entropy based on the multimodal score distribution of the local region where the particle is located:

[0173]

[0174] In the formula, H local (p i ) is particle p i Information entropy of the surrounding local score distribution; j is the score type index, j∈{1,2,3,4}, corresponding to local concept score, global concept score, local visual score, and global visual score, respectively; S j (p i ) is particle p i The score value of the j-th item; H global The mean entropy of the score distribution for all particles; i This indicates that the average value is taken over all particle numbers i.

[0175] Define the local-global entropy difference ΔH(p) i )=H local (p i )-H global The weights of each weighting term are adjusted as follows:

[0176] β′ j (p i )=β j ·(1+λ·△H(p i ));

[0177] In the formula, △H(p) i ) is particle p ithe local-global entropy difference of the anomaly score distribution; λ is the entropy difference adjustment coefficient, reflecting the amplification effect of score difference on weight; β j represents the base weight of the jth score; β′ j i is the jth weighted coefficient after dynamic adjustment for particle p i .

[0178] Calculate the final weighted score S of the particle:

[0179] S(p i )=β′1(p i )·LC′(p i )+β′2(p i )·GC′(p i )+β′3(p i )·LV(p i )+β′4(p i )·GV(p i );

[0180] In the formula, S(p i ) is the final weighted score of particle p i ; LV(p i ) is the local visual score of particle p i .

[0181] Determine whether it constitutes an abnormal hotspot area, and the abnormal prediction condition is as follows:

[0182]

[0183] In the formula, τ1 is an anomaly score threshold value for screening significant score areas; is the temperature change gradient at the position of particle p i , indicating the trend of temperature change; θ1 is a temperature change gradient threshold value, reflecting the risk boundary of rapid change area; M heat (p i ) is a historical failure mask; θ2 is a temperature mask heat intensity threshold value for screening high heat areas, and areas with insufficient heat change are excluded to improve recognition accuracy; ρ is a behavior correlation determination threshold value for screening areas most relevant to known fault types to prevent false positives.

[0184] Introduce a context semantic consistency factor to enhance the adaptability of the score to the context of operation and maintenance events and prevent local abnormal amplification misjudgment; introduce a behavior potential vector a(p i ​), and by taking the dot product with the score results as the adjustment coefficient, the score heat map can improve the prediction ability of future possible failure trends; the score distribution difference between the particle peripheral region and the overall space is dynamically adjusted to fuse the weight, the score information entropy difference is introduced and mapped as a coefficient adjustment function, the weight expression of the abnormal concentration area is adaptively strengthened, and the balance between model sensitivity and robustness is improved.

[0185] (4) Structured noise prediction module

[0186] The structured noise prediction module is configured to receive the current particle state set and the abnormal hotspot area, construct a future particle state sketch for prediction, and generate a predicted particle state based on a structured noise reverse initialization and diffusion reasoning method, to simulate the system response that different control instructions may cause at a future time.

[0187] As shown in Figure 4 , in one embodiment, the structured noise prediction module includes:

[0188] A particle state sketch generator configured to construct a future particle state sketch within a prediction period based on the current particle state set, the abnormal hotspot area, and the semantic heat map, including particle position, attribute trend, semantic label distribution, etc., to support the generation of particle state evolution profiles at multiple time granularities (e.g., 5 minutes, 30 minutes, and 2 hours) respectively;

[0189] A structured noise initializer configured to perform structured noise reverse initialization on the future particle state sketch encoding to generate an initial state The calculation formula is:

[0190]

[0191] In the formula, z0 is the latent variable representation of the future particle state sketch; α t is the noise scheduling coefficient of diffusion time step t; ∈ is a standard Gaussian structured noise;

[0192] A reverse diffusion reasoner configured to take the initial state as the starting point to perform a reverse diffusion process to generate a predicted particle state sequence P (t+△t) , and simulate the evolution trajectory of the system response under different control instructions respectively;

[0193] An abnormal enhancement mechanism configured to add an abnormal weight coefficient to the particles located in the abnormal hotspot area during the particle prediction process, so that the prediction process focuses on the high-temperature and high-entropy area, thereby improving the sensitivity and accuracy of the abnormal trend response;

[0194] A closed-loop feedback update unit configured to compare the actual feedback state of the device with the predicted particle state to construct a prediction error function:

[0195]

[0196] wherein E(t) is the prediction error function value at time t, representing the total deviation between the predicted state of all particles at time point t and the real state by the structured noise prediction module; N is the total number of particles, i.e. the number of particles participating in the prediction evaluation in the current frame or the current prediction period;

[0197] is the predicted state vector of the i-th particle, representing the particle attribute state predicted by the system according to the structured noise model and the diffusion back calculation algorithm, generally in the form of a vector, such as: containing the predicted values of the spatial coordinates (x i ,y i ,z i ) and key physical indicators (temperature T, humidity H, power consumption P);

[0198] is the actual feedback state vector of the i-th particle, representing the real state corresponding to the position of the i-th particle collected by the physical sensor in real time, also in the form of a vector, containing the measurement data corresponding to the predicted state vector;

[0199] When the prediction error function value E(t) exceeds the set threshold value, it indicates that the prediction model is invalid, and model updating or parameter relearning needs to be triggered, at which time the structured noise generation parameters and the future particle state sketch are automatically updated to realize adaptive adjustment of the prediction model, forming a closed-loop control chain of prediction—execution—feedback—correction.

[0200] (5) Instruction generation and control module

[0201] The instruction generation and control module is used to combine the abnormal hotspot area, the predicted particle state and the knowledge graph rule to perform causal relationship analysis and control instruction optimization, and output a device control instruction set;

[0202] In a preferred embodiment, the instruction generation and control module comprises:

[0203] a causal path modeler, configured to combine the weighted scoring results of the abnormal hotspot area, the semantic heat map text and the device behavior rules defined in the knowledge graph to construct a causal path graph of event—causal relationship—controllable variable;

[0204] a control instruction candidate generator, configured to identify the controllable variables from the causal path graph, call the candidate device control instructions a i corresponding to the controllable variables from the historical operation and maintenance instruction knowledge base, and simulate the system response of each candidate device control instruction at different time granularities in combination with the predicted particle state;

[0205] a semantic response evaluation module, configured to invoke the multi-modal language model to generate a natural language summary of the device response of each candidate control instruction in the simulated predicted state, and compare the response with the predicted particle scene for semantic consistency, and calculate a response credibility score R(a i );

[0206] an instruction conflict detection module, configured to detect combinations of operation mutual exclusion, resource occupation conflict or device state limitation in the candidate device control instruction set, and eliminate instruction groups that cannot be executed in parallel;

[0207] an execution risk evaluation module, configured to combine the local temperature gradient, power consumption fluctuation amplitude and historical abnormal frequency in the predicted particle state to construct an instruction execution risk score ρ(a i ) as a penalty factor of the control instruction utility function;

[0208] a fusion optimization module, configured to comprehensively calculate the control instruction utility function U(a i ) by combining the final weighted score S, the response credibility score R(a i ) and the execution risk score ρ(a i ):

[0209] U(a i )=λ1·S+λ2·R(a i )-λ3·ρ(a i );

[0210] wherein λ1, λ2 and λ3 are weighting coefficients set by the system;

[0211] an instruction output selector, configured to sort the candidate device control instruction set according to the priority of the control instruction utility function U(a i ), and output the optimal device control instruction set executable by the current system to the device control end.

[0212] (6) a three-dimensional visualization interaction module

[0213] The three-dimensional visualization interaction module is configured to receive the current particle state set, the semantic heat map and the predicted particle state, and realize three-dimensional visualization interaction display on the client side through GPU acceleration and WebGL rendering technology, at least including real-time particle distribution rendering, abnormal hotspot area highlight display, trend evolution trajectory animation and device control instruction interactive operation.

[0214] As shown in Figure 5 , another embodiment of the present application provides a multi-modal intelligent linkage 3D visualization data center operation and maintenance method, which includes the following contents:

[0215] The physical sensor data, video monitoring data and operation and maintenance log data of the data center are collected, different modal data is uniformly mapped to the same three-dimensional space coordinate system through time synchronization and space alignment processing, and a multi-modal fusion feature tensor is constructed;

[0216] A particle space structure of the data center is constructed based on a Voronoi diagram and a Delaunay triangulation method, a mapping relationship between the particles and data items in the multi-modal fusion feature tensor is established, and the visualization resolution level of the particles is dynamically adjusted according to the distance between the particles and the current perspective, and a current particle state set is output;

[0217] Local concept scores, global concept scores, local visual scores and global visual scores of each particle in the current particle state set are calculated, a semantic heat map is generated according to the weighted score results, and an abnormal hotspot area meeting an abnormal prediction condition is identified;

[0218] A future particle state sketch for prediction is constructed, and a prediction particle state is generated based on a structured noise reverse initialization and diffusion reasoning method, for simulating the system response that may be triggered by different control instructions at a future time;

[0219] The abnormal hotspot area, the prediction particle state and the knowledge graph rule are combined for causal relationship analysis and control instruction optimization, and a device control instruction set is output;

[0220] GPU acceleration and WebGL rendering technology are combined to realize three-dimensional visualization interaction display on the client side.

[0221] In summary, the present application has integrated capabilities of multi-modal intelligent fusion, three-dimensional structure modeling, prediction simulation and reasoning, and linkage control decision, which significantly improves the intelligent, real-time and interactive nature of the data center operation and maintenance system, and has broad application prospects and promotion value.

[0222] In the above embodiments, all or part of them can be realized by software, hardware, firmware or any other combination. When realized by software, all or part of them can be realized in the form of a computer program product, which includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another.

[0223] In addition, each of the function units in each embodiment of the present application can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0224] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various changes or replacements within the technical scope disclosed in the present application, and these should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-modal intelligent linkage 3D visualization data center operation and maintenance system, characterized in that, The system comprises: a multi-modal data fusion module for collecting physical sensor data, video monitoring data and operation and maintenance log data of the data center, mapping different modal data to the same three-dimensional space coordinate system through time synchronization and space alignment processing, and constructing a multi-modal fusion feature tensor; a three-dimensional particle modeling module for constructing a particle space structure of the data center based on a Voronoi diagram and a Delaunay triangulation method, establishing a mapping relationship between particles and data items in the multi-modal fusion feature tensor, and dynamically adjusting the visualization resolution level of the particles according to the distance between the particles and the current perspective, and outputting a current particle state set; a multi-modal scoring analysis module for calculating the local concept score, global concept score, local visual score and global visual score of each particle in the current particle state set, generating a semantic heat map according to the weighted scoring result, and identifying an abnormal hotspot area that meets an abnormal prediction condition; a structured noise prediction module for constructing a future particle state sketch for prediction, and generating a predicted particle state based on a structured noise reverse initialization and diffusion reasoning method, for simulating the system response that may be triggered by different control instructions at a future time; an instruction generation and control module for combining the abnormal hotspot area, the predicted particle state and the knowledge graph rules to perform causal relationship analysis and control instruction optimization, and outputting a device control instruction set; a three-dimensional visualization interaction module for realizing three-dimensional visualization interaction display on the client side through GPU acceleration and WebGL rendering technology, including at least real-time particle distribution rendering, abnormal hotspot area highlight display, trend evolution trajectory animation and device control instruction interactive operation. 2.The multi-modal intelligent linked 3D visualization data center operation and maintenance system of claim 1, wherein, The multi-modal data fusion module comprises: a sensor data unit for acquiring physical sensor data of various devices in the data center, including at least temperature, humidity and power consumption, synchronizing the physical sensor data according to the time stamp, and mapping the physical sensor data to a three-dimensional space coordinate system based on the spatial installation position of each sensor to form physical features; a video monitoring data unit for acquiring video monitoring data of the data center and performing image frame level decoding, extracting visual features of a target area in the image, and mapping the visual features to a three-dimensional space in combination with the internal and external parameter information of the corresponding camera; an operation and maintenance log data unit for performing semantic analysis on the operation and maintenance log text and structuring it into event triples, and mapping it to a three-dimensional space coordinate system according to a preset spatial semantic mapping rule; A data fusion unit is configured to fuse the physical features, visual features and semantic features in a unified three-dimensional spatial coordinate system and time dimension to generate a multi-modal fusion feature tensor wherein x, y, z represent three-dimensional spatial coordinates, and t represents time. 3.The multi-modal intelligent linked 3D visualization data center operation and maintenance system of claim 1, wherein, In the three-dimensional particle modeling module, the construction method of the particle space structure comprises: taking the physical sensor nodes deployed in the space of the data center as space seed points, using a weighted Voronoi diagram to divide the three-dimensional space region, wherein the weight value of each physical sensor node is related to the sensor perception coverage range, the historical abnormal record frequency or the importance of the region; extracting a centroid set in the weighted Voronoi region, and performing Delaunay triangulation on the centroid set to construct a three-dimensional particle network structure; After the construction, a one-to-one mapping relationship is established according to the tensor values in the multi-modal fusion feature tensor in the region where each particle is located; the three-dimensional particle modeling module monitors the local variation gradient of the multi-modal fusion feature tensor during system operation, and dynamically adjusts the particle distribution structure based on the preset reconstruction threshold, refines the particles in the high-variability region, and performs particle aggregation reconstruction in the stable region.

4. The multi-modal intelligent linked 3D visualization data center operation and maintenance system according to claim 3, characterized in that, The three-dimensional particle modeling module determines the visualization resolution level of each particle based on the spatial distance between the particle and the current view center point and the temperature variation gradient of the region where the particle is located, and the resolution level scheduling mechanism specifically includes: For each particle p i , the Euclidean distance d i of the particle to the perspective center point is calculated The particle rendering weight function is constructed: In the formula, W i is a particle rendering weight; σ is a Sigmoid function; and β and γ are preset adjustment coefficients of the system. According to the particle rendering weight W i a value in the range [0, 1] corresponding to a mapping to different levels of visualization resolution and determining whether to preserve, aggregate or refine the rendering granularity of the particle in the current frame; During system operation, the visualization resolution level of each particle can be dynamically updated, supporting visual continuous transition and smooth presentation, ensuring that the particle rendering in the abnormal hotspot region is delicate, while the region far from the view angle and slowly changing is dynamically simplified.

5. The multi-modal intelligent linked 3D visualization data center operation and maintenance system according to claim 4, characterized in that, The current particle state set at least includes: Three-dimensional coordinate positions (x i ,y i ,z i ) of the particles for spatial localization; temperature values T monitored in real time i , humidity values H i and power consumption values P i for characterizing the physical environmental state of the space region corresponding to the particles; a fusion feature vector v i , composed of the visual features and semantic features extracted by the region where the particles are located in the multi-modal fusion feature tensor; a visualisation level tag L i for indicating a visualisation resolution level of the particle at the current viewing angle; State entropy value For measuring the normalized uncertainty of temperature, humidity and power consumption changes of particles in the last continuous multiple time instants, the calculation formula is: where p k is the distribution probability of any physical property k of the particles within a time window. Life cycle counter C i for recording the number of state updates since the particle was spawned, triggering a particle resampling process when the system-set upper limit on the life cycle of an example is exceeded; Abnormal cluster label A i , for identifying whether a particle is clustered to belong to any abnormal hotspot region or potential failure domain in abnormal pattern recognition, for supporting priority calculation of device regulation instructions and abnormal tracking. 6.The multi-modal intelligent linked 3D visualization data center operation and maintenance system of claim 1, wherein, In the multi-modal score analysis module, the local concept score LC is based on the weighted average method of fusing local semantic heat and mask area proportion, for measuring the semantic concept saliency of the local region where the particle is located; The global concept score GC is calculated by the image-text embedding similarity of the Alpha-CLIP model, for evaluating the global semantic consistency of the particle image region and the target semantic text; The local visual score LV is obtained by calculating the feature similarity of the support image and the current image in the local region of the particle, and the local average aggregation of the spatial matching matrix is included in the calculation process; The global visual score GV is obtained by measuring the similarity of the feature distribution of the particle region in different images by EMD based on the overall embedding distribution between the support image and the current image extracted by the ViT or DINO model.

7. The multi-modal intelligent linked 3D visualization data center operation and maintenance system according to claim 6, characterized in that, In the multi-modal score analysis module, a context semantic consistency factor γ(p i ) is introduced, which is scored by a multi-modal large model on the consistency of particle semantics and the global description of the current operation and maintenance scene, with a value range of [0, 1], used to adjust the local concept score and the global concept score to form a revised score item: LC'(p i ) = LC(p i ) · γ(p i ), GC'(p i ) = GC(p i ) · γ(p i ); where p i represents the i-th particle, LC(p i ) is the local concept score of particle p i , GC(p i ) is the global concept score of particle p i ; LC'(p i ) is the modified local concept score of particle p i , GC'(p i ) is the modified global concept score of particle p i ; a(p i ) is introduced as a semantic-behavior association vector, which is obtained by text and video modalities collaborative reasoning, to predict the potential causal relationship between the area where the particle is located and the historical operation and maintenance events, and the behavior enhancement weight term R(p i ) is calculated: R(p i ) = <a(p i ), Emb event > ; where Emb event is the semantic embedding vector for the historically known failure event; <·, ·> denotes the vector dot product; According to the multi-modal score distribution of the local region where the particle is located, the particle information entropy is calculated: H global = mean i (H local (p i )); where H local (p i ) is the score of particle p i ; j is the score type index, j ∈ {1, 2, 3, 4}; S j (p i ) is the jth score value of particle p i ; H global is the average entropy of the score distribution of all particles; mean i denotes the average over all particle indices i. Define the local-global entropy difference ΔH(p) i )=H local (p i )-H global The weights of each weighting term are adjusted as follows: β' j (p i ) = β j · (1 + λ · ΔΗ(ρ i )) ; where ΔH(p i ) is the local-global entropy difference of particle p i ; λ is the entropy difference adjustment coefficient; β j represents the base weight of the jth score; β′ j (p i ) is the jth weight coefficient of the particle p i after dynamic adjustment; The final weighted score S of the particle is calculated: S(p i ) = β1(p i ) · LC(p i ) + β2(p i ) · GC(p i ) + β3(p i ) · LV(p i ) + β4(p i ) · GV(p i ); where S(p i ) is the final weighted score of particle p i . LV(p i ) is the local visual score of particle p i ; GV(p i ) is the global visual score of particle p i ; Determine whether it constitutes an abnormal hotspot region, and the abnormal prediction condition is as follows: In the formula, τ1 is the anomaly scoring threshold; For particle p i Temperature gradient at location; θ1 is the temperature gradient threshold; M heat (p i ) is the historical fault mask; θ2 is the temperature mask heat intensity threshold; ρ is the behavior correlation determination threshold. 8.The multi-modal intelligent linked 3D visualization data center operation and maintenance system of claim 1, wherein, The structured noise prediction module includes: A particle state sketch generator is configured to construct a future particle state sketch in a prediction period according to the current particle state set, the abnormal hotspot region, and the semantic heat map, including particle position, attribute trend, and semantic label distribution, and support generating particle state evolution profile at multiple time granularities; a structured noise initializer for encoding a future particle state sketch with structured noise reverse initialization to generate an initial state The computational formula is: where z0is a latent variable representation of the future particle state sketch; a t is the noise schedule coefficient for diffusion time step t; ∈ is a standard Gaussian structured noise; a back-diffusion reasoner to perform a back-diffusion process to generate a sequence of predicted particle states P starting from an initial state (t +△t) and simulate the evolution trajectories of the system response under different control instructions, respectively;​ An abnormal enhancement mechanism is configured to add an abnormal weight coefficient to the particles located in the abnormal hotspot region during the particle prediction process, so as to enhance the focus in the high-temperature and high-entropy region during the prediction process; A closed-loop feedback update unit is configured to compare the actual feedback state of the device with the predicted particle state to construct a prediction error function: In the formula, E(t) is a predicted error function value at time t; N is the total number of particles; is a predicted state vector of the i th particle; is an actual feedback state vector of the i th particle; When the prediction error function value E(t) exceeds the set threshold, the structured noise generation parameters and the future particle state sketch are automatically updated. 9.The multi-modal intelligent linked 3D visualization data center operation and maintenance system of claim 7, wherein, The instruction generation and control module includes: A causal path modeler is configured to construct a causal path graph of event-causality-controllable variable by combining the weighted score result of the abnormal hotspot region, the semantic heat map text, and the device behavior rules defined in the knowledge graph; The control instruction candidate generator is configured to identify controllable variables from the causal path graph, and call candidate device control instructions a corresponding to the controllable variables from a historical operation and maintenance instruction knowledge base i And simulate system responses of each candidate device control instruction at different time granularities in combination with the predicted particle state. a semantic response evaluation module, configured to invoke the multi-modal language model to generate a natural language summary of the device response of each candidate control instruction in the simulated predicted state, and compare the response with the predicted particle scene for semantic consistency, and calculate a response credibility score R(a i ); and an instruction conflict detection module, configured to detect combinations of operation mutual exclusion, resource occupation conflict or device state limitation in the candidate device control instruction set, and eliminate instruction groups that cannot be executed in parallel. The execution risk assessment module is configured to combine the local temperature gradient in the predicted particle state, the power consumption fluctuation amplitude, and the historical abnormal frequency to construct an instruction execution risk score ρ(a i ) as a penalty factor of the control instruction utility function. A fusion preference module is used to combine the final weighted score S, the response trust score R(a i ) and the execution risk score p(a i ) to compute a control instruction utility function U(a i ): U(a i ) = λ1 · S + λ2 · R(a i ) - λ3 · ρ(a i ); In the formula, λ1, λ2, and λ3 are weighting coefficients set by the system; an instruction output selector, configured to sort the candidate device control instruction set according to the priority of the control instruction utility function U(a i ) and output the optimal device control instruction set executable by the current system to the device regulation end.

10. The multi-modal intelligent linkage 3D visualization data center operation and maintenance method based on the multi-modal intelligent linkage 3D visualization data center operation and maintenance system of any one of claims 1-9, characterized in that, The method includes: Physical sensor data, video monitoring data and operation and maintenance log data of the data center are collected, different modal data is uniformly mapped to the same three-dimensional space coordinate system through time synchronization and space alignment processing, and a multi-modal fusion feature tensor is constructed; A particle space structure of the data center is constructed based on a Voronoi diagram and a Delaunay triangulation method, a mapping relationship between particles and data items in the multi-modal fusion feature tensor is established, and a visual resolution level of the particles is dynamically adjusted according to a distance between the particles and a current view angle, and a current particle state set is output; Local concept scores, global concept scores, local visual scores and global visual scores of each particle in the current particle state set are calculated, a semantic heat map is generated according to a weighted score result, and an abnormal hotspot region meeting an abnormal prediction condition is identified; A future particle state sketch for prediction is constructed, and a predicted particle state is generated based on a structured noise reverse initialization and diffusion reasoning method, for simulating a system response possibly caused by different control instructions at a future time; Causal relationship analysis and control instruction optimization are performed in combination with the abnormal hotspot region, the predicted particle state and a knowledge graph rule, and a device control instruction set is output; Three-dimensional visual interactive display is realized on a client side through GPU acceleration and WebGL rendering technology.

Images