Data center management and control system and method

Abstract

This invention provides a data center management and control system and method. Data center management and control comprises a data layer, an analysis layer, a decision-making layer, and an execution layer. The data layer includes a data acquisition module and a multimodal feature fusion module; the analysis layer includes a multi-level collaborative tracking module, a dynamic prediction and warning module, and a large-scale visual model analysis module; the decision-making layer includes a voice interaction processing module, a knowledge base matching module, and an operation and maintenance decision-making module; and the execution layer includes a control instruction execution module and an operation and maintenance result feedback module. This system establishes a complete closed loop of "data acquisition-multimodal feature fusion-analysis-decision-execution-feedback optimization," which doubles up the timeliness and accuracy of safety warnings, significantly optimizes the efficiency and intelligence level of fault handling, significantly improves energy efficiency, and collaboratively enhances the reliability and operation and maintenance efficiency of edge scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of data centers and edge computer rooms, and in particular to a data center management and control system and method. Background Art

[0002] Currently, with the continuous development of business, data centers are facing more complex environments and increasingly demanding reliability, which urgently requires solutions for intelligent monitoring and operation and maintenance efficiency. Traditional data centers have the following technical problems:

[0003] 1. Efficiency bottleneck: Operations and maintenance personnel rely on dedicated terminals to manually query dynamic environment data, which requires complex steps. In emergency situations, such as "log in to the terminal → query the alarm list → locate the device → perform operations," the process can lead to response delays and make it difficult to complete the operation within the critical time.

[0004] 2. Broken decision-making process: Dynamic environment data is separated from operational experience, and fault diagnosis relies solely on manual analysis. For example, when a high temperature alarm occurs, manual analysis is required to correlate multiple data sources, such as air conditioner operating status and equipment load, causing the processing process to significantly lag behind the development of the fault.

[0005] 3. Failure in edge scenarios: Remote data centers are prone to communication interruptions due to unstable networks, and cloud-based control solutions lack localized intelligent processing capabilities. Once the network is disconnected, emergency commands cannot be executed. For example, the inability to remotely shut down abnormal equipment can cause cascading failures. Summary of the Invention

[0006] In view of this, the purpose of the present invention is to provide a data center management and control system and method to build a complete closed loop of "data collection-multimodal feature fusion-analysis and decision-execution-feedback optimization".

[0007] In the first aspect, an embodiment of the present invention provides a data center management and control system, which includes: a data layer, an analysis layer, a decision layer and an execution layer; the data layer includes: a data acquisition module and a multimodal feature fusion module, the analysis layer includes: a multi-level collaborative tracking module, a dynamic prediction and warning module and a visual large model analysis module, the decision layer includes: a voice interaction processing module, a knowledge base matching module and an operation and maintenance decision generation module, and the execution layer includes: a control instruction execution module and an operation and maintenance result feedback module; the data acquisition module is used to collect multimodal data; wherein, the multimodal data includes: device body data, environment-related data and dynamic relationship data; the multimodal feature fusion module is used to use the modal attention mechanism based on the multimodal fusion engine to perform cross-modal alignment and dynamic weighting of voice instructions, dynamic environment data and device images; the multi-level collaborative tracking module is used to perform primary tracking and secondary tracking of the target , and fuse the detection results of the first-level tracking and the second-level tracking; the dynamic prediction and warning module is used to warn the target through two-dimensional detection, time window deduplication and hierarchical warning; the visual large model analysis module is used to combine the visual large model to perform in-depth information analysis on the images obtained by the camera in real time, and identify the equipment status and abnormal conditions; the voice interaction processing module is used to adopt a voice and control collaborative architecture to achieve effective collaboration between voice and control; the knowledge base matching module is used to build a system based on the knowledge graph and update the dynamic environment fault handling knowledge graph in real time; the operation and maintenance decision generation module is used to integrate the visual large model analysis results, voice interaction information and knowledge base matching suggestions to generate operation and maintenance decisions; the control instruction execution module is used to generate corresponding control instructions according to the operation and maintenance decisions, and execute the control instructions through the edge collaborative execution module; the operation and maintenance result feedback module is used to optimize the operation and maintenance closed loop and update the knowledge graph.

[0008] In an optional embodiment of the present application, the above-mentioned data acquisition module is used to collect status data and network parameters through a specified protocol, and synchronously record the equipment model configuration and location information; the data acquisition module is used to use dynamic environment sensors to collect environmental parameters, the microphone array to capture voice commands and perform voice activity detection preprocessing, and the camera to obtain image information; the data acquisition module is used to track topology changes in real time through RFID positioning and linkage with the asset system, and record the physical connection relationship and logical dependency relationship between devices.

[0009] In an optional embodiment of the present application, the above-mentioned multi-level collaborative tracking module is used to detect and associate the initial target, and output the detection results of the first-level tracking with the tracking ID; the multi-level collaborative tracking module is used to use the centroid distance and intersection-over-union ratio joint matching strategy to perform secondary verification of the first-level tracking detection results, maintain the target trajectory history and calculate the velocity vector, and output the detection results of the second-level tracking.

[0010] In an optional embodiment of the present application, the above-mentioned dynamic prediction and warning module is used to simultaneously determine whether the current position of the target and the future position based on the velocity vector enter the safe area, and to issue an early warning; the dynamic prediction and warning module is used to record the last alarm timestamp of the target; when the alarm situation of the same target is detected again, it is determined whether to issue a warning based on the recorded last alarm timestamp of the target; the dynamic prediction and warning module is used to issue alarms in different ways for the current position and future position of the target.

[0011] In an optional embodiment of the present application, the above-mentioned operation and maintenance result feedback module is used to mark samples and update training data based on the diagnosis results confirmed by the operation and maintenance personnel; the operation and maintenance result feedback module is used to automatically update the fault causal relationship according to the new fault situation and integrate the new fault handling plan into the knowledge graph training.

[0012] In an optional embodiment of the present application, the above-mentioned multimodal feature fusion module is used to enable speech semantics and dynamic context data to interact through a cross-attention layer based on a multi-head attention mechanism.

[0013] In an optional embodiment of the present application, the multimodal feature fusion module is used to dynamically adjust the weights of voice commands and dynamic environment data based on the command type and data urgency.

[0014] In an optional embodiment of the present application, the above-mentioned operation and maintenance decision generation module is used to normalize the device image feature vector from the visual large model analysis module, the semantic feature vector from the voice interaction processing module, and the fault handling solution probability vector output by the knowledge base matching module; the operation and maintenance decision generation module is used to fuse the normalized information as different evidence bodies to obtain the fused probability distribution result; the operation and maintenance decision generation module is used to input the fused probability distribution result into a pre-trained decision conversion model to output the operation and maintenance decision.

[0015] In an optional embodiment of the present application, the above-mentioned data acquisition module is connected to the multimodal feature fusion module; the multimodal feature fusion module is also connected to the multi-level collaborative tracking module, the dynamic prediction and warning module and the visual large model analysis module respectively; the multi-level collaborative tracking module is connected to the voice interaction processing module, and the dynamic prediction and warning module and the visual large model analysis module are both connected to the knowledge base matching module; the voice interaction processing module and the knowledge base matching module are both connected to the operation and maintenance decision generation module; the operation and maintenance decision generation module, the control instruction execution module and the operation and maintenance result feedback module are connected in sequence, and the operation and maintenance result feedback module is also connected to the data acquisition module.

[0016] In the second aspect, an embodiment of the present invention further provides a data center management and control method, which is applied to the above-mentioned data center management and control system. The data center management and control method includes: a data acquisition module collects multimodal data; wherein the multimodal data includes: device body data, environment-related data and dynamic relationship data; a multimodal feature fusion module is based on a multimodal fusion engine, and uses a modal attention mechanism to perform cross-modal alignment and dynamic weighting of voice commands, dynamic environment data and device images; a multi-level collaborative tracking module performs primary tracking and secondary tracking on the target, and fuses the detection results of the primary tracking and secondary tracking; a dynamic prediction and warning module performs dual-dimensional detection, time window deduplication and hierarchical warning on the target. The visual big model analysis module combines the visual big model to conduct in-depth information analysis on the images obtained by the camera in real time, and identify the equipment status and abnormal conditions; the voice interaction processing module adopts a voice and control collaborative architecture to achieve effective collaboration between voice and control; the knowledge base matching module builds a system based on the knowledge graph and updates the knowledge graph of dynamic environment fault handling in real time; the operation and maintenance decision generation module integrates the visual big model analysis results, the information obtained by voice interaction and the knowledge base matching suggestions to generate operation and maintenance decisions; the control instruction execution module generates corresponding control instructions according to the operation and maintenance decisions, and executes the control instructions through the edge collaborative execution module; the operation and maintenance result feedback module optimizes the operation and maintenance closed loop and updates the knowledge graph.

[0017] The embodiments of the present invention bring the following beneficial effects:

[0018] The embodiments of the present invention provide a data center management and control system and method, which constructs a complete closed loop of "data collection-multimodal feature fusion-analysis and decision-making-execution-feedback optimization", doubly improves the timeliness and accuracy of safety warnings, significantly optimizes fault handling efficiency and intelligence level, significantly improves energy consumption efficiency optimization, and synergistically improves edge scenario reliability and operation and maintenance efficiency.

[0019] Other features and advantages of the present disclosure will be set forth in the following description, or some features and advantages may be inferred or unambiguously determined from the description, or may be learned by practicing the above-mentioned technology of the present disclosure.

[0020] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] Figure 1 A schematic diagram of the structure of a data center management and control system provided by an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of a data acquisition module provided in an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of a multimodal feature fusion module provided by an embodiment of the present invention;

[0025] Figure 4 A schematic diagram of a multi-level collaborative tracking module provided by an embodiment of the present invention;

[0026] Figure 5 A schematic diagram of a dynamic prediction and warning module provided by an embodiment of the present invention;

[0027] Figure 6 A schematic diagram of voice collection and processing provided by an embodiment of the present invention;

[0028] Figure 7 A schematic diagram of image data acquisition and processing provided by an embodiment of the present invention;

[0029] Figure 8 A schematic diagram of cross-modal interaction modeling provided by an embodiment of the present invention;

[0030] Figure 9 A schematic diagram of an intelligent decision output and modal weight dynamic adjustment mechanism provided by an embodiment of the present invention;

[0031] Figure 10 A schematic diagram of an operation and maintenance decision generation method provided by an embodiment of the present invention;

[0032] Figure 11 A schematic diagram of a knowledge graph construction and multi-hop reasoning mechanism provided by an embodiment of the present invention;

[0033] Figure 12 A flowchart of a data center management and control method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0034] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0035] Currently, with the maturing of edge computing and large-scale lightweight model technologies, it's becoming possible to build intelligent systems combining voice interaction, dynamic and environmental data fusion, and localized decision-making. However, existing solutions generally lack closed-loop collaboration from data collection to intelligent decision-making, making them difficult to cope with complex and ever-changing operational and maintenance scenarios.

[0036] Based on this, the embodiments of the present invention provide a data center management and control system and method, which specifically provides a data center management and control method and system based on multimodal artificial intelligence fusion, and constructs a complete closed loop of "data collection-multimodal feature fusion-analysis and decision-making-execution-feedback optimization".

[0037] To facilitate understanding of this embodiment, a data center management and control system disclosed in an embodiment of the present invention is first introduced in detail.

[0038] Example 1:

[0039] The embodiment of the present invention provides a data center management and control system, see Figure 1 The structural diagram of a data center management and control system shown in the figure includes: a data layer, an analysis layer, a decision layer and an execution layer; the data layer includes: a data acquisition module and a multimodal feature fusion module, the analysis layer includes: a multi-level collaborative tracking module, a dynamic prediction and warning module and a visual large model analysis module, the decision layer includes: a voice interaction processing module, a knowledge base matching module and an operation and maintenance decision generation module, and the execution layer includes: a control instruction execution module and an operation and maintenance result feedback module.

[0040] like Figure 1 As shown, the data acquisition module is connected to the multimodal feature fusion module; the multimodal feature fusion module is also connected to the multi-level collaborative tracking module, the dynamic prediction and warning module and the visual large model analysis module respectively; the multi-level collaborative tracking module is connected to the voice interaction processing module, and the dynamic prediction and warning module and the visual large model analysis module are both connected to the knowledge base matching module; the voice interaction processing module and the knowledge base matching module are both connected to the operation and maintenance decision generation module; the operation and maintenance decision generation module, the control instruction execution module and the operation and maintenance result feedback module are connected in sequence, and the operation and maintenance result feedback module is also connected to the data acquisition module.

[0041] like Figure 1 This embodiment builds a complete closed loop of "data acquisition - multimodal feature fusion - analysis and decision-making - execution - feedback optimization." The data layer encompasses the data acquisition module and the multimodal feature fusion module. The analysis layer includes a multi-level collaborative tracking module, a dynamic prediction and warning module, and a large-scale visual model analysis module. The decision-making layer includes a voice interaction processing module, a knowledge base matching module, and an operation and maintenance decision generation module. The execution layer includes a control instruction execution module and an operation and maintenance result feedback module.

[0042] 1. Data layer:

[0043] 1. The data acquisition module is used to collect multimodal data; wherein, the multimodal data includes: device body data, environment-related data and dynamic relationship data.

[0044] In some embodiments, the data acquisition module is used to collect status data and network parameters through specified protocols, and synchronously record device model configuration and location information; the data acquisition module uses dynamic environment sensors to collect environmental parameters, the microphone array to capture voice commands and perform voice activity detection preprocessing, and the camera to obtain image information; the data acquisition module is used to track topology changes in real time through RFID positioning and linkage with the asset system, and record the physical connection relationship and logical dependency relationship between devices.

[0045] See also Figure 2 The diagram shows a data acquisition module that builds an intelligent sensing network covering all elements, collects multi-dimensional data in real time, and dynamically tracks device relationships. The multi-dimensional data includes:

[0046] (1) Equipment data: Collect the server CPU (Central Processing Unit) load, memory usage, hard disk IO (Input / Output) and other operating status through the BMC (Baseboard Management Controller) / IPMI (Intelligent Platform Management Interface) protocol; obtain network parameters such as switch port traffic and bandwidth utilization through the SNMP (Simple Network Management Protocol) protocol; and simultaneously record the equipment model configuration (such as CPU model, memory capacity) and location information (cabinet U-position coordinates, computer room area code).

[0047] (2) Environmental data: Dynamic environment sensors collect environmental parameters such as temperature and humidity, power load, and PUE (Power Usage Effectiveness) values, and transmit them using the OPC UA (OPC Unified Architecture) protocol; microphone arrays capture voice commands and pre-process them through voice activity detection; cameras obtain image information such as the status of device indicator lights and cabinet space occupancy.

[0048] (3) Dynamic relationship data: Through RFID (Radio Frequency Identification) positioning and linkage with the asset system, topology changes such as server installation / deinstallation and device migration can be tracked in real time; the physical connection relationship between devices (such as server-switch port mapping) and logical dependency relationships (such as active-standby clusters and storage mount paths) can be recorded.

[0049] Specifically, the microphone array consists of multiple microphones arranged in a specific geometric layout. The voice signals collected by each microphone are connected to the voice activity detection module through lines; the dynamic environment sensor is installed in the equipment or environment of the data center through physical connection, and the collected data (such as temperature, humidity, power parameters, etc.) is transmitted to the subsequent processing module through the network line based on the OPC UA protocol; the camera is installed in a suitable monitoring position in the data center, and the captured image data is transmitted to the image preprocessing module through the video line.

[0050] 2. The multimodal feature fusion module is used to align and dynamically weight voice commands, dynamic environment data, and device images across modalities based on the multimodal fusion engine and the modal attention mechanism.

[0051] See also Figure 3 The figure shows a schematic diagram of a multimodal feature fusion module. This module receives integrated data from the data aggregation layer (including device data, environmental data, and dynamic relationship data). Leveraging the multimodal fusion engine, it employs a modal attention mechanism to address cross-modal alignment and dynamic weighting of voice commands, dynamic environment data, and device images. In query scenarios, dynamic environment data accounts for 70% of the weight, while in emergency control scenarios, voice commands account for 80% of the weight, improving semantic understanding accuracy in complex scenarios. Furthermore, a fully connected layer maps the speech text word embeddings (300-dimensional numerical vectors), dynamic environment temporal features (128-dimensional numerical vectors output by LSTM (Long Short-Term Memory) networks), device image features (256-dimensional numerical vectors output by CNN (Convolutional Neural Network) networks), and relationship graph embeddings (128-dimensional numerical vectors output by TransE (Translating Embedding) networks) into a unified 512-dimensional tensor, providing standardized input for subsequent processing.

[0052] 2. Analysis layer:

[0053] 1. The multi-level collaborative tracking module is used to perform primary and secondary tracking of the target and fuse the detection results of primary and secondary tracking.

[0054] In some embodiments, the multi-level collaborative tracking module is used to detect and associate the initial target, and output the detection results of the first-level tracking with the tracking ID; the multi-level collaborative tracking module is used to use the centroid distance and intersection-over-union ratio joint matching strategy to perform secondary verification of the first-level tracking detection results, maintain the target trajectory history and calculate the velocity vector, and output the detection results of the second-level tracking.

[0055] See also Figure 4 The schematic diagram of a multi-level collaborative tracking module shown in FIG. mainly includes:

[0056] (1) Level 1 tracking: The ByteTrack algorithm (a multi-target tracking algorithm) of YOLOv11 (a real-time target detection algorithm model) is used to quickly detect and associate the initial target, output the detection results with tracking IDs, and ensure the real-time performance of the system. In the specific implementation, the video image is input into the YOLOv11 model, and the convolutional neural network layer of the model is used to extract features. The ByteTrack algorithm is then used to associate the detected targets, assign a unique tracking ID to each target, and output the detection results.

[0057] (2) Secondary Tracking: Using a combined matching strategy of centroid distance and IOU (Intersection over Union), the primary results are rechecked, the target trajectory history is maintained, and the velocity vector is calculated to improve tracking stability. In the secondary tracking module, the primary tracking output is received, the centroid distance and IOU of each target are calculated, and matching verification is performed based on the set threshold. At the same time, the target trajectory history information is recorded, and the velocity vector is obtained by calculating the change in the target position between adjacent frames.

[0058] (3) Collaborative optimization: Fusion of two-level tracking results. Initial screening ensures real-time performance (frame rate ≥ 15fps). Enhanced tracking improves stability in complex scenarios and reduces target loss rate by 30%. The data fusion module fuses the results of the first-level and second-level tracking. Initial screening is performed to remove some obviously erroneous detection results to ensure that the system frame rate is not less than 15fps. Further processing is then performed to enhance the tracking effect and improve stability in complex scenarios.

[0059] 2. The dynamic prediction and warning module is used to warn the target through dual-dimensional detection, time window deduplication and graded warning.

[0060] In some embodiments, the dynamic prediction and warning module is used to simultaneously determine whether the current position of the target and the future position based on the velocity vector enter the safe area, and to issue an early warning; the dynamic prediction and warning module is used to record the last alarm timestamp of the target; when the alarm situation of the same target is detected again, it is determined whether to issue a warning based on the recorded last alarm timestamp of the target; the dynamic prediction and warning module is used to issue alarms in different ways for the current position and future position of the target.

[0061] See also Figure 5 The schematic diagram of a dynamic prediction and warning module shown in FIG. mainly includes:

[0062] (1) Dual-dimensional detection: Simultaneously determine whether the target's current position and future position based on the velocity vector (such as the predicted position in 5 seconds) have entered the safe zone, providing early warning. This mechanism receives the target's position information and velocity vector information, calculates the target's current position and the predicted position in 5 seconds, and then compares them with the boundary of the safe zone.

[0063] (2) Time window deduplication: Record the target's last alarm timestamp and set a 120-second time window to avoid duplicate alarms and improve system availability. In the alarm recording module, the target's alarm timestamp is recorded each time an alarm occurs. When the same target's alarm is detected again, determine whether the difference between the current time and the last alarm timestamp is within the 120-second time window.

[0064] (3) Graded warning: Distinguish between "current intrusion" (red frame + sound alarm) and "predicted intrusion" (yellow frame + interface prompt), supporting differentiated response. Based on the results of dual-dimensional detection, if the target's current location enters the safe area, the "current intrusion" warning is triggered, the target is marked with a red frame on the display interface and an alarm is sounded; if the target's future location enters the safe area, the "predicted intrusion" warning is triggered, marked with a yellow frame and an interface prompt.

[0065] 3. The visual large model analysis module is used to combine the visual large model to conduct in-depth information analysis on the images obtained by the camera in real time, and identify device status and abnormal conditions.

[0066] The large-scale visual model analysis module, combined with the large-scale visual model, can deeply analyze real-time camera image information to identify device status and anomalies. For example, if a user takes a photo of the computer room using their mobile phone camera, the large-scale visual model can identify device information in the image. The large-scale visual model is deployed on the server side. Image data collected by the camera is transmitted to the server via the network. The large-scale visual model uses pre-trained model parameters to perform feature extraction, classification, and recognition on the image, and outputs information about device status and anomalies.

[0067] 3. Decision-making level:

[0068] 1. The voice interaction processing module is used to adopt a voice and control collaborative architecture to achieve effective collaboration between voice and control.

[0069] The voice interaction processing module adopts a voice-control collaborative architecture to achieve effective coordination between voice and control. It utilizes WiFi (Wireless Fidelity) / Bluetooth dual-link communication and localized large-scale model deployment, automatically switching to the Bluetooth link to maintain command transmission when the network is disconnected. A microphone captures user voice commands, and through voice recognition and semantic understanding, it enables real-time question-and-answer and status prediction of the computer room equipment. The voice acquisition module transmits the collected voice signal to the voice recognition module, which converts the voice into text and transmits it to the semantic understanding module. The semantic understanding module analyzes the text semantics and generates corresponding control commands or query requests. Simultaneously, the WiFi and Bluetooth communication modules monitor the network connection status in real time. When the WiFi connection is interrupted, they automatically switch to the Bluetooth link for command transmission.

[0070] 2. The knowledge base matching module is used to build a system based on the knowledge graph and update the knowledge graph of dynamic environment fault handling in real time.

[0071] The knowledge base matching module can build a system based on the knowledge graph and update the dynamic environment fault handling knowledge graph in real time. The TransE knowledge graph embedding algorithm is used to map fault entities and relationships into 128-dimensional vectors, supporting deep reasoning of up to 5 hops. A lightweight BERT (‌B‌idirectional‌‌E‌ncoder‌‌ R‌epresentations from‌‌ T‌ransformers, a pre-trained language model) model is used to evaluate the reasoning chain and determine its rationality. Relevant handling solutions and suggestions are searched in the knowledge graph based on the fault situation. The knowledge graph is stored in the database. When the system detects a fault, the fault information is converted into a 128-dimensional vector, searched and reasoned in the knowledge graph, and the relationships between fault entities are calculated using the TransE algorithm. The lightweight BERT model is used to evaluate the inferred handling solutions and screen out reasonable solutions.

[0072] 3. The operation and maintenance decision generation module is used to integrate the analysis results of the large visual model, information obtained by voice interaction, and knowledge base matching suggestions to generate operation and maintenance decisions.

[0073] The O&M decision generation module integrates the results of large-scale visual model analysis, information obtained through voice interaction, and knowledge base matching recommendations to generate reasonable O&M decisions. For example, when the system detects a high temperature alarm in cabinet 3, the knowledge graph directly retrieves the one-hop relationship from "high temperature → fan failure" and generates a "restart the fan" solution. If the fault persists after restarting the fan, the large-scale model's multi-hop reasoning is triggered, combining historical data to generate a two-hop reasoning chain, recommending "checking the UPS (uninterruptible power supply)." The O&M decision generation module receives information from the large-scale visual model analysis module, the voice interaction processing module, and the knowledge base matching module. It then performs a comprehensive analysis based on pre-set decision rules and algorithms to generate a specific O&M decision solution.

[0074] 4. Execution Layer:

[0075] 1. The control instruction execution module is used to generate corresponding control instructions based on operation and maintenance decisions, and execute the control instructions through the edge collaborative execution module.

[0076] The control instruction execution module generates corresponding control instructions based on operation and maintenance decisions and executes them through the edge collaborative execution module. This module uses dual-link communication to support localized emergency control instruction execution in network outage scenarios, ensuring system reliability. The control instruction generation module transmits the generated control instructions to the edge collaborative execution module. The edge collaborative execution module communicates with the device via WiFi or Bluetooth links. When the network is normal, the control instructions are sent via WiFi links. In the event of a network outage, the localized emergency control instructions are sent via Bluetooth links to control the device to perform the corresponding operation.

[0077] 2. The operation and maintenance result feedback module is used to optimize the operation and maintenance closed loop and update the knowledge graph.

[0078] In some embodiments, the operation and maintenance result feedback module is used to mark samples and update training data based on the diagnostic results confirmed by the operation and maintenance personnel; the operation and maintenance result feedback module is used to automatically update the fault causal relationship according to the new fault situation and integrate the new fault handling plan into the knowledge graph training.

[0079] The O&M Results Feedback module enables closed-loop O&M optimization. O&M personnel confirm diagnostic results (e.g., "Fan fault confirmed") via voice. The system automatically labels the sample and updates the training data. Models are fine-tuned weekly to improve the accuracy of identifying new faults. The voice confirmation information is received by the Voice Interaction Processing Module and transmitted to the Data Management Module, which labels and stores the sample. The updated training data is then provided to the Model Training Module weekly for fine-tuning of the relevant models.

[0080] The Operation and Maintenance Result Feedback Module can also update the knowledge graph: based on newly occurring faults, it automatically updates fault causal relationships and incorporates new fault resolution solutions into knowledge graph training. When the system handles a new fault, the fault handling module transmits the new fault information and resolution solution to the knowledge graph update module, which then updates the fault causal relationships in the knowledge graph and uses the new information for knowledge graph training.

[0081] Example 2:

[0082] This embodiment provides another data center management and control system, which is implemented on the basis of the above embodiment, and focuses on describing the specific methods of the data center management and control system.

[0083] 1. Multimodal data acquisition and processing:

[0084] 1. Voice collection and processing: see Figure 6 The diagram below shows a method for speech collection and processing. A microphone array is used to collect speech data, which is then converted into text using an automatic speech recognition (ASR) model. Finally, semantic features are extracted. The speech signal collected by the microphone array first passes through a voice activity detection module to remove silence, then is transmitted to the speech recognition model. The text output by the speech recognition model then passes through a semantic analysis module to extract semantic features.

[0085] For example, when the operation and maintenance personnel say the command "Query the load status of Server 1", the microphone array captures the voice signal, the voice activity detection module determines the start and end of the voice, removes the silent segments in the middle, and transmits the effective voice to the ASR model to convert it into the text "Query the load status of Server 1". The semantic analysis module recognizes the keywords "query", "server 1", and "load", and extracts the corresponding semantic features for subsequent processing.

[0086] 2. Dynamic Environment Data Collection and Processing: Dynamic environment sensors collect dynamic environment data such as temperature, humidity, and power parameters, transmit it via the OPC UA protocol, and use LSTM to extract time series features. The dynamic environment sensors encapsulate the collected data according to the OPC UA protocol and transmit it over the network to the data processing server, where the LSTM model is used to process the data and extract time series features.

[0087] For example, a temperature and humidity sensor in a data center cabinet collects minute-by-minute temperature and humidity data. The sensor packages this data into data frames using the OPC UA protocol and sends it to a data processing server via the computer room network. After the server receives the data, an LSTM model learns from the continuous temperature and humidity data sequence, extracting temporal features such as trends and fluctuations in temperature and humidity over time for subsequent analysis and decision-making.

[0088] 3. Image data acquisition and processing: see Figure 7 The diagram below shows a schematic diagram of image data acquisition and processing. This diagram captures image information through a camera and uses a CNN to extract device image features. The camera image undergoes noise reduction and cropping in an image preprocessing module before being fed into a CNN model. The CNN model extracts device image features through operations such as convolutional and pooling layers.

[0089] For example, a camera captures an image of equipment inside a cabinet. This image may contain noise due to lighting conditions and unwanted background. The image preprocessing module performs a median filter to reduce noise and crops out irrelevant background areas, retaining only the main device. The processed image is then fed into a CNN model, which uses multi-layer convolution operations to extract features such as the device's outline, interfaces, and indicator light status to determine the device's operating status.

[0090] 2. Modal Fusion and Diagnosis Prediction:

[0091] 1. Cross-modal interaction modeling:

[0092] In some embodiments, the multimodal feature fusion module is used to enable speech semantics and dynamic context data to interact through a cross-attention layer based on a multi-head attention mechanism.

[0093] See also Figure 8 The diagram below illustrates a cross-modal interaction model. Using the MM-Transformer (Multi-Scale Visual Transformer) architecture and a multi-head attention mechanism, speech semantics and context data interact through cross-attention layers. For example, when querying "current server load," the CPU utilization characteristics of the context data are coupled with the speech keyword "load" through attention weights.

[0094] For example, when an operations and maintenance personnel asks about the CPU load of server A, the speech semantic features are encoded and fed into the MM-Transformer architecture. Simultaneously, features such as CPU utilization from the corresponding dynamic context data for server A are also fed into the architecture. The multi-head attention mechanism processes both features separately. At the cross-attention layer, the "CPU load" keyword in the speech is correlated with the CPU utilization features from the dynamic context data. Attention weights are calculated, effectively fusing the two together to produce a fused feature that combines both speech semantics and dynamic context data information.

[0095] 2. Intelligent decision output and dynamic adjustment mechanism of modal weight:

[0096] In some embodiments, the multimodal feature fusion module is used to dynamically adjust the weights of voice commands and dynamic environment data based on command type and data urgency.

[0097] See also Figure 9 The diagram shows an intelligent decision-making output and dynamic adjustment mechanism for modal weights. A three-tier trigger mechanism can be used to achieve adaptive adjustment of modal weights, dynamically adjusting the weights of voice commands and dynamic environment data based on factors such as command type and data urgency. For example, query commands automatically increase the dynamic environment data weight to 60%-70%, control commands increase the voice semantic weight to 55%-65%, and emergency alerts directly lock the voice weight to 80% and disable dynamic fine-tuning.

[0098] When an operator issues a control command to "adjust the fan speed of server B," the command type determination module identifies it as a control command. The data urgency assessment module then assesses the current data urgency. Both information is then transmitted to the modal weight adjustment module. Based on pre-set rules, this module increases the weight of speech semantics to 60% and adjusts the weight of dynamic environment data to 40%. It then weights the speech semantic and dynamic environment data features and outputs them for subsequent intelligent decision-making.

[0099] 3. Operation and maintenance decision generation:

[0100] In some embodiments, the operation and maintenance decision generation module is used to normalize the device image feature vector from the visual large model analysis module, the semantic feature vector from the voice interaction processing module, and the fault handling plan probability vector output by the knowledge base matching module; the operation and maintenance decision generation module is used to fuse the normalized information as different evidence bodies to obtain the fused probability distribution result; the operation and maintenance decision generation module is used to input the fused probability distribution result into a pre-trained decision conversion model to output the operation and maintenance decision.

[0101] The Operation and Maintenance Decision Generation module integrates the results of large-scale visual model analysis, information obtained through voice interaction, and knowledge base matching recommendations to generate reasonable operation and maintenance decisions. This module is based on a lightweight decision-making Transformer model and incorporates Dempster-Shafer Evidence Theory (a mathematical reasoning framework for handling uncertainty and incomplete information) to achieve multi-source information fusion decision-making.

[0102] See also Figure 10 The diagram shown in the figure shows a schematic diagram of operation and maintenance decision generation, which mainly includes:

[0103] (1) Information preprocessing: The device image feature vector from the visual large model analysis module, the semantic feature vector from the voice interaction processing module, and the fault handling solution probability vector output by the knowledge base matching module are first normalized to map the data to the [0, 1] interval to eliminate the dimensionality effect.

[0104] (2) Multi-source information fusion: Using the DS evidence theory, the three normalized information mentioned above are treated as different evidence bodies. By calculating the basic probability distribution function, the degree of support for different operation and maintenance decision options (such as "restart equipment", "check circuit", "replace parts", etc.) is determined. Then, the Dempster (multi-source evidence fusion algorithm) synthesis rule is used to fuse these evidences to obtain the fused probability distribution result.

[0105] (3) Decision generation: The fused probability distribution results are input into the decision Transformer model. This model is based on pre-trained operation and maintenance decision knowledge and combines the real-time operating status of the current data center (such as equipment load, network traffic, etc.) as context information. The input information is weighted through the self-attention mechanism and the operation and maintenance decision plan with the highest probability is finally output.

[0106] For example, when the system detects an "abnormal power alarm for cabinet 5" and the operator asks via voice, "What's wrong with cabinet 5?", the visual large model analysis module analyzes the image captured by the camera in cabinet 5 and obtains a feature vector indicating the abnormal flashing of the power indicator light. The voice interaction processing module identifies the key information "cabinet 5" in the voice command and generates a semantic feature vector. The knowledge base matching module searches the knowledge graph to find the one-hop relationship "abnormal power → power module failure" and outputs a probability vector for the "check power module connection" solution. These three pieces of information are normalized and fused using DS evidence theory. They are then input into the decision-making Transformer model and combined with the real-time operating status of the current cabinet, such as the load and power status of other equipment, to ultimately generate an operation and maintenance decision solution: "Check the power module connection in cabinet 5. If the connection is normal, further test the power module voltage." This solution is then implemented by the subsequent control instruction execution module.

[0107] 4. Adaptive Feedback and Learning

[0108] (1) Operation and maintenance closed-loop optimization:

[0109] After the operator confirms the diagnosis results by voice, the system automatically updates the training data and regularly fine-tunes the model to improve the accuracy of identifying new faults. Voice confirmation triggers the data update process. The system records and organizes the relevant data and then fine-tunes the model at a predetermined interval (once a week) to optimize model parameters and improve the model's ability to identify new faults.

[0110] When the system diagnoses a server hard drive failure and provides recommended solutions, the operations and maintenance personnel confirm via voice that "it is indeed a hard drive failure." The voice interaction processing module receives this confirmation and passes it to the data management module, which records and organizes relevant data about the failure (such as the fault symptoms, diagnostic process, and solution results). At a fixed time each week, the model training module receives updated training data from the data management module and fine-tunes the fault diagnosis model, ensuring that the model can more accurately identify and diagnose similar failures in the future.

[0111] (2) Knowledge graph construction and multi-hop reasoning mechanism:

[0112] See also Figure 11 The diagram below shows a knowledge graph construction and multi-hop reasoning mechanism. This hybrid mechanism uses a "large model supervision + graph multi-hop reasoning" approach to implement root cause analysis and multi-hop reasoning. For example, when the system detects a high temperature alarm in cabinet 3, the knowledge graph directly retrieves the one-hop relationship from "high temperature → fan failure" and generates a "restart the fan" solution. If the fault persists after restarting the fan, the large model multi-hop reasoning is triggered, combining historical data to generate a two-hop reasoning chain, recommending "checking the UPS power supply module."

[0113] When the system detects an overtemperature alarm in cabinet 3, it first searches the knowledge graph for relationships related to "high temperature." If it finds a single-hop relationship from "high temperature to fan failure," it directly generates a solution: "restart the fan." If the high temperature issue persists after executing this solution, the system triggers multi-hop reasoning with the large model. The large model combines historical data on similar high-temperature failures with other dynamic data about the current cabinet (such as power supply and equipment load) to generate more in-depth recommendations, such as "check the UPS power supply module."

[0114] In summary, the embodiment of the present invention mainly provides a data center management and control system, which mainly includes:

[0115] 1. Multimodal Dynamic Fusion Framework: This framework employs the MM-Transformer architecture combined with a multi-head attention mechanism for unified cross-modal modeling. This framework encompasses a unique method for interacting between speech semantics and dynamic context data through a cross-attention layer. Furthermore, it dynamically adjusts the precise strategies for weighting voice commands and dynamic context data based on different scenarios (such as query and emergency control). For example, the framework sets a 70% weighting for dynamic context data in query scenarios and an 80% weighting for voice commands in emergency control scenarios. Furthermore, it implements a complete integrated process for "voice query + real-time analysis + image recognition," encompassing key steps from multimodal data acquisition, feature extraction, fusion, and output.

[0116] The multimodal dynamic fusion framework pioneered the unified cross-modal modeling of voice, dynamic environment data and image data, and used intelligent algorithms to dynamically focus on key information, effectively improving the accuracy and efficiency of data fusion.

[0117] 2. Multi-level collaborative tracking architecture: Utilizing YOLOv11's ByteTrack algorithm for first-level tracking, we implement the specific parameter settings and processes for rapid initial target detection and association. A combined centroid distance and IOU (Intersection over Union) matching strategy is used for precise matching calculation and parameter settings for second-level tracking. This includes innovative algorithms for fusing the results of these two levels, including a collaborative mechanism for ensuring real-time performance through initial screening and enhancing system stability through enhanced tracking.

[0118] The multi-level collaborative tracking architecture cleverly combines YOLO's built-in tracking and custom enhanced tracking technology to form an efficient and collaborative tracking system, significantly enhancing tracking robustness in complex scenarios and drastically reducing target loss rate.

[0119] 3. Two-dimensional dynamic prediction and warning mechanism: A specific algorithm for calculating the target's future position based on the velocity vector, along with strict rules for determining whether the target's current and future positions enter the safe zone. The system also includes the basis for setting a 120-second time window deduplication strategy, as well as a detailed process for recording the target's last alarm timestamp and determining duplicate alarms. Clear trigger conditions and precise display control methods distinguish between "current intrusion" (red box + audible alarm) and "predicted intrusion" (yellow box + interface prompt).

[0120] This dual-dimensional dynamic prediction and warning mechanism transcends the limitations of traditional warning models. It integrates real-time location detection with future location predictions. Through deep fusion analysis of multimodal data, it accurately warns of potential risks 5 seconds in advance. Combined with time window deduplication and a tiered warning strategy, this significantly improves the timeliness and accuracy of warnings.

[0121] 4. Knowledge-Based Multimodal Decision-Making: This section describes the detailed implementation of mapping fault entities and relationships into 128-dimensional vectors using the TransE knowledge graph embedding algorithm, including the basis for setting vector dimensions and the mapping algorithm details. It also describes the specific evaluation metrics and process for evaluating the reasoning chain using a lightweight BERT model. Furthermore, it describes the complete logic and steps from fault occurrence to knowledge graph retrieval, large-scale model multi-hop reasoning, and resolution plan generation. For example, this section describes the resolution plan generation process for different scenarios for a high-temperature alarm in cabinet 3.

[0122] Multimodal decision-making based on a knowledge base meticulously constructs a knowledge graph for dynamic environmental fault handling, and deeply integrates it with a large-scale model supervision mechanism to achieve automated derivation of fault diagnosis and handling solutions. This effectively reduces reliance on expert experience, improves fault handling efficiency for novice operators by 40%, and significantly reduces human error.

[0123] The data center management and control system provided by the embodiment of the present invention has the following advantages:

[0124] 1. Double improvement in the timeliness and accuracy of safety warnings:

[0125] This multi-level collaborative tracking architecture innovatively integrates YOLOv11's ByteTrack algorithm with a combined centroid distance-interference-of-union (IOU) matching strategy. This two-level tracking solution effectively improves stability in complex scenarios and reduces target loss by 30%. Through a dual-dimensional dynamic prediction and early warning mechanism, it breaks through the traditional early warning model that relies solely on current location. By integrating real-time detection with future location prediction, it can provide early warning of potential risks up to 5 seconds in advance. Through multimodal data fusion analysis, combined with time window deduplication and a hierarchical early warning strategy, the timeliness and accuracy of early warnings are significantly improved.

[0126] 2. Significantly improved fault handling efficiency and intelligence level:

[0127] Relying on a multimodal dynamic fusion framework, the system uses the MM-Transformer architecture and a modal attention mechanism to achieve cross-modal unified modeling of speech semantics, dynamic environment data, and device images. Upon receiving a "Cabinet 3 High Temperature Alarm" command, the dynamic environment data weight is automatically increased to 70%. Combined with the image features of abnormal fan speeds analyzed by the large visual model, a multi-level action plan ("Restart the fan → Check the UPS power supply") is generated within 10 seconds, achieving a 50% improvement in efficiency compared to traditional manual analysis processes.

[0128] A multimodal decision-making mechanism based on a knowledge base, using TransE knowledge graph embedding and lightweight BERT inference, builds a knowledge network for dynamic environmental fault handling. When a novice operator handles a "power module abnormality" fault, the system automatically retrieves the two-hop relationship chain from "voltage fluctuation to capacitor aging" and generates capacitor replacement recommendations, improving operator efficiency by 40% and reducing human error rates by 35%.

[0129] 3. Energy efficiency optimization is significantly improved:

[0130] By fusion-analyzing energy consumption patterns through multimodal data, combined with dynamic load adjustment and energy-saving strategies, the system optimizes data center energy efficiency. The system uses LSTM to extract time-series features of energy consumption. When it detects that device utilization falls below 30% during off-peak hours, it automatically triggers the "low load → server hibernation" association in the knowledge graph, generating a decision to shut down redundant equipment and simultaneously adjusting the air conditioning system to economize mode. This reduces the data center's Power Use Effectiveness (PUE) to below 1.4, achieving over 30% energy savings compared to traditional solutions.

[0131] 4. Collaborative improvement of edge scenario reliability and operation and maintenance efficiency:

[0132] The system utilizes a voice-interactive and edge-cooperative architecture, leveraging WiFi / Bluetooth dual-link communication and localized large-scale model deployment to enable emergency control during network outages in edge data centers with unstable networks. For example, if a remote data center loses network connectivity, operations personnel can send a voice command via Bluetooth to "shut down the abnormal server." The system, relying on its local knowledge base, directly executes the power-off operation, avoiding the cascading failure risks associated with traditional cloud-based solutions due to communication interruptions. Measured data shows a 99.2% command execution success rate in edge scenarios, an 85% improvement over traditional solutions.

[0133] Example 3:

[0134] This embodiment provides a data center management and control method, which is implemented on the basis of the above embodiment and applied to the data center management and control system provided in the above embodiment. Figure 12 The flowchart of a data center management and control method shown in FIG. 1 includes the following steps:

[0135] Step S1202: The data collection module collects multimodal data; wherein the multimodal data includes: device body data, environment-related data, and dynamic relationship data;

[0136] In step S1204, the multimodal feature fusion module uses a modal attention mechanism based on the multimodal fusion engine to perform cross-modal alignment and dynamic weighting of the voice command, dynamic environment data, and device image.

[0137] Step S1206: The multi-level collaborative tracking module performs primary tracking and secondary tracking on the target, and fuses the detection results of the primary tracking and the secondary tracking;

[0138] Step S1208: The dynamic prediction and warning module issues warnings to the target through dual-dimensional detection, time window deduplication, and graded warnings.

[0139] Step S1210: The visual large model analysis module performs in-depth information analysis on the images acquired by the camera in real time in combination with the visual large model to identify the device status and abnormal conditions;

[0140] Step S1212: The voice interaction processing module adopts a voice and control collaborative architecture to achieve effective collaboration between voice and control.

[0141] In step S1214, the knowledge base matching module builds a system based on the knowledge graph and updates the knowledge graph for dynamic environment fault handling in real time;

[0142] Step S1216: The operation and maintenance decision generation module integrates the visual large model analysis results, the information obtained by voice interaction, and the knowledge base matching suggestions to generate an operation and maintenance decision;

[0143] Step S1218: The control instruction execution module generates corresponding control instructions according to the operation and maintenance decision, and executes the control instructions through the edge collaborative execution module;

[0144] In step S1220, the operation and maintenance result feedback module performs operation and maintenance closed-loop optimization and knowledge graph update.

[0145] The embodiment of the present invention provides a data center management and control method, constructing a complete closed loop of "data collection-multimodal feature fusion-analysis and decision-making-execution-feedback optimization", doubly improving the timeliness and accuracy of security warnings, significantly optimizing fault handling efficiency and intelligence level, significantly improving energy consumption efficiency optimization, and synergistically improving edge scenario reliability and operation and maintenance efficiency.

[0146] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the data center management and control method described above can refer to the corresponding process in the aforementioned embodiment and will not be repeated here.

[0147] In addition, in the description of the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to mechanical connections or electrical connections; they may refer to direct connections or indirect connections through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0148] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the method of the present invention. The aforementioned storage medium includes various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0149] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0150] Finally, it should be noted that the above embodiments are only specific implementation methods of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above embodiments within the technical scope disclosed by the present invention, or replace some of the technical features therein with equivalents. Such modifications, changes or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A data center management and control system, characterized in that: The data center management and control system includes: a data layer, an analysis layer, a decision layer, and an execution layer; the data layer includes: a data acquisition module and a multimodal feature fusion module; the analysis layer includes: a multi-level collaborative tracking module, a dynamic prediction and warning module, and a visual large model analysis module; the decision layer includes: a voice interaction processing module, a knowledge base matching module, and an operation and maintenance decision generation module; the execution layer includes: a control instruction execution module and an operation and maintenance result feedback module; The data acquisition module is used to collect multimodal data; wherein the multimodal data includes: device body data, environment-related data and dynamic relationship data; The multimodal feature fusion module is used to perform cross-modal alignment and dynamic weighting of voice commands, dynamic environment data and device images based on a multimodal fusion engine using a modal attention mechanism; The multi-level collaborative tracking module is used to perform primary tracking and secondary tracking of the target and fuse the detection results of the primary tracking and the secondary tracking; The dynamic prediction and warning module is used to warn the target through dual-dimensional detection, time window deduplication and graded warning; The visual large model analysis module is used to perform in-depth information analysis on the images acquired by the camera in real time in combination with the visual large model to identify the device status and abnormal conditions; The voice interaction processing module is used to adopt a voice and control collaborative architecture to achieve effective collaboration between voice and control; The knowledge base matching module is used to build a system based on the knowledge graph and update the knowledge graph of dynamic environment fault handling in real time; The operation and maintenance decision generation module is used to generate operation and maintenance decisions by integrating the visual large model analysis results, voice interaction information and knowledge base matching suggestions; The control instruction execution module is used to generate corresponding control instructions according to the operation and maintenance decision, and execute the control instructions through the edge collaborative execution module; The operation and maintenance result feedback module is used to perform operation and maintenance closed-loop optimization and knowledge graph update.

2. The data center management and control system according to claim 1, characterized in that: The data acquisition module is used to collect status data and network parameters through a specified protocol, and simultaneously record device model configuration and location information; The data acquisition module is used for the dynamic environment sensor to collect environmental parameters, the microphone array to capture voice commands and perform voice activity detection preprocessing, and the camera to obtain image information; The data acquisition module is used to track topology changes in real time through RFID positioning and linkage with the asset system, and record the physical connection relationship and logical dependency relationship between devices.

3. The data center management and control system according to claim 1, characterized in that: The multi-level collaborative tracking module is used to detect and associate the initial target and output the detection result of the first-level tracking with the tracking ID; The multi-level collaborative tracking module is used to use the centroid distance and intersection-over-union ratio joint matching strategy to perform secondary verification on the first-level tracking detection results, maintain the target trajectory history and calculate the velocity vector, and output the second-level tracking detection results.

4. The data center management and control system according to claim 1, wherein: The dynamic prediction and warning module is used to simultaneously determine whether the target's current position and the future position based on the velocity vector enter the safe area and provide early warning; The dynamic prediction and warning module is used to record the target's last alarm timestamp; When an alarm condition of the same target is detected again, whether to issue an early warning is determined based on the recorded timestamp of the target's last alarm; The dynamic prediction and warning module is used to warn the target's current position and future position in different ways.

5. The data center management and control system according to claim 1, characterized in that: The operation and maintenance result feedback module is used to mark samples and update training data based on the diagnosis results confirmed by the operation and maintenance personnel; The operation and maintenance result feedback module is used to automatically update the fault causal relationship according to the new fault situation and integrate the new fault handling plan into the knowledge graph training.

6. The data center management and control system according to claim 1, characterized in that: The multimodal feature fusion module is used to enable speech semantics and dynamic context data to interact through a cross-attention layer based on a multi-head attention mechanism.

7. The data center management and control system according to claim 1, characterized in that: The multimodal feature fusion module is used to dynamically adjust the weights of voice commands and dynamic environment data based on command type and data urgency.

8. The data center management and control system according to claim 1, wherein: The operation and maintenance decision generation module is used to normalize the device image feature vector from the visual large model analysis module, the semantic feature vector from the voice interaction processing module, and the fault handling solution probability vector output by the knowledge base matching module; The operation and maintenance decision generation module is used to fuse the normalized information as different evidence bodies to obtain a fused probability distribution result; The operation and maintenance decision generation module is used to input the fused probability distribution result into a pre-trained decision conversion model and output the operation and maintenance decision.

9. The data center management and control system according to any one of claims 1 to 8, characterized in that: The data acquisition module is connected to the multimodal feature fusion module; The multimodal feature fusion module is also connected to the multi-level collaborative tracking module, the dynamic prediction and warning module and the visual large model analysis module respectively; The multi-level collaborative tracking module is connected to the voice interaction processing module, and the dynamic prediction and warning module and the visual large model analysis module are both connected to the knowledge base matching module; The voice interaction processing module and the knowledge base matching module are both connected to the operation and maintenance decision generation module; The operation and maintenance decision generation module, the control instruction execution module and the operation and maintenance result feedback module are connected in sequence, and the operation and maintenance result feedback module is also connected to the data acquisition module.

10. A data center management and control method, characterized in that: Applied to the data center management and control system according to any one of claims 1 to 9, the data center management and control method includes: The data acquisition module acquires multimodal data; wherein the multimodal data includes: device body data, environment-related data and dynamic relationship data; The multimodal feature fusion module is based on the multimodal fusion engine and uses the modal attention mechanism to perform cross-modal alignment and dynamic weighting of voice commands, dynamic environment data, and device images. The multi-level collaborative tracking module performs primary and secondary tracking of the target and fuses the detection results of primary and secondary tracking; The dynamic prediction and warning module issues warnings to targets through dual-dimensional detection, time window deduplication, and graded warnings; The visual big model analysis module combines the visual big model to conduct in-depth information analysis on the images acquired by the camera in real time to identify device status and abnormal conditions; The voice interaction processing module adopts a voice and control collaborative architecture to achieve effective collaboration between voice and control; The knowledge base matching module builds a system based on the knowledge graph and updates the knowledge graph of dynamic environment fault handling in real time; The operation and maintenance decision generation module integrates the analysis results of the large visual model, information obtained through voice interaction, and knowledge base matching suggestions to generate operation and maintenance decisions; The control instruction execution module generates corresponding control instructions according to the operation and maintenance decision, and executes the control instructions through the edge collaborative execution module; The operation and maintenance result feedback module performs operation and maintenance closed-loop optimization and knowledge graph updates.

Images