Intelligent machine room operation and maintenance management method and device

Abstract

The invention discloses an intelligent machine room operation and maintenance management method and device, and relates to the technical field of machine room operation and maintenance management. The method comprises the following steps: acquiring text data, a screenshot image, a shot image and time sequence data; respectively inputting the text data, the screenshot image, the shot image and the time sequence data into a complex multi-mode multi-task hybrid neural network and a random forest model for fault types so as to obtain first equipment fault type information and first equipment residual life prediction information; first maintenance priority ranking information and second equipment fault type information; inputting the time sequence data and the text data into a residual life support vector regression model to obtain residual life prediction information of the second equipment; obtaining final equipment fault type information through the first equipment fault type information and the second equipment fault type information; and obtaining final residual life prediction information through the first equipment residual life prediction information and the second equipment residual life prediction information.

Description

Technical Field

[0001] This application relates to the technical field of computer room operation and maintenance management, and particularly relates to an intelligent computer room operation and maintenance management method and an intelligent computer room operation and maintenance management device. Background Art

[0002] In the prior art, computer room operation and maintenance management mainly relies on traditional monitoring and manual analysis means. The specific methods include the following: Single data source monitoring: The prior art usually only relies on a single type of data source for monitoring, such as the log data of devices, simple alarm information, or basic performance indicators (such as CPU usage rate, memory occupancy rate, etc.).

[0003] The monitoring means are mostly threshold alarms, that is, an alarm is triggered when a certain indicator exceeds a preset threshold.

[0004] Manual inspection and analysis: Operation and maintenance personnel need to regularly or irregularly inspect the computer room devices, check the running status, connection status, indicator light status, etc. of the devices.

[0005] When a device fails or malfunctions, operation and maintenance personnel need to manually analyze the log data, screenshots or taken images to determine the type and cause of the failure.

[0006] Rule-based fault diagnosis: In the prior art, fault diagnosis often relies on preset rules or experience, such as judging the type of failure according to specific error codes in the log.

[0007] This method lacks flexibility and adaptability and is difficult to handle complex and changeable fault situations.

[0008] Simple life prediction: For the prediction of the remaining life of devices, the prior art mostly adopts statistical methods or simple linear regression models.

[0009] These methods often ignore the complexity and non-linear characteristics of device operation, resulting in inaccurate prediction results. Summary of the Invention

[0010] The purpose of the present invention is to provide an intelligent computer room operation and maintenance management method to at least solve one of the above technical problems.

[0011] In one aspect of the present invention, there is provided an intelligent computer room operation and maintenance management method, and the intelligent computer room operation and maintenance management method includes: Obtain the text data, screen capture images, taken images, and time series data of the devices to be monitored; Obtain a trained complex multi-modal multi-task hybrid neural network; Obtain a trained random forest model for fault types and a trained support vector regression model for remaining life; Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain first device fault type information, first device remaining life prediction information, and first maintenance priority ranking information; Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain second device fault type information; Input the time series data and text data into the trained support vector regression model for remaining life to obtain second device remaining life prediction information; Obtain the final device fault type information through the first device fault type information and the second device fault type information; Obtain the final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

[0012] Optionally, the intelligent computer room operation and maintenance management method further includes: Train the complex multi-modal multi-task hybrid neural network; Train the random forest model for fault types and the support vector regression model for remaining life.

[0013] Optionally, the training of the complex multi-modal multi-task hybrid neural network includes: Obtain a training set, which includes the text data of each device in the computer room, the captured images transmitted by each camera device in the computer room, the screen capture images intercepted by the screen capture tool, and the time series data of each device; Construct a complex multi-modal multi-task hybrid neural network, which includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer; Train the complex multi-modal multi-task hybrid neural network with the training set.

[0014] Optionally, the feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; among them, the text feature extraction layer is used to extract the text features of the text data; the image feature extraction layer is used to extract the image features of the screen capture images and the captured images; the time series feature extraction layer is used to extract the time series features of the time series data; The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully-connected layer. Among them, the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully-connected layer is used to fuse the intermediate fusion features and the hypergraph features to form final fusion features; The multi-task learning layer includes a shared layer and task layers. The shared layer is used to receive the final fusion features and extract the general feature representation of the final fusion features; the task layers include a fault type task module, a remaining life prediction task module, and a maintenance priority module. The fault type task module is used to output first device fault type information according to the general feature representation; the remaining life prediction task module is used to output first device remaining life prediction information according to the general feature representation; the maintenance priority module is used to output first maintenance priority ranking information according to the general feature representation.

[0015] Optionally, the complex multi-modal multi-task hybrid neural network includes a device fault type information task loss function, where the device fault type information task loss function adopts the following formula: ; Where, is the device fault type information task loss function, M is the total number of samples, is the true class label of sample a, is the class sample frequency of, is the frequency adjustment parameter, is the sample a is predicted to be the class probability of, is the probability that sample a is predicted to be class c, is a hyperparameter that controls the intensity of non-target class suppression.

[0016] Optionally, the complex multi-modal multi-task hybrid neural network includes a device remaining life prediction information task loss function, where the device remaining life prediction information task loss function adopts the following formula: ; Where, is the device remaining life prediction information task loss function, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, is a constant to prevent division by zero, is a tuning parameter that controls the impact of time deviation on error scaling, is the timestamp of sample a, is the average of all sample timestamps, is a hyperparameter, is the rate of change of the predicted remaining life over time, is the rate of change of the true remaining life over time.

[0017] Optionally, the complex multi-modal multi-task hybrid neural network includes a maintenance priority ranking information task loss function, where the maintenance priority ranking information task loss function uses the following formula: ; where, is the maintenance priority ranking information task loss function, S is the set of positive sample pairs and D is the set of negative sample pairs , is the maintenance priority score of sample a, is the maintenance priority score of sample b, is the maintenance priority score of sample c, is the maintenance priority score of sample d, is the dynamic margin of the positive sample pair (a, b), is the dynamic margin of the negative sample pair (c, d).

[0018] Optionally, the complex multi-modal multi-task hybrid neural network includes a total loss function, where the total loss function uses the following formula: ; where, is the equipment failure type information task loss function, is the equipment remaining life prediction information task loss function, is the maintenance priority ranking information task loss function, is the loss weight of the failure type classification task, is the loss weight of the remaining life prediction task, is the loss weight of the maintenance priority ranking task.

[0019] This application also provides an intelligent computer room operation and maintenance management device, and the intelligent computer room operation and maintenance management device includes: An initial data acquisition module, which is used to acquire text data, screen capture images, captured images, and time series data of the device to be monitored; A neural network acquisition module, which is used to acquire a trained complex multi-modal multi-task hybrid neural network; A model acquisition module, which is used to acquire a trained random forest model for fault types and a trained support vector regression model for remaining life respectively; A first prediction result acquisition module, which is used to input the text data, screen capture images, captured images and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain first device fault type information, first device remaining life prediction information and first maintenance priority ranking information; A second prediction result acquisition module, which is used to input the text data, screen capture images, captured images and time series data of the device to be monitored into the trained random forest model for fault types to obtain second device fault type information; input the time series data and text data into the trained support vector regression model for remaining life to obtain second device remaining life prediction information; A final device fault type information acquisition module, which is used to obtain final device fault type information through the first device fault type information and the second device fault type information; A final device fault type information acquisition module, and the final device fault type information is used to obtain final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

[0020] The intelligent computer room operation and maintenance management method of the present application realizes comprehensive and in-depth monitoring and analysis of the device operation status and fault conditions by introducing a complex multi-modal multi-task hybrid neural network, a random forest model for fault types and a support vector regression model for remaining life, improves the fault diagnosis efficiency and prediction accuracy, and provides more intelligent and efficient support for computer room operation and maintenance management. Brief Description of the Drawings

[0021] Figure 1 It is a schematic flowchart of the intelligent computer room operation and maintenance management method according to an embodiment of the present application. Detailed Embodiment

[0022] To make the objectives, technical solutions, and advantages of the present application more clear, the following will describe in more detail the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. In the drawings, the same or similar reference numerals represent the same or similar elements or elements with the same or similar functions from beginning to end. The described embodiments are some, but not all, of the embodiments of the present application. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present application and should not be construed as a limitation to the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present application. The following will describe the embodiments of the present application in detail in conjunction with the accompanying drawings.

[0023] As Figure 1 shown, the intelligent computer room operation and maintenance management method includes: Step 1: Obtain the text data, screen capture images, captured images, and time series data of the device to be monitored; Step 2: Obtain a trained complex multi-modal multi-task hybrid neural network; Step 3: Obtain a trained random forest model for fault types and a trained remaining life support vector regression model; Step 4: Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network to obtain the first device fault type information, the first device remaining life prediction information, and the first maintenance priority ranking information; Step 5: Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain the second device fault type information; Step 6: Input the time series data and text data into the trained remaining life support vector regression model to obtain the second device remaining life prediction information; Step 7: Obtain the final device fault type information through the first device fault type information and the second device fault type information; Step 8: Obtain the final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

[0024] By introducing a complex multi-modal multi-task hybrid neural network, a random forest model for fault types, and a remaining life support vector regression model, the intelligent computer room operation and maintenance management method of the present application realizes comprehensive and in-depth monitoring and analysis of the device operation status and fault conditions, improves the fault diagnosis efficiency and prediction accuracy, and provides more intelligent and efficient support for computer room operation and maintenance management.

[0025] In this embodiment, the intelligent computer room operation and maintenance management method further includes: Training the complex multi-modal multi-task hybrid neural network; Training the random forest model for fault types and the support vector regression model for remaining life.

[0026] In this embodiment, the training of the complex multi-modal multi-task hybrid neural network includes: Obtaining a training set, where the training set includes text data of each device in the computer room, captured images transmitted by each camera device in the computer room, screen capture images captured by a screen capture tool, and time series data of each device; Constructing a complex multi-modal multi-task hybrid neural network, where the complex multi-modal multi-task hybrid neural network includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer; Training the complex multi-modal multi-task hybrid neural network with the training set.

[0027] In this embodiment, the feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; among them, the text feature extraction layer is used to extract text features of text data; the image feature extraction layer is used to extract image features of screen capture images and captured images; the time series feature extraction layer is used to extract time series features of time series data; The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully connected layer. Among them, the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully connected layer is used to fuse the intermediate fusion features and the hypergraph features to form final fusion features; The multi-task learning layer includes a shared layer and a task layer. The shared layer is used to receive the final fusion features and extract the general feature representation of the final fusion features; the task layer includes a fault type task module, a remaining life prediction task module, and a maintenance priority module. The fault type task module is used to output the first device fault type information according to the general feature representation; the remaining life prediction task module is used to output the first device remaining life prediction information according to the general feature representation; the maintenance priority module is used to output the first maintenance priority ranking information according to the general feature representation.

[0028] In this embodiment, the text feature extraction layer can use a pre-trained BERT model (or a similar Transformer architecture) to encode text data and extract context-related semantic features.

[0029] In this embodiment, ResNet-50 or EfficientNet is used to extract features from screenshot images and captured images.

[0030] In this embodiment, a Transformer encoder is used to extract features from time series data.

[0031] In this embodiment, the primary fusion layer fuses text features, image features, and time series features into a primary feature vector (i.e., the primary feature) by concatenation or weighted summation.

[0032] In this embodiment, an inter-modal attention mechanism (Cross-Modal Attention) is used to perform self-attention calculation on the primary features to capture the interaction relationships between different modalities.

[0033] In this embodiment, the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features, including: The intermediate fusion features are split into multiple sub-features, and each sub-feature serves as a node. For example, text features, image features, and time series features can be used as different nodes respectively. It can be understood that more detailed division can be carried out. For example, in an embodiment of the present application, the text data includes the frequency distribution of error codes, alarm information, and operation logs, and the screenshot images include the status of interface elements, operating parameters, and abnormal colors, which can also be used as nodes.

[0034] Construct a hyperedge connection matrix, and define hyperedges according to the relevance of events. For example, connect the text, image, and time series feature nodes related to a certain event together.

[0035] Use a multi-layer GCN (Graph Convolutional Network) to process the graph structure to obtain hypergraph features.

[0036] In this embodiment, after concatenating the intermediate fusion features and the hypergraph features, they are further fused through a fusion fully connected layer to form the final fusion features.

[0037] In this embodiment, the fault type task module uses a fully connected layer and a Softmax activation function to output the probability distribution of the device fault type, and selects the one with the highest probability as the first device fault type information; The remaining useful life prediction task module uses a fully connected layer and a linear activation function to output the continuous value of the remaining useful life of the device.

[0038] The maintenance priority module uses a fully connected layer and a Sigmoid activation function to output a score for the maintenance priority (between 0 and 1).

[0039] In this embodiment, the complex multi-modal multi-task hybrid neural network uses a multi-task learning framework to simultaneously optimize the loss functions of three tasks, and adopts a weighted summation method to balance the loss contributions of different tasks, ensuring that the model can achieve good performance on multiple tasks.

[0040] It can be understood that the training method of the complex multi-modal multi-task hybrid neural network can adopt the existing training method, which will not be elaborated here. For example, setting a training set, a test set, etc., and setting hyperparameters such as an appropriate learning rate and momentum.

[0041] In this embodiment, the complex multi-modal multi-task hybrid neural network can use an Adam optimizer or an SGD optimizer during the training process, combined with a learning rate scheduling strategy (such as cosine annealing or learning rate warm-up) to accelerate the convergence of the model.

[0042] In this embodiment, evaluating the performance of the model on the test set also belongs to the prior art and will not be elaborated here. For example, calculating evaluation metrics such as accuracy, recall, F1 score (classification task), mean squared error (regression task), etc.

[0043] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a device fault type information task loss function, where the device fault type information task loss function adopts the following formula: ; Wherein, is the device fault type information task loss function, M is the total number of samples, is the true class label of sample a, is the class sample frequency of, is the frequency adjustment parameter, is the sample a is predicted to be the class probability of, is the probability that sample a is predicted to be class c, is a hyperparameter that controls the intensity of non-target class suppression.

[0044] In this embodiment, by introducing a class frequency dynamic adjustment mechanism, the deficiency of the traditional loss function in the class imbalance problem is effectively solved. Its core advantage is that it can dynamically adjust the loss weights according to the sample frequencies of each class, so that samples of rare classes receive higher attention during training.

[0045] Specifically, the term in the loss function amplifies the loss contribution of rare class samples through the exponential decay of the class frequency , thus alleviating the model bias problem caused by class imbalance. In addition, the clarity of the classification boundary is enhanced by suppressing the probability of non-target classes , and the misclassification of uncertain samples by the model is reduced. This mechanism not only improves the model's recognition ability for rare classes but also enhances the overall classification accuracy. In practical applications, it can significantly improve the performance of equipment fault type classification, especially in scenarios where the fault type distribution is uneven, and its advantages are particularly obvious. In addition, the design of this loss function has high flexibility and scalability, and can adapt to different data distributions and task requirements by adjusting the frequency adjustment parameter and the non-target class suppression parameter μ.

[0046] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a loss function for the equipment remaining life prediction information task, where the loss function for the equipment remaining life prediction information task adopts the following formula: ; where is the loss function for the equipment remaining life prediction information task, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, is a constant to prevent division by zero, is a tuning parameter that controls the impact of time bias on error scaling, is the timestamp of sample a, is the average of all sample timestamps, is a hyperparameter, is the rate of change of the predicted remaining life over time, is the rate of change of the true remaining life over time.

[0047] By introducing a time-dependent error scaling mechanism, the performance of the remaining life prediction task is significantly improved. Its advantage is that it can dynamically adjust the loss weight according to the time distribution of the prediction error, making the model pay more attention to the prediction accuracy of recent data. Specifically, the term amplifies the error contribution of recent samples through the exponential function of the time bias , thus emphasizing the importance of recent data in model training. In addition, the trend consistency penalty is introduced Ensures the consistency between the predicted trend and the actual trend, enabling the model to capture the variation law of the remaining life of the device over time. This mechanism not only improves the model's prediction ability for recent data but also enhances the model's adaptability to time-series data. In practical applications, it can significantly improve the accuracy of predicting the remaining life of the device, especially in scenarios where the data changes greatly over time, and its advantages are particularly obvious. In addition, the design of this loss function has high robustness and interpretability, and can be adjusted by adjusting the time adjustment parameter β and the trend penalty parameter ν to adapt to different data characteristics and task requirements.

[0048] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a maintenance priority ranking information task loss function, where the maintenance priority ranking information task loss function adopts the following formula: ; Where is the maintenance priority ranking information task loss function, S is the set of positive sample pairs , D is the set of negative sample pairs , is the maintenance priority score of sample a, is the maintenance priority score of sample b, is the maintenance priority score of sample c, is the maintenance priority score of sample d, is the dynamic margin of the positive sample pair (a, b). In this embodiment, ; is the dynamic margin of the negative sample pair (c, d). In this embodiment, ; is the basic margin; is the adjustment parameter, controlling the influence of feature similarity on the margin; is the feature similarity between sample a and b; is the feature similarity between sample c and d; is the feature vector of sample a; is the feature vector of sample b; is the feature vector of sample c; is the feature vector of sample d.

[0049] By introducing the dynamic margin adjustment mechanism, the performance of the maintenance priority ranking task is significantly improved. It can dynamically adjust the ranking margin according to the feature similarity of the sample pairs, enabling the model to handle the ranking relationship more meticulously. Specifically, the term in the loss function passes through the feature similarity The linear function sets a smaller margin for similar sample pairs and a larger margin for dissimilar sample pairs, thereby enhancing the robustness and accuracy of sorting.

[0050] In addition, the consistency of the sorting relationship is ensured through bidirectional sorting constraints (the loss terms for positive and negative sample pairs), enabling the model to optimize the sorting performance of both positive and negative sample pairs simultaneously. This mechanism not only improves the model's ability to capture subtle sorting differences but also enhances the model's adaptability to complex sorting tasks. In practical applications, it can significantly improve the accuracy of the maintenance priority sorting.

[0051] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a total loss function, where the total loss function adopts the following formula: ; Where is the task loss function for device fault type information, is the task loss function for device remaining life prediction information, is the task loss function for maintenance priority sorting information, is the loss weight for the fault type classification task, is the loss weight for the remaining life prediction task, is the loss weight for the maintenance priority sorting task.

[0052] Each loss function and the total loss function of this application significantly improve the performance of the model in device operation and maintenance management tasks by introducing new strategies and mechanisms. The device fault type information loss function effectively solves the problem of class imbalance and improves the accuracy of fault type classification; the device remaining life prediction information task loss function enhances the model's ability to capture recent data and trend changes and improves the accuracy of remaining life prediction; the maintenance priority sorting information task loss function refines the sorting relationship and improves the robustness of maintenance priority sorting. These advantages make the improved loss function have significant advantages in practical applications and can meet the requirements of complex device operation and maintenance management scenarios.

[0053] In this embodiment, the text data, screen capture images, captured images, and time series data of the device to be monitored are input into the trained random forest model for fault types to obtain the second device fault type information as follows: Extract features from the input data (text data, screen capture images, captured images, and time series data). For text data, the bag-of-words model or the TF-IDF method can be used to extract features; for image data, a convolutional neural network can be used to extract features; for time series data, statistical features (such as mean, variance, peak, etc.) can be used as features.

[0054] The CART (Classification and Regression Tree) algorithm is used to construct a decision tree. At each node, the optimal feature and split point are selected to divide the data set into two subsets, such that the data within the subsets belongs to the same class as much as possible. The splitting is recursively performed until the stopping conditions are met (such as the number of samples in the node is less than a certain threshold, the purity of the node reaches a certain requirement, etc.).

[0055] Multiple decision trees are constructed, and each decision tree is trained using a different random subset. The random subset includes a randomly selected subset of features and a subset of samples randomly drawn from the original data set (sampling with replacement).

[0056] Each decision tree performs classification prediction on the input data, and the final prediction result is determined by the voting results of all decision trees.

[0057] In this embodiment, the labeled fault type data can be used to train the random forest model, and the parameters of the model, such as the number of decision trees, the maximum depth, etc., are adjusted to improve the classification accuracy of the model.

[0058] In this embodiment, in the prediction of the remaining life prediction information of the second device, the remaining life support vector regression model can find an optimal hyperplane to minimize the error between the input time series data and the remaining life of the device.

[0059] In this embodiment, the remaining life data, time series data, and text data of the device can be used to train the support vector regression model, and the appropriate kernel function and parameters are selected to optimize the regression performance of the model.

[0060] In this embodiment, the text data includes the log files of the device, which can include various log information, such as operation information, alarm information, maintenance information, and other information. It can be understood that in other embodiments, other information can also be included.

[0061] The screenshot image refers to the screenshot during the operation of the device, and the captured image refers to the appearance image of the device.

[0062] In this embodiment, the time series data can include CPU usage rate, memory occupancy rate, disk I / O speed, network traffic, working temperature of device components, stability indicators of power supply, vibration, and noise level. It can be understood that in other embodiments, other information can also be included.

[0063] In this embodiment, the final device failure type information is obtained through the first device failure type information and the second device failure type information. For example, when the first device failure type information and the second device failure type information are the same, either one can be used as the standard. Alternatively, rules can be set such that when the voting results of all decision trees exceed a threshold (e.g., 90%), the second device failure type information is used as the standard. It can be understood that other methods can also be used to determine it.

[0064] In this embodiment, the final remaining life prediction information is obtained through the first device remaining life prediction information and the second device remaining life prediction information, which is similar to the determination of the above device failure type information. For example, when the first device remaining life prediction information and the second device remaining life prediction information are the same, either one can be used as the standard. Alternatively, it can be obtained through a formula. For example, it can be obtained through the following formula: ; where α is a weight coefficient, and the optimal value can be determined through methods such as cross-validation, and 0 < α < 1; is the final remaining life prediction information; is the first device remaining life prediction information; is the second device remaining life prediction information.

[0065] For example, for a certain router, the first device remaining life prediction information is 30 hours, the second device remaining life prediction information is 50 hours, and α is 0.4, then (0.4 × 30) + (0.6 × 50) = 42 hours.

[0066] This application also provides an intelligent computer room operation and maintenance management device. The intelligent computer room operation and maintenance management device includes an initial data acquisition module, a neural network acquisition module, a model acquisition module, a first prediction result acquisition module, a second prediction result acquisition module, a final device failure type information acquisition module, and a final device failure type information acquisition module. Among them, The initial data acquisition module is used to acquire the text data, screen capture images, captured images, and time series data of the device to be monitored; The neural network acquisition module is used to acquire a trained complex multi-modal multi-task hybrid neural network; The model acquisition module is used to respectively acquire a trained random forest model for failure types and a trained support vector regression model for remaining life; The first prediction result acquisition module is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain the first device failure type information, the first device remaining life prediction information, and the first maintenance priority ranking information; The second prediction result obtaining module is configured to input the text data, screen capture images, captured images, and time series data of the device to be monitored into a trained random forest model for fault types to obtain second device fault type information; input the time series data and text data into a trained remaining life support vector regression model to obtain second device remaining life prediction information; The final device fault type information obtaining module is configured to obtain final device fault type information through the first device fault type information and the second device fault type information; The final device fault type information is used to obtain final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information In addition, it is obvious that the word "including" does not exclude other units or steps. The multiple units, modules or devices stated in the apparatus claims can also be implemented by one unit or a general apparatus through software or hardware.

[0067] Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it on the basis of the present invention, which will be obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention fall within the scope of the present invention claimed.

Claims

1. An intelligent computer room operation and maintenance management method, characterized in that: The intelligent computer room operation and maintenance management method includes: Obtain text data, screenshot images, photographic images, and time series data of the device to be monitored; Obtain a trained complex multimodal multitask hybrid neural network; Obtain the trained random forest model for fault types and the trained support vector regression model for remaining life; Inputting text data, screenshot images, photographed images, and time series data of the device to be monitored into the complex multimodal multitasking hybrid neural network, thereby obtaining first device fault type information, first device remaining life prediction information, and first maintenance priority ranking information; Inputting text data, screenshot images, photographed images, and time series data of the device to be monitored into a trained random forest model for fault types to obtain second device fault type information; Inputting the time series data and the text data into a trained remaining life support vector regression model to obtain remaining life prediction information of the second device; Acquire final device failure type information through first device failure type information and second device failure type information; The final remaining life prediction information is obtained through the first device remaining life prediction information and the second device remaining life prediction information.

2. The intelligent computer room operation and maintenance management method according to claim 1, characterized in that: The intelligent computer room operation and maintenance management method further comprises: Training the complex multi-modal multi-task hybrid neural network; The random forest model for the fault type and the remaining life support vector regression model were trained.

3. The intelligent computer room operation and maintenance management method according to claim 2, characterized in that: The training of the complex multi-modal multi-task hybrid neural network includes: Acquire a training set, wherein the training set includes text data of each device in the computer room, images captured by each camera in the computer room, screenshot images captured by a screenshot tool, and time series data of each device; Constructing a complex multimodal multitask hybrid neural network, wherein the complex multimodal multitask hybrid neural network includes a feature extraction layer, an interactive fusion layer, and a multitask learning layer; The complex multi-modal multi-task hybrid neural network is trained using the training set.

4. The intelligent computer room operation and maintenance management method according to claim 3, characterized in that: The feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; wherein the text feature extraction layer is used to extract text features of text data; the image feature extraction layer is used to extract image features of screenshot images and photographed images; and the time series feature extraction layer is used to extract time series features of time series data; The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer and a fusion fully connected layer, wherein the primary fusion layer is used to fuse text features, image features and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through the inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully connected layer is used to fuse the intermediate fusion features and the hypergraph features to form the final fusion features; The multi-task learning layer includes a shared layer and a task layer. The shared layer is used to receive the final fusion feature and extract the general feature representation of the final fusion feature; the task layer includes a fault type task module, a remaining life prediction task module and a maintenance priority module. The fault type task module is used to output the first equipment fault type information according to the general feature representation; the remaining life prediction task module is used to output the first equipment remaining life prediction information according to the general feature representation; the maintenance priority module is used to output the first maintenance priority sorting information according to the general feature representation.

5. The intelligent computer room operation and maintenance management method according to claim 4, characterized in that: The complex multi-modal multi-task hybrid neural network includes an equipment fault type information task loss function, wherein the equipment fault type information task loss function adopts the following formula: ； in, is the equipment fault type information task loss function, M is the total number of samples, is the true category label of sample a, For Category The sample frequency, is the frequency adjustment parameter, For sample a Predicted as category The probability of is the probability that sample a is predicted to be category c, is a hyperparameter that controls the strength of non-target category suppression.

6. The intelligent computer room operation and maintenance management method according to claim 5, characterized in that: The complex multi-modal multi-task hybrid neural network includes a task loss function for predicting the remaining life of an equipment, wherein the task loss function for predicting the remaining life of an equipment adopts the following formula: ； in, The information loss function for the equipment remaining life prediction task, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, To prevent division by zero, To adjust the parameters and control the effect of time deviation on error scaling, is the timestamp of sample a, is the average of all sample timestamps, is a hyperparameter, To predict the rate of change of remaining life over time, is the rate of change of the true remaining life with time.

7. The intelligent computer room operation and maintenance management method according to claim 6, characterized in that: The complex multi-modal multi-task hybrid neural network includes a maintenance priority sorting information task loss function, wherein the maintenance priority sorting information task loss function adopts the following formula: ； in, is the loss function of the maintenance priority sorting information task, S is the set of positive sample pairs , D is the set of negative sample pairs , Score the maintenance priority of sample a, Score the maintenance priority of sample b, Score the maintenance priority of sample c, Score the maintenance priority of sample d, is the dynamic margin of the positive sample pair (a, b), is the dynamic margin of the negative sample pair (c, d).

8. The intelligent computer room operation and maintenance management method according to claim 7, characterized in that: The complex multi-modal multi-task hybrid neural network includes a total loss function, wherein the total loss function adopts the following formula: ； in, is the equipment fault type information task loss function, The information loss function for the equipment remaining life prediction task, Loss function for the maintenance priority sorting information task, is the loss weight for the fault type classification task, is the loss weight for the remaining life prediction task, Loss weights for repair priority sorting tasks.

9. An intelligent computer room operation and maintenance management device, characterized in that: The intelligent computer room operation and maintenance management device comprises: An initial data acquisition module, the initial data acquisition module is used to acquire text data, screenshot images, photographed images and time series data of the device to be monitored; A neural network acquisition module, wherein the neural network acquisition module is used to acquire a trained complex multi-modal multi-task hybrid neural network; A model acquisition module, wherein the model acquisition module is used to respectively acquire a trained random forest model for fault types and a trained remaining life support vector regression model; A first prediction result acquisition module, the first prediction result acquisition module is used to input the text data, screenshot images, photographed images and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain first device fault type information, first device remaining life prediction information and first maintenance priority sorting information; A second prediction result acquisition module, the second prediction result acquisition module is used to input the text data, screenshot images, photographed images and time series data of the device to be monitored into a trained random forest model for fault types to obtain second device fault type information; input the time series data and text data into a trained remaining life support vector regression model to obtain remaining life prediction information of the second device; A final device fault type information acquisition module, wherein the final device fault type information acquisition module is used to acquire final device fault type information through the first device fault type information and the second device fault type information; A final equipment failure type information acquisition module, wherein the final equipment failure type information is used to acquire final remaining life prediction information through the first equipment remaining life prediction information and the second equipment remaining life prediction information.

Images