An intelligent computer room operation and maintenance management method and device

Abstract

The present application discloses an intelligent computer room operation and maintenance management method and device, relating to the technical field of computer room operation and maintenance management. The method includes: obtaining text data, screen capture images, captured images, and time series data; respectively inputting the text data, screen capture images, captured images, and time series data into a complex multi-modal multi-task hybrid neural network and a random forest model for fault types to obtain first device fault type information, first device remaining life prediction information, first maintenance priority ranking information, and second device fault type information; inputting the time series data and text data into a remaining life support vector regression model to obtain second device remaining life prediction information; obtaining final device fault type information through the first device fault type information and the second device fault type information; and obtaining final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

Description

Technical Field

[0001] This application relates to the technical field of computer room operation and maintenance management, and particularly relates to an intelligent computer room operation and maintenance management method and an intelligent computer room operation and maintenance management device. Background Art

[0002] In the prior art, computer room operation and maintenance management mainly relies on traditional monitoring and manual analysis means. The specific methods include the following several:

[0003] Single data source monitoring:

[0004] The prior art usually only relies on a single type of data source for monitoring, such as the log data of devices, simple alarm information, or basic performance metrics (such as CPU usage rate, memory occupancy rate, etc.).

[0005] The monitoring means are mostly threshold alarms, that is, an alarm is triggered when a certain index exceeds a preset threshold.

[0006] Manual inspection and analysis:

[0007] Operation and maintenance personnel need to regularly or irregularly inspect the computer room equipment, and check the running status, connection status, indicator light status, etc. of the equipment.

[0008] When a device fails or is abnormal, operation and maintenance personnel need to manually analyze the log data, screenshots or take images to determine the type and cause of the failure.

[0009] Rule-based fault diagnosis:

[0010] In the prior art, fault diagnosis often relies on preset rules or experience, such as judging the type of failure according to specific error codes in the log.

[0011] This method lacks flexibility and self-adaptability and is difficult to cope with complex and changeable fault situations.

[0012] Simple life prediction:

[0013] For the prediction of the remaining life of devices, the prior art mostly adopts statistical methods or simple linear regression models.

[0014] These methods often ignore the complexity and non-linear characteristics of device operation, resulting in inaccurate prediction results. Summary of the Invention

[0015] The purpose of the present invention is to provide an intelligent computer room operation and maintenance management method to at least solve one of the above technical problems.

[0016] In one aspect of the present invention, there is provided an intelligent computer room operation and maintenance management method, and the intelligent computer room operation and maintenance management method includes:

[0017] Obtain the text data, screen capture images, captured images, and time series data of the device to be monitored;

[0018] Obtain a trained complex multi-modal multi-task hybrid neural network;

[0019] Obtain a trained random forest model for fault types and a trained support vector regression model for remaining life;

[0020] Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network to obtain first device fault type information, first device remaining life prediction information, and first maintenance priority ranking information;

[0021] Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain second device fault type information;

[0022] Input the time series data and text data into the trained support vector regression model for remaining life to obtain second device remaining life prediction information;

[0023] Obtain the final device fault type information through the first device fault type information and the second device fault type information;

[0024] Obtain the final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

[0025] Optionally, the intelligent computer room operation and maintenance management method further includes:

[0026] Train the complex multi-modal multi-task hybrid neural network;

[0027] Train the random forest model for fault types and the support vector regression model for remaining life.

[0028] Optionally, the training of the complex multi-modal multi-task hybrid neural network includes:

[0029] Obtain a training set, which includes the text data of each device in the computer room, the captured images transmitted by each camera device in the computer room, the screen capture images intercepted by the screen capture tool, and the time series data of each device;

[0030] Construct a complex multi-modal multi-task hybrid neural network, which includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer;

[0031] Train the complex multi-modal multi-task hybrid neural network with the training set.

[0032] Optionally, the feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; wherein, the text feature extraction layer is used to extract text features of text data; the image feature extraction layer is used to extract image features of screenshot images and captured images; the time series feature extraction layer is used to extract time series features of time series data;

[0033] The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully connected layer, wherein the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully connected layer is used to fuse the intermediate fusion features and the hypergraph features to form final fusion features;

[0034] The multi-task learning layer includes a shared layer and task layers, the shared layer is used to receive the final fusion features and extract the general feature representation of the final fusion features; the task layers include a fault type task module, a remaining life prediction task module, and a maintenance priority module, the fault type task module is used to output first device fault type information according to the general feature representation; the remaining life prediction task module is used to output first device remaining life prediction information according to the general feature representation; the maintenance priority module is used to output first maintenance priority ranking information according to the general feature representation.

[0035] Optionally, the complex multi-modal multi-task hybrid neural network includes a device fault type information task loss function, wherein the device fault type information task loss function adopts the following formula:

[0036] ;

[0037] Wherein, is the device fault type information task loss function, M is the total number of samples, is the true class label of sample a, is the class sample frequency of, is the frequency adjustment parameter, is the sample a is predicted to be the class probability of, is the probability that sample a is predicted to be class c, is a hyperparameter that controls the intensity of non-target class suppression.

[0038] Optionally, the complex multimodal multitask hybrid neural network includes a device remaining life prediction information task loss function, where the device remaining life prediction information task loss function is expressed by the following formula:

[0039] ;

[0040] where, is the device remaining life prediction information task loss function, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, is a constant to prevent division by zero, is a tuning parameter that controls the influence of time deviation on error scaling, is the timestamp of sample a, is the average of all sample timestamps, is a hyperparameter, is the rate of change of the predicted remaining life over time, is the rate of change of the true remaining life over time.

[0041] Optionally, the complex multimodal multitask hybrid neural network includes a maintenance priority ranking information task loss function, where the maintenance priority ranking information task loss function is expressed by the following formula:

[0042] ;

[0043] where, is the maintenance priority ranking information task loss function, S is the set of positive sample pairs and D is the set of negative sample pairs , is the maintenance priority score of sample a, is the maintenance priority score of sample b, is the maintenance priority score of sample c, is the maintenance priority score of sample d, is the dynamic margin of the positive sample pair (a, b), is the dynamic margin of the negative sample pair (c, d).

[0044] Optionally, the complex multimodal multitask hybrid neural network includes a total loss function, where the total loss function is expressed by the following formula:

[0045] ;

[0046] where, is the task loss function for equipment failure type information, is the task loss function for equipment remaining life prediction information, is the task loss function for maintenance priority ranking information, is the loss weight for the fault type classification task, is the loss weight for the remaining life prediction task, is the loss weight for the maintenance priority ranking task.

[0047] This application also provides an intelligent computer room operation and maintenance management device, and the intelligent computer room operation and maintenance management device includes:

[0048] An initial data acquisition module, which is used to acquire text data, screen capture images, captured images, and time series data of the device to be monitored;

[0049] A neural network acquisition module, which is used to acquire a trained complex multi-modal multi-task hybrid neural network;

[0050] A model acquisition module, which is used to respectively acquire a trained random forest model for fault types and a trained support vector regression model for remaining life;

[0051] A first prediction result acquisition module, which is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain first device fault type information, first device remaining life prediction information, and first maintenance priority ranking information;

[0052] A second prediction result acquisition module, which is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain second device fault type information; input the time series data and text data into the trained support vector regression model for remaining life to obtain second device remaining life prediction information;

[0053] A final device fault type information acquisition module, which is used to obtain final device fault type information through the first device fault type information and the second device fault type information;

[0054] A final device fault type information acquisition module, which is used to obtain final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information.

[0055] The intelligent computer room operation and maintenance management method of the present application realizes comprehensive and in-depth monitoring and analysis of the device operation status and fault conditions by introducing a complex multi-modal multi-task hybrid neural network, a random forest model for fault types, and a support vector regression model for remaining life, improving the fault diagnosis efficiency and prediction accuracy, and providing more intelligent and efficient support for computer room operation and maintenance management. Brief Description of the Drawings

[0056] Figure 1 is a schematic flowchart of the intelligent computer room operation and maintenance management method according to an embodiment of the present application. Detailed Embodiments

[0057] To make the purpose, technical solutions, and advantages of the implementation of the present application clearer, the technical solutions in the embodiments of the present application will be described in more detail below with reference to the accompanying drawings in the embodiments of the present application. In the drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, of the embodiments of the present application. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present application and should not be construed as limiting the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts fall within the scope of protection of the present application. The embodiments of the present application will be described in detail below with reference to the drawings.

[0058] As Figure 1 shown, the intelligent computer room operation and maintenance management method includes:

[0059] Step 1: Obtain the text data, screen capture images, captured images, and time series data of the device to be monitored;

[0060] Step 2: Obtain the trained complex multi-modal multi-task hybrid neural network;

[0061] Step 3: Obtain the trained random forest model for fault types and the trained support vector regression model for remaining life;

[0062] Step 4: Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network to obtain the first device fault type information, the first device remaining life prediction information, and the first maintenance priority ranking information;

[0063] Step 5: Input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain the second device fault type information;

[0064] Step 6: Input the time series data and text data into the trained remaining life support vector regression model to obtain the remaining life prediction information of the second device;

[0065] Step 7: Obtain the final device fault type information based on the first device fault type information and the second device fault type information;

[0066] Step 8: Obtain the final remaining life prediction information based on the first device remaining life prediction information and the second device remaining life prediction information.

[0067] The intelligent computer room operation and maintenance management method of this application realizes comprehensive and in-depth monitoring and analysis of the device operation status and fault conditions by introducing a complex multi-modal multi-task hybrid neural network, a random forest model for fault types, and a remaining life support vector regression model, improves the fault diagnosis efficiency and prediction accuracy, and provides more intelligent and efficient support for computer room operation and maintenance management.

[0068] In this embodiment, the intelligent computer room operation and maintenance management method further includes:

[0069] Train the complex multi-modal multi-task hybrid neural network;

[0070] Train the random forest model for fault types and the remaining life support vector regression model.

[0071] In this embodiment, the training of the complex multi-modal multi-task hybrid neural network includes:

[0072] Obtain a training set, where the training set includes the text data of each device in the computer room, the captured images transmitted by each camera device in the computer room, the screen capture images captured by the screen capture tool, and the time series data of each device;

[0073] Construct a complex multi-modal multi-task hybrid neural network, where the complex multi-modal multi-task hybrid neural network includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer;

[0074] Train the complex multi-modal multi-task hybrid neural network with the training set.

[0075] In this embodiment, the feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; among them, the text feature extraction layer is used to extract the text features of the text data; the image feature extraction layer is used to extract the image features of the screen capture images and the captured images; the time series feature extraction layer is used to extract the time series features of the time series data;

[0076] The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully-connected layer. Among them, the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully-connected layer is used to fuse the intermediate fusion features and the hypergraph features to form the final fusion features.

[0077] The multi-task learning layer includes a shared layer and task layers. The shared layer is used to receive the final fusion features and extract the general feature representation of the final fusion features; the task layers include a fault type task module, a remaining life prediction task module, and a maintenance priority module. The fault type task module is used to output the first device fault type information according to the general feature representation; the remaining life prediction task module is used to output the first device remaining life prediction information according to the general feature representation; the maintenance priority module is used to output the first maintenance priority ranking information according to the general feature representation.

[0078] In this embodiment, the text feature extraction layer can use a pre-trained BERT model (or a similar Transformer architecture) to encode text data and extract context-related semantic features.

[0079] In this embodiment, ResNet-50 or EfficientNet is used to extract features from screenshot images and captured images.

[0080] In this embodiment, a Transformer encoder is used to extract features from time series data.

[0081] In this embodiment, the primary fusion layer fuses text features, image features, and time series features into a primary feature vector (i.e., the primary feature) by means of concatenation or weighted summation.

[0082] In this embodiment, an inter-modal attention mechanism (Cross-Modal Attention) is used to perform self-attention calculation on the primary features to capture the interaction relationships between different modalities.

[0083] In this embodiment, the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features, and constructing hypergraph features through the sub-features includes:

[0084] Split the intermediate fusion features into multiple sub - features, with each sub - feature serving as a node. For example, text features, image features, and time - series features can be used as different nodes respectively. It can be understood that more detailed divisions can be made. For example, in an embodiment of the present application, the text data includes the frequency distribution of error codes, alarm information, operation logs, and the screenshot images include the status of interface elements, operating parameters, and abnormal colors, which can also be used as nodes.

[0085] Construct a hyper - edge connection matrix, and define hyper - edges according to the relevance of events. For example, connect the text, image, and time - series feature nodes related to a certain event together.

[0086] Use a multi - layer GCN (Graph Convolutional Network) to process the graph structure, thereby obtaining hyper - graph features.

[0087] In this embodiment, after splicing the intermediate fusion features and hyper - graph features, further fuse them through a fusion fully - connected layer to form the final fusion features.

[0088] In this embodiment, the fault type task module uses a fully - connected layer and a Softmax activation function to output the probability distribution of the device fault type, and selects the one with the highest probability as the first device fault type information;

[0089] The remaining life prediction task module uses a fully - connected layer and a linear activation function to output the continuous value of the device's remaining life.

[0090] The maintenance priority module uses a fully - connected layer and a Sigmoid activation function to output the score of the maintenance priority (between 0 and 1).

[0091] In this embodiment, the complex multi - modal multi - task hybrid neural network uses a multi - task learning framework to simultaneously optimize the loss functions of the three tasks, and adopts a weighted summation method to balance the loss contributions of different tasks, ensuring that the model can achieve good performance on multiple tasks.

[0092] It can be understood that the training method of the complex multi - modal multi - task hybrid neural network can adopt the existing training method, which will not be elaborated here. For example, set the training set, test set, etc., and set appropriate hyper - parameters such as the learning rate and momentum.

[0093] In this embodiment, the complex multi - modal multi - task hybrid neural network can use an Adam optimizer or an SGD optimizer during the training process, combined with a learning rate scheduling strategy (such as cosine annealing or learning rate warm - up) to accelerate the convergence of the model.

[0094] In this embodiment, evaluating the performance of the model on the test set also belongs to the prior art and will not be elaborated here. For example, evaluation metrics such as accuracy, recall, F1-score (classification task), mean squared error (regression task), etc. are calculated.

[0095] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a device fault type information task loss function, where the device fault type information task loss function adopts the following formula:

[0096] ;

[0097] Where is the device fault type information task loss function, M is the total number of samples, is the true class label of sample a, is the class sample frequency of, is the frequency adjustment parameter, is the sample a is predicted to be the class probability of, is the probability that sample a is predicted to be class c, is a hyperparameter that controls the intensity of non-target class suppression.

[0098] In this embodiment, by introducing a class frequency dynamic adjustment mechanism, the deficiency of the traditional loss function in the class imbalance problem is effectively solved. Its core advantage lies in being able to dynamically adjust the loss weights according to the sample frequencies of each class, enabling samples of rare classes to receive higher attention during the training process.

[0099] Specifically, the term in the loss function amplifies the loss contribution of rare class samples through the exponential decay of the class frequency , thus alleviating the model bias problem caused by class imbalance. In addition, the clarity of the classification boundary is enhanced by suppressing the probability of non-target classes, reducing the misclassification of uncertain samples by the model. This mechanism not only improves the model's ability to identify rare classes but also enhances the overall classification accuracy. In practical applications, it can significantly improve the performance of device fault type classification, especially in scenarios where the fault type distribution is uneven, and its advantages are particularly obvious. In addition, the design of this loss function has high flexibility and scalability, and can adapt to different data distributions and task requirements by adjusting the frequency adjustment parameter and the non-target class suppression parameter μ.

[0100] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a loss function for the equipment remaining life prediction information task, where the loss function for the equipment remaining life prediction information task adopts the following formula:

[0101] ;

[0102] where, is the loss function for the equipment remaining life prediction information task, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, is a constant to prevent division by zero, is a tuning parameter that controls the influence of time deviation on error scaling, is the timestamp of sample a, is the average of the timestamps of all samples, is a hyperparameter, is the rate of change of the predicted remaining life over time, is the rate of change of the true remaining life over time.

[0103] By introducing a time-dependent error scaling mechanism, the performance of the remaining life prediction task is significantly improved. Its advantage lies in being able to dynamically adjust the loss weights according to the time distribution of the prediction errors, making the model pay more attention to the prediction accuracy of recent data. Specifically, the term magnifies the error contribution of recent samples through the exponential function of the time deviation , thus emphasizing the importance of recent data in model training. In addition, by introducing a trend consistency penalty , the consistency between the prediction trend and the actual trend is ensured, enabling the model to capture the changing pattern of the equipment remaining life over time. This mechanism not only improves the model's prediction ability for recent data but also enhances the model's adaptability to time series data. In practical applications, it can significantly improve the accuracy of equipment remaining life prediction, especially in scenarios where the data changes greatly over time, and its advantages are particularly obvious. In addition, the design of this loss function has high robustness and interpretability, and can be adjusted by the time adjustment parameter β and the trend penalty parameter ν to adapt to different data characteristics and task requirements.

[0104] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a loss function for the maintenance priority ranking information task, where the loss function for the maintenance priority ranking information task adopts the following formula:

[0105] ;

[0106] Among them, is the task loss function for maintenance priority ranking information, S is the set of positive sample pairs , D is the set of negative sample pairs , is the maintenance priority score of sample a, is the maintenance priority score of sample b, is the maintenance priority score of sample c, is the maintenance priority score of sample d, is the dynamic margin of the positive sample pair (a, b). In this embodiment, ; is the dynamic margin of the negative sample pair (c, d). In this embodiment, ; is the basic margin; is the adjustment parameter, controlling the influence of feature similarity on the margin; is the feature similarity between samples a and b; is the feature similarity between samples c and d; is the feature vector of sample a; is the feature vector of sample b; is the feature vector of sample c; is the feature vector of sample d.

[0107] By introducing a dynamic margin adjustment mechanism, the performance of the maintenance priority ranking task is significantly improved. It can dynamically adjust the ranking margin according to the feature similarity of sample pairs, enabling the model to handle the ranking relationship more delicately. Specifically, in the loss function, term sets a smaller margin for similar sample pairs and a larger margin for dissimilar sample pairs through a linear function of the feature similarity , thus enhancing the robustness and accuracy of the ranking.

[0108] In addition, the consistency of the ranking relationship is ensured through two-way ranking constraints (the loss terms of positive and negative sample pairs), enabling the model to optimize the ranking performance of both positive and negative sample pairs simultaneously. This mechanism not only improves the model's ability to capture subtle ranking differences but also enhances the model's adaptability to complex ranking tasks. In practical applications, it can significantly improve the accuracy of maintenance priority ranking.

[0109] In this embodiment, the complex multi-modal multi-task hybrid neural network includes a total loss function. Among them, the total loss function adopts the following formula:

[0110] ;

[0111] Among them, is the task loss function for device failure type information, is the task loss function for equipment remaining life prediction information, is the task loss function for maintenance priority ranking information, is the loss weight for the fault type classification task, is the loss weight for the remaining life prediction task, is the loss weight for the maintenance priority ranking task.

[0112] Each loss function and the total loss function of this application significantly improve the performance of the model in equipment operation and maintenance management tasks by introducing new strategies and mechanisms. The device failure type information loss function effectively solves the problem of class imbalance and improves the accuracy of fault type classification; the equipment remaining life prediction information task loss function enhances the model's ability to capture recent data and trend changes and improves the accuracy of remaining life prediction; the maintenance priority ranking information task loss function refines the ranking relationship and improves the robustness of maintenance priority ranking. These advantages make the modified loss function have significant advantages in practical applications and can meet the requirements of complex equipment operation and maintenance management scenarios.

[0113] In this embodiment, the text data, screen capture images, captured images, and time series data of the device to be monitored are input into a trained random forest model for fault types to obtain the second device fault type information as follows:

[0114] Extract features from the input data (text data, screen capture images, captured images, and time series data). For text data, the bag-of-words model or TF-IDF method can be used to extract features; for image data, a convolutional neural network can be used to extract features; for time series data, statistical features (such as mean, variance, peak value, etc.) can be used as features.

[0115] Use the CART (Classification and Regression Tree) algorithm to construct decision trees. At each node, select the optimal feature and split point to divide the data set into two subsets so that the data within the subsets belongs to the same category as much as possible. Recursively perform splitting until the stopping condition is met (such as the number of samples in the node is less than a certain threshold, the purity of the node reaches a certain requirement, etc.).

[0116] Construct multiple decision trees, and each decision tree is trained using different random subsets. The random subsets include randomly selected feature subsets and sample subsets randomly drawn from the original data set (sampling with replacement).

[0117] Each decision tree classifies and predicts the input data, and the final prediction result is determined by the voting results of all decision trees.

[0118] In this embodiment, the labeled fault type data can be used to train the random forest model, and the parameters of the model, such as the number of decision trees, the maximum depth, etc., can be adjusted to improve the classification accuracy of the model.

[0119] In this embodiment, in the prediction of the remaining life prediction information of the second device, the remaining life support vector regression model can find an optimal hyperplane to minimize the error between the input time series data and the remaining life of the device.

[0120] In this embodiment, the remaining life data, time series data, and text data of the device can be used to train the support vector regression model, and the appropriate kernel function and parameters can be selected to optimize the regression performance of the model.

[0121] In this embodiment, the text data includes the log files of the device, which can include various log information, such as operation information, alarm information, maintenance information, and other information. It can be understood that in other embodiments, other information can also be included.

[0122] The screenshot image refers to the screenshot during the operation of the device, and the captured image refers to the external image of the device.

[0123] In this embodiment, the time series data can include CPU usage rate, memory occupancy, disk I / O speed, network traffic, working temperature of device components, stability indicators of power supply, vibration, and noise level. It can be understood that in other embodiments, other information can also be included.

[0124] In this embodiment, the final device fault type information is obtained through the first device fault type information and the second device fault type information. For example, when the first device fault type information and the second device fault type information are the same, either one can be used as the standard. Or rules can be set. When the voting results of all decision trees exceed a threshold (such as 90%), the second device fault type information is used as the standard. It can be understood that other methods can also be used to determine.

[0125] In this embodiment, the final remaining life prediction information is obtained through the first device remaining life prediction information and the second device remaining life prediction information, which is similar to the determination of the above device fault type information. For example, when the first device remaining life prediction information and the second device remaining life prediction information are the same, either one can be used as the standard. Or it can be obtained through a formula. For example, it can be obtained through the following formula:

[0126] ; where

[0127] α is a weight coefficient, and the optimal value can be determined by methods such as cross-validation, where 0 < α < 1; is the final remaining life prediction information; is the remaining life prediction information of the first device; is the remaining life prediction information of the second device.

[0128] For example, for a certain router, the remaining life prediction information of the first device is 30 hours, the remaining life prediction information of the second device is 50 hours, and α is 0.4, then (0.4 × 30) + (0.6 × 50) = 42 hours.

[0129] This application also provides an intelligent computer room operation and maintenance management device. The intelligent computer room operation and maintenance management device includes an initial data acquisition module, a neural network acquisition module, a model acquisition module, a first prediction result acquisition module, a second prediction result acquisition module, and a final device failure type information acquisition module. Among them,

[0130] The initial data acquisition module is used to acquire the text data, screen capture images, captured images, and time series data of the device to be monitored;

[0131] The neural network acquisition module is used to acquire a trained complex multi-modal multi-task hybrid neural network;

[0132] The model acquisition module is used to respectively acquire a trained random forest model for failure types and a trained support vector regression model for remaining life;

[0133] The first prediction result acquisition module is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain the first device failure type information, the first device remaining life prediction information, and the first maintenance priority ranking information;

[0134] The second prediction result acquisition module is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the trained random forest model for failure types to obtain the second device failure type information; input the time series data and text data into the trained support vector regression model for remaining life to obtain the second device remaining life prediction information;

[0135] The final device failure type information acquisition module is used to obtain the final device failure type information through the first device failure type information and the second device failure type information;

[0136] The final device failure type information is used to obtain the final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information

[0137] In addition, it is obvious that the term "including" does not exclude other units or steps. A plurality of units, modules or devices stated in the apparatus claims may also be implemented by one unit or a general apparatus through software or hardware.

[0138] Although the present invention has been described in detail with general descriptions and specific embodiments above, some modifications or improvements can be made based on the present invention, which are obvious to those skilled in the art. Therefore, these modifications or improvements made without departing from the spirit of the present invention all fall within the scope of protection required by the present invention.

Claims

1. An intelligent computer room operation and maintenance management method, characterized in that, The intelligent computer room operation and maintenance management method includes: Obtaining text data, screenshot images, captured images, and time series data of the device to be monitored; wherein, the time series data includes CPU usage rate, memory occupancy rate, disk I / O speed, network traffic, working temperature of device components, stability index of power supply, vibration, and noise level; Obtaining a trained complex multi-modal multi-task hybrid neural network; Obtaining a trained random forest model for fault types and a trained support vector regression model for remaining life; Inputting the text data, screenshot images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network to obtain first device fault type information, first device remaining life prediction information, and first maintenance priority ranking information; Inputting the text data, screenshot images, captured images, and time series data of the device to be monitored into the trained random forest model for fault types to obtain second device fault type information; Inputting the time series data and text data into the trained support vector regression model for remaining life to obtain second device remaining life prediction information; Obtaining final device fault type information through the first device fault type information and the second device fault type information; Obtaining final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information; The complex multi-modal multi-task hybrid neural network includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer; The feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; wherein, the text feature extraction layer is used to extract text features of text data; the image feature extraction layer is used to extract image features of screenshot images and captured images; the time series feature extraction layer is used to extract time series features of time series data; The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully connected layer. Among them, the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully connected layer is used to fuse the intermediate fusion features and the hypergraph features to form final fusion features; The multi-task learning layer includes a shared layer and task layers. The shared layer is used to receive the final fused feature and extract the general feature representation of the final fused feature. The task layers include a fault type task module, a remaining life prediction task module, and a maintenance priority module. The fault type task module is used to output the first device fault type information according to the general feature representation. The remaining life prediction task module is used to output the first device remaining life prediction information according to the general feature representation. The maintenance priority module is used to output the first maintenance priority ranking information according to the general feature representation.

2. The intelligent computer room operation and maintenance management method according to claim 1, wherein, The intelligent computer room operation and maintenance management method further includes: Training the complex multi-modal multi-task hybrid neural network; Training the random forest model for fault types and the support vector regression model for remaining life.

3. The intelligent computer room operation and maintenance management method according to claim 2, characterized in that, The training of the complex multi-modal multi-task hybrid neural network includes: Obtaining a training set, which includes the text data of each device in the computer room, the captured images transmitted by each camera device in the computer room, the screen capture images intercepted by the screen capture tool, and the time series data of each device; Constructing a complex multi-modal multi-task hybrid neural network; Training the complex multi-modal multi-task hybrid neural network with the training set.

4. The intelligent computer room operation and maintenance management method according to claim 3, characterized in that, The complex multi-modal multi-task hybrid neural network includes a task loss function for device fault type information, where the task loss function for device fault type information adopts the following formula: ； Among them, is the task loss function of device fault type information, M is the total number of samples, is the true class label of sample a, is the class sample frequency of, is the frequency adjustment parameter, is the sample a is predicted to be the class probability of, is the probability that sample a is predicted to be class c, is a hyperparameter that controls the intensity of non-target class suppression.

5. The intelligent computer room operation and maintenance management method according to claim 4, wherein, The complex multi-modal multi-task hybrid neural network includes a task loss function for device remaining life prediction information, where the task loss function for device remaining life prediction information adopts the following formula: ； Among them, is the task loss function of the equipment remaining life prediction information, M is the total number of samples, is the true remaining life value of sample a, is the predicted remaining life value of sample a, is a constant to prevent division by zero, is a tuning parameter to control the impact of time deviation on error scaling, is the timestamp of sample a, the average of all sample timestamps, a hyperparameter, the rate of change of the predicted remaining life over time, and the rate of change of the true remaining life over time.

6. The intelligent computer room operation and maintenance management method according to claim 5, wherein, The complex multi-modal multi-task hybrid neural network includes a task loss function for maintenance priority ranking information, where the task loss function for maintenance priority ranking information adopts the following formula: ； Among them, is the task loss function for sorting the maintenance priority information, S is the set of positive sample pairs, D is the set of negative sample pairs, is the maintenance priority score of sample a, is the maintenance priority score of sample b, is the maintenance priority score of sample c, is the maintenance priority score of sample d. Among them, > , < , is the dynamic margin of the positive sample pair (a, b), is the dynamic margin of the negative sample pair (c, d).

7. The intelligent computer room operation and maintenance management method according to claim 6, characterized in that, The complex multi-modal multi-task hybrid neural network includes a total loss function, where the total loss function adopts the following formula: ； Among them, is the task loss function of equipment failure type information, is the task loss function of equipment remaining life prediction information, is the task loss function of maintenance priority ranking information, is the loss weight of the fault type classification task, is the loss weight of the remaining life prediction task, is the loss weight of the maintenance priority ranking task.

8. An intelligent computer room operation and maintenance management device, characterized in that, The intelligent computer room operation and maintenance management device includes: An initial data acquisition module, which is used to acquire the text data, screen capture images, captured images, and time series data of the device to be monitored. Among them, the time series data includes CPU usage rate, memory occupancy rate, disk I / O speed, network traffic, working temperature of device components, stability index of power supply, vibration, and noise level; A neural network acquisition module, which is used to acquire the trained complex multi-modal multi-task hybrid neural network; A model acquisition module, which is used to respectively acquire the trained random forest model for fault types and the trained support vector regression model for remaining life; A first prediction result acquisition module, which is used to input the text data, screen capture images, captured images, and time series data of the device to be monitored into the complex multi-modal multi-task hybrid neural network, so as to obtain the first device fault type information, the first device remaining life prediction information, and the first maintenance priority ranking information; The second prediction result acquisition module is configured to input the text data, screen capture images, captured images, and time series data of the device to be monitored into a trained random forest model for fault types to obtain second device fault type information; input the time series data and text data into a trained remaining life support vector regression model to obtain second device remaining life prediction information; The final device fault type information acquisition module is configured to obtain final device fault type information through the first device fault type information and the second device fault type information; The final device fault type information acquisition module, the final device fault type information is used to obtain final remaining life prediction information through the first device remaining life prediction information and the second device remaining life prediction information; The complex multi-modal multi-task hybrid neural network includes a feature extraction layer, an interactive fusion layer, and a multi-task learning layer; The feature extraction layer includes a text feature extraction layer, an image feature extraction layer, and a time series feature extraction layer; among them, the text feature extraction layer is used to extract text features of text data; the image feature extraction layer is used to extract image features of screen capture images and captured images; the time series feature extraction layer is used to extract time series features of time series data; The interactive fusion layer includes a primary fusion layer, an intermediate fusion layer, a hypergraph fusion layer, and a fusion fully connected layer. Among them, the primary fusion layer is used to fuse text features, image features, and time series features into primary features; the intermediate fusion layer performs self-attention calculation on the primary features through an inter-modal attention mechanism to obtain intermediate fusion features; the hypergraph fusion layer splits the intermediate fusion features into multiple sub-features and constructs hypergraph features through the sub-features; the fusion fully connected layer is used to fuse the intermediate fusion features and the hypergraph features to form final fusion features; The multi-task learning layer includes a shared layer and a task layer. The shared layer is used to receive the final fusion features and extract the general feature representation of the final fusion features; the task layer includes a fault type task module, a remaining life prediction task module, and a maintenance priority module. The fault type task module is used to output first device fault type information according to the general feature representation; the remaining life prediction task module is used to output first device remaining life prediction information according to the general feature representation; the maintenance priority module is used to output first maintenance priority ranking information according to the general feature representation.

Images