Machine room state data processing method and device based on artificial intelligence and storage medium

Abstract

The invention relates to the technical field of artificial intelligence, and provides a machine room state data processing method based on artificial intelligence, and the method comprises the steps: carrying out the preprocessing of multi-modal data and time series data of a related machine room, and obtaining the preprocessed multi-modal data and the preprocessed time series data; feature extraction is carried out on the preprocessed multi-modal data and the preprocessed time series data based on an artificial intelligence algorithm, and multi-modal data features and time series data features are obtained; performing feature alignment on the multi-modal data features and the time series data features; based on the multi-modal data features and the time series data features after feature alignment, the preprocessed multi-modal data and the preprocessed time series data are fused to obtain fused features; and performing data analysis according to the fusion features to determine the current state of the machine room. According to the technical scheme, the accuracy of judging the safety event of the machine room can be improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an artificial intelligence-based computer room status data processing method, device and storage medium. Background Art

[0002] With the rapid development of big data and artificial intelligence, the processing and analysis of multimodal data and time series data have become increasingly important. Multimodal data usually includes multiple types of data such as images, audio, and text, while time series data emphasizes the changes of data over time.

[0003] In traditional computer room operation and maintenance scenarios, multimodal data mainly refers to video data and audio data in the computer room, while time series data mainly refers to status monitoring data obtained by various sensors in the computer room. By analyzing the multimodal data and time series data of the computer room, we can intuitively understand the changing trends of the computer room environment and equipment status. Existing data analysis methods often separate these two types of data, and the two are not linked. That is, multimodal data such as video and audio are usually only used for post-event review, and time series data is only used for analysis when the sensor is working normally, but often due to sensor failure or abnormality, it leads to wrong judgments on the equipment status. Summary of the invention

[0004] The present application provides a computer room status data processing method, device and storage medium based on artificial intelligence, which can improve the accuracy of judging computer room security events.

[0005] On the one hand, the present application provides a method for processing computer room status data based on artificial intelligence, the method comprising: Preprocessing the multimodal data and time series data of the relevant computer room respectively, and obtaining preprocessed multimodal data and preprocessed time series data respectively; Based on the artificial intelligence algorithm, feature extraction is performed on the preprocessed multimodal data and the preprocessed time series data to obtain multimodal data features and time series data features respectively; Performing feature alignment on the multimodal data features and the time series data features; Based on the feature-aligned multimodal data features and time series data features, the preprocessed multimodal data and the preprocessed time series data are fused to obtain fused features; Data analysis is performed based on the fusion features to determine the current status of the computer room.

[0006] On the other hand, the present application provides a computer room status data processing device based on artificial intelligence, the device comprising: A preprocessing module, used to preprocess the multimodal data and time series data of the relevant computer room, respectively, to obtain preprocessed multimodal data and preprocessed time series data; A feature extraction module, used to extract features from the preprocessed multimodal data and the preprocessed time series data based on an artificial intelligence algorithm, to obtain multimodal data features and time series data features respectively; An alignment module, used for aligning the multimodal data features with the time series data features; A fusion module, used for fusing the preprocessed multimodal data and the preprocessed time series data to obtain fusion features based on the multimodal data features and time series data features after feature alignment; The determination module is used to perform data analysis based on the fusion features to determine the current status of the computer room.

[0007] In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the technical solution of the above-mentioned method for processing computer room status data based on artificial intelligence when executing the computer program.

[0008] In a fourth aspect, the present application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the technical solution of the above-mentioned method for processing computer room status data based on artificial intelligence.

[0009] From the technical solution provided by the present application, it can be seen that after extracting features from the pre-processed multimodal data and the pre-processed time series data based on the artificial intelligence algorithm, and obtaining the multimodal data features and the time series data features respectively, the multimodal data features and the time series data features are feature aligned, and based on the multimodal data features and the time series data features after feature alignment, the pre-processed multimodal data and the pre-processed time series data are fused to obtain fused features. Compared with the existing data analysis methods that often separate these two types of data and do not link the two, the technical solution of the present application improves the accuracy of the judgment of computer room security events through the fusion of data features, reduces false alarms and omissions caused by a single data source, and provides a more reliable decision-making basis for operation and maintenance personnel. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0011] Figure 1 It is a flow chart of a method for processing computer room status data based on artificial intelligence provided by an embodiment of the present application; Figure 2 It is a structural schematic diagram of a computer room status data processing device based on artificial intelligence provided in an embodiment of the present application; Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0013] In this specification, adjectives such as first and second may be used only to distinguish one element or action from another element or action, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be interpreted as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0014] In this specification, for the convenience of description, the sizes of various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0015] In traditional computer room operation and maintenance scenarios, multimodal data mainly refers to video data and audio data in the computer room, while time series data mainly refers to status monitoring data obtained by various sensors in the computer room. By analyzing the multimodal data and time series data of the computer room, we can intuitively understand the changing trends of the computer room environment and equipment status. Existing data analysis methods often separate these two types of data, and the two are not linked. That is, multimodal data such as video and audio are usually only used for post-event review, and time series data is only used for analysis when the sensor is working normally, but often due to sensor failure or abnormality, it leads to wrong judgments on the equipment status.

[0016] In view of the above problems in the prior art, this application proposes a computer room status data processing method based on artificial intelligence, the flow chart of which is shown in the attached Figure 1 As shown, it mainly includes steps S101 to S105, which are described in detail as follows: Step S101: preprocess the multimodal data and time series data of the relevant computer room respectively to obtain preprocessed multimodal data and preprocessed time series data respectively.

[0017] In the embodiment of the present application, the multimodal data related to the computer room includes data such as video, image or audio about the computer room workplace, and the time series data related to the computer room includes data generated by some sensors (for example, temperature sensors, smoke sensors, monitoring equipment, etc.) in the computer room workplace. Preprocessing the multimodal data and time series data related to the computer room mainly includes denoising and unifying the scale of these data, for example, adjusting the multimodal data and time series data to conform to the standard normal distribution and compressing them to a specific range by means of standardization and normalization, etc.

[0018] Step S102: Based on the artificial intelligence algorithm, feature extraction is performed on the preprocessed multimodal data and the preprocessed time series data to obtain multimodal data features and time series data features respectively.

[0019] If the preprocessed multimodal data is a preprocessed image, then as an embodiment of the present application, feature extraction is performed on the preprocessed multimodal data and the preprocessed time series data based on an artificial intelligence algorithm, and obtaining multimodal data features and time series data features respectively can be achieved through the following steps S1021 to S1023, which are described in detail as follows: Step S1021: extracting features of the preprocessed image based on a deep convolutional neural network to obtain an image data feature map.

[0020] The deep convolutional neural network can be a model such as VGG16, ResNet or EfficientNet. After training, these models use their multiple convolutional layers and pooling layers to perform operations such as convolution and pooling to extract high-dimensional features of the preprocessed image and obtain image data feature maps.

[0021] Step S1022: Slide the sliding window on the image data feature map to obtain multiple targets on the preprocessed image.

[0022] In a complex environment such as a computer room, the target detection model can effectively identify and locate key objects in the image (such as equipment, indicator lights, alarms, etc.). Traditional convolutional neural networks are no longer sufficient for a single classification task because it is necessary to detect multiple targets in the image and accurately calibrate their positions. Therefore, in an embodiment of the present application, after obtaining the image data feature map, a sliding window can be slid on the image data feature map to obtain multiple targets on the preprocessed image. As an embodiment of the present application, obtaining multiple targets on the preprocessed image by sliding a sliding window on the image data feature map can be achieved through steps S1 to S3, as described in detail as follows: Step S1: Generate candidate boxes for multiple targets by sliding the sliding window across the image data feature map.

[0023] The sliding window slides along each position (i.e. each pixel position) of the feature map, and multiple candidate boxes (anchor boxes) are generated at each position. These candidate boxes have different scales and aspect ratios, designed to accommodate targets of different sizes and shapes. Assume that the size of the sliding window is (generally ), each time it slides, a window is generated on the feature map. The window size usually corresponds to the receptive field of the convolutional layer. Each window position generates multiple candidate boxes of different scales and aspect ratios. These candidate boxes are called anchor boxes. Specifically, suppose that at a certain position in the feature map Slide a window and generate K anchor boxes at that position. The size and ratio of each anchor box can be determined by a predefined scale and aspect ratio. For example, 3 scales can be defined and 3 aspect ratios , then generate at each location Candidate boxes for a target.

[0024] Step S2: Classify and regress the candidate boxes of multiple targets to obtain the classification score and regression value of each target.

[0025] At each sliding window position, the classification score and bounding box regression value of each anchor box need to be calculated. This can be achieved by performing classification and bounding box regression operations on candidate boxes of multiple targets. The classification operation is to determine whether each candidate box contains the target, which is usually represented by a binary classification (background / foreground) model. , output a classification score , indicating the probability that the anchor box contains the target. For the kth anchor box, the classification score Indicates the probability that the box is a foreground:

[0026] in, is the feature extracted from the convolutional layer, and are the parameters of the classification layer, is the sigmoid function, which is used to map the output to a probability value.

[0027] The bounding box regression operation is used to adjust the position of each anchor box so that it fits the true bounding box of the target more accurately. , output a regression value , indicating the amount of adjustment required for the anchor frame, including: the offset of the center coordinates and size adjustment For the kth anchor box, the bounding box regressor It is calculated by the following regression function:

[0028] in, and are the parameters of the regression layer, is the convolution feature, Represents the amount of adjustment required to generate the target box from the anchor box.

[0029] Step S3: Use the non-maximum suppression algorithm to remove duplicate candidate boxes in the candidate boxes to obtain the category and position of the target in the preprocessed image.

[0030] Specifically, by calculating the intersection over union (IoU) between each pair of candidate boxes, we remove boxes with high overlap and retain the box with the maximum score. The calculation formula for the intersection over union is:

[0031] in, and There are two candidate boxes. represents the area of ​​the box, Represents the ratio of the overlapping area of ​​two boxes to the union area.

[0032] Step S1023: Use the LSTM model to extract features from the preprocessed time series data to obtain time series data features.

[0033] Specifically, the LSTM model is used to extract features from the preprocessed time series data, and the features of the time series data can be: input the preprocessed time series data into the LSTM (Long Short-Term Memory) model and initialize the state of the LSTM model; at each time step, calculate the forget gate, input gate, candidate memory unit, update memory unit and output gate of the LSTM model, and calculate the hidden state at the current moment; according to the task requirements, use the hidden state at the final moment or the entire hidden state sequence as the time series data feature.

[0034] If the preprocessed multimodal data is a preprocessed video stream, then as another embodiment of the present application, feature extraction is performed on the preprocessed multimodal data and the preprocessed time series data based on an artificial intelligence algorithm, and obtaining multimodal data features and time series data features respectively can be achieved through the following steps S'1021 to S'1025, which are described in detail as follows: Step S'1021: Perform a three-dimensional convolution operation on the preprocessed video stream, wherein the convolution kernel of the three-dimensional convolution is set to , in the time dimension Frame is the sliding step.

[0035] It should be noted that the video stream can be regarded as a four-dimensional tensor, and the convolution kernel size of the three-dimensional convolution is , where the spatial dimension Used to extract spatial features of a single video frame of a video stream (such as device outline, indicator light color, etc.), temporal dimension Used to capture dynamic changes across multiple consecutive frames (such as fan speed changes, temperature diffusion trends, etc.). The sliding stride (Stride) in the 3D convolution operation is equal to the time dimension of the convolution kernel size. , which means that after each processing After a frame, the convolution kernel slides to the next position on the time axis.

[0036] In convolution operations (two-dimensional or three-dimensional convolution operations), the selection of convolution kernel size is closely related to factors such as computational efficiency, receptive field, and task adaptability. Specifically, larger convolution kernels (such as 5×5) will increase the number of parameters and computational cost, and easily lead to overfitting. Conversely, smaller convolution kernels (such as 1×1) are mainly used for channel dimension adjustment and cannot effectively extract spatial features. In feature extraction tasks, edge continuity and local texture details are key. Convolution kernels of appropriate size can effectively capture pixel correlations within local neighborhoods (such as edge direction, color gradients) while avoiding blurring problems introduced by large convolution kernels. Based on the considerations of balancing computational efficiency and receptive field and adapting to the feature extraction of this application, the above convolution kernel size is The spatial dimension Specifically, it can be 3×3, and the time dimension It can be taken as 5. The three-dimensional convolution operation performed on the preprocessed video stream can be expressed as:

[0037]

[0038] Among them, W is the convolution kernel weight, is the video frame of the preprocessed video stream, b is the bias term, c is the output channel index, is the output of the 3D convolution operation.

[0039] Step S'1022: The output of the three-dimensional convolution operation is activated through the LeakyReLU activation function to generate an activated feature map.

[0040] Since the traditional activation function ReLU outputs 0 when the input is <0, the gradient in the negative area is 0, and the back propagation cannot update the parameters, that is, the gradient disappears. Therefore, in order to solve the above-mentioned gradient disappearance problem, the output of the three-dimensional convolution operation is passed through the LeakyReLU activation function in the embodiment of the present application to generate an activated feature map, wherein the leakage factor of the LeakyReLU activation function is set to , the expression of the LeakyReLU activation function is:

[0041] The above leakage factor The negative activation ratio of the current batch can be determined based on the relationship between the negative activation ratio and the preset threshold. Specifically:

[0042] in, is the basic leakage factor, which can be taken as 0.2. is the Sigmoid function, N is the batch size, The above expression shows that if the negative activation ratio of the current batch is greater than the preset threshold, then (in, is a negative activation ratio), otherwise, Equal to the basic leakage factor .

[0043] Step S'1023: Perform global average pooling on the activated feature map and calculate the mean statistics of each channel.

[0044] Specifically, global average pooling is performed on the activated feature map to calculate the mean statistics of each channel for:

[0045] in, is the size of the feature map after activation, For the cth channel at position The characteristic value of .

[0046] Step S'1024: Set the mean statistic Input Softmax function to generate normalized attention weights .

[0047] Specifically, ,and

[0048] in, is the total number of channels of the feature map after activation, .

[0049] Step S'1025: Press the formula The activated feature map is channel-weighted to obtain the enhanced feature map.

[0050] Step S103: aligning the multimodal data features and the time series data features.

[0051] The features of modalities not only differ in "data form" (for example, images are two-dimensional data, while time series data has time dependence), but also in their representation methods and abstraction levels. For example, images are represented by pixel values ​​and color spaces, mainly capturing spatial structural information, texts are represented by words and syntactic structures in the vocabulary, and mainly focus on the grammatical and semantic information of the language, while time series data has continuity and time dependence in the time dimension, and mainly captures changes and trends in time series. Feature alignment helps the features of different modalities to match each other in the same space, so that the information of each modality can complement each other and work together. Therefore, after steps S101 and S102, it is necessary to align the multimodal data features and the time series data features. As an embodiment of the present application, feature alignment of multimodal data features and time series data features can be achieved through steps S1031 to S1033, as described in detail as follows: Step S1031: Map the multimodal data features and the time series data features to a semantic space of the same dimension.

[0052] Assumptions Represents multimodal data features (e.g., images, video streams, etc.), Represents the time series data features, then the multimodal data features and time series data features can be mapped to the semantic space of the same dimension according to the following formula:

[0053] in, and is the weight matrix, and is the bias term, and They are the multimodal data features and time series data features after mapping.

[0054] Step S1032: Calculate the similarity between the multimodal data features and the time series data features in the semantic space of the same dimension.

[0055] In an embodiment of the present application, the similarity between the multimodal data features and the time series data features in the semantic space of the same dimension may be the cosine similarity.

[0056] Step S1033: Adjust weights and biases through an optimization algorithm to minimize the distance between multimodal data features and time series data features.

[0057] Specifically, adjusting the weights and biases through the optimization algorithm to minimize the distance between the multimodal data features and the time series data features can be performed through steps S10331 to S10333, which are described in detail as follows: Step S10331: Construct a loss function that measures the degree of alignment between different modal features.

[0058] In order to maximize the similarity between features of different modalities, the goal is to make similar sample pairs (such as matching images and time series data) have high similarity, while different sample pairs (such as unmatched images and time series data) have low similarity. Therefore, the loss function that measures the degree of alignment between features of different modalities is It can be defined by contrastive loss as follows:

[0059] in, is a label, indicating whether the multimodal data features and time series data features match. Indicates matching, Indicates mismatch. is the cosine similarity between multimodal data features and time series data features, Is a threshold. When the sample pair is mismatched, the similarity cannot exceed this threshold. Usually The value is a constant, indicating that when the similarity of unmatched samples is lower than a certain value, they are considered "unmatched".

[0060] from From the expression of , we can know that: , we hope to minimize the difference in similarity, that is, to make their cosine similarity as close to 1 as possible; when , we hope that the cosine similarity is less than , i.e., maximizing the difference in similarity, ensuring that they are correctly distinguished.

[0061] Step S10332: Optimize weights and biases using a gradient descent algorithm.

[0062] Optimizing the weight matrix and and bias and The goal is to minimize the loss function , as follows: S1: Calculate the weight matrix and and bias and gradient.

[0063] The core step of gradient descent is to calculate the gradient of the loss function with respect to the model parameters (weights and biases). If the selected optimization algorithm is stochastic gradient descent (SGD), the parameter update rule is:

[0064] in, is the learning rate, which controls the step size of parameter updates, and is the loss function number The gradient of the weights is and is the loss function Gradient with respect to bias.

[0065] S2: The specific form of calculating the gradient.

[0066] In order to calculate the gradient, the loss function needs to be About parameters , , and Find the partial derivative, specifically:

[0067] S3: Update parameters , , and .

[0068] Once the gradient is calculated, the gradient descent algorithm can be used to update , , and Through the back propagation algorithm, the weights and biases can be adjusted in each iteration so that the loss function Minimize and thus optimize semantic alignment.

[0069] Step S10333: iterative training.

[0070] Repeat the above steps S10331 to S10333, calculate the gradient and update the parameters in each training iteration until the loss function converges to an optimal value.

[0071] As another embodiment of the present application, feature alignment of multimodal data features and time series data features can be achieved through steps S'1031 to S'1033, which are described in detail as follows: Step S'1031: perform dimensionality compression on multimodal data features and time series data features.

[0072] Since there are great differences between multimodal data (such as video streams, images, etc.) and time series data (such as sensor data), the dimensions of multimodal data features and time series data features may be different, or even have great differences. For example, multimodal data features may be multimodal features such as shape, color or texture features with a dimension of 256 for video stream data, while time series data features may be sensor time series features such as temperature, humidity or voltage with a dimension of 64. Considering that the premise of the subsequent calculation of the dynamic time-warping curved path of the multimodal feature sequence and the time series feature sequence is that the dimensions of the multimodal data features and the time series data features need to be the same, dimensional compression can be performed on the multimodal data features and the time series data features. Specifically, PCA dimensionality reduction can be performed on the multimodal features, retaining the principal components whose variance contribution rate exceeds a preset threshold, such as 85%, and performing autoencoder compression on the time series data features so that the compressed dimension is equal to the principal component dimension of the multimodal data features.

[0073] Step S'1032: Calculate the dynamic time warping curved path of the multimodal feature sequence and the temporal feature sequence to obtain the optimal curved path.

[0074] Assume that the multimodal feature sequence is , the time series feature sequence is , calculate the dynamic time warping curved path of the multimodal feature sequence and the temporal feature sequence, and get the optimal curved path according to the formula:

[0075] Finding the Optimal Curved Path , so that the path cumulative distance is minimized, where is the feature distance metric (e.g. Euclidean distance).

[0076] Step S'1033: perform cubic spline interpolation on the shorter sequences of multimodal data features and time series data features with the same principal component dimension along the optimal curved path, wherein the interpolation point density of the cubic spline interpolation is dynamically adjusted according to the difference in sampling frequencies of the two sequences, and the adjustment ratio is the inverse of the frequency difference.

[0077] Assuming multimodal feature sequence and time series feature sequences middle, , then interpolate the multimodal feature sequence, otherwise, interpolate the time series feature sequence. If the sampling frequency of the multimodal data is , the time series data is , then the interpolation point density of cubic spline interpolation can be calculated according to the following formula:

[0078]

[0079] in, To prevent the smoothing coefficient from dividing by zero, the actual number of interpolation points , L is the original length of the interpolated sequence.

[0080] As another embodiment of the present application, feature alignment of multimodal data features and time series data features can be achieved through steps S''1031 to S''1036, which are described in detail as follows: Step S''1031: Map the multimodal data feature X to a query matrix through linear transformation , map the time series data feature Y into a key matrix K and a value matrix V.

[0081] Assuming multimodal data characteristics (m time steps, dimension is ), time series data characteristics (n time steps, dimension is ), the time series data feature Y is mapped into the key matrix K and the value matrix V according to the following linear transformation formula: ,in, , , .

[0082] in, is the dimension of the attention head.

[0083] Step S''1032: Calculate the attention score matrix ,in, It is a relative position encoding matrix that is dynamically adjusted based on the timestamp interval.

[0084] In the embodiment of the present application, the relative position encoding matrix , whose elements Represents the time step and The relative position weight between them. A solution based on dynamic adjustment of timestamp interval is:

[0085] in, is the decay function, .

[0086] Step S''1033: The attention score matrix S performs Softmax normalization and is weighted with V to obtain preliminary alignment features .

[0087] Step S''1034: Split the preliminary alignment feature Z into multiple attention heads.

[0088] Take the example of splitting into 8 attention heads and each attention head has a dimension of 32, and calculate the output of each attention head independently for: ,

[0089] in,

[0090] Step S''1035: Output of each attention head Perform LayerNorm standardization and get .

[0091] Specifically, ,in, , , and Can be loaded from predefined parameter tables according to the equipment type in the computer room.

[0092] Step S''1036: Concatenate the outputs of all attention heads and generate the final alignment features through residual connections.

[0093] Specifically, according to Concatenate the outputs of all attention heads and connect them according to the residual Generate the final alignment feature, where is the multimodal data feature, is the output projection matrix, .

[0094] Step S104: Based on the feature-aligned multimodal data features and time series data features, the preprocessed multimodal data and the preprocessed time series data are fused to obtain fused features.

[0095] As an embodiment of the present application, based on the feature-aligned multimodal data features and the time series data features, fusing the preprocessed multimodal data and the preprocessed time series data to obtain fused features can be implemented through steps S1041 to S1043, which are described in detail as follows: Step S1041: Calculate the similarity between the multimodal data features and the time series data features.

[0096] Specifically, the dot product can be used to calculate the similarity between multimodal data features and time series data features. :

[0097] in, is the feature vector of the position in the multimodal data feature map, C is the channel of the multimodal data feature map (assuming that the multimodal data is an image), It is the feature representation of the kth word with feature dimension D in the time series data.

[0098] Step S1042: Based on the similarity between the multimodal data features and the time series data features, the softmax function is used to calculate the attention weight between the multimodal data and the time series data.

[0099] Specifically, the attention weights between multimodal data and temporal data It can be calculated by the following formula:

[0100] Wherein, L is the length of the time series data corresponding to the time series data feature.

[0101] Step S1043: Calculate the weighted fusion representation between the multimodal data features and the time series data features by using the attention weight between the multimodal data and the time series data.

[0102] Specifically, the weighted fusion representation between multimodal data features and time series data features is calculated by using the attention weights between multimodal data and time series data. This can be achieved through the following formula:

[0103] From the weighted fusion representation From the expression, we can see that, in fact, the multimodal data features The weighted and time-series data are associated with each word to generate a spatially aligned representation , which combines the common information of multimodal data and time series data.

[0104] As another embodiment of the present application, based on the feature-aligned multimodal data features and time series data features, fusing the preprocessed multimodal data and the preprocessed time series data to obtain fused features can be implemented through steps S'1041 to S'1043, as described in detail as follows: Step S'1041: Mapping the aligned multimodal data features and time series data features to a first low-dimensional embedding representation and a second low-dimensional embedding representation respectively.

[0105] Step S'1042: cross-map the first low-dimensional embedding representation and the second low-dimensional embedding representation through a shared mapping function to obtain cross-mapped embedding features.

[0106] Step S'1043: Reconstruct the cross-mapped embedded features back into the feature space of the original modality.

[0107] Considering that the dimension of the fused feature may be too large, or even cause "dimensionality disaster", after the fused feature is obtained in step S104, the fused feature may be reduced in dimension.

[0108] Step S105: Perform data analysis based on the fusion features to determine the current status of the computer room.

[0109] Specifically, as an embodiment of the present application, data analysis is performed based on fusion features to determine the current state of the computer room. Classification algorithms, such as support vector machines (SVM), random forests, etc., are used to analyze the fusion features to determine the safety status of the computer room as normal, warning, or dangerous, and feature patterns of different safety states are learned based on training data to accurately classify new data. As an embodiment of the present application, data analysis is performed based on fusion features to determine the current state of the computer room. Anomaly detection algorithms can also be used. For example, isolation forests, local outlier factors (LOF), etc. can be used to analyze fusion features, identify abnormal patterns in the data, determine whether there is a security incident, and issue an early warning signal when abnormal data points that are significantly different from normal data patterns are detected.

[0110] From the above attached Figure 1 It can be seen from the example of the computer room status data processing method based on artificial intelligence that after extracting features from the pre-processed multimodal data and the pre-processed time series data based on the artificial intelligence algorithm, and obtaining the multimodal data features and time series data features respectively, the multimodal data features and time series data features are feature aligned, and based on the multimodal data features and time series data features after feature alignment, the pre-processed multimodal data and the pre-processed time series data are fused to obtain fused features. Compared with the existing data analysis methods that often separate these two types of data and do not link the two, the technical solution of the present application improves the accuracy of judging computer room security events through the fusion of data features, reduces false positives and false negatives caused by a single data source, and provides a more reliable decision-making basis for operation and maintenance personnel.

[0111] Please see attached Figure 2 , is an artificial intelligence-based computer room status data processing device provided in an embodiment of the present application, the device may include a preprocessing module 201, a feature extraction module 202, an alignment module 203, a fusion module 204 and a determination module 205, which are described in detail as follows: A preprocessing module 201 is used to preprocess the multimodal data and time series data of the relevant computer room to obtain preprocessed multimodal data and preprocessed time series data respectively; A feature extraction module 202 is used to extract features from the preprocessed multimodal data and the preprocessed time series data based on an artificial intelligence algorithm to obtain multimodal data features and time series data features respectively; An alignment module 203 is used to align the multimodal data features with the time series data features; A fusion module 204 is used to fuse the pre-processed multi-modal data and the pre-processed time series data based on the multi-modal data features and the time series data features after feature alignment to obtain fused features; The determination module 205 is used to perform data analysis based on the fusion features to determine the current status of the computer room.

[0112] From the above attached Figure 2 It can be seen from the example of the computer room status data processing device based on artificial intelligence that after extracting features from the pre-processed multimodal data and the pre-processed time series data based on the artificial intelligence algorithm, and obtaining the multimodal data features and time series data features respectively, the multimodal data features and time series data features are feature aligned, and based on the multimodal data features and time series data features after feature alignment, the pre-processed multimodal data and the pre-processed time series data are fused to obtain fused features. Compared with the existing data analysis methods that often separate these two types of data and do not link the two, the technical solution of the present application improves the accuracy of judging computer room security events through the fusion of data features, reduces false positives and false negatives caused by a single data source, and provides a more reliable basis for decision-making for operation and maintenance personnel.

[0113] Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Figure 3 As shown, the electronic device 3 of this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program of a computer room status data processing method based on artificial intelligence. When the processor 30 executes the computer program 32, the steps in the embodiment of the computer room status data processing method based on artificial intelligence are implemented, such as Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are implemented, for example Figure 2 The functions of the preprocessing module 201, the feature extraction module 202, the alignment module 203, the fusion module 204 and the determination module 205 are shown.

[0114] Exemplarily, the computer program 32 of the computer room status data processing method based on artificial intelligence mainly includes: preprocessing the multimodal data and time series data of the relevant computer room respectively, and obtaining the preprocessed multimodal data and the preprocessed time series data respectively; extracting the features of the preprocessed multimodal data and the preprocessed time series data respectively based on the artificial intelligence algorithm, and obtaining the multimodal data features and the time series data features respectively; aligning the multimodal data features and the time series data features; based on the multimodal data features and the time series data features after feature alignment, fusing the preprocessed multimodal data and the preprocessed time series data to obtain the fusion features; performing data analysis according to the fusion features to determine the current state of the computer room. The computer program 32 can be divided into one or more modules / units, one or more modules / units are stored in the memory 31, and are executed by the processor 30 to complete the present application. One or more modules / units can be a series of computer program instruction segments that can complete specific functions, and the instruction segments are used to describe the execution process of the computer program 32 in the electronic device 3. For example, the computer program 32 can be divided into the functions of a preprocessing module 201, a feature extraction module 202, an alignment module 203, a fusion module 204 and a determination module 205 (modules in the virtual device), and the specific functions of each module are as follows: the preprocessing module 201 is used to preprocess the multimodal data and time series data of the relevant computer room respectively, and obtain the preprocessed multimodal data and the preprocessed time series data respectively; the feature extraction module 202 is used to extract features from the preprocessed multimodal data and the preprocessed time series data respectively based on the artificial intelligence algorithm, and obtain multimodal data features and time series data features respectively; the alignment module 203 is used to align the multimodal data features and the time series data features; the fusion module 204 is used to fuse the preprocessed multimodal data and the preprocessed time series data based on the multimodal data features and the time series data features after feature alignment to obtain fused features; the determination module 205 is used to perform data analysis based on the fused features to determine the current state of the computer room.

[0115] The electronic device 3 may include but is not limited to a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of the electronic device 3 and does not constitute a limitation of the electronic device 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0116] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0117] The memory 31 may be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 31 may also be an external storage device of the electronic device 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 3. Further, the memory 31 may also include both an internal storage unit and an external storage device of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 may also be used to temporarily store data that has been output or is to be output.

[0118] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0119] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0120] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0121] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0122] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program of the computer room status data processing method based on artificial intelligence can be stored in a storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments, that is, preprocessing the multimodal data and time series data of the relevant computer room respectively to obtain preprocessed multimodal data and preprocessed time series data respectively; extracting features from the preprocessed multimodal data and preprocessed time series data based on the artificial intelligence algorithm to obtain multimodal data features and time series data features respectively; aligning the multimodal data features and time series data features; based on the multimodal data features and time series data features after feature alignment, fusing the preprocessed multimodal data and preprocessed time series data to obtain fusion features; performing data analysis based on the fusion features to determine the current state of the computer room. Among them, the computer program includes computer program code, which can be in source code form, object code form, executable file or some intermediate form, etc. The storage medium may include: any entity or device that can carry computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the storage medium does not include electric carrier signal and telecommunication signal.

[0125] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application is described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the specific implementation method of the present application, and is not used to limit the protection scope of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present invention.

Claims

1. A computer room status data processing method based on artificial intelligence, characterized in that: The method comprises: Preprocessing the multimodal data and time series data of the relevant computer room respectively, and obtaining preprocessed multimodal data and preprocessed time series data respectively; Based on the artificial intelligence algorithm, feature extraction is performed on the preprocessed multimodal data and the preprocessed time series data to obtain multimodal data features and time series data features respectively; Performing feature alignment on the multimodal data features and the time series data features; Based on the feature-aligned multimodal data features and time series data features, the preprocessed multimodal data and the preprocessed time series data are fused to obtain fused features; Data analysis is performed based on the fusion features to determine the current status of the computer room.

2. The method for processing computer room status data based on artificial intelligence as claimed in claim 1, characterized in that: The preprocessed multimodal data includes a preprocessed image, and the artificial intelligence algorithm is used to extract features from the preprocessed multimodal data and the preprocessed time series data to obtain multimodal data features and time series data features, respectively, including: Extracting features of the preprocessed image based on a deep convolutional neural network to obtain an image data feature map; Sliding a sliding window on the image data feature map to obtain multiple targets on the preprocessed image; The LSTM model is used to extract features of the preprocessed time series data to obtain the features of the time series data.

3. The method for processing computer room status data based on artificial intelligence as claimed in claim 2, characterized in that: The step of sliding a sliding window on the image data feature map to obtain a plurality of targets on the preprocessed image comprises: Generate candidate frames of the multiple objects by sliding a sliding window on the image data feature map; Performing classification and bounding box regression operations on the candidate boxes of the multiple objects to obtain a classification score and a regression value for each object; A non-maximum suppression algorithm is used to remove duplicate candidate boxes in the candidate boxes to obtain the category and position of the target in the preprocessed image.

4. The method for processing computer room status data based on artificial intelligence as claimed in claim 1, characterized in that: The step of aligning the multimodal data features with the time series data features includes: Mapping the multimodal data features and the time series data features to a semantic space of the same dimension; Calculating the similarity between the multimodal data features and the time series data features in the semantic space of the same dimension, wherein the similarity includes a weight and a bias; The weights and biases are adjusted by an optimization algorithm to minimize the distance between the multimodal data features and the time series data features.

5. The method for processing computer room status data based on artificial intelligence as claimed in claim 1, characterized in that: The method of fusing the preprocessed multimodal data and the preprocessed time series data based on the feature-aligned multimodal data features and the time series data features to obtain fused features includes: Calculating the similarity between the multimodal data features and the time series data features; Based on the similarity between the multimodal data features and the time series data features, using a softmax function to calculate the attention weight between the multimodal data and the time series data; By using the attention weights between the multimodal data and the time series data, a weighted fusion representation between the multimodal data features and the time series data features is calculated.

6. The method for processing computer room status data based on artificial intelligence as claimed in claim 1, characterized in that: The method of fusing the preprocessed multimodal data and the preprocessed time series data based on the feature-aligned multimodal data features and the time series data features to obtain fused features includes: Mapping the feature-aligned multimodal data features and time series data features to a first low-dimensional embedding representation and a second low-dimensional embedding representation, respectively; Cross-mapping the first low-dimensional embedding representation and the second low-dimensional embedding representation through a shared mapping function to obtain a cross-mapped embedding feature; The cross-mapped embedded features are reconstructed back into the feature space of the original modality.

7. The method for processing computer room status data based on artificial intelligence according to any one of claims 1 to 6, characterized in that: The method further comprises: The fusion features are subjected to dimensionality reduction.

8. An artificial intelligence-based computer room status data processing device, characterized in that: The device comprises: A preprocessing module, used to preprocess the multimodal data and time series data of the relevant computer room, respectively, to obtain preprocessed multimodal data and preprocessed time series data; A feature extraction module, used to extract features from the preprocessed multimodal data and the preprocessed time series data based on an artificial intelligence algorithm, to obtain multimodal data features and time series data features respectively; An alignment module, used for aligning the multimodal data features with the time series data features; A fusion module, used for fusing the preprocessed multimodal data and the preprocessed time series data to obtain fusion features based on the multimodal data features and time series data features after feature alignment; The determination module is used to perform data analysis based on the fusion features to determine the current status of the computer room.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images