Museum air conditioner abnormity identification method fusing multi-modal data

Abstract

The invention relates to a museum air conditioner abnormity identification method fused with multi-modal data, and the method comprises the following steps: 1, collecting the historical data of the operation of an air conditioner, the historical data comprising work order data, voiceprint data and current and voltage data; 2, work order data features, voiceprint data features and current and voltage data features are extracted based on the collected historical data to serve as samples; step 3, constructing a multi-modal convolutional neural network; 4, dividing the samples into a training set and a test set, training the multi-modal convolutional neural network by using the training set, and evaluating the performance by using the test set to obtain a fault prediction model; and 5, predicting the abnormal condition of the air conditioner in the museum based on the fault prediction model. According to the method, various modal data such as voiceprints, work orders and current and voltage are fused, information in multiple aspects can be comprehensively considered, the accuracy of fault recognition is remarkably improved, and intelligent and efficient operation of a museum air conditioning system is promoted.

Description

Technical Field

[0001] The present invention relates to the technical field of air-conditioning fault diagnosis, and in particular to a museum air-conditioning anomaly recognition method integrating multi-modal data. Background Art

[0002] As a gathering place for culture and art, the stability of the environmental control system of a museum is crucial to the protection of exhibits. Abnormal operation of the air conditioning system not only affects the comfort level, but may also cause damage to the exhibits. Existing air conditioning fault diagnosis methods usually rely on single modal data and cannot make full use of information from different sources, resulting in insufficient accuracy and timeliness of fault identification. Therefore, it is particularly important to develop a museum air conditioning anomaly identification technology based on multimodal data. Summary of the invention

[0003] The present invention provides a museum air conditioning anomaly recognition method integrating multimodal data to solve the above technical problems.

[0004] In order to solve the above technical problems, the present invention provides a museum air conditioning anomaly identification method integrating multimodal data, comprising the following steps:

[0005] Step 1: Collect historical data of air conditioner operation, including work order data, voiceprint data, and current and voltage data;

[0006] Step 2: Based on the collected historical data, extract the work order data features, voiceprint data features, and current and voltage data features as samples;

[0007] Step 3: Construct a multimodal convolutional neural network;

[0008] Step 4: Divide the samples into a training set and a test set, use the training set to train the multimodal convolutional neural network, and use the test set to evaluate the performance to obtain a fault prediction model;

[0009] Step 5: Predict abnormal conditions of the museum air conditioning based on the fault prediction model.

[0010] Preferably, the method for extracting the features of the voiceprint data comprises: extracting the features of the sound signal in the voiceprint data using Mel-frequency cepstral coefficients.

[0011] Preferably, the method for extracting the features of the work order data includes: utilizing text processing technology to extract keywords and sentiment analysis results in the work order data, and generating a numerical feature vector.

[0012] Preferably, the method for extracting the characteristics of the current and voltage data includes: calculating statistical characteristics of the current and voltage respectively, and the statistical characteristics include at least a mean and a variance.

[0013] Preferably, the multimodal convolutional neural network includes a convolution layer, a feature fusion layer, a fully connected layer and an output layer in sequence, and the convolution layer contains three channels, the work order channel is used to extract text features, the sound channel is used to extract sound features, and the current and voltage channel is used to extract time series features.

[0014] Preferably, the feature fusion layer performs feature fusion by splicing the outputs of convolutional layers from different channels.

[0015] Preferably, the output layer adopts a Softmax activation function, and the output is the probability of the fault type.

[0016] Preferably, the output of the output layer is:

[0017] F output =W f ·F+b f

[0018] Among them, W f is the weight matrix of the fully connected layer, b f is the bias term, and F is the fused feature.

[0019] Preferably, a cross entropy loss function is used to train the multimodal convolutional neural network.

[0020] Preferably, the performance evaluation indicators include: accuracy, recall and F1 score indicators.

[0021] Compared with the prior art, the museum air conditioning anomaly identification method integrating multimodal data provided by the present invention has the following advantages:

[0022] 1. The present invention can comprehensively consider various information by integrating voiceprints, work orders, current and voltage and other modal data, and can effectively reduce the false alarm rate by combining multiple data features, avoiding misjudgment caused by relying on single modal data, thereby improving the accuracy of fault identification and promoting the intelligent and efficient operation of the museum's air-conditioning system;

[0023] 2. The present invention adopts a multimodal convolutional neural network, which can quickly process information from different data sources, monitor and diagnose the status of the air-conditioning system in real time, discover potential faults in time, reduce response time, and integrate various types of information to form a more comprehensive fault identification view, providing a basis for subsequent analysis and optimization;

[0024] 3. The present invention can realize intelligent management of museum air conditioners by establishing a data-driven fault identification system, reduce the need for manual monitoring, improve management efficiency, and adapt to air conditioning systems in different environments. It can capture the characteristics of specific environments and equipment through training models and has strong generalization ability;

[0025] 4. The present invention combines historical work order data and real-time monitoring data to provide data-driven support for decision makers, helping them to formulate more effective maintenance and operation strategies, and by ensuring the normal operation of the air-conditioning system, improving the comfort and stability of the museum environment, thereby improving the visitor's visiting experience;

[0026] 5. The present invention helps to reduce the repair costs and equipment downtime caused by sudden failures by early identification and diagnosis of faults, thereby reducing overall maintenance costs. At the same time, it promotes the development of multimodal learning and intelligent monitoring technologies in actual scenarios, and provides a foundation for technological innovation in related fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The present invention is a flowchart of a method for identifying anomalies of museum air conditioners by integrating multimodal data in a specific embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to describe the technical solution of the above invention in more detail, specific embodiments are listed below to demonstrate the technical effects; it should be emphasized that these embodiments are used to illustrate the present invention but not to limit the scope of the present invention.

[0029] The museum air conditioning anomaly identification method provided by the present invention by integrating multimodal data is as follows: Figure 1 As shown, the following steps are included:

[0030] Step 1: Collect historical data of air conditioner operation, including work order data, voiceprint data, and current and voltage data;

[0031] Step 2: Based on the collected historical data, extract the work order data features, voiceprint data features, and current and voltage data features as samples for subsequent analysis;

[0032] Step 3: Construct a multimodal convolutional neural network;

[0033] Step 4: Divide the samples into a training set and a test set, use the training set to train the multimodal convolutional neural network, and use the test set to evaluate the performance to obtain a fault prediction model;

[0034] Step 5: Predict the abnormal situation of the museum air conditioning based on the fault prediction model, that is, input the real-time monitoring data into the trained fault prediction model, and determine the abnormal situation of the museum air conditioning according to the output content of the fault prediction model.

[0035] By integrating multiple modal data such as voiceprints, work orders, current and voltage, the present invention can comprehensively consider various aspects of information and combine multiple data features to effectively reduce the false alarm rate and avoid misjudgment caused by relying on a single modal data, thereby improving the accuracy of fault identification and promoting the intelligent and efficient operation of the museum's air conditioning system. The present invention combines multimodal data processing, deep learning algorithms and Internet of Things technology to improve the operating efficiency and fault response capability of the air conditioning system.

[0036] In some embodiments, the method of extracting the voiceprint data features includes: extracting the sound signal features in the voiceprint data using Mel-Frequency Cepstral Coefficients (MFCC).

[0037]

[0038] Among them, MFCC(n) represents the nth MFCC coefficient, M m is the frequency domain feature obtained after FFT (Fast Fourier Transform), M is the number of Mel filters (usually 20 to 40 filters), n is the MFCC coefficient index, and m is the Mel frequency index. is the constant factor of the discrete cosine transform.

[0039] In some embodiments, the method for extracting the work order data features includes: extracting keywords and sentiment analysis results from the work order data using text processing technology to generate a numerical feature vector. In some embodiments, the method for extracting the current and voltage data features includes: calculating the statistical features of the current and voltage respectively, and the statistical features include at least the mean and variance. The extracted voiceprint data features, work order data features, and current and voltage data features can be directly used as input data for the multimodal convolutional neural network.

[0040] In some embodiments, the multimodal convolutional neural network (MCNN) includes a convolution layer, a feature fusion layer, a fully connected layer and an output layer in sequence, and the convolution layer contains three channels, the work order channel is used to extract text features, the sound channel is used to extract sound features, and the current and voltage channel is used to extract time series features.

[0041] For 2D convolutional layer (two-dimensional convolutional layer):

[0042]

[0043] Among them, Z i，j represents the element of the output feature map obtained after the convolution operation (i.e., the position (i, j) in the output image), W m，n X is the element of the convolution kernel, with a size of M×N, which is a trainable parameter for the convolution operation. i+m，j+nis the element of the input feature map, with a size of H × W. b is the bias term, a constant value of the convolution kernel output, and also a trainable parameter.

[0044] For 1D convolutional layer (one-dimensional convolutional layer):

[0045]

[0046] Among them, Z i is the element of the output feature map obtained after the convolution operation, indicating the value at the position after the convolution operation, W j is the element of the convolution kernel, with a size of K, and is a trainable parameter. i+j is the element of the input feature map, with a size of N (e.g., the length of the time series). i+j represents the local area where the convolution kernel acts. b is the bias term, which is a constant value output by the convolution kernel and helps adjust the offset of the convolution calculation result.

[0047] In some embodiments, the feature fusion layer performs feature fusion by splicing the outputs of convolutional layers from different channels (different modalities).

[0048] F=[F sound , F text , F current ] (4)

[0049] Among them, F sound It is the feature representation of the sound mode, and is extracted by the MFCC method. text It is the feature representation of the work order mode, which is the numerical feature after text processing. current It is the feature representation of the current and voltage mode, based on the data collected by the current and voltage sensors, and is a vector processed by the convolutional layer feature extraction method.

[0050] In some embodiments, the output of the output layer is:

[0051] F output =W f ·F+b f (5)

[0052] Among them, W f is the weight matrix of the fully connected layer, b f is the bias term, and F is the fused feature.

[0053] In this embodiment, the output layer adopts a Softmax activation function, and the output is the probability of the fault type.

[0054]

[0055] Among them, P(y i) is the predicted category y i The probability of F output,i is the output value of the i-th category, and C is the total number of categories.

[0056] In some embodiments, the multimodal convolutional neural network is trained using a cross entropy loss function.

[0057]

[0058] Among them, L is the loss value, which is the difference between the model prediction result and the true label, and is usually used to guide the optimization algorithm to adjust the model parameters. N is the number of samples, which is the number of samples in the training set. i is the actual category label, is the model prediction value, It is a logarithmic loss, which amplifies the loss of incorrect predictions by taking the logarithm of the predicted probability. This can encourage the model to better approximate the correct category. The purpose of the cross entropy loss function is to minimize the difference between the predicted value and the true label, ensuring that the model can output accurate category probabilities.

[0059] In some embodiments, the performance evaluation metrics include: precision, recall, and F1 score metrics.

[0060]

[0061] in:

[0062]

[0063] TP, FP, and FN represent the number of true positives, false positives, and false negatives, respectively.

[0064] The present invention adopts a multimodal convolutional neural network, which can quickly process information from different data sources, monitor and diagnose the status of the air-conditioning system in real time, promptly discover potential faults, and reduce response time; by establishing a data-driven fault identification system, it can realize the intelligent management of museum air-conditioning, reduce the need for manual monitoring, improve management efficiency, and can adapt to air-conditioning systems in different environments. By training models to capture the characteristics of specific environments and equipment, it has strong generalization capabilities.

[0065] In summary, the museum air conditioning anomaly identification method integrating multimodal data provided by the present invention comprises the following steps: Step 1: Collecting historical data of air conditioning operation, wherein the historical data includes work order data, voiceprint data and current and voltage data; Step 2: Extracting work order data features, voiceprint data features and current and voltage data features based on the collected historical data as samples; Step 3: Constructing a multimodal convolutional neural network; Step 4: Dividing the samples into a training set and a test set, training the multimodal convolutional neural network using the training set, and evaluating the performance using the test set to obtain a fault prediction model; Step 5: Predicting abnormal conditions of museum air conditioning based on the fault prediction model. The present invention can comprehensively consider various aspects of information by integrating multiple modal data such as voiceprints, work orders, and current and voltage, and can effectively reduce the false alarm rate by combining multiple data features, avoid misjudgment caused by relying on single modal data, thereby improving the accuracy of fault identification and promoting the intelligent and efficient operation of museum air conditioning systems. The present invention combines multimodal data processing, deep learning algorithms and Internet of Things technologies to improve the operating efficiency and fault response capabilities of air conditioning systems.

[0066] Obviously, those skilled in the art can make various changes and modifications to the invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the invention fall within the scope of the claims of the invention and their equivalents, the invention is also intended to include these modifications and variations.

Claims

1. A museum air conditioning anomaly identification method integrating multimodal data, characterized in that: The steps include: Step 1: Collect historical data of air conditioner operation, including work order data, voiceprint data, and current and voltage data; Step 2: Based on the collected historical data, extract the work order data features, voiceprint data features, and current and voltage data features as samples; Step 3: Construct a multimodal convolutional neural network; Step 4: Divide the samples into a training set and a test set, use the training set to train the multimodal convolutional neural network, and use the test set to evaluate the performance to obtain a fault prediction model; Step 5: Predict abnormal conditions of the museum air conditioning based on the fault prediction model.

2. The museum air conditioning anomaly identification method integrating multimodal data according to claim 1, characterized in that: The method for extracting the voiceprint data features includes: extracting the sound signal features in the voiceprint data using Mel-frequency cepstral coefficients.

3. The museum air conditioning anomaly identification method integrating multimodal data according to claim 1, characterized in that: The method for extracting the features of the work order data includes: using text processing technology to extract keywords and sentiment analysis results in the work order data, and generating a numerical feature vector.

4. The museum air conditioning anomaly identification method integrating multimodal data according to claim 1, characterized in that: The method for extracting the current and voltage data features includes: calculating statistical features of the current and voltage respectively, and the statistical features include at least a mean and a variance.

5. The museum air conditioning anomaly identification method integrating multimodal data as claimed in claim 1, characterized in that: The multimodal convolutional neural network includes a convolution layer, a feature fusion layer, a fully connected layer and an output layer in sequence. The convolution layer contains three channels. The work order channel is used to extract text features, the sound channel is used to extract sound features, and the current and voltage channel is used to extract time series features.

6. The museum air conditioning anomaly identification method integrating multimodal data as claimed in claim 5, characterized in that: The feature fusion layer performs feature fusion by splicing the outputs of convolutional layers from different channels.

7. The museum air conditioning anomaly identification method integrating multimodal data as claimed in claim 5, characterized in that: The output layer adopts a Softmax activation function, and the output is the probability of the fault type.

8. The museum air conditioning anomaly identification method integrating multimodal data according to claim 7, characterized in that: The output of the output layer is: F output =W f ·F+b f Among them, W f is the weight matrix of the fully connected layer, b f is the bias term, and F is the fused feature.

9. The museum air conditioning anomaly identification method integrating multimodal data as claimed in claim 1, characterized in that: The multimodal convolutional neural network is trained using a cross entropy loss function.

10. The museum air conditioning anomaly identification method integrating multimodal data according to claim 1, characterized in that: The performance evaluation metrics include: precision, recall, and F1 score.

Images