A method for detecting the operating status of an automatic weather station based on multimodal data

Abstract

The present application discloses a method for detecting the operating status of an automatic weather station based on multimodal data, which relates to the field of meteorological monitoring technology. The method comprises: obtaining meteorological training samples of the automatic weather station in different operating states, sequentially cascading a multimodal feature extraction module, a multimodal feature fusion module and a time series anomaly detection module to construct a network architecture for obtaining an automatic weather station operating status detection model, and using the meteorological training samples as input and the operating status of the automatic weather station as output to train the automatic weather station operating status detection model. Multimodal meteorological time series data is obtained to form a time series to be detected, and the trained automatic weather station operating status detection model is used to determine the operating status of the automatic weather station at the current operating moment based on the time series to be detected. By integrating multimodal meteorological time series data, the detection accuracy and efficiency of the operating status of the automatic weather station are effectively improved, ensuring the stable operation of the automatic weather station.

Description

Technical Field

[0001] The present application relates to the field of meteorological monitoring technology, and in particular to a method for detecting the operating status of an automatic weather station based on multimodal data. Background Art

[0002] Weather stations are a vital component of modern meteorological observation networks, capable of automatically and continuously collecting a wide range of meteorological data. Compared to traditional weather stations, automatic weather stations offer more comprehensive meteorological observation capabilities, collecting data on a wide range of environmental meteorological factors. Consequently, they are increasingly being used.

[0003] During actual operation, automatic weather stations are prone to abnormal conditions due to various factors, including equipment failures, environmental interference, and human factors. This can degrade the quality of their observations and impact the reliability and accuracy of weather forecasts. Traditionally, abnormal condition detection methods monitor the station's operating electrical parameters, such as voltage, current, and temperature. When these parameters are abnormal, the station's operating status is identified and promptly investigated. However, this detection method lacks accuracy and reliability. Summary of the Invention

[0004] In response to the above-mentioned problems and technical requirements, this application proposes a method for detecting the operating status of an automatic weather station based on multimodal data. The technical solution of this application is as follows:

[0005] A method for detecting the operating status of an automatic weather station based on multimodal data comprises the following steps:

[0006] Obtain meteorological training samples of the automatic weather station under different operating states. The meteorological training samples of the automatic weather station under each operating state include a time series consisting of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment include the working electrical parameters of the automatic weather station, meteorological element data observed by the automatic weather station, environmental audio characteristics of the working environment of the automatic weather station, and environmental video data of the working environment of the automatic weather station. Each group of meteorological training samples includes a meteorological numerical sequence consisting of the working electrical parameters and meteorological element data at different moments, an environmental audio sequence consisting of environmental audio characteristics at different moments, and an environmental video sequence consisting of environmental video data at different moments.

[0007] The multimodal feature extraction module, the multimodal feature fusion module and the time series anomaly detection module are cascaded in sequence to construct a network architecture of the automatic weather station operation status detection model, and the meteorological training samples are used as input and the operation status of the automatic weather station is used as output to train the automatic weather station operation status detection model; after the meteorological training samples are input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence in the meteorological training samples respectively, the multimodal feature fusion module performs information interaction and fusion on the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence to obtain a time series feature sequence, the time series anomaly detection module includes a time series VAE model and a classifier, and the time series VAE model compresses and reconstructs the fused time series feature sequence to obtain a reconstruction error sequence Classifier pair reconstruction error sequence Perform classification to obtain the prediction classification results of the automatic weather station's operating status;

[0008] The multimodal meteorological time series data of the automatic weather station at any operating time and multiple consecutive historical operating times before it are obtained to form a time series to be detected. The operating status of the automatic weather station at the current operating time is determined based on the time series to be detected using the trained automatic weather station operating status detection model.

[0009] A further technical solution is to train an automatic weather station operation status detection model including:

[0010] Initialize the model parameters of the automatic weather station operation status detection model, use meteorological training samples to train the automatic weather station operation status detection model, and obtain the automatic weather station operation status detection model by minimizing the loss function;

[0011] The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error sequence The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data, time regularization loss Used to characterize the reconstruction error sequence The ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the lower the temporal regularization loss. The smaller.

[0012] Its further technical solution is time regularization loss Including timing reconstruction loss L and timing alignment loss L2; ​​reconstruction error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the temporal similarity with the time series z composed of multimodal meteorological time series data, the smaller the temporal alignment loss L2.

[0013] Its further technical solution is that the time series reconstruction loss L = L1 + L3, the reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstruction error sequence The higher the independence at different moments, the smaller the covariance regularization loss L3.

[0014] Its further technical solution is that the smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The sequence length.

[0015] Its further technical solution is covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm.

[0016] Its further technical solution is to reduce the timing alignment loss Indicates the time series z and reconstruction error series composed of multimodal meteorological time series data calculated using the dynamic time warping method The temporal similarity at each moment.

[0017] Its further technical solution is that the multimodal feature extraction module uses the Informer model to extract features from meteorological numerical sequences to obtain numerical time series features; uses the Audio Spectrogram Transformer model to extract features from environmental audio sequences to obtain audio time series features; and uses the Swin Transformer model to extract features from environmental video sequences to obtain video time series features.

[0018] Its further technical solution is that the multimodal feature fusion module obtains the feature encoding of the temporal features of the meteorological numerical sequence, environmental audio sequence and environmental video sequence respectively through the transformer-based self-attention mechanism, and uses the cross-modal attention mechanism and the multi-head spatiotemporal attention mechanism to interactively fuse the information of each feature encoding to obtain the temporal feature sequence.

[0019] A further technical solution is to obtain meteorological training samples of the automatic weather station in each operating state, including:

[0020] During the operation of the automatic weather station, original working electrical parameters, observed original meteorological element data, and original environmental audio data and original environmental video data of the working environment of the automatic weather station at different times are obtained;

[0021] The original meteorological element data at each moment is processed and screened for missing values ​​and outliers, and the scale of the screened original meteorological element data is normalized to obtain the normalized original meteorological element data;

[0022] The original working electrical parameters at each moment are screened for missing values ​​and outliers, and the scales of the screened original working electrical parameters are normalized to obtain normalized original working electrical parameters;

[0023] Perform noise reduction on the original ambient audio data at each moment and extract audio features to obtain original ambient audio features;

[0024] Adjusting the image size of the original environment video data at each moment and performing image enhancement to obtain processed original environment video data;

[0025] The original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment are time-aligned according to the same time resolution to obtain multimodal meteorological time series data at multiple consecutive moments, which constitute the meteorological training samples of the automatic weather station in its current operating state.

[0026] The beneficial technical effects of this application are:

[0027] This application proposes a method for detecting the operating status of an automatic weather station based on multimodal data. By constructing an end-to-end automatic weather station operating status detection model, it effectively integrates multimodal meteorological time series data. Compared to traditional abnormal status detection methods that only use single-modal data, this method fully utilizes the temporal characteristics of multimodal meteorological time series data, effectively improving the detection accuracy of the automatic weather station's operating status.

[0028] Taking into account the data characteristics of each modal meteorological time series data and the fusion characteristics between each modal meteorological time series data, a transform-based deep learning model is used to extract the time series features of each modal meteorological time series data respectively. This can not only ensure the full extraction of the unique time series features of each modal meteorological time series data, but also ensure the full fusion of the extracted meteorological time series data of each modality, thus laying a solid foundation for further improving the accuracy of the automatic weather station operation status detection results.

[0029] By introducing a time regularization loss based on the reconstruction loss and KL divergence of the traditional VAE model, the reconstructed error sequence reconstructed by the automatic weather station operation status detection model is ensured to capture the temporal characteristics of the time series composed of multimodal meteorological time series data. The smoothness loss is used to improve the sensitivity of the automatic weather station operation status detection model to abnormal data. The covariance regularization loss is used to enhance the independence of the reconstructed error sequence at different times, ensuring that the automatic weather station operation status detection model can still capture the global dynamic characteristics of the time series while decorrelating, thereby effectively representing the long-term dependence of the time series. The temporal alignment loss is used to ensure the temporal similarity between the reconstructed error sequence and the time series composed of multimodal meteorological time series data, which helps to accurately capture the temporal characteristics of the time series composed of multimodal meteorological time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the method for detecting the operating status of an automatic weather station.

[0031] Figure 2 It is the network architecture of the automatic weather station operation status detection model.

[0032] Figure 3 It is the process of processing meteorological training samples by the automatic weather station operation status detection model. DETAILED DESCRIPTION

[0033] The specific implementation of this application will be further described below with reference to the accompanying drawings.

[0034] This application proposes a method for detecting the operating status of an automatic weather station based on multimodal data. Figure 1 The operating status of the automatic weather station includes two states: normal and abnormal. When the operating status of the automatic weather station is abnormal, the acquired multimodal meteorological time series data will be abnormal. Therefore, this application detects the operating status of the automatic weather station based on the multimodal meteorological time series data. The specific steps are as follows:

[0035] Step 1: Obtain meteorological training samples for the automatic weather station under different operating states. The meteorological training samples for each operating state of the automatic weather station include a time series consisting of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment includes the operating electrical parameters of the automatic weather station, meteorological element data observed by the automatic weather station, ambient audio characteristics of the operating environment of the automatic weather station, and ambient video data of the operating environment of the automatic weather station. Each set of meteorological training samples includes a meteorological numerical sequence consisting of the operating electrical parameters and meteorological element data at different moments, an ambient audio sequence consisting of ambient audio characteristics at different moments, and an ambient video sequence consisting of ambient video data at different moments.

[0036] Meteorological training samples are constructed by acquiring multimodal meteorological time series data of the historical operating moments of the automatic weather station, each meteorological training sample corresponding to an operating state. In one embodiment, the process of constructing meteorological training samples of the automatic weather station in each operating state is as follows:

[0037] (1) When the automatic weather station is operating in the corresponding operating state, the original working electrical parameters and the original meteorological element data observed by the automatic weather station at different times are obtained. The original environmental audio data of the working environment of the automatic weather station is obtained using an audio acquisition device such as a microphone. The original environmental video data is obtained using the camera of the automatic weather station. Among them, the original working electrical parameters and original meteorological element data are obtained from the observation system of the automatic weather station. The working electrical parameters include the data reflecting the equipment status information such as the mainboard voltage, mainboard temperature, and signal strength of the automatic weather station; the meteorological element data includes the meteorological data such as temperature, humidity, air pressure, wind speed, and precipitation observed by the automatic weather station. The data types of the working electrical parameters and meteorological element data are both in numerical format. The environmental audio data includes sound information indicating the working environment of the automatic weather station, such as wind and rain sounds. The data type of the environmental audio data is in audio format. The environmental video data is the image of the working environment of the automatic weather station taken. The data type of the environmental video data is in video format.

[0038] (2) Data preprocessing is performed on the obtained original working electrical parameters, original meteorological element data, original environmental audio data and original environmental video data respectively.

[0039] The acquired numerical data, such as raw operating electrical parameters and raw meteorological element data, may contain missing values ​​and data anomalies. First, we address missing and outlier values ​​in the raw operating electrical parameters and raw meteorological element data: We filter missing and outlier values ​​for the raw meteorological element data at each moment; and filter missing and outlier values ​​for the raw operating electrical parameters at each moment. Given the time-dependent nature of the raw operating electrical parameters and raw meteorological element data, we can employ missing value processing methods such as time series interpolation or forward filling. For outliers, we can use the sliding window IQR method to identify and process time series anomalies.

[0040] The filtered raw operating electrical parameters and raw meteorological element data are then normalized, standardizing the numerical data to the same scale. This results in normalized raw meteorological element data and normalized raw operating electrical parameters. Normalizing the raw operating electrical parameters and raw meteorological element data facilitates accurate data feature analysis. A sliding window MinMaxScaler method can be used to normalize the raw data based on the maximum and minimum values ​​of the time series data.

[0041] The acquired raw ambient audio data typically contains irrelevant noise information, and it is necessary to perform noise reduction processing on the raw ambient audio data at each moment and extract audio features to obtain the raw ambient audio features. Extracting audio features involves dividing the denoised raw ambient audio data into frames and then extracting audio features for each frame of raw ambient audio data. Any audio feature extraction method can be selected based on actual needs, such as using the MFCC method to convert the audio signal into Mel frequencies and then performing cepstrum analysis. The specific content belongs to the existing technology and will not be repeated in this application.

[0042] Each frame of the acquired raw environmental video data is an environmental image. Multiple environmental images are captured at each moment. Each environmental image is relatively large, and each target to be detected within it is also relatively large. Therefore, even with such large image sizes, the target information can be recognized. To improve the model's efficiency, the image size of the raw environmental video data at each moment is adjusted, for example, to 224×224, while preserving the valid information. Image enhancement is then performed to further improve image quality, resulting in the processed raw environmental video data. For example, the ACE method is used to enhance image contrast.

[0043] (3) Since the sampling frequency of multimodal meteorological time series data of each data type is inconsistent, for example, numerical format data samples one working electrical parameter and original meteorological element data per second, audio format data samples hundreds or thousands of environmental audio data per second, and video format data samples dozens of environmental images per second. Therefore, it is necessary to time-align the original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment according to the same time resolution. The original meteorological element data, original working electrical parameters, original environmental audio data, and original environmental video data with the lowest time resolution are selected as the unified time resolution for sampling. The data with high time resolution are interpolated to extract the data at the specified sampling moment, and the multimodal meteorological time series data with synchronized meteorological time series data of each modality at multiple consecutive moments are obtained to form the meteorological training samples of the automatic weather station in the current operating state.

[0044] Step 2: cascade the multimodal feature extraction module, multimodal feature fusion module and time series anomaly detection module in sequence to build the network architecture of the automatic weather station operation status detection model. Please refer to Figure 2 The network architecture diagram shown in Figure 1 is shown in Figure 2. The automatic weather station operation status detection model is an end-to-end model architecture that takes meteorological training samples as input and the operation status of the automatic weather station as output. The automatic weather station operation status detection model is trained to obtain the meteorological training samples. Figure 3 As shown, the specific process is as follows:

[0045] (1) After the meteorological training samples are input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the temporal features of the meteorological numerical sequence, environmental audio sequence, and environmental video sequence in the meteorological training samples respectively.

[0046] The multimodal feature extraction module extracts time series features based on the data characteristics of each modal meteorological time series data. In one embodiment, the multimodal feature extraction module uses the Informer model to extract features from meteorological numerical sequences to obtain numerical time series features; uses the Audio Spectrogram Transformer model to extract features from environmental audio sequences to obtain audio time series features; and uses the Swin Transformer model to extract features from environmental video sequences to obtain video time series features. The unified use of a transform-based model to extract time series features ensures that the time series features of each modal meteorological time series data are fully extracted and that the time series features of each modal meteorological time series data are effectively integrated.

[0047] (2) The multimodal feature fusion module interactively fuses the temporal features of the meteorological numerical sequence, environmental audio sequence, and environmental video sequence to obtain a temporal feature sequence.

[0048] In one embodiment, the multimodal feature fusion module uses a transformer-based self-attention mechanism to generate feature encodings for the temporal features of the meteorological numerical sequence, the environmental audio sequence, and the environmental video sequence. It then uses a cross-modal attention mechanism and a multi-head spatiotemporal attention mechanism to interactively fuse the information encoded in each feature to generate a temporal feature sequence. The cross-modal attention mechanism can accurately capture the correlation between different modalities, thereby improving the effectiveness of anomaly detection.

[0049] (3) The time series anomaly detection module includes a time series VAE model and a classifier. The time series VAE model includes an encoder and a decoder. The encoder uses a multi-layer GRU network to map the fused time series feature sequence to the compressed potential representation sequence in the latent space, thereby capturing the spatiotemporal features in the time series; the decoder reconstructs the reconstruction error sequence from the latent space. Ensure that the model can recover key information from the compressed latent representation sequence. The classifier reconstructs the error sequence The classification is performed to obtain the predicted classification result of the operation status of the automatic weather station. The classifier can adopt any pre-trained model of the existing technology.

[0050] The automatic weather station operation status detection model is trained multiple times using meteorological training samples. The specific process is as follows: the model parameters of the automatic weather station operation status detection model are initialized, the model is trained using meteorological training samples, and the model parameters are adjusted by minimizing the loss function and using an optimizer to obtain the automatic weather station operation status detection model. The optimizer uses an adaptive gradient descent algorithm, such as the Adam optimizer.

[0051] In one embodiment, considering the temporal characteristics of multimodal meteorological time series data, the loss function of the automatic weather station operation status detection model of the present application introduces time regularization loss on the basis of the loss function of the traditional VAE model. The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error sequence The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data, time regularization loss Used to characterize the reconstruction error sequence The ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the lower the temporal regularization loss. The smaller.

[0052] In one embodiment, the temporal regularization loss It includes the timing reconstruction loss L and the timing alignment loss L2. The timing reconstruction loss L is used to characterize the reconstruction error sequence The timing dynamic characteristics of the time series, the timing alignment loss L2 is used to characterize the reconstruction error sequence The time series similarity with the time series z composed of multimodal meteorological time series data, minimizing the time series alignment loss L2 can make the reconstructed error sequence Better represent the time series characteristics of the time series z composed of multimodal meteorological time series data. Therefore, the reconstructed error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the temporal similarity with the time series z composed of multimodal meteorological time series data, the smaller the temporal alignment loss L2.

[0053] Timing dynamics include reconstruction error sequence Smooth dynamic characteristics and reconstruction error sequence The global dynamic characteristics of the time series reconstruction loss L = L1 + L3, the reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstruction error sequence The higher the independence at different moments, the smaller the covariance regularization loss L3, and the reconstruction error sequence The more accurate the global dynamic characteristics captured.

[0054] The smoothness loss L1 can characterize the reconstruction error sequence Smoothness, when the reconstruction error sequence When the data at a certain moment in the process undergoes an abnormal mutation, the error sequence is reconstructed. The smoothness of the calculated smoothness loss L1 will become worse. In one embodiment, the reconstruction error sequence is reflected by calculating the distance between the reconstruction errors at adjacent moments. Smoothness, smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The sequence length of . Minimizing the smoothness loss L1 can effectively capture the reconstruction error sequence The smooth dynamic characteristics of , which is particularly important for anomaly detection.

[0055] In order to enhance the ability of the automatic weather station operation status detection model to capture the long-term dependency between multimodal meteorological time series data at different times, the independence of the reconstruction error at each moment is enhanced to achieve the separability of abnormal data. In one embodiment, the covariance regularization loss L3 is used to enhance the reconstruction error sequence. Independence at different moments, covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm. The covariance regularization loss is achieved by reconstructing the error sequence The covariance matrix of is regularized to achieve the purpose of decorrelation, thereby reducing the redundant information and dependencies between different moments. At the same time, since the covariance matrix takes into account the reconstruction error sequence The global dynamic characteristics of the regularization process will not completely eliminate the long-term dependence, but will preserve the reconstruction error sequence by constraining the overall covariance. Therefore, the covariance regularization loss L3 is adopted to enable the automatic weather station operation status detection to capture the global dynamic characteristics while decorrelating, ensuring the effective representation of long-term dependencies.

[0056] In order to improve the reconstruction error sequence For the accurate expression of the time series z composed of multimodal meteorological time series data, the time series alignment loss L2 is used to characterize the reconstruction error sequence The temporal similarity of the time series z composed of multimodal meteorological time series data. In one embodiment, the temporal alignment loss Indicates the time series z and reconstruction error series composed of multimodal meteorological time series data calculated using the dynamic time warping method Dynamic Time Warping (DTW) is an alignment technique that can handle the speed inconsistency problem in time series data. By dynamically adjusting the time series z composed of multimodal meteorological time series data and reconstructing the error sequence The distance at each moment is used to achieve timing alignment, thereby ensuring the reconstruction of the error sequence It can more accurately represent the time series characteristics of the time series z composed of multimodal meteorological time series data.

[0057] Step 3: Obtain multimodal meteorological time series data for the automatic weather station at any operating moment and for multiple consecutive historical operating moments before the operating moment to construct the time series to be tested. The method for constructing the time series to be tested is the same as the method for constructing meteorological training samples in Step 2. The trained automatic weather station operating status detection model is used to determine the operating status of the automatic weather station at the current operating moment based on the time series to be tested.

[0058] The above description is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and variations directly derived or imagined by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the scope of protection of the present application.

Claims

1. A method for detecting the operating status of an automatic weather station based on multimodal data, characterized in that: The automatic weather station operating status detection method comprises: Obtain meteorological training samples of the automatic weather station under different operating states. The meteorological training samples of the automatic weather station under each operating state include a time series consisting of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment include the working electrical parameters of the automatic weather station, meteorological element data observed by the automatic weather station, environmental audio characteristics of the working environment of the automatic weather station, and environmental video data of the working environment of the automatic weather station. Each group of meteorological training samples includes a meteorological numerical sequence consisting of the working electrical parameters and meteorological element data at different moments, an environmental audio sequence consisting of environmental audio characteristics at different moments, and an environmental video sequence consisting of environmental video data at different moments. A multimodal feature extraction module, a multimodal feature fusion module and a time series anomaly detection module are sequentially cascaded to construct a network architecture of an automatic weather station operation status detection model, and a meteorological training sample is used as input and the operation status of the automatic weather station is used as output to train the automatic weather station operation status detection model; after the meteorological training sample is input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence in the meteorological training sample respectively, the multimodal feature fusion module performs information interaction fusion on the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence to obtain a time series feature sequence, the time series anomaly detection module includes a time series VAE model and a classifier, and the time series VAE model compresses and reconstructs the fused time series feature sequence to obtain a reconstruction error sequence Classifier pair reconstruction error sequence Perform classification to obtain the prediction classification results of the automatic weather station's operating status; The multimodal meteorological time series data of the automatic weather station at any operating time and multiple consecutive historical operating times before it are obtained to form a time series to be detected. The operating status of the automatic weather station at the current operating time is determined based on the time series to be detected using a trained automatic weather station operating status detection model.

2. The method for detecting the operating status of an automatic weather station according to claim 1, wherein: The automatic weather station operation status detection model obtained by training includes: Initialize the model parameters of the automatic weather station operation status detection model, use meteorological training samples to train the automatic weather station operation status detection model, and obtain the automatic weather station operation status detection model by minimizing the loss function; The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error sequence The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data, time regularization loss Used to characterize the reconstruction error sequence The ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the lower the temporal regularization loss. The smaller.

3. The method for detecting the operating status of an automatic weather station according to claim 2, wherein: Temporal Regularization Loss Including timing reconstruction loss L and timing alignment loss L2; ​​reconstruction error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the temporal similarity with the time series z composed of multimodal meteorological time series data, the smaller the temporal alignment loss L2.

4. The method for detecting the operating status of an automatic weather station according to claim 3, wherein: Time series reconstruction loss L = L1 + L3, reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstruction error sequence The higher the independence at different moments, the smaller the covariance regularization loss L3.

5. The method for detecting the operating status of an automatic weather station according to claim 4, wherein: The smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The sequence length.

6. The method for detecting the operating status of an automatic weather station according to claim 4, wherein: The covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm.

7. The method for detecting the operating status of an automatic weather station according to claim 3, wherein: The timing alignment loss Indicates the time series z and reconstruction error series composed of multimodal meteorological time series data calculated using the dynamic time warping method The temporal similarity at each moment.

8. The method for detecting the operating status of an automatic weather station according to claim 1, wherein: The multimodal feature extraction module uses the Informer model to extract features from the meteorological numerical sequence to obtain numerical time series features; uses the AudioSpectrogram Transformer model to extract features from the environmental audio sequence to obtain audio time series features; and uses the Swin Transformer model to extract features from the environmental video sequence to obtain video time series features.

9. The method for detecting the operating status of an automatic weather station according to claim 1, wherein: The multimodal feature fusion module obtains the feature encoding of the temporal features of the meteorological numerical sequence, environmental audio sequence and environmental video sequence respectively through the transformer-based self-attention mechanism, and uses the cross-modal attention mechanism and the multi-head spatiotemporal attention mechanism to interactively fuse the information of each feature encoding to obtain a temporal feature sequence.

10. The method for detecting the operating status of an automatic weather station according to claim 1, wherein: The obtaining of meteorological training samples of the automatic weather station in each operating state comprises: During the operation of the automatic weather station in the operating state, obtaining original working electrical parameters, observed original meteorological element data, and original environmental audio data and original environmental video data of the working environment of the automatic weather station at different times; The original meteorological element data at each moment is processed and screened for missing values ​​and outliers, and the scale of the screened original meteorological element data is normalized to obtain the normalized original meteorological element data; The original working electrical parameters at each moment are screened for missing values ​​and outliers, and the scales of the screened original working electrical parameters are normalized to obtain normalized original working electrical parameters; Perform noise reduction on the original ambient audio data at each moment and extract audio features to obtain original ambient audio features; Adjusting the image size of the original environment video data at each moment and performing image enhancement to obtain processed original environment video data; The original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment are time-aligned according to the same time resolution to obtain multimodal meteorological time series data at multiple consecutive moments to constitute the meteorological training samples of the automatic weather station in the current operating state.

Images