Automatic meteorological station operation state detection method based on multi-modal data

Abstract

The invention discloses an automatic meteorological station operation state detection method based on multi-modal data, and relates to the technical field of meteorological monitoring, and the method comprises the steps: obtaining meteorological training samples of an automatic meteorological station in different operation states, and the multi-modal feature extraction module, the multi-modal feature fusion module and the time sequence anomaly detection module are cascaded in sequence to construct a network architecture of the automatic meteorological station operation state detection model, and the automatic meteorological station operation state detection model is trained by taking the meteorological training sample as input and the operation state of the automatic meteorological station as output. And obtaining multi-modal meteorological time sequence data to form a to-be-detected time sequence, and determining the operation state of the automatic meteorological station at the current operation moment based on the to-be-detected time sequence by using the trained automatic meteorological station operation state detection model. By integrating the multi-mode meteorological time sequence data, the detection precision and efficiency of the running state of the automatic meteorological station are effectively improved, and stable running of the automatic meteorological station is guaranteed.

Description

Technical Field

[0001] The present application relates to the technical field of meteorological monitoring, and in particular to a method for detecting the operating status of an automatic weather station based on multimodal data. Background Art

[0002] Weather stations are an important part of the modern meteorological observation network, which can automatically and continuously collect various meteorological data. Compared with traditional weather stations, automatic weather stations have more comprehensive meteorological observation capabilities and can collect data on a variety of different environmental meteorological elements, so they are gradually being promoted and applied.

[0003] During the actual operation, the automatic weather station is prone to abnormal conditions due to various influences such as equipment failure, environmental interference or human factors, which leads to the decline of the observation quality of the automatic weather station and affects the reliability and accuracy of meteorological observation and forecast. The traditional abnormal state detection method is to monitor the working electrical parameters of the automatic weather station such as voltage, current, and temperature, and judge the working state of the automatic weather station to be abnormal and check it in time when the working electrical parameters are abnormal, but the accuracy and reliability of this detection method are not high. Summary of the invention

[0004] In view of the above problems and technical requirements, this application proposes a method for detecting the operating status of an automatic weather station based on multimodal data. The technical solution of this application is as follows:

[0005] A method for detecting the operating status of an automatic weather station based on multimodal data comprises the following steps:

[0006] Obtain meteorological training samples of the automatic weather station under different operating states. The meteorological training samples of the automatic weather station under each operating state include a time series composed of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment include working electrical parameters of the automatic weather station, meteorological element data observed by the automatic weather station, environmental audio characteristics of the working environment of the automatic weather station, and environmental video data of the working environment of the automatic weather station. Each group of meteorological training samples includes a meteorological numerical sequence composed of working electrical parameters and meteorological element data at different moments, an environmental audio sequence composed of environmental audio characteristics at different moments, and an environmental video sequence composed of environmental video data at different moments.

[0007] A multimodal feature extraction module, a multimodal feature fusion module and a time series anomaly detection module are cascaded in sequence to construct a network architecture of an automatic weather station operation status detection model, and the meteorological training sample is used as input, and the operation status of the automatic weather station is used as output to train the automatic weather station operation status detection model; after the meteorological training sample is input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence in the meteorological training sample respectively, and the multimodal feature fusion module performs information interactive fusion on the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence to obtain a time series feature sequence, and the time series anomaly detection module includes a time series VAE model and a classifier, and the time series VAE model compresses and reconstructs the fused time series feature sequence to obtain a reconstruction error sequence The classifier reconstructs the error sequence Classification is performed to obtain the classification results of the operation status prediction of the automatic weather station;

[0008] The multimodal meteorological time series data of the automatic weather station at any operating time and multiple consecutive historical operating times before it are obtained to form a time series to be detected. The operating status of the automatic weather station at the current operating time is determined based on the time series to be detected using the trained automatic weather station operating status detection model.

[0009] Its further technical solution is that the automatic weather station operation status detection model obtained by training includes:

[0010] Initialize the model parameters of the automatic weather station operation status detection model, use the meteorological training samples to train the automatic weather station operation status detection model, and obtain the automatic weather station operation status detection model by minimizing the loss function;

[0011] The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error series The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data and the time regularization loss Used to characterize the reconstruction error sequence The ability to capture the time series characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the greater the temporal regularization loss. The smaller.

[0012] Its further technical solution is that the time regularization loss Including timing reconstruction loss L and timing alignment loss L2; ​​reconstruction error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the time series similarity with the time series z composed of multimodal meteorological time series data, the smaller the time series alignment loss L2.

[0013] Its further technical solution is that the time series reconstruction loss L = L1 + L3, the reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstructed error sequence The higher the independence at different moments, the smaller the covariance regularization loss L3.

[0014] Its further technical solution is that the smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The length of the sequence.

[0015] Its further technical solution is covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm.

[0016] Its further technical solution is that the timing alignment loss It represents the time series z and reconstruction error series composed of multimodal meteorological time series data calculated by dynamic time warping method. The temporal similarity at each moment.

[0017] Its further technical solution is that the multimodal feature extraction module uses the Informer model to extract features from the meteorological numerical sequence to obtain numerical time series features; uses the Audio Spectrogram Transformer model to extract features from the environmental audio sequence to obtain audio time series features; and uses the Swin Transformer model to extract features from the environmental video sequence to obtain video time series features.

[0018] Its further technical solution is that the multimodal feature fusion module obtains the feature encoding of the temporal features of the meteorological numerical sequence, environmental audio sequence and environmental video sequence respectively through the transformer-based self-attention mechanism, and uses the cross-modal attention mechanism and the multi-head spatiotemporal attention mechanism to interactively fuse the information of each feature encoding to obtain the temporal feature sequence.

[0019] A further technical solution is to obtain meteorological training samples of the automatic weather station in each operating state, including:

[0020] During the operation of the automatic weather station in the operating state, original working electrical parameters of the automatic weather station at different times, original meteorological element data observed, and original environmental audio data and original environmental video data of the working environment are obtained;

[0021] The original meteorological element data at each moment are processed and screened for missing values ​​and abnormal values, and the scale of the screened original meteorological element data is normalized to obtain the normalized original meteorological element data;

[0022] The original working electrical parameters at each moment are processed and screened for missing values ​​and abnormal values, and the scales of the screened original working electrical parameters are normalized to obtain normalized original working electrical parameters;

[0023] Performing noise reduction processing on the original ambient audio data at each moment and extracting audio features to obtain original ambient audio features;

[0024] Adjust the image size of the original environment video data at each moment and perform image enhancement to obtain processed original environment video data;

[0025] The original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment are time-aligned according to the same time resolution to obtain multimodal meteorological time series data at multiple consecutive moments to constitute the meteorological training samples of the automatic weather station in the current operating state.

[0026] The beneficial technical effects of this application are:

[0027] The present application proposes a method for detecting the operating status of an automatic weather station based on multimodal data, which effectively integrates multimodal meteorological time series data by constructing an end-to-end automatic weather station operating status detection model. Compared with the traditional abnormal state detection method that only uses single-modal data, the automatic weather station operating status detection method of the present application makes full use of the time series characteristics of multimodal meteorological time series data, effectively improving the detection accuracy of the operating status of the automatic weather station.

[0028] Taking into account the data characteristics of each modal meteorological time series data and the fusion characteristics between each modal meteorological time series data, a transform-based deep learning model is used to extract the time series characteristics of each modal meteorological time series data respectively. This can not only ensure the full extraction of the unique time series characteristics of each modal meteorological time series data, but also ensure the full fusion of the extracted meteorological time series data of each modality, thus laying a solid foundation for further improving the accuracy of the automatic weather station operation status detection results.

[0029] By introducing the time regularization loss on the basis of the reconstruction loss and KL divergence of the traditional VAE model, it is ensured that the reconstructed error sequence reconstructed by the automatic weather station operation status detection model can capture the time series characteristics of the time series composed of multimodal meteorological time series data. The smoothness loss is used to improve the sensitivity of the automatic weather station operation status detection model to abnormal data. The covariance regularization loss is used to enhance the independence of the reconstructed error sequence at different times, ensuring that the automatic weather station operation status detection model can still capture the global dynamic characteristics of the time series while decorrelating, thereby effectively representing the long-term dependence of the time series. The time series alignment loss is used to ensure the time series similarity between the reconstructed error sequence and the time series composed of multimodal meteorological time series data, which helps to accurately capture the time series characteristics of the time series composed of multimodal meteorological time series data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of the method for detecting the operating status of an automatic weather station.

[0031] Figure 2 It is the network architecture of the automatic weather station operation status detection model.

[0032] Figure 3 It is the process of processing meteorological training samples by the automatic weather station operation status detection model. DETAILED DESCRIPTION

[0033] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0034] This application proposes a method for detecting the operating status of an automatic weather station based on multimodal data. Please refer to Figure 1 The operating state of the automatic weather station includes two states: normal and abnormal. When the operating state of the automatic weather station is abnormal, the acquired multimodal meteorological time series data will be abnormal. Therefore, this application detects the operating state of the automatic weather station based on the multimodal meteorological time series data. The specific steps are as follows:

[0035] Step 1, obtain meteorological training samples of the automatic weather station in different operating states. The meteorological training samples of the automatic weather station in each operating state include a time series composed of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment include the working electrical parameters of the automatic weather station, the meteorological element data observed by the automatic weather station, the environmental audio characteristics of the working environment of the automatic weather station, and the environmental video data of the working environment of the automatic weather station. Each group of meteorological training samples includes a meteorological numerical sequence composed of working electrical parameters and meteorological element data at different moments, an environmental audio sequence composed of environmental audio characteristics at different moments, and an environmental video sequence composed of environmental video data at different moments.

[0036] Meteorological training samples are constructed by acquiring multimodal meteorological time series data of the historical operating moments of the automatic weather station, each meteorological training sample corresponds to an operating state. In one embodiment, the process of constructing meteorological training samples of the automatic weather station in each operating state is as follows:

[0037] (1) When the automatic weather station is running in the corresponding operating state, the original working electrical parameters and the observed original meteorological element data of the automatic weather station at different times are obtained. The original environmental audio data of the working environment of the automatic weather station is obtained using an audio acquisition device such as a microphone. The original environmental video data is obtained using the camera of the automatic weather station. Among them, the original working electrical parameters and original meteorological element data are obtained from the observation system of the automatic weather station. The working electrical parameters include data reflecting the equipment status information such as the mainboard voltage, mainboard temperature, and signal strength of the automatic weather station; the meteorological element data include meteorological data such as temperature, humidity, air pressure, wind speed, and precipitation observed by the automatic weather station. The data types of the working electrical parameters and meteorological element data are both in numerical format. The environmental audio data includes sound information indicating the working environment of the automatic weather station, such as wind and rain, and the data type of the environmental audio data is in audio format. The environmental video data is the image of the working environment of the automatic weather station taken, and the data type of the environmental video data is in video format.

[0038] (2) Preprocess the acquired original working electrical parameters, original meteorological element data, original environmental audio data and original environmental video data respectively.

[0039] There may be missing values ​​and data anomalies in the obtained numerical format data such as the original working electrical parameters and original meteorological element data. First, the missing values ​​and anomalies of the original working electrical parameters and original meteorological element data are processed separately: the original meteorological element data at each moment are processed and screened for missing values ​​and anomalies; the original working electrical parameters at each moment are processed and screened for missing values ​​and anomalies. In view of the time series characteristics of the original working electrical parameters and original meteorological element data related to time, missing value processing methods such as time series interpolation or forward filling can be used. For outliers, the sliding window IQR method can be used to identify and process time series abnormal data.

[0040] Then, the filtered original working electrical parameters and original meteorological element data are normalized, and the numerical format data are standardized to the same scale to obtain the normalized original meteorological element data and the normalized original working electrical parameters. Normalizing the original working electrical parameters and the original meteorological element data helps to accurately analyze the data characteristics. The sliding window MinMaxScaler method can be used to normalize the original data according to the maximum and minimum values ​​of the time series data.

[0041] The original ambient audio data obtained usually contains irrelevant noise information. It is necessary to perform noise reduction processing on the original ambient audio data at each moment and extract audio features to obtain the original ambient audio features. Extracting audio features includes dividing the original ambient audio data after noise reduction into frames, and then extracting audio features for each frame of original ambient audio data. Any audio feature extraction method can be selected according to actual needs, such as using the MFCC method to convert the audio signal into Mel frequency, and then performing cepstrum analysis. The specific content belongs to the prior art and will not be repeated in this application.

[0042] Each frame of the acquired raw environmental video data is an environmental image, and each moment contains multiple environmental images. The size of each environmental image is large, and the size of each target to be detected in the environmental image is also large. Therefore, the information of the target to be detected can be recognized without such a large image size. In order to improve the efficiency of the model, the image size of the raw environmental video data at each moment is adjusted while ensuring the effective information in the image, such as adjusting it to an image of 224×224 size. Then, image enhancement is performed to further improve the image quality to obtain the processed raw environmental video data, for example, the ACE method is used to improve the image contrast.

[0043] (3) Since the sampling frequency of multimodal meteorological time series data of each data type is inconsistent, for example, the numerical format data samples one working electrical parameter and original meteorological element data per second, the audio format data samples hundreds or thousands of environmental audio data per second, and the video format data samples dozens of environmental images per second. Therefore, it is necessary to time-align the original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment according to the same time resolution. The original meteorological element data, original working electrical parameters, original environmental audio data, and original environmental video data with the lowest time resolution are selected as the unified time resolution for sampling. The data with high time resolution are interpolated to extract the data at the specified sampling time, and the multimodal meteorological time series data with each modal meteorological time series data at multiple consecutive moments are obtained. The synchronized multimodal meteorological time series data constitutes the meteorological training samples of the automatic weather station in the current operating state.

[0044] Step 2: cascade the multimodal feature extraction module, the multimodal feature fusion module and the time series anomaly detection module in sequence to construct the network architecture of the automatic weather station operation status detection model. Please refer to Figure 2 The network architecture diagram shown in Figure 1. The automatic weather station operation status detection model is an end-to-end model architecture that takes meteorological training samples as input and the operation status of the automatic weather station as output. The automatic weather station operation status detection model is trained to obtain the meteorological training samples. Figure 3 As shown, the specific process is as follows:

[0045] (1) After the meteorological training samples are input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the temporal features of the meteorological numerical sequence, environmental audio sequence, and environmental video sequence in the meteorological training samples respectively.

[0046] The multimodal feature extraction module extracts time series features based on the data characteristics of each modal meteorological time series data. In one embodiment, the multimodal feature extraction module uses the Informer model to extract features from the meteorological numerical sequence to obtain numerical time series features; uses the Audio Spectrogram Transformer model to extract features from the environmental audio sequence to obtain audio time series features; and uses the Swin Transformer model to extract features from the environmental video sequence to obtain video time series features. The unified use of a transform-based model to extract time series features can ensure that the time series features of each modal meteorological time series data are fully extracted and that the time series features of the subsequent meteorological time series data of each modality are effectively integrated.

[0047] (2) The multimodal feature fusion module interactively fuses the temporal features of the meteorological numerical sequence, environmental audio sequence, and environmental video sequence to obtain a temporal feature sequence.

[0048] In one embodiment, the multimodal feature fusion module obtains the feature encoding of the temporal features of the meteorological numerical sequence, the environmental audio sequence, and the environmental video sequence respectively through the transformer-based self-attention mechanism, and uses the cross-modal attention mechanism and the multi-head spatiotemporal attention mechanism to interactively fuse the information of each feature encoding to obtain the temporal feature sequence. The cross-modal attention mechanism can accurately capture the correlation between different modalities, thereby improving the effect of anomaly detection.

[0049] (3) The time series anomaly detection module includes a time series VAE model and a classifier. The time series VAE model includes an encoder and a decoder. The encoder uses a multi-layer GRU network to map the fused time series feature sequence to the latent space to obtain a compressed latent representation sequence, thereby capturing the spatiotemporal features in the time series; the decoder reconstructs the reconstruction error sequence from the latent space. Ensure that the model can recover key information from the compressed latent representation sequence. The classifier reconstructs the error sequence The classification is performed to obtain the predicted classification result of the operation status of the automatic weather station. The classifier can adopt any pre-trained model of the existing technology.

[0050] The automatic weather station operation status detection model is trained multiple times using meteorological training samples. The specific process is as follows: initialize the model parameters of the automatic weather station operation status detection model, train the automatic weather station operation status detection model using meteorological training samples, and adjust the model parameters by minimizing the loss function and the optimizer to obtain the automatic weather station operation status detection model. The optimizer uses an adaptive gradient descent algorithm, such as the Adam optimizer.

[0051] In one embodiment, considering the time series characteristics of multimodal meteorological time series data, the loss function of the automatic weather station operation status detection model of the present application introduces time regularization loss on the basis of the loss function of the traditional VAE model. The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error series The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data and the time regularization loss Used to characterize the reconstruction error sequence The ability to capture the time series characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the greater the temporal regularization loss. The smaller.

[0052] In one embodiment, the temporal regularization loss It includes the timing reconstruction loss L and the timing alignment loss L2. The timing reconstruction loss L is used to characterize the reconstruction error sequence The timing dynamic characteristics of the time series, the timing alignment loss L2 is used to characterize the reconstruction error sequence The time series similarity with the time series z composed of multimodal meteorological time series data, minimizing the time series alignment loss L2 can make the reconstructed error sequence It better represents the time series characteristics of the time series z composed of multimodal meteorological time series data. Therefore, the reconstructed error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the time series similarity with the time series z composed of multimodal meteorological time series data, the smaller the time series alignment loss L2.

[0053] Timing dynamics include reconstruction error sequence Smooth dynamic characteristics and reconstruction error sequence of The global dynamic characteristics of the time series reconstruction loss L = L1 + L3, the reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstructed error sequence The higher the independence at different times, the smaller the covariance regularization loss L3 is, and the reconstruction error sequence The more accurately the global dynamic characteristics are captured.

[0054] The smoothness loss L1 can characterize the reconstruction error sequence The smoothness of the reconstruction error sequence When the data at a certain moment in the process undergoes an abnormal mutation, the error sequence is reconstructed. The smoothness of the reconstructed error sequence is deteriorated, and the larger the calculated smoothness loss L1 is. In one embodiment, the distance between the reconstructed errors at adjacent moments is calculated to reflect the reconstructed error sequence. Smoothness, smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The sequence length of . Minimizing the smoothness loss L1 can effectively capture the reconstruction error sequence The smooth dynamic characteristics of , which is particularly important for anomaly detection.

[0055] In order to enhance the ability of the automatic weather station operation status detection model to capture the long-term dependence between multimodal meteorological time series data at different times, the independence of the reconstruction error at each moment is enhanced to achieve the separability of abnormal data. In one embodiment, the covariance regularization loss L3 is used to enhance the reconstruction error sequence Independence at different times, covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm. The covariance regularization loss is achieved by reconstructing the error sequence The covariance matrix of is regularized to achieve decorrelation, thereby reducing redundant information and dependencies between different moments. At the same time, since the covariance matrix takes into account the reconstruction error sequence The regularization process does not completely eliminate long-term dependencies, but preserves the reconstruction error sequence by constraining the overall covariance. Therefore, the covariance regularization loss L3 is adopted to enable the automatic weather station operation status detection to capture the global dynamic characteristics while decorrelating, ensuring the effective representation of long-term dependencies.

[0056] In order to improve the reconstruction error sequence For the accurate expression of the time series z composed of multimodal meteorological time series data, the time series alignment loss L2 is used to characterize the reconstruction error sequence The temporal similarity of the time series z composed of multimodal meteorological time series data. In one embodiment, the temporal alignment loss It represents the time series z and reconstruction error series composed of multimodal meteorological time series data calculated by dynamic time warping method. Dynamic Time Warping (DTW) is an alignment technique that can handle the problem of inconsistent speed in time series data. By dynamically adjusting the time series z composed of multimodal meteorological time series data and reconstructing the error series The distance at each moment is used to achieve timing alignment, thereby ensuring the reconstruction of the error sequence It can more accurately represent the time series characteristics of the time series z composed of multimodal meteorological time series data.

[0057] Step 3: Obtain multimodal meteorological time series data of the automatic weather station at any operating time and multiple consecutive historical operating times before it to form a time series to be detected. The construction method of the time series to be detected is consistent with the construction method of the meteorological training samples in step 2. The operating status of the automatic weather station at the current operating time is determined based on the time series to be detected using the trained automatic weather station operating status detection model.

[0058] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. A method for detecting the operating status of an automatic weather station based on multimodal data, characterized in that: The automatic weather station operation status detection method comprises: Obtain meteorological training samples of the automatic weather station under different operating states. The meteorological training samples of the automatic weather station under each operating state include a time series composed of multimodal meteorological time series data at multiple consecutive moments. The multimodal meteorological time series data at any moment include working electrical parameters of the automatic weather station, meteorological element data observed by the automatic weather station, environmental audio characteristics of the working environment of the automatic weather station, and environmental video data of the working environment of the automatic weather station. Each group of meteorological training samples includes a meteorological numerical sequence composed of working electrical parameters and meteorological element data at different moments, an environmental audio sequence composed of environmental audio characteristics at different moments, and an environmental video sequence composed of environmental video data at different moments. A multimodal feature extraction module, a multimodal feature fusion module and a time series anomaly detection module are sequentially cascaded to construct a network architecture of an automatic weather station operation status detection model, and a meteorological training sample is used as input, and the operation status of the automatic weather station is used as output to train the automatic weather station operation status detection model; after the meteorological training sample is input into the automatic weather station operation status detection model, the multimodal feature extraction module extracts the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence in the meteorological training sample respectively, the multimodal feature fusion module performs information interactive fusion on the time series features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence to obtain a time series feature sequence, the time series anomaly detection module includes a time series VAE model and a classifier, and the time series VAE model compresses and reconstructs the fused time series feature sequence to obtain a reconstruction error sequence The classifier reconstructs the error sequence Classification is performed to obtain the classification results of the operation status prediction of the automatic weather station; The multimodal meteorological time series data of the automatic weather station at any operating time and multiple consecutive historical operating times before it are obtained to form a time series to be detected, and the operating status of the automatic weather station at the current operating time is determined based on the time series to be detected using a trained automatic weather station operating status detection model.

2. The automatic weather station operation status detection method according to claim 1, characterized in that: The automatic weather station operation status detection model obtained by training includes: Initialize the model parameters of the automatic weather station operation status detection model, use the meteorological training samples to train the automatic weather station operation status detection model, and obtain the automatic weather station operation status detection model by minimizing the loss function; The loss function includes D KL Represents the time series z composed of the input multimodal meteorological time series data and the reconstruction error series The KL divergence between them, E represents the reconstruction error sequence The reconstruction error between the time series z composed of multimodal meteorological time series data and the time regularization loss Used to characterize the reconstruction error sequence The ability to capture the time series characteristics of the time series z composed of multimodal meteorological time series data and reconstruct the error series The stronger the ability to capture the temporal characteristics of the time series z composed of multimodal meteorological time series data, the greater the temporal regularization loss. The smaller.

3. The automatic weather station operation status detection method according to claim 2, characterized in that: Temporal Regularization Loss Including timing reconstruction loss L and timing alignment loss L2; ​​reconstruction error sequence The better the timing dynamic characteristics, the smaller the timing reconstruction loss L, and the reconstruction error sequence The higher the time series similarity with the time series z composed of multimodal meteorological time series data, the smaller the time series alignment loss L2.

4. The automatic weather station operation status detection method according to claim 3, characterized in that: Time series reconstruction loss L = L1 + L3, reconstruction error sequence The higher the smoothness at different moments, the smaller the smoothness loss L1; the reconstructed error sequence The higher the independence at different moments, the smaller the covariance regularization loss L3.

5. The automatic weather station operation status detection method according to claim 4, characterized in that: The smoothness loss in, is the reconstruction error sequence at time t, is the reconstruction error sequence at time t+1, T is the reconstruction error sequence The length of the sequence.

6. The method for detecting the operating status of an automatic weather station according to claim 4, characterized in that: The covariance regularization loss in, is the reconstruction error sequence The covariance matrix of , I is the identity matrix, ||·|| F is the Froubenius norm.

7. The automatic weather station operation status detection method according to claim 3, characterized in that: The timing alignment loss It represents the time series z and reconstruction error series composed of multimodal meteorological time series data calculated by dynamic time warping method. The temporal similarity at each moment.

8. The method for detecting the operating status of an automatic weather station according to claim 1, characterized in that: The multimodal feature extraction module uses the Informer model to extract features from the meteorological numerical sequence to obtain numerical time series features; uses the AudioSpectrogram Transformer model to extract features from the environmental audio sequence to obtain audio time series features; and uses the Swin Transformer model to extract features from the environmental video sequence to obtain video time series features.

9. The method for detecting the operating status of an automatic weather station according to claim 1, characterized in that: The multimodal feature fusion module obtains the feature encoding of the temporal features of the meteorological numerical sequence, the environmental audio sequence and the environmental video sequence respectively through the transformer-based self-attention mechanism, and uses the cross-modal attention mechanism and the multi-head spatiotemporal attention mechanism to interactively fuse the information of each feature encoding to obtain the temporal feature sequence.

10. The automatic weather station operation status detection method according to claim 1, characterized in that: The obtaining of meteorological training samples of the automatic weather station in each operating state comprises: During the operation of the automatic weather station in the operation state, obtaining original working electrical parameters of the automatic weather station at different times, original meteorological element data observed, and original environmental audio data and original environmental video data of the working environment; The original meteorological element data at each moment are processed and screened for missing values ​​and abnormal values, and the scale of the screened original meteorological element data is normalized to obtain the normalized original meteorological element data; The original working electrical parameters at each moment are processed and screened for missing values ​​and abnormal values, and the scales of the screened original working electrical parameters are normalized to obtain normalized original working electrical parameters; Performing noise reduction processing on the original ambient audio data at each moment and extracting audio features to obtain original ambient audio features; Adjust the image size of the original environment video data at each moment and perform image enhancement to obtain processed original environment video data; The original meteorological element data at each moment, the original working electrical parameters at each moment, the original environmental audio features at each moment, and the processed original environmental video data at each moment are time-aligned at the same time resolution to obtain multimodal meteorological time series data at multiple consecutive moments to constitute the meteorological training samples of the automatic weather station in the current operating state.

Images