Infrasound data enhancement method and device based on diffusion model

The improved backdiffusion process, achieved through a diffusion model and a hybrid attention mechanism, addresses the issues of insufficient infrasound data acquisition and GAN model instability, generating a high-quality infrasound dataset and enhancing the model's performance and generalization ability.

CN118916608APending Publication Date: 2024-11-08BEIJING UNIV OF POSTS & TELECOMM
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410709631.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-06-03
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

In existing technologies, infrasound data acquisition is difficult, data is insufficient, GAN model training is unstable, the data distribution after diffusion is uneven, and the model cannot effectively capture the peak value and frequency domain characteristics of infrasound data during the back diffusion process.

Method used

Infrasound data augmentation is performed using a diffusion model. By adjusting the noise granularity during the diffusion process and introducing a hybrid attention mechanism, the UNet model is combined for backdiffusion. The noise granularity is dynamically adjusted and a hybrid attention mechanism is used during the backdiffusion process to improve feature extraction capabilities.

Benefits of technology

It improves the accuracy and quality of infrasound data generation, enhances the generalization ability of the model, and can better capture the time and frequency domain features of infrasound data, generating high-quality datasets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118916608A_ABST
    Figure CN118916608A_ABST
Patent Text Reader

Abstract

The invention provides an infrasound data enhancement method and device based on a diffusion model. The method comprises a data cleaning device, a diffusion device, a reverse diffusion device and a data enhancement device. The data cleaning device processes original data through event detection to obtain a training data set. The diffusion device adds noise to the training data set by using a diffusion process to obtain a noise-added data set; the noise adding granularity is dynamically adjusted according to factors such as event occurrence time, so that noise adding data tends to specific distribution. The back diffusion device uses a back diffusion process to restore the noise-added data to obtain a restored data set; in order to solve the problem that a united network (UNet) cannot capture peak features and frequency domain features of a training data set, a mixed attention mechanism is introduced, including a time domain self-attention mechanism and a frequency domain self-attention mechanism, and the effect of data enhancement is improved. The data enhancement device generates random noise through a noise generator and inputs the random noise to the back diffusion device, and finally an enhanced data set is obtained.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method and device for enhancing time series data with novel structure and better diversity, and more specifically, to a method and device for enhancing infrasound data based on a diffusion model, which is an innovative data enhancement method and device involving multiple technical fields such as infrasound signal data processing, deep learning technology, diffusion model, and attention mechanism. Background Art

[0002] Infrasound Data Enhancement:

[0003] Sound waves propagating in the atmosphere can be divided into three types: infrasound, audible sound, and ultrasonic waves. Among them, infrasound is a low-frequency sound wave with a frequency range of 0.002-20Hz. It is generated by weak vibrations on the ground and in the air, and has the characteristics of wide sources, low frequency, low attenuation, long propagation, and strong penetration. Therefore, infrasound is of great significance in monitoring and studying natural disasters, such as earthquakes, volcanic eruptions, storms, etc.

[0004] With the continuous development of deep learning and machine learning technologies, a large number of algorithms have been applied to event detection and classification of infrasound data. For example, the STA / LTA algorithm can be used for infrasound event detection, while models such as support vector machine (SVM) and long short-term memory network (LSTM) can be used for event classification. However, these algorithm models require a large amount of data sets to support them, and infrasound data collection is difficult, so the event data obtained by scholars is very limited.

[0005] In order to solve the problem of insufficient data, the current solution is to use existing event data for data enhancement. Data enhancement can generate more training data by performing a series of transformations and expansions on existing data, thereby improving the robustness and generalization ability of the model. For example, more data sets can be generated by translating, rotating, flipping, adding noise, and other operations on the original data to meet the training needs of machine learning or deep learning models. Data enhancement can also be trained on existing small sample data sets through deep learning models, and the trained model can be used to generate high-quality sample data with similar characteristics to the original data set, thereby achieving the effect of expanding the data set.

[0006] Diffusion Model:

[0007] The Diffusion Model is a generative model used to model and sample high-dimensional data. The Diffusion Model was originally proposed by OpenAI. Its core idea is to regard high-dimensional data as a process that evolves from a simple distribution through a series of random perturbations, and to model high-dimensional data by modeling this evolution process. The Diffusion Model achieves modeling by introducing the diffusion process and the reverse diffusion equation. In the Diffusion Model, each data point is regarded as a point in a high-dimensional space, which is constantly perturbed and gradually evolves into the target distribution.

[0008] The core of the diffusion model is the diffusion process based on the Langevin equation, which simulates the perturbation and evolution of data points through a series of time steps. After the distribution of the initial data is given, Gaussian noise can be continuously added to the distribution. The standard deviation of this noise is determined by the hyperparameter setting, and the mean is also determined by the hyperparameter and the number of diffusions. This process is a Markov process. As the number of diffusions increases, the distribution of data points will gradually become more blurred and smoother, and eventually tend to an independent Gaussian distribution.

[0009] The back-diffusion process of the diffusion model simulates the evolution of data points by modeling the back-diffusion equation, which is the reverse process of the Langevin equation, which describes how to sample data points from the target distribution.

[0010] In short, it is to add noise to the data through the diffusion process until the data has completely lost its original appearance, and then gradually restore the noisy data through a deep learning network model. The generation process is to randomly input noise, and then use the trained deep learning model to restore the noise data to the sample data we need.

[0011] Attention Mechanism:

[0012] The attention mechanism is a technique used to strengthen the model's attention to certain parts of the input data, making it more effective in processing the input data. Its core idea is to introduce a weight distribution mechanism into the model so that the model can assign different weights to different parts of the input data.

[0013] Typically, the attention mechanism can be divided into the following steps:

[0014] 1. Create attention weights: Based on the input data, the attention weights for each input position are calculated in some way.

[0015] 2. Weighted aggregation: The input data is weighted and aggregated according to the corresponding attention weights to obtain a weighted representation.

[0016] 3. Output: Send the weighted representation to the next layer of network for processing.

[0017] Common attention mechanisms include Softmax attention, Sigmoid attention, self-attention, non-self-attention, etc. Among them, Softmax attention and Sigmoid attention are mainly used to calculate attention weights, while self-attention and non-self-attention are specific ways to implement the attention mechanism. Summary of the invention

[0018] The technical problems to be solved by the present invention are:

[0019] In view of the shortcomings of existing research methods, the present invention makes targeted improvements. In view of the problems that the GAN model training in the prior art is unstable and the degree of refinement of the generated data cannot be controlled, the present invention uses a diffusion model to enhance infrasound data. The core idea of ​​the diffusion model is to add noise to the data through a diffusion process, and then use a deep learning model to denoise and restore the noisy data through a reverse diffusion process. Both the diffusion process and the reverse diffusion process are Markov processes, and only rely on a deep learning model in the reverse diffusion process. Based on a strong mathematical foundation, the problem of unstable GAN model training is solved. Because the samples are generated by the gradual reverse diffusion of noise, rather than being generated directly from a noise vector in one step. And the diffusion model can also achieve more precise control.

[0020] In view of the random noise of the data after diffusion in the original diffusion model, the final diffused data does not tend to a specific distribution well. In order to solve the problem of uneven distribution of the data after noise in the original process, the noise granularity is dynamically adjusted according to relevant factors such as the time of event occurrence, so that the noisy data tends to a specific distribution. In view of the problem that the model using the joint network (UNet) in the original reverse diffusion process cannot capture the peak characteristics and frequency domain characteristics of the training data set, a hybrid attention mechanism is introduced, including the time domain self-attention mechanism and the frequency domain self-attention mechanism, to improve the feature extraction ability of the original model in the time and frequency domains, thereby improving the effect of data enhancement.

[0021] The complete technical solution provided by the present invention is:

[0022] The present invention mainly consists of four devices, namely a data cleaning device, a diffusion device, a reverse diffusion device and a data enhancement device.

[0023] Data cleaning device

[0024] Infrasound data refers to sound waves below 20Hz. Unlike ordinary pictures and voice text data, the construction of infrasound data sets requires relevant processing according to their characteristics. Therefore, the collected data needs to be filtered first, and a bandpass filter with a lower limit of 0Hz and an upper limit of 20Hz is selected to filter out unnecessary interference. In addition, the infrasound data generated by different events present different characteristics. For example, the infrasound data generated by sudden events such as lightning and volcanic eruptions are short and have obvious mutations, while the infrasound data generated by long-term events such as typhoons and micro-air pressure have a longer duration and a larger amplitude. Therefore, for sudden events, the STA / LTA algorithm is used for event detection, and the relevant data is intercepted according to the detected event start time and end time. For long-term events, the data length is directly intercepted according to the required unified data length. Finally, the part of the data with insufficient length is filled with 0 on both sides to ensure that all data lengths are consistent.

[0025] Before training the deep learning model, the processed data needs to be standardized and normalized. The commonly used normalization method is to subtract the mean of each data value and divide it by the variance. There are two normalization methods: linear normalization and nonlinear normalization. The present invention adopts the maximum value normalization method to ensure that the difference of the restored data will not be amplified, thereby ensuring the validity of the data.

[0026] Therefore, the key to the present invention is to perform special processing on the infrasound data set to facilitate the training of the deep learning model. The processing includes steps such as filtering, event detection and data normalization to ensure the quality and validity of the data.

[0027] Diffusion device

[0028] The diffusion device mainly adds noise to the data set processed by the data cleaning device through the diffusion process in the original diffusion model. It adds noise to the initial data distribution to obtain a series of data. This process can be described by a Markov process, and the data at each moment is only related to the previous moment. The specific formula is:

[0029]

[0030] where β1,…,β T is a hyperparameter of the Gaussian distribution variance, which is generally set to a linear interpolation of 0.0001 to 0.02. After T diffusions, the data will become an independent Gaussian distribution.

[0031] The infrasound data generated by different events have different characteristics. For example, sudden events such as earthquakes and volcanic eruptions generate higher peak values ​​of infrasound data, while long-term events such as typhoons and micro-air pressure generate lower peak values ​​of infrasound data. In order to make the final data tend to the same distribution, a weight can be added to each piece of data. The weight w is the maximum peak value of all sudden events divided by the maximum peak value of the current data. For long-term events, they can be divided into three time periods: the initial period, the peak period, and the final period. The initial period refers to the time period when the event just occurs, the peak period is the duration period of the event, and the final period refers to the time period when the event is about to end. The data intensity of the initial and final time periods is relatively low, so their weights w can be set to 1.2, while the data intensity of the peak time period is higher, and its weight can be set to 1. The formula for the improved diffusion process is as follows:

[0032]

[0033] where α t =1-β t ; z t-1 is the random noise generated by the noise generator; w is the calculated weight value.

[0034] Reverse diffusion device

[0035] The reverse diffusion device mainly trains the deep learning model through the reverse diffusion process in the original diffusion model to estimate the noise added to the original data set by the diffusion process, and restores the data according to the noise output by the model. The specific formula is as follows:

[0036]

[0037] α t 1-β t ,∈ θ (x t , t) is the noise estimated by the deep learning model, σ t z is the noise randomly generated by the noise generator.

[0038] Given an original data x0, the data is denoised through the diffusion process, and finally the data x is obtained. t By training the deep learning model, we can estimate the noise ∈ added to the diffusion process at this moment. θ (x t , t). After restoring the data at time t according to the above formula, multiple back-diffusion processes are performed step by step to finally obtain the restored data x0. In the back-diffusion process, the UNet model combined with the hybrid attention mechanism is selected as the neural network model. The main reason is that it can effectively process the spatial information in the data and extract features at different levels.

[0039] The UNet model is a convolutional neural network structure, which consists of a downsampling module, an intermediate smoothing module, and an upsampling module. It can effectively extract multi-scale features in the data and combine these features for tasks such as classification or segmentation. And the modules are all composed of residual blocks. In this invention, the original residual blocks are improved, and a hybrid attention mechanism is added before the model output. Specifically, Figure 3 As shown in the figure. Because infrasound data is a kind of time series data, the correlation between the previous and the next is strong, and in addition to the time domain features, infrasound data also has strong frequency domain features. The hybrid attention mechanism can effectively improve the problem of insufficient feature extraction ability of the original model. The diffusion model is a sequence model whose input is an initial data distribution, and after multiple iterations and diffusions, a series of outputs are obtained. These outputs may contain a lot of redundant information, and the hybrid attention mechanism can help the model better capture the key features in each output, thereby improving the performance of the model.

[0040] Therefore, adding a hybrid attention mechanism in the back diffusion process can improve the model performance, enable the model to better capture the time domain and frequency domain features in the data, and help the model more accurately predict the noise added at a certain moment.

[0041] Data Enhancement Device

[0042] Model training is performed based on the processed data set and the constructed diffusion device and reverse diffusion device, and the trained model is saved.

[0043] The specific processing flow of the data enhancement device is as follows:

[0044] 1. Generate random noise with a normal distribution (N(0,1)) with a mean of 0 and a variance of 1 through a noise generator, and the dimension is the same as the original dataset.

[0045] 2. Input the noise data into the reverse diffusion device to perform a data restoration.

[0046] 3. Repeat step 2 until high-quality data is generated.

[0047] Beneficial effects brought by the technical solution of the present invention:

[0048] The present invention improves the diffusion process equation in the diffusion model and adds the concept of weight, so that the diffused data tends to be more similar to the same distribution, thereby improving the accuracy of data enhancement. At the same time, in the reverse diffusion model, the present invention adds a hybrid attention mechanism, so that the model can more accurately identify the time domain features and frequency domain features in the infrasound data, and more accurately predict the noise added at a certain moment, thereby improving the quality of the generated data. The hybrid attention mechanism can help the model better capture the key information in the input data, improve the performance and generalization ability of the model, and thus better generate infrasound data. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a device structure diagram of an infrasound data enhancement method and device based on a diffusion model of the present invention.

[0050] Figure 2 The present invention is a data processing flow chart of an infrasound data enhancement method and device based on a diffusion model.

[0051] Figure 3 The invention discloses a UNet network structure used in a method and device for enhancing infrasound data based on a diffusion model.

[0052] Figure 4 It is a residual block improved by the infrasound data enhancement method and device based on a diffusion model of the present invention. Figure 5 This is a structural diagram of a hybrid attention mechanism used in a diffusion model-based infrasound data enhancement method and device of the present invention. DETAILED DESCRIPTION

[0053] The present invention realizes a time series data enhancement method and device based on a diffusion model, improves the diffusion process method of the diffusion model and combines the hybrid attention mechanism into the model required in the reverse diffusion process. This method relies on the hyperparameter β t In this experiment, the selected diffusion times are 300, β t [0.0001, 0.02] is divided into 300 points. The pseudo code of the specific algorithm is as follows: Algorithm: Diffusion process

[0054]

[0055] Algorithm: Back diffusion process

[0056]

[0057]

Claims

1. A method and device for infrasound data enhancement based on a diffusion model, characterized in that: It includes the following devices: (1) Data cleaning device: The original data set is collected by multi-element infrasound array detection equipment located in multiple locations. According to the time of the event, the data collected by the detection equipment in the corresponding time period is obtained and marked to construct the original data set. Event types include but are not limited to lightning, typhoon mudslides, chemical explosions, aurora, etc. Use STA / LTA [1] The event detection algorithm extracts events from the raw data and truncates the event data to a fixed length, which is then standardized to obtain a consistent and comparable training dataset. (2) Diffusion device: By improving the diffusion process of the original diffusion model, the noise granularity can be dynamically adjusted according to factors such as the time of event occurrence. The training data set output by device (1) is subjected to noise processing so that the noised data set tends to the same distribution. (3) Reverse diffusion device: network model (UNet) in the reverse diffusion process of the original diffusion model [2] Improvements are made by introducing a hybrid attention mechanism to improve the feature extraction capability in the time and frequency domains and improve the data enhancement effect. The improved network model can better extract features in the time and frequency domains, and thus accurately restore the noisy data set output by the device (2). (4) Data enhancement device: A noise generator is used to generate random noise that follows a normal distribution (N(0, 1)) with a mean of 0 and a variance of 1, and the dimension is the same as the original data set. The random noise is input into the reverse diffusion device, and an enhanced data set consistent with the original data is obtained through processing.

2. The method and device for infrasound data enhancement based on a diffusion model according to claim 1, characterized in that: The specific process in the device (2) is as follows: (2.1) Determine the two hyperparameters required for the diffusion process in the diffusion model: the number of diffusions (T) and the hyperparameters used to control the diffusion rate (β1, ..., β T ). T represents the number of diffusions that the original data set needs to undergo, (β1,…,β T ) is used to control the diffusion rate and the degree of data distortion according to its setting size. (2.2) To dynamically adjust the noise granularity, the dynamic weight of each data is calculated. For sudden events, the weight (w) is obtained by dividing the maximum peak value of all data by the maximum peak value of the current data. For long-lasting events, the weight is dynamically determined according to the initial, peak and end stages of the event to adapt to the intensity changes in different time periods during the development of the event. (2.3) Diffusion of the original data set, using the formula where α t =1-β t ; z t-1 is the random noise generated by the noise generator; w is the weight value calculated by process (2.2).

3. The method and device for infrasound data enhancement based on a diffusion model according to claim 1, characterized in that: The network model structure involved in the device (3) is as follows: (3.1) Downsampling module (Down module): contains one-dimensional convolutional layer and residual block, which is used for data feature extraction. (3.2) Intermediate smoothing module (Mid module): It consists of multiple residual blocks and is used to perform convolution operations on the features extracted by the Down module to extract higher-level features. (3.3) Upsampling module (Up module): It contains a deconvolution layer and a residual block to restore the feature data extracted by the Down module and the Mid module.

4. The model structure according to claim 3, characterized in that: The original residual block and the improved residual block structure involved in the processes (3.1), (3.2) and (3.3) are as follows: (4.1) Original residual block structure: (4.1.1) Time embedding module: maps time t to a tensor t′ of the same shape as the input data x. The input data is batch normalized and processed by one-dimensional convolution, and then added to the time embedding tensor to generate a tensor p with time features. (4.1.2) Convolution module: Receive the tensor p output by the time embedding module as input, and extract richer feature representations through batch normalization and one-dimensional convolution operations. To prevent feature disappearance, the feature data extracted by the convolution module is added to the input data in (4.1.1) to obtain the output o, retaining the original input data information and merging the features with the original data. (4.2) Improved residual block structure: (4.2.1) Attention module: The time domain and frequency domain features of the data are further extracted through the hybrid attention mechanism. The convolution module output o of (4.1.2) is input into the hybrid attention mechanism, combining the features of the time domain and frequency domain to obtain more advanced and abstract features through weighting and integration.

5. The improved residual block according to claim 4, characterized in that: The structure of the hybrid attention mechanism involved in the structure (4.2.1) is as follows: (4.2.1.1) Temporal self-attention mechanism: Based on the input x, we first calculate the query (q), key (k), and value (v) respectively. Then, we perform a dot product on q and k to obtain the weight α in the temporal domain. t Finally, the weight α in the time domain t Multiply it with the value v to get the output o in the time domain t . (4.2.1.2) Frequency domain self-attention mechanism: Perform Fourier transform on q, k and v respectively to obtain frequency domain related Q, K and V, perform dot multiplication on Q and K represented in the frequency domain, and obtain the weight α in the frequency domain f This weight represents the degree of similarity in the frequency domain and is used to calculate the attention weight in the frequency domain. f Multiply it by the value V to get the output o in the frequency domain f ′. This output represents the result obtained by weighting the frequency domain attention mechanism. Finally, the frequency domain output o f ' Perform inverse Fourier transform to get the output o f . (4.2.1.3) Hybrid attention mechanism: The output of the temporal self-attention mechanism is t And the output o of the frequency domain self-attention mechanism f Add them together to get the final output o.