Traffic infrastructure monitoring data probability abnormal value diagnosis method based on conditional diffusion model

Abstract

The invention provides a traffic infrastructure monitoring data probability abnormal value diagnosis method based on a conditional diffusion model. The method comprises the steps of calculating a hidden state by a conditional embedding network, adding white noise in a forward process, training a prediction model in combination with the hidden state, optimizing model parameters of a conditional network and a denoising network, generating a prediction result based on a trained network model, calculating a prediction mean value and a standard deviation, calculating a data point abnormal probability and the like. According to the method, by means of conditional processing and the like, an accurate prediction model can be obtained when abnormal data exists in a detection data set, and the abnormal degree of data points can be quantified while the abnormal data are diagnosed by calculating the abnormal probability; an effective abnormal value diagnosis method can be provided for the field of traffic infrastructure structure health monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of structural health monitoring, and in particular relates to a method for diagnosing probabilistic outliers of traffic infrastructure monitoring data based on a conditional diffusion model. Background Art

[0002] Abnormal data diagnosis is an important measure to ensure that the structural health monitoring of transportation infrastructure has a clean and reliable data set. Due to the instability of the data transmission system, or measurement noise and sensor failure, abnormal data will inevitably exist in the monitoring data. The existence of outliers may seriously affect the reliability of the monitoring system and the accuracy of subsequent damage diagnosis, status assessment, early warning and other work. Among the existing methods, techniques based on computer vision, density clustering and statistical regression are widely used. However, traditional computer vision methods usually require a large number of labeled data sets, and the recognition result is a piece of data rather than a specific data point; density clustering methods usually identify low-density areas as outliers, but such methods usually require pre-setting of hyperparameters such as the number of clusters, and the robustness of the method is poor; methods based on statistical regression are easily affected by outliers during model training, resulting in inaccurate model predictions, and there is no clear criterion for determining how much the actual value differs from the predicted value to be abnormal data.

[0003] The diffusion model is based on the diffusion process theory. It captures the dynamic characteristics of data changes by simulating the diffusion process of data in time series. Unlike traditional models, the conditional diffusion model can better handle high-dimensional and nonlinear data. Through conditional processing, it can reduce the impact of abnormal data in the training model process, and still obtain a reliable prediction model when there are outliers in the training data. By defining the abnormal probability of data points, the degree of abnormality of data points can be quantitatively determined. Considering the shortcomings of existing abnormal data diagnosis methods, combined with the needs of practical problems and the advantages of conditional diffusion models, a probabilistic outlier diagnosis method for traffic infrastructure monitoring data based on conditional diffusion model is established. Summary of the invention

[0004] The purpose of the present invention is to solve the problem of outlier diagnosis of traffic infrastructure monitoring data in the prior art, and proposes a probabilistic outlier diagnosis method for traffic infrastructure monitoring data based on a conditional diffusion model.

[0005] The present invention is implemented by the following technical scheme. The present invention proposes a method for diagnosing probabilistic outliers of traffic infrastructure monitoring data based on a conditional diffusion model. The method comprises the following steps:

[0006] Step 1: Construct a traffic infrastructure monitoring data set that needs to remove outliers, and determine the time step of each type of time series monitoring data according to the sampling frequency, so as to train the time series prediction model for outlier diagnosis;

[0007] Step 2: Establish a conditional diffusion model, determine the network architecture of the conditional embedding network module and the denoising network module, and train a time series prediction model based on the monitoring data set;

[0008] Step 3: Predict the monitoring data points at each moment based on the time series prediction model, calculate the residuals through the true value and the predicted value, and estimate the variance parameters of the prediction error;

[0009] Step 4: Define the abnormal probability of the data point based on the probability distribution of the predicted data point, calculate the abnormal probability of each data point, and identify the data point with an abnormal probability greater than 0.5 as an outlier.

[0010] Furthermore, in step 2, the network architecture of the conditional embedding network module is selected as an RNN network model, which is composed of GRU gated units and includes 4 hidden layers, each of which contains 30 hidden neurons. The time-related features at the prediction time t and the observations before multiple times t are used as context windows to form the covariate c t ; Combine it with the hidden state h at the previous moment t-2 As the input of the conditional embedding network module, the output of the conditional embedding network module is h t-1 :

[0011] h t-1 =GRU(c t ,h t-2 )

[0012] The specific expression is:

[0013] r t-1 =σ[W r ·(h t-2 ,c t )+b r ]

[0014] z t-1 =σ[W z ·(h t-2 ,c t )+b z ]

[0015]

[0016] in is the sigmoid activation function, is the Tanh activation function, ⊙ represents element-by-element multiplication, C θ = {W r ,W z ,W h ,b r ,b z ,bh} is the model parameter of the conditional embedding network module, which is initialized from the standard normal distribution is randomly generated, and then the parameters are optimized through the subsequent model training process, and h 0 =0.

[0017] Furthermore, in step 2, the original monitoring data at time t is Gradually increase Gaussian white noise, and assume that this process satisfies Markov property. After T steps of diffusion, the original monitoring data becomes pure Gaussian white noise, that is:

[0018]

[0019] The reverse process from Start typing h under given conditions t-1 Then, the original monitoring data is generated by gradually removing the noise According to the Bayesian formula, under given conditions Time conditional probability distribution Satisfies the following relationship:

[0020]

[0021] A neural network is used to fit the above inverse process, and the fitting result is Set to:

[0022]

[0023] The KL divergence of the two is used as the loss function:

[0024]

[0025] Where C is a constant that is independent of the network model parameter θ; Select the form for:

[0026]

[0027] Further simplifying it into KL divergence yields:

[0028]

[0029] where ε θ Represents the model parameters of the denoising network module.

[0030] Furthermore, in step 2, when the hidden state h corresponding to the input time t t-1 When used as input, the loss function of the time series forecasting model is:

[0031]

[0032] After shuffling the order, repeat T times and randomly select an m from (1,2,…,T), and use the gradient descent algorithm to update the network model parameters. Here, the network model parameters include the model parameters ε of the denoising network module. θ and the model parameters C of the conditional embedding network module θ Two parts:

[0033]

[0034] When the error loss is less than the threshold Δ=0.01, the update is stopped and the model training is completed.

[0035] Furthermore, in step 3, after the model is trained, given the hidden state h t-1 Under the condition of Medium Sampling

[0036]

[0037] When n=2,3,…,T When m=1, z=0; after repeating the sampling process T times, the predicted output of the model under given conditions at a certain time t can be obtained.

[0038] Furthermore, in step 3, the number of samples is set to N, and the predicted mean and standard deviation at a certain moment are calculated as follows:

[0039]

[0040]

[0041] Furthermore, in step 3, the variance of the prediction error is estimated by resampling. A certain proportion of data is randomly extracted from the original data L times, and the variance is calculated by unbiased estimation and the mean is obtained:

[0042]

[0043] Furthermore, in step 4, when the prediction mean and variance as well as the variance of the prediction error are given, the prediction distribution at a certain moment approximately satisfies the Gaussian distribution:

[0044]

[0045] The abnormal probability of the data point at this moment is:

[0046]

[0047] When the calculated abnormal probability is greater than 0.5, it is identified as an outlier.

[0048] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for diagnosing probabilistic outliers of traffic infrastructure monitoring data based on a conditional diffusion model are implemented.

[0049] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for diagnosing probabilistic outliers in traffic infrastructure monitoring data based on a conditional diffusion model.

[0050] Beneficial effects of the present invention:

[0051] The method for monitoring probabilistic outliers of traffic infrastructure monitoring data based on the conditional diffusion model described in the present invention can more accurately learn the implicit pattern characteristics of the monitoring data and is more robust to abnormal data. It can also quantify the degree of abnormality of data points while determining abnormal data by calculating the probability of abnormality. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0053] Figure 1 It is a flow chart of the method for diagnosing probabilistic outliers of traffic infrastructure monitoring data based on the conditional diffusion model of the present invention;

[0054] Figure 2 This is a diagnostic effect diagram of the method for diagnosing probabilistic outliers in traffic infrastructure monitoring data based on the conditional diffusion model described in the present invention; wherein the two sub-diagrams are respectively the diagnostic effects of two different types of monitoring data. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0056] The purpose of the present invention is to solve the shortcomings of the existing outlier diagnosis methods for traffic infrastructure monitoring data and propose a probabilistic outlier diagnosis method for traffic infrastructure monitoring data based on the conditional diffusion model. The method of the present invention can effectively diagnose outliers that may exist in traffic infrastructure monitoring data.

[0057] Combination Figure 1 , Figure 2 The present invention proposes a method for diagnosing probabilistic outliers in traffic infrastructure monitoring data based on a conditional diffusion model, which specifically includes the following steps:

[0058] Step 1: Construct a traffic infrastructure monitoring data set that needs to remove outliers, and determine the time step and other parameters of each type of time series monitoring data according to the sampling frequency, so as to train the time series prediction model for outlier diagnosis;

[0059] Step 2: Establish a conditional diffusion model, determine the network architecture of the conditional embedding network module and the denoising network module, and train a time series prediction model based on the monitoring data set;

[0060] Step 3: Predict the monitoring data points at each moment based on the time series prediction model, calculate the residuals through the true value and the predicted value, and estimate the variance parameters of the prediction error;

[0061] Step 4: Define the abnormal probability of the data point based on the probability distribution of the predicted data point, calculate the abnormal probability of each data point, and identify the data point with an abnormal probability greater than 0.5 as an outlier.

[0062] The step 2 is specifically as follows:

[0063] Step 2.1: The network architecture of the selected conditional embedding network module is an RNN network model, which is composed of GRU gated units and contains 4 hidden layers. Each hidden layer contains 30 hidden neurons. The time-related features at the prediction time t and the 60 observations before time t are used as context windows to form the covariate c. t ; Combine it with the hidden state h at the previous moment t-2 As the input of the conditional embedding network module, the output of the conditional embedding network module is h t-1 :

[0064] h t-1 =GRU(c t ,h t-2 )

[0065] The specific expression is:

[0066] r t-1 =σ[w r ·(h t-2 ,c t )+br ]

[0067] z t-1 =σ[W z ·(h t-2 ,c t )+b z ]

[0068]

[0069] in is the sigmoid activation function, is the Tanh activation function, ⊙ represents element-by-element multiplication, C θ = {W r ,W z ,W h ,b r ,b z ,b h} is the model parameter of the conditional embedding network module, which is initialized from the standard normal distribution is randomly generated, and then the parameters are optimized through the subsequent model training process, and h 0 =0.

[0070] Step 2.2: Original monitoring data at time t Gradually increase Gaussian white noise, and assume that this process satisfies Markov property. After T steps of diffusion, the original monitoring data becomes pure Gaussian white noise, that is:

[0071]

[0072] The reverse process from Start typing h under given conditions t-1 Then, the original monitoring data is generated by gradually removing the noise According to the Bayesian formula, under given conditions Time conditional probability distribution Satisfies the following relationship:

[0073]

[0074] A neural network (consisting of eight residual layer jump connections) is used to fit the above inverse process, and the fitting result is Set to:

[0075]

[0076] The KL divergence of the two (probability distribution and fitting results) is used as the loss function:

[0077]

[0078] Where C is a constant that is independent of the network model parameter θ; Select the form for:

[0079]

[0080] Further simplifying it into KL divergence yields:

[0081]

[0082] where ε θ Represents the model parameters of the denoising network module;

[0083] When the hidden state h corresponding to the input time t t-1 When used as input, the loss function of the time series forecasting model is:

[0084]

[0085] In order to further ensure the robustness of the model, the order is shuffled and repeated T times to randomly select an m from (1,2,…,T), and the gradient descent algorithm is used to update the network model parameters. Here, the network model parameters include the model parameters ε of the denoising network module θ and the model parameters C of the conditional embedding network module θ Two parts:

[0086]

[0087] When the error loss is less than the threshold Δ=0.01, the update is stopped and the model training is completed.

[0088] The step three is specifically as follows:

[0089] After the model is trained, given the hidden state h t-1 Under the condition of Medium Sampling

[0090]

[0091] When m=2,3,…,T When m=1, z=0; after repeating the sampling process T times, the predicted output of the model under given conditions at a certain time t can be obtained. Set the number of samples to N, and calculate the predicted mean and standard deviation at this moment as follows:

[0092]

[0093] The variance of the prediction error is estimated by resampling. A certain proportion of data is randomly extracted from the original data for L = 10,000 times, where G = 0.8N. The variance is calculated using an unbiased estimate and the mean is obtained:

[0094]

[0095] The step 4 is specifically as follows:

[0096] When the forecast mean and variance and the variance of the forecast error are given, the forecast distribution at a certain moment approximately satisfies the Gaussian distribution:

[0097]

[0098] The abnormal probability of the data point at this moment is:

[0099]

[0100] When the calculated abnormal probability is greater than 0.5, it is identified as a possible outlier.

[0101] Example

[0102] Figure 2 The application effect of the present invention in the actual monitoring data set is given. For a bridge monitoring database, the bridge wind speed monitoring data is used as the input-output of the conditional diffusion prediction model to perform outlier diagnosis on the bridge wind speed monitoring data. The specific operation steps are as follows:

[0103] The step 1 is specifically as follows: a data matrix consisting of wind speeds at two different measuring points in a bridge monitoring database is used as input-output. The sampling frequency of the monitoring data is 10 Hz, so the time step of the time series prediction model is set to 100 ms.

[0104] The second step is specifically: determine the day of the week, hour of the day, and other time-related features of the predicted data point according to the corresponding time point when the monitoring data is collected, and the monitoring data of the previous 60 time points as the lag features to form the covariate c t , the input contains 4 hidden layers, each hidden layer has 30 nodes in the GRU gated unit conditional network to generate the hidden state h at the current moment t-1 . Train the conditional diffusion prediction model, select the total number of diffusion steps T = 100, and calculate the noise coefficient β min =0.0001,β max=0.1, then gradually add Gaussian white noise to complete the forward process. The denoising network is mainly composed of 8 conditional residual blocks, each of which is composed of a one-dimensional convolutional neural network and a ReLU activation function, and Fourier position embedding is used to encode the noise. According to the aforementioned loss function, the gradient descent method is used to update the network model parameters, and the training is stopped when the error loss is less than Δ = 0.01.

[0105] The specific step three is: after the model is trained, given the hidden state h t-1 Under the condition of multiple sampling from the trained model, the predicted output of the model under given conditions at a certain time t is obtained Calculate the mean and variance of the current forecast value and estimate the standard deviation of the residual.

[0106] The step 4 is specifically as follows: calculating the abnormal probability of the data point based on the predicted distribution of the data point, and identifying the data point with an abnormal probability greater than 0.5 as abnormal data.

[0107] The present invention also proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for diagnosing probabilistic outliers of traffic infrastructure monitoring data based on a conditional diffusion model are implemented.

[0108] The present invention also proposes a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for diagnosing probabilistic outliers in traffic infrastructure monitoring data based on a conditional diffusion model.

[0109] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0110] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0111] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0112] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0113] The above is a detailed introduction to the probabilistic outlier diagnosis method for traffic infrastructure monitoring data based on the conditional diffusion model proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A probabilistic outlier diagnosis method for traffic infrastructure monitoring data based on a conditional diffusion model, characterized in that: The method comprises the following steps: Step 1: Construct a traffic infrastructure monitoring data set that needs to remove outliers, and determine the time step of each type of time series monitoring data according to the sampling frequency, so as to train the time series prediction model for outlier diagnosis; Step 2: Establish a conditional diffusion model, determine the network architecture of the conditional embedding network module and the denoising network module, and train a time series prediction model based on the monitoring data set; Step 3: Predict the monitoring data points at each moment based on the time series prediction model, calculate the residuals through the true value and the predicted value, and estimate the variance parameters of the prediction error; Step 4: Define the abnormal probability of the data point based on the probability distribution of the predicted data point, calculate the abnormal probability of each data point, and identify the data point with an abnormal probability greater than 0.5 as an outlier.

2. The method according to claim 1, characterized in that In step 2, the network architecture of the conditional embedding network module is selected as an RNN network model, which is composed of GRU gated units and contains 4 hidden layers. Each hidden layer contains 30 hidden neurons. The time-related features at the prediction time t and the observations before multiple times t are used as context windows to form the covariate c t ; Combine it with the hidden state h at the previous moment t-2 As the input of the conditional embedding network module, the output of the conditional embedding network module is h t-1 : h t-1 =GRU(c t ,h t-2 ) The specific expression is: r t-1 =σ[W r ·(h t-2 ,c t )+b r ] z t-1 =σ[W z ·(h t-2 ,c t )+b z ] in is the sigmoid activation function, is the Tanh activation function, ⊙ represents element-by-element multiplication, C θ = {W r ,W z ,W h ,b r ,b z ,b h } is the model parameter of the conditional embedding network module, which is initialized from the standard normal distribution The parameters are randomly generated in , and then optimized through the subsequent model training process, and h0=0.

3. The method according to claim 2, characterized in that In step 2, the original monitoring data at time t is Gradually increase Gaussian white noise, and assume that this process satisfies Markov property. After T steps of diffusion, the original monitoring data becomes pure Gaussian white noise, that is: The reverse process from Start typing h under given conditions t-1 Then, the original monitoring data is generated by gradually removing the noise According to the Bayesian formula, under given conditions Time conditional probability distribution Satisfies the following relationship: A neural network is used to fit the above inverse process, and the fitting result is Set to: The KL divergence of the two is used as the loss function: Where C is a constant that is independent of the network model parameter θ; Select the form for: Further simplifying it into KL divergence yields: where ε θ Represents the model parameters of the denoising network module.

4. The method according to claim 3, characterized in that In step 2, when the hidden state h corresponding to the input time t t-1 When used as input, the loss function of the time series forecasting model is: After shuffling the order, repeat T times and randomly select an m from (1,2,…,T), and use the gradient descent algorithm to update the network model parameters. Here, the network model parameters include the model parameters ε of the denoising network module. θ and the model parameters C of the conditional embedding network module θ Two parts: When the error loss is less than the threshold Δ=0.01, the update is stopped and the model training is completed.

5. The method according to claim 4, characterized in that In step 3, after the model is trained, given the hidden state h t-1 Under the condition of Medium Sampling When m=2,3,…,T When m=1, z=0; after repeating the sampling process T times, the predicted output of the model under given conditions at a certain time t can be obtained.

6. The method according to claim 5, characterized in that In step 3, set the number of samples to N, and calculate the predicted mean and standard deviation at a certain moment as follows:

7. The method according to claim 6, characterized in that In step 3, the variance of the prediction error is estimated by resampling. A certain proportion of data is randomly extracted from the original data L times, and the variance is calculated by unbiased estimation and the mean is obtained:

8. The method according to claim 7, characterized in that In step 4, when the forecast mean and variance and the variance of the forecast error are given, the forecast distribution at a certain moment approximately satisfies the Gaussian distribution: The abnormal probability of the data point at this moment is: When the calculated abnormal probability is greater than 0.5, it is identified as an outlier.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium for storing computer instructions, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images