A Physical and Data-Driven Approach to Environmental Data Denoising and Feature Enhancement

Abstract

This invention discloses a method for environmental data denoising and feature enhancement based on a dual-driven approach of physics and data, belonging to the field of data processing technology. The method includes the following steps: S1, collecting raw environmental data of the region and removing outliers to obtain standard environmental data; S2, constructing a physical prior model based on the standard environmental data of each grid in the region, and determining the spatial feature vector of the grid; S3, determining the temporal feature vector of the grid based on the standard environmental data of each grid in the region; S4, enhancing the spatial and temporal feature vectors of the grid to identify grids exceeding or meeting standards in the region. This invention uses dynamic fusion and neighborhood aggregation to enhance spatiotemporal features, obtaining enhanced environmental interaction features, and ultimately identifying grids exceeding and meeting standards. This provides scientific support for pollution source tracing and precise governance, and provides a refined basis for regional pollution control.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a method for environmental data denoising and feature enhancement based on a dual-drive approach of physics and data. Background Technology

[0002] With the development of environmental monitoring technology, accurate monitoring of environmental pollutants such as water and air has become a core requirement for ecological and environmental protection. Current environmental monitoring largely relies on sensor networks to collect continuous pollutant concentration data. However, in actual monitoring processes, data is easily affected by sensor noise, environmental interference (such as wind, rain, and electromagnetic interference), and equipment malfunctions, leading to distortion and abnormal fluctuations in the raw data. Furthermore, the migration and diffusion of pollutants in the monitored area exhibit significant physical laws. Traditional data processing methods, such as denoising techniques (e.g., wavelet denoising and Kalman filtering), are often purely data-driven, smoothing data only from a mathematical perspective without considering the physical laws of pollutant migration. This can cause the denoised data to deviate from the actual pollution state, such as filtering out true pollution peaks or retaining noise that does not conform to physical logic, leading to misjudgments. Summary of the Invention

[0003] To address the above problems, this invention proposes a method for environmental data denoising and feature enhancement based on a dual-drive approach of physics and data.

[0004] The technical solution of this invention is: a method for environmental data denoising and feature enhancement based on both physical and data-driven approaches, comprising the following steps:

[0005] S1. Collect the raw environmental data of the area and remove outliers to obtain standard environmental data;

[0006] S2. Based on the standard environmental data of each grid in the region, construct a physical prior model and determine the spatial feature vector of the grid;

[0007] S3. Determine the temporal feature vector of each grid based on the standard environmental data of each grid in the region;

[0008] S4. Enhance the spatial and temporal feature vectors of the grid to identify grids that exceed or meet the standards in the region.

[0009] Furthermore, S2 includes the following sub-steps:

[0010] S21. Divide the region into grids;

[0011] S22. Construct a physical prior model based on the standard environmental data of each grid;

[0012] S23. Determine the spatial feature vector of each grid based on the physical prior model of each grid.

[0013] Furthermore, in S22, the expression for the physical prior model is:

[0014] ;

[0015] in, Represents a grid The concentration of pollutants (mg / L). Indicates time (s). This represents the spatial gradient operator (1 / m). Represents a grid The water flow velocity vector (m / s). Represents a grid Pollutant diffusion coefficient (m 2 / s), Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface flow (m) between 3 / s), Represents a grid volume (m 3 ), Indicates adjacent grids The concentration of pollutants (mg / L). Represents a grid The intensity of pollutant source and sink terms (mg / (L·s)). Represents a grid The first-order degradation coefficient of pollutants (1 / s).

[0016] The beneficial effect of the above-mentioned further scheme is that, in this invention, the left side of the model represents the rate of change of pollutant concentration within the grid per unit time; the terms on the right side represent the physical processes that lead to the concentration change.

[0017] The core physical processes (convection, diffusion, source-sink, and degradation) of the environmental data in the physical prior model of this invention are incorporated with a grid coupling term to adapt to discrete grid scenarios. The physical meaning of the grid coupling term is the material exchange between adjacent grids, which conforms to the spatial migration patterns of pollutants in actual water bodies.

[0018] Convection term : Describes the transport process of pollutants carried by water flow. If the velocity vector... If the pollutant flows out of the grid along with the water flow, its concentration will decrease; otherwise, its concentration will increase.

[0019] diffusion term This describes the molecular and turbulent diffusion processes of pollutants from high-concentration areas to low-concentration areas. A larger diffusion coefficient indicates a stronger diffusion effect and a faster rate of concentration homogenization.

[0020] Mesh Coupling Term : Describe the mesh With adjacent grid The process of pollutant exchange between adjacent grids. If the concentration of adjacent grids... Pollutants will flow into the grid. If the concentration increases, it will increase; conversely, it will flow out.

[0021] Source and Exchange This describes the input (source, such as a sewage outlet) or output (sink, such as sediment adsorption) of pollutants within the grid. Source terms are positive, and sink terms are negative. The self-growth and decay of physical quantities in the flow field are generally described by the sources and sinks distributed within the flow field. These sources and sinks are distribution functions of the field.

[0022] Degradation items Describes the natural degradation process of pollutants under physical, chemical, and biological processes. Degradation coefficient. The larger the value, the faster the degradation rate and the more significant the concentration reduction.

[0023] Furthermore, S23 includes the following sub-steps:

[0024] S231. The physical prior model is discretized using the finite volume method to obtain the physical constraint operators of the mesh at each discrete time step, and to generate the discrete operator sequence of the mesh.

[0025] S232. Extract the mean and standard deviation of the discrete operator sequence, and concatenate them to form the spatial feature vector of the grid.

[0026] The beneficial effects of the above-mentioned further scheme are as follows: In this invention, the physical meaning of the constraint operator is the residual between the observed concentration change rate and the concentration change rate predicted by the physical model. If the constraint operator = 0, it indicates that the observed data completely conforms to the physical model; if the constraint operator ≠ 0, it indicates that there is noise or model error in the observed data. The constraint operator is obtained by discretizing the physical prior model using the finite volume method, which is a commonly used numerical method in the prior art for solving convection-diffusion equations. The operational principle of the constraint operator is residual calculation, which is the standard method for implementing physical constraints in Physical Information Neural Networks (PINN).

[0027] Because the physical prior model describes the dynamic changes of pollutant concentration over time and space, while the discrete time step is a breakdown of the time dimension (e.g., one time step per hour). The concentration of each grid cell changes at different time steps, as do the observed values ​​and the physical model predictions. Therefore, it is necessary to calculate a constraint operator separately for each time step of each grid cell. This operator is the residual between the observed rate of change of concentration and the rate of change of concentration predicted by the physical model at that time step, with each time step corresponding to a single value.

[0028] Discrete operator sequences are time-varying sequences of residuals between physical prior models and observed data, representing a quantitative characteristic of physical constraints.

[0029] Furthermore, S3 includes the following sub-steps:

[0030] S31. Based on standard environmental data, obtain the pollutant concentration sequence of the grid at each time point;

[0031] S32. Extract the mean and standard deviation of the pollutant concentration sequence, and splice them together to form the time feature vector of the grid.

[0032] Furthermore, S4 includes the following sub-steps:

[0033] S41. Normalize the spatial and temporal feature vectors of the grid.

[0034] S42. Dynamically fuse the normalized spatial feature vector and temporal feature vector to obtain the dynamic coupling characteristics of the mesh.

[0035] S42. Based on the dynamic coupling characteristics of the mesh and its adjacent meshes, the environmental enhancement interaction characteristics of the mesh are obtained;

[0036] S43. Input the environmental enhancement interaction features of all grids into the fully connected layer to obtain the out-of-range probability of each grid.

[0037] S44. Grids with a probability of exceeding the standard greater than or equal to the threshold are considered as exceeding the standard grids, and grids with a probability of exceeding the standard less than the threshold are considered as meeting the standard grids.

[0038] The beneficial effects of the above-mentioned further scheme are as follows: In this invention, spatial feature vectors and temporal feature vectors are mapped to the same interval, eliminating the influence of dimensional differences on subsequent fusion; dynamic weights are calculated based on the physical constraint strength (mean of discrete operators), adaptively balancing the contributions of physical features and data features (the higher the physical conformity, the greater the weight of the physical feature). Based on the cross-grid diffusion characteristics of pollutants, the fusion features of adjacent grids are aggregated through interface flow weights to obtain environmental enhancement interaction features containing spatial correlation information; the enhanced interaction features are input into a fully connected layer, mapped to the exceedance probability of [0,1] through the Sigmoid activation function, and the compliant and exceedance grids are distinguished based on the threshold.

[0039] Furthermore, in S42, the mesh Dynamic coupling characteristics The expression is:

[0040] ;

[0041] ;

[0042] in, Represents a grid The spatiotemporal weights Represents a grid Normalized spatial eigenvectors Represents a grid Normalized time feature vector, This represents the mean of the discrete operator sequence in grid 1. This represents the mean of the discrete operator sequence in grid 2. Represents a grid The mean of the discrete operator sequence, Indicating the exponential algorithm, This represents the maximum value algorithm.

[0043] make sure ∈(0,1], It is a 4-dimensional initial fusion feature vector, with spatial and temporal features each having 2 dimensions, which are then concatenated after coupling. This represents the first element. This represents the second element. This represents the third element. This represents the 4th element.

[0044] Furthermore, in S42, the mesh Enhanced interactive features The expression is:

[0045] ;

[0046] in, Represents a grid The dynamic coupling characteristics, Represents a grid The dynamic coupling characteristics, Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface flow (m) between 3 / s), Represents a grid With adjacent grid Interface flow (m) between 3 / s).

[0047] The beneficial effects of the above-mentioned further scheme are as follows: In this invention, water pollutants exhibit cross-grid diffusion characteristics; therefore, feature aggregation of neighboring grids is introduced (based on interface flow weighting in the physical model, reflecting that the greater the flow, the stronger the correlation between grids). For the current grid... All adjacent grids First calculate adjacent grids For the current grid The influence weight is then used to assign this weight to adjacent grid cells. The features are scored, and finally, the features of all adjacent grid cells are summed to aggregate the feature information of the neighborhood. The neighbors... initial fusion features Multiply by the influence weight calculated earlier to determine the impact of the neighbor's features on the current grid. How great is its contribution?

[0048] In the middle, molecules Indicates the current grid and neighbors Interface traffic between (e.g.) =2m 3 / s indicates that 2 cubic meters of water (carrying pollutants) flow from [the source] per second. Flow to ). Denominator Indicates the current grid The sum of the interface traffic of this neighbor and all its neighbors. The overall ratio indicates the traffic of this neighbor and... The more pollutants exchanged, the better. The stronger the influence, the better. Using different dummy variables k to traverse "all adjacent grids" in the denominator clearly distinguishes between them. For the single neighboring grid whose contribution is to be calculated; It can be any one of the adjacent grids.

[0049] The beneficial effects of this invention are as follows: This invention constrains the data processing process through a physical prior model, ensuring that the denoised and enhanced features do not deviate from the actual laws of pollutant migration and diffusion, and tracks the pollution diffusion path through the correlation information between grids; it also extracts time feature vectors reflecting the time statistical laws of concentration based on standard environmental data; this invention performs enhancement processing such as dynamic fusion and neighborhood aggregation on spatiotemporal features to obtain environmental enhancement interactive features, and finally determines the grids that exceed the standard and the grids that meet the standard, providing scientific support for pollution source tracing and precise governance, and providing a refined basis for regional pollution control. Attached Figure Description

[0050] Figure 1This is a flowchart of an environmental data denoising and feature enhancement method based on both physical and data-driven approaches. Detailed Implementation

[0051] The embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0052] like Figure 1 As shown, this invention provides a method for environmental data denoising and feature enhancement based on both physical and data-driven approaches, comprising the following steps:

[0053] S1. Collect the raw environmental data of the area and remove outliers to obtain standard environmental data;

[0054] S2. Based on the standard environmental data of each grid in the region, construct a physical prior model and determine the spatial feature vector of the grid;

[0055] S3. Determine the temporal feature vector of each grid based on the standard environmental data of each grid in the region;

[0056] S4. Enhance the spatial and temporal feature vectors of the grid to identify grids that exceed or meet the standards in the region.

[0057] In this embodiment of the invention, S2 includes the following sub-steps:

[0058] S21. Divide the region into grids;

[0059] S22. Construct a physical prior model based on the standard environmental data of each grid;

[0060] S23. Determine the spatial feature vector of each grid based on the physical prior model of each grid.

[0061] In this embodiment of the invention, in S22, the expression of the physical prior model is:

[0062] ;

[0063] in, Represents a grid The concentration of pollutants (mg / L). Indicates time (s). This represents the spatial gradient operator (1 / m). Represents a grid The water flow velocity vector (m / s). Represents a grid Pollutant diffusion coefficient (m 2 / s), Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface flow (m) between 3 / s), Represents a grid volume (m 3 ), Indicates adjacent grids The concentration of pollutants (mg / L). Represents a grid The intensity of pollutant source and sink terms (mg / (L·s)). Represents a grid The first-order degradation coefficient of pollutants (1 / s).

[0064] The beneficial effect of the above-mentioned further scheme is that, in this invention, the left side of the model represents the rate of change of pollutant concentration within the grid per unit time; the terms on the right side represent the physical processes that lead to the concentration change.

[0065] The core physical processes (convection, diffusion, source-sink, and degradation) of the environmental data in the physical prior model of this invention are incorporated with a grid coupling term to adapt to discrete grid scenarios. The physical meaning of the grid coupling term is the material exchange between adjacent grids, which conforms to the spatial migration patterns of pollutants in actual water bodies.

[0066] Convection term : Describes the transport process of pollutants carried by water flow. If the velocity vector... If the pollutant flows out of the grid along with the water flow, its concentration will decrease; otherwise, its concentration will increase.

[0067] diffusion term This describes the molecular and turbulent diffusion processes of pollutants from high-concentration areas to low-concentration areas. A larger diffusion coefficient indicates a stronger diffusion effect and a faster rate of concentration homogenization.

[0068] Mesh Coupling Term : Describe the mesh With adjacent grid The process of pollutant exchange between adjacent grids. If the concentration of adjacent grids... Pollutants will flow into the grid. If the concentration increases, it will increase; conversely, it will flow out.

[0069] Source and Exchange This describes the input (source, such as a sewage outlet) or output (sink, such as sediment adsorption) of pollutants within the grid. Source terms are positive, and sink terms are negative. The self-growth and decay of physical quantities in the flow field are generally described by the sources and sinks distributed within the flow field. These sources and sinks are distribution functions of the field.

[0070] Degradation items Describes the natural degradation process of pollutants under physical, chemical, and biological processes. Degradation coefficient. The larger the value, the faster the degradation rate and the more significant the concentration reduction.

[0071] In this embodiment of the invention, S23 includes the following sub-steps:

[0072] S231. The physical prior model is discretized using the finite volume method to obtain the physical constraint operators of the mesh at each discrete time step, and to generate the discrete operator sequence of the mesh.

[0073] S232. Extract the mean and standard deviation of the discrete operator sequence, and concatenate them to form the spatial feature vector of the grid.

[0074] In this invention, the constraint operator represents the residual between the observed rate of concentration change and the rate of concentration change predicted by the physical model. If the constraint operator = 0, it indicates that the observed data perfectly matches the physical model; if the constraint operator ≠ 0, it indicates that the observed data contains noise or model error. The constraint operator is obtained by discretizing the physical prior model using the finite volume method, a commonly used numerical method in the prior art for solving convection-diffusion equations. The operational principle of the constraint operator is residual calculation, which is the standard method for implementing physical constraints in Physical Information Neural Networks (PINN).

[0075] Because the physical prior model describes the dynamic changes of pollutant concentration over time and space, while the discrete time step is a breakdown of the time dimension (e.g., one time step per hour). The concentration of each grid cell changes at different time steps, as do the observed values ​​and the physical model predictions. Therefore, it is necessary to calculate a constraint operator separately for each time step of each grid cell. This operator is the residual between the observed rate of change of concentration and the rate of change of concentration predicted by the physical model at that time step, with each time step corresponding to a single value.

[0076] Discrete operator sequences are time-varying sequences of residuals between physical prior models and observed data, representing a quantitative characteristic of physical constraints.

[0077] In this embodiment of the invention, S3 includes the following sub-steps:

[0078] S31. Based on standard environmental data, obtain the pollutant concentration sequence of the grid at each time point;

[0079] S32. Extract the mean and standard deviation of the pollutant concentration sequence, and splice them together to form the time feature vector of the grid.

[0080] In this embodiment of the invention, S4 includes the following sub-steps:

[0081] S41. Normalize the spatial and temporal feature vectors of the grid.

[0082] S42. Dynamically fuse the normalized spatial feature vector and temporal feature vector to obtain the dynamic coupling characteristics of the mesh.

[0083] S42. Based on the dynamic coupling characteristics of the mesh and its adjacent meshes, the environmental enhancement interaction characteristics of the mesh are obtained;

[0084] S43. Input the environmental enhancement interaction features of all grids into the fully connected layer to obtain the out-of-range probability of each grid.

[0085] S44. Grids with a probability of exceeding the standard greater than or equal to the threshold are considered as exceeding the standard grids, and grids with a probability of exceeding the standard less than the threshold are considered as meeting the standard grids.

[0086] In this invention, spatial feature vectors and temporal feature vectors are mapped to the same interval to eliminate the influence of dimensional differences on subsequent fusion. Dynamic weights are calculated based on the strength of physical constraints (mean of discrete operators) to adaptively balance the contributions of physical features and data features (the higher the physical conformity, the greater the weight of the physical feature). Based on the cross-grid diffusion characteristics of pollutants, the fusion features of adjacent grids are aggregated through interface flow weights to obtain environmental enhancement interaction features containing spatial correlation information. The enhanced interaction features are input into a fully connected layer and mapped to the exceedance probability of [0,1] through the Sigmoid activation function. Based on the threshold, compliant and exceedance grids are distinguished.

[0087] In this embodiment of the invention, in S42, the mesh Dynamic coupling characteristics The expression is:

[0088] ;

[0089] ;

[0090] in, Represents a grid The spatiotemporal weights Represents a grid Normalized spatial eigenvectors Represents a grid Normalized time feature vector, This represents the mean of the discrete operator sequence in grid 1. This represents the mean of the discrete operator sequence in grid 2. Represents a grid The mean of the discrete operator sequence, Indicating the exponential algorithm, This represents the maximum value algorithm.

[0091] make sure ∈(0,1], It is a 4-dimensional initial fusion feature vector, with spatial and temporal features each having 2 dimensions, which are then concatenated after coupling. This represents the first element. This represents the second element. This represents the third element. This represents the 4th element.

[0092] In this embodiment of the invention, in S42, the mesh Enhanced interactive features The expression is:

[0093] ;

[0094] in, Represents a grid The dynamic coupling characteristics, Represents a grid The dynamic coupling characteristics, Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface flow (m) between 3 / s), Represents a grid With adjacent grid Interface flow (m) between 3 / s).

[0095] In this invention, water pollutants exhibit cross-grid diffusion characteristics; therefore, feature aggregation of neighboring grids is introduced (based on interface flow weighting in the physical model, reflecting that the greater the flow, the stronger the correlation between grids). For the current grid... All adjacent grids First calculate adjacent grids For the current grid The influence weight is then used to assign this weight to adjacent grid cells. The features are scored, and finally, the features of all adjacent grid cells are summed to aggregate the feature information of the neighborhood. The neighbors... initial fusion features Multiply by the influence weight calculated earlier to determine the impact of the neighbor's features on the current grid. How great is its contribution?

[0096] In the middle, molecules Indicates the current grid and neighbors Interface traffic between (e.g.) =2m 3 / s indicates that 2 cubic meters of water (carrying pollutants) flow from [the source] per second. Flow to ). Denominator Indicates the current grid The sum of the interface traffic of this neighbor and all its neighbors. The overall ratio indicates the traffic of this neighbor and... The more pollutants exchanged, the better. The stronger the influence, the better. Using different dummy variables k to traverse "all adjacent grids" in the denominator clearly distinguishes between them. For the single neighboring grid whose contribution is to be calculated; It can be any one of the adjacent grids.

[0097] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the principles of the invention, and should be understood that the scope of protection of the invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of this invention.

Claims

1. A method for environmental data denoising and feature enhancement based on a dual-driven approach of physics and data, characterized in that, Includes the following steps: S1. Collect the raw environmental data of the area and remove outliers to obtain standard environmental data; S2. Based on the standard environmental data of each grid in the region, construct a physical prior model and determine the spatial feature vector of the grid; S3. Determine the temporal feature vector of each grid based on the standard environmental data of each grid in the region; S4. Enhance the spatial and temporal feature vectors of the grid to identify grids that exceed or meet the standards in the region; S2 includes the following sub-steps: S21. Divide the region into grids; S22. Construct a physical prior model based on the standard environmental data of each grid; S23. Determine the spatial feature vector of each grid based on the physical prior model of each grid. In S22, the expression for the physical prior model is: ； in, Represents a grid The concentration of pollutants, Indicates time, Represents the spatial gradient operator. Represents a grid Water flow velocity vector Represents a grid The pollutant diffusion coefficient, Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface traffic between them Represents a grid volume, Indicates adjacent grids The concentration of pollutants, Represents a grid The intensity of pollutant source and sink items, Represents a grid The first-order degradation coefficient of pollutants; S23 includes the following sub-steps: S231. The physical prior model is discretized using the finite volume method to obtain the physical constraint operators of the mesh at each discrete time step, and to generate the discrete operator sequence of the mesh. S232. Extract the mean and standard deviation of the discrete operator sequence, and concatenate them to form the spatial feature vector of the grid.

2. The environmental data denoising and feature enhancement method based on dual physical and data-driven approaches according to claim 1, characterized in that, S3 includes the following sub-steps: S31. Based on standard environmental data, obtain the pollutant concentration sequence of the grid at each time point; S32. Extract the mean and standard deviation of the pollutant concentration sequence, and splice them together to form the time feature vector of the grid.

3. The environmental data denoising and feature enhancement method based on dual physical and data-driven approaches according to claim 1, characterized in that, S4 includes the following sub-steps: S41. Normalize the spatial and temporal feature vectors of the grid. S42. Dynamically fuse the normalized spatial feature vector and temporal feature vector to obtain the dynamic coupling characteristics of the mesh. S42. Based on the dynamic coupling characteristics of the mesh and its adjacent meshes, the environmental enhancement interaction characteristics of the mesh are obtained; S43. Input the environmental enhancement interaction features of all grids into the fully connected layer to obtain the out-of-range probability of each grid. S44. Grids with a probability of exceeding the standard greater than or equal to the threshold are considered as exceeding the standard grids, and grids with a probability of exceeding the standard less than the threshold are considered as meeting the standard grids.

4. The environmental data denoising and feature enhancement method based on dual physical and data-driven approaches according to claim 3, characterized in that, In S42, the grid Dynamic coupling characteristics The expression is: ； ； in, Represents a grid The spatiotemporal weights Represents a grid Normalized spatial eigenvectors Represents a grid Normalized time feature vector, This represents the mean of the discrete operator sequence in grid 1. This represents the mean of the discrete operator sequence in grid 2. Represents a grid The mean of the discrete operator sequence, Indicating the exponential algorithm, This represents the maximum value algorithm.

5. The environmental data denoising and feature enhancement method based on dual physical and data-driven approaches according to claim 3, characterized in that, In S42, the grid Enhanced interactive features The expression is: ； in, Represents a grid The dynamic coupling characteristics, Represents a grid The dynamic coupling characteristics, Represents a grid The set of adjacent grids, Represents a grid With adjacent grid Interface traffic between them Represents a grid With adjacent grid Interface traffic between them.

Images