Environmental parameter field fine reconstruction method based on preliminary estimation and deep neural network correction

Abstract

The invention discloses an environmental parameter field fine reconstruction method based on preliminary estimation and deep neural network correction, and belongs to the technical field of environmental parameter field reconstruction and deep learning application. The method comprises the following steps: firstly, based on sparse measuring point data, generating a rough field through a field reconstruction algorithm; then, taking the rough field as a main feature, and carrying out multi-channel fusion on the main feature and a space coordinate feature to form an input feature map; and then, inputting the fused feature map into a deep correction neural network, correcting the rough field by using the strong nonlinear mapping capability of the network, and outputting a high-precision and high-resolution fine field. In the model training stage, a composite loss function including data loss and physical constraint loss is adopted for optimization, and key parameters of a field reconstruction algorithm can be further taken as trainable variables to realize joint optimization with neural network parameters. The method effectively solves the problems that a traditional interpolation method is low in precision and weak in detail capturing capability under the sparse data condition, and a pure data driving model lacks physical reasonability, and remarkably improves the accuracy, robustness and generalization capability of environment parameter field reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of environmental parameter field reconstruction and deep learning application technology, and particularly relates to a method for refining sparse measurement point data fields by combining field reconstruction algorithm preprocessing with deep correction neural network, and can realize the joint optimization of the field reconstruction algorithm and neural network model parameters. Background Technology

[0002] In highly sensitive applications such as smart cities, industrial safety, and indoor environmental quality monitoring, rapid, real-time sensing and accurate quantification of the fine spatial distribution of environmental parameters such as volatile organic compounds (VOCs), CO2 concentration, and temperature are crucial for ensuring efficient system operation and protecting personnel safety. However, traditional monitoring methods are limited by computing resources and deployment costs, often only allowing for sparse sensor networks and failing to achieve dense measurements covering the entire space. Therefore, how to rapidly reconstruct high-resolution, high-precision environmental parameter fields based on sparse measurement point data under resource-constrained conditions to meet the needs of real-time decision-making and response has become a core technological challenge that urgently needs to be addressed in this field.

[0003] Traditional field reconstruction methods mainly include deterministic interpolation methods (such as inverse distance weighted (IDW)) and geostatistical methods (such as Kriging). While these methods can utilize spatial correlations for field estimation, they often struggle to effectively incorporate complex physical mechanisms and perform poorly under sparse data and nonlinear field distributions. To overcome these limitations, the Kernel series of field reconstruction algorithms has gained widespread attention in the field of environmental monitoring in recent years. The advantage of this algorithm lies in its ability to combine physical constraints and spatial correlations to a certain extent through carefully designed kernel functions and model structures, thereby improving the physical plausibility of the field reconstruction.

[0004] However, despite these advantages, the inherent limitations of the methods described above remain apparent in applications requiring extremely high reconstruction quality (accuracy) and response speed (real-time performance):

[0005] 1. Insufficient ability to capture and refine details: Whether it is traditional interpolation methods (IDW, Kriging) or estimation based on kernel functions, their models (or kernel functions) are usually based on prior assumptions, making it difficult to capture local nonlinear gradients and fine field structures under complex boundary conditions or strong convection diffusion environments. As a result, the output field data is not fine enough to meet the needs of real-time decision-making.

[0006] 2. Poor parameter dependence and adaptability: These traditional models generally exhibit poor robustness under highly sparse or dynamically changing data distributions. Their key parameters (such as Kriging's variogram and kernel width) typically rely on time-consuming manual experience or offline statistical analysis, lacking a fast adaptive optimization mechanism, making it difficult to ensure rapid and continuous high-precision operation of the model under real-time environmental changes.

[0007] 3. Difficulty in integrating physical constraints: In addition, these existing algorithms have difficulty in flexibly integrating additional general physical constraints, such as field conservation or gradient consistency, which further limits their physical rationality in fine-grained reconstruction.

[0008] Therefore, a novel field reconstruction method is needed, which should have strong nonlinear correction capabilities and parameter adaptive optimization capabilities to achieve rapid, high-precision, and detailed reconstruction of environmental parameter fields under resource-constrained sparse measurement point conditions. Summary of the Invention

[0009] This invention provides a method for fine reconstruction of environmental parameter fields at sparse measurement points based on field reconstruction algorithm preprocessing and deep neural network correction. In this method, a field reconstruction algorithm is used to generate a coarse field, which, along with spatial coordinate features, serves as input to a neural network. The deep neural network learns and performs nonlinear corrections, ultimately outputting a high-precision fine field. This invention improves the accuracy and robustness of field reconstruction under sparse data conditions.

[0010] This invention provides a method for fine reconstruction of sparse measurement point environmental parameter fields based on field reconstruction algorithm preprocessing and deep neural network correction. The method mainly includes the following functional modules and steps:

[0011] 1. Coarse Field Generation: Based on sparse measurement point data, coarse field data F is generated through a field reconstruction algorithm. coarse .

[0012] 2. Feature Fusion and Input: The coarse field F is then fused with the input. coarse As the primary input feature, it is fused with spatial coordinate features (optionally including sparse point indicator features) through multiple channels and then fed into a deep modified neural network N.

[0013] 3. Nonlinear correction: The neural network N modulates the coarse field F. coarse Perform nonlinear corrections to output high-precision, high-resolution fine field data F. fine .

[0014] 4. Composite Loss Optimization: During the training phase, a loss term L is used that includes at least the data loss term L. data and physical constraint loss term L phys The composite loss function L is used to optimize the network.

[0015] 5. Joint Optimization (Preferred): Preferably, the key parameter θ of the field reconstruction algorithm is... k Incorporating it as a trainable variable into the deep learning optimization framework, along with the neural network parameter θ N Achieve joint optimization.

[0016] The beneficial effects of the above technical solution of the present invention are as follows:

[0017] (1) Significantly improve reconstruction accuracy: By combining field reconstruction algorithm preprocessing with deep neural network correction, the shortcomings of single interpolation method in capturing details under sparse data are effectively solved, and the accuracy of field reconstruction is significantly improved.

[0018] (2) Enhancing robustness and physical rationality: Introducing the physical constraint loss term L phy Ensured the prediction field F fine It remains physically plausible even in areas far from the measurement points, enhancing the model's robustness and generalization ability.

[0019] (3) Optimize feature utilization efficiency: By fusing coarse field and spatial coordinate features into a multi-channel input, the neural network can learn an adaptive correction strategy that depends on location and physical initial value, thereby improving the training efficiency and generalization ability of the model.

[0020] (4) Achieve joint parameter optimization: Preferably, by optimizing the field reconstruction algorithm parameters θ k By incorporating joint optimization, adaptive optimization of the physical model and the deep correction model is achieved, further enhancing the system's ability to operate with high precision in dynamic environments.

[0021] (5) Technical universality: As a general field reconstruction method, this framework can be flexibly applied to the fine reconstruction of various environmental parameters (such as temperature, humidity, and pollutant concentration), and has broad application prospects. Attached Figure Description

[0022] To more clearly illustrate the technical solution of the present invention, the accompanying drawings are briefly described below.

[0023] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of the structure of a deep modified neural network according to an embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram illustrating the joint optimization of the physical model and the depth model in an embodiment of the present invention;

[0026] Figure 4 This is a schematic diagram illustrating the principle of the composite loss function in an embodiment of the present invention; Detailed Implementation

[0027] To make the technical problems, technical solutions and advantages of the present invention clearer, a detailed description will be given below in conjunction with the accompanying drawings and specific embodiments.

[0028] The technical problems this invention aims to solve are: existing single interpolation field reconstruction methods suffer from low accuracy and difficulty in capturing complex nonlinear field details under sparse measurement point conditions; and existing single neural network methods lack physical constraints, are highly dependent on training data, and have weak generalization ability. This invention provides a fine reconstruction method for environmental parameter fields at sparse measurement points based on field reconstruction algorithm preprocessing and deep neural network correction. In this method, a field reconstruction algorithm is used to generate a coarse field, which, along with spatial coordinate features, serves as input to a neural network. The deep neural network learns and performs nonlinear corrections, ultimately outputting a high-precision fine field. This invention improves the accuracy and robustness of field reconstruction under sparse data conditions.

[0029] The environmental parameters described in this invention include, but are not limited to, temperature, humidity, CO2 concentration, and pollutant concentration.

[0030] The technical solution of the present invention will be further described in detail below through specific embodiments.

[0031] This invention provides a method for fine field reconstruction based on field reconstruction algorithm preprocessing and deep neural network correction. A schematic flowchart of the method is shown below. Figure 1 As shown, the main steps include:

[0032] Step S1: Data Acquisition, Gridding, and Initialization Processing

[0033] 1. Data Acquisition: Obtain sparse environmental parameter measurement point data of the area to be reconstructed (not limited to a specific sampling method and sampling quantity, outlier removal, and missing value completion), denoted as:

[0034]

[0035] Where (x) i y i , z i v i ) represents spatial coordinates, v i Here, K represents the measured value, and K represents the number of sparse points. The method of this invention is not limited to a specific data sampling method or the number of sparse measurement points K.

[0036] 2. Data Preprocessing and Initialization: Preprocessing the collected sparse measurement point data P... sparseData preprocessing operations are performed to ensure the quality and consistency of the input data. These preprocessing operations include, but are not limited to: outlier identification and removal, missing value statistics and completion, and spatial coordinate system unification and necessary normalization operations.

[0037] 3. Mesh definition: Define the reconstructed spatial mesh G for the target region.

[0038] 4. Training Data Preparation: During the model training phase, prepare the fine-field ground truth F corresponding to the grid G. true The true value can be obtained through high-density measurement or high-fidelity physical simulation.

[0039] Step S2: Field Reconstruction Model Preprocessing

[0040] This module aims to generate a coarse field F using sparse measurement point data and a field reconstruction algorithm. coarse , as the initial input features of the neural network.

[0041] 1. Field Reconstruction Model Selection: The field reconstruction algorithm is used to reconstruct the sparse measurement point data P described in step S1. sparse Perform field reconstruction calculations to generate coarse field data F on grid G. coarse Preferably, the field reconstruction algorithm can employ the Kernel series field reconstruction algorithm, as it can better combine physical constraints and spatial correlations when generating a coarse field. The Kernel series field reconstruction algorithms are mainly divided into the following two types based on whether the sampling environment has wind speed and direction information:

[0042] (1) Kernel DM+V algorithm: suitable for light or no wind environments. This algorithm only requires the coordinates of the sampling points and the corresponding sensor data, and uses the Kernel function and interpolation techniques to estimate the field distribution.

[0043] (2) Kernel DM+V / W algorithm: suitable for mechanical ventilation or natural wind environments. In addition to sampling point coordinates and sensor data, this algorithm also requires wind speed and direction information as input features so as to incorporate the physical effects of convection and diffusion in the field reconstruction calculation.

[0044] 2. Coarse Field Calculation: Using the selected field reconstruction model, calculate the estimated field value V for each point g in the grid G. coarse This generates the entire coarse field F. coarse .

[0045] F coarse Function: F coarse Inheriting the initial physical constraints or spatial correlations of the field reconstruction model, it provides a physically meaningful basic feature for subsequent deep correction neural networks, effectively reducing the difficulty of neural network learning.

[0046] Step S3: Construction of Input Features for Deeply Modified Neural Network

[0047] This step aims to construct the multi-channel input feature tensor I. NN In order to maximize the use of spatial information:

[0048] 1. Construction of basic input features

[0049] (1) Channel feature construction: The coarse field data F coarse As the main input channel (channel 1).

[0050] (2) Spatial coordinate characteristics: The absolute spatial coordinates (x, y, g) of grid points g∈G. i y i , z i v i These serve as auxiliary input channels (channels 2, 3, and 4). The spatial coordinate features help the neural network learn the nonlinear mapping relationship between the field distribution and the absolute position.

[0051] (3) Tensor stacking: Stack the features of each channel to form an input feature tensor.

[0052] 2. Preferred embodiment: Sparse point indicator feature M sparse Construction and integration

[0053] As a preferred optimization method, the present invention can further introduce sparse point indicator features M. sparse As input channel 5. The M sparse It can be a distance map, representing the distance from a grid point to the nearest sparse measurement point, used to guide the neural network to execute different correction strategies in different confidence regions.

[0054] (1) Feature construction principle: Sparse points indicate features M sparse The aim is to quantize the target grid point g to the nearest sparse measurement point P. sparse The spatial distance is used to evaluate the predicted value v of the grid point. coarse The confidence level of the distance. The closer the distance, the higher the confidence level of the point. coarse The more reliable the network, the smaller the correction magnitude that the neural network N needs to perform; the farther the distance, the lower the confidence level, and the more the neural network N should rely on the physical constraints L. phys And implement more radical revisions.

[0055] (2) Construction process:

[0056] 1) Calculate the Euclidean distance: For any point g = (x, y, z) on the grid G, calculate its distance to all sparse measurement points P. i ∈P sparse Euclidean distance di .

[0057] 2) Determine the nearest distance: Take the minimum value among all distances as the original M for that grid point. sparse Value D(g):

[0058]

[0059] 3) Generate distance map: Repeat the above steps to generate the original distance map D for all points in grid G. raw .

[0060] 4) Min-Max Normalization: In order to normalize D raw The numerical range is unified to F coarse The intervals that match the spatial coordinate features (e.g., [0, 1]) are normalized using the Min-Max method to generate the final M. sparse Feature map:

[0061]

[0062] Where D min and D max D respectively raw The minimum and maximum distances in the range.

[0063] (3) Fusion: In this preferred embodiment, M sparse As an additional channel and They are stacked together to form the final input feature tensor I. NN .

[0064] Step S4: Model Training and Composite Loss Function Optimization

[0065] This invention employs a deep neural network model N to learn a nonlinear correction mapping from a coarse field to a fine field.

[0066] 1. Network Structure: A convolutional neural network (CNN) architecture is employed to process gridded spatial data. Preferably, the network includes residual connections and / or skip connections (e.g., a U-Net structure) to enhance the ability to capture spatial details.

[0067] For details regarding the input feature construction and network structure of the deep modified neural network, please refer to [link to documentation]. Figure 2 As shown. This network receives data from F. coarse The network takes a multi-channel fused input consisting of spatial coordinates (x, y); features are extracted by the encoder (downsampling) and restored by the decoder (upsampling), with skip connections used to preserve spatial details, and finally outputs a correction or fine field F. fine .

[0068] 2. Training objective: Minimize the output F of the neural network. fine With truth value F true The error between them.

[0069] Preferred Implementation (Joint Optimization): Preferably, during model training, the key parameter θ of the Kernel series field reconstruction algorithm is simultaneously incorporated. k The key parameter θ is incorporated as a trainable variable into the deep learning optimization framework. k Including but not limited to: kernel width, smoothness parameter, anisotropy factor, or regularization term coefficient. By defining the Kernel algorithm as a custom layer in the differentiable computation graph, the physical model parameter θ can be realized. k With depth correction model parameters θ N Joint optimization is used to further improve the overall performance of the model.

[0070] The Kernel parameter θ k With depth correction model parameters θ N For details on the joint optimization mechanism, please refer to [link / reference]. Figure 3 As shown. This mechanism will use the key parameter θ of the Kernel algorithm. k And the parameters θ of the neural network N N Simultaneously included in the optimization scope. The composite loss L passes through the optimizer, simultaneously receiving L with respect to θ. k and θ N The gradient is used to achieve end-to-end adaptive joint updates of the physical model and the deep model.

[0071] 3. Composite Loss Function: Optimization is performed using the composite loss function L, whose expression is:

[0072] L = L date +λL phys

[0073] in:

[0074] λ is a balance coefficient used to adjust the weights of data fitting and physical constraints.

[0075] L data For the data loss term, F is used in this example. fine With truth value F true The mean squared error (MSE) between them. phys The physical constraint loss term, used as a regularization term, is employed to constrain the prediction field F. fine The expected physical properties can be achieved in ways including, but not limited to, the following two:

[0076] (1) Field gradient smoothness constraint (regularization term)

[0077] This constraint is achieved by limiting the fine field F. fineSpatial gradient changes encourage smoothness and continuity of field distribution, thereby enhancing the robustness of the model in regions far from sparse measurement points.

[0078]

[0079] in, and F fine The first and second spatial derivatives (gradient and Laplacian operator) can be implemented in the computational graph of a neural network through finite difference or convolution kernel operations.

[0080] (2) Field conservation constraint (physical residual)

[0081] This constraint is achieved by F fine Substituting the equations into the physical equations describing the diffusion and convection processes of environmental parameters (such as CO2 concentration or pollutants), minimizing the residuals on both sides of the equations ensures that F fine It satisfies the basic physical conservation laws.

[0082] 1) Physical equations: It is assumed that the distribution of the environmental parameter C follows the convection-diffusion equation or its simplified form, i.e.

[0083] 2) Residual calculation: The output F of the deep correction neural network is calculated. fine Substitute the predefined physical operators In the calculation of physical residual R phys :

[0084]

[0085] 3) Definition of loss term: The mean square error of the physical residuals is used as the physical constraint loss term.

[0086]

[0087] Introducing L phys Its core function is to enhance the model's generalization ability and robustness to sparse data, ensuring F... fine It still has physical rationality in areas far from the measuring point.

[0088] For the calculation principle of the composite loss function L, please refer to [link / reference needed]. Figure 4 As shown.

[0089] Step S5: Fine-field reconstruction (application)

[0090] This step involves high-precision field reconstruction in real-time applications after the model training is complete.

[0091] 1. Real-time data input: Real-time acquisition of new sparse measurement point data Psparse .

[0092] 2. Coarse Field and Feature Construction: Repeat steps S2 and S3 to generate the corresponding coarse field F. coarse and input feature tensor I NN .

[0093] 3. Forward reasoning: I NN The input is fed into a pre-trained deep modified neural network N to perform forward inference.

[0094] 4. Fine-field output: The output of the neural network N is the high-precision, high-resolution fine-field F after nonlinear correction. fine .

[0095] The beneficial effects of this invention are as follows:

[0096] (1) Improved reconstruction accuracy: This invention creatively adopts a field reconstruction model as a preprocessor and combines it with a deep correction neural network for nonlinear correction. This combination effectively compensates for the shortcomings of single interpolation field reconstruction methods in terms of insufficient detail capture and inaccurate gradient prediction under sparse data, thereby improving the accuracy of fine reconstruction of environmental parameter fields.

[0097] (2) Enhancing the robustness and physical rationality of the model: This method introduces a composite loss function L = L in the neural network training. date +λL phys The use of physical constraint loss terms (such as field gradient smoothness or conservation constraints) ensures the output F of the neural network. fine It can maintain physical plausibility even in areas far from sparse measurement points, avoiding the distortion that may occur in purely data-driven models.

[0098] (3) Optimizing feature utilization efficiency: This method fully utilizes spatial information by fusing the output of the field reconstruction model (coarse field) with the absolute spatial coordinates of the grid points through multiple channels. This enables the neural network to learn an adaptive correction strategy that depends on location and initial physical values, thereby improving the training efficiency and generalization ability of the model.

[0099] (4) Achieving joint optimization of the physical model: Preferably, this invention incorporates key parameters of the field reconstruction algorithm (e.g., kernel function width) into the deep learning optimization framework during training, achieving joint and adaptive optimization of the physical model parameters and the deep modified neural network parameters. This overcomes the limitations of traditional field reconstruction algorithms where parameters are fixed or rely on manual experience, enabling the coarse field F... coarse It is inherently closer to the true value, further improving the upper limit of reconstruction accuracy and adaptability of the entire system.

[0100] (5) Technical universality: As a general field reconstruction method, this framework can be flexibly applied to the fine reconstruction of various environmental parameters (such as temperature, humidity, and pollutant concentration), and has broad application prospects.

[0101] The above description is a specific illustration of the embodiments of the present invention. It should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principles described in the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for fine reconstruction of environmental parameter field based on field reconstruction preprocessing and deep neural network correction with sparse measurement points, characterized in that, The method comprises the following steps: S1 Rough field generation: based on sparse measurement point data, generate rough field data F through field reconstruction algorithm coarse ; S2 Feature fusion and input: the rough field data F coarse As the main input feature, and multi-channel fusion with spatial coordinate features, form a fusion feature map; S3, a nonlinear correction: inputting the fused feature map into a deep correction neural network N, performing nonlinear correction by the neural network N, and outputting fine field data F with high precision and high resolution fine ; S4 composite loss optimization: during the training of the neural network N, a composite loss function L is adopted to optimize the parameters θ of the neural network N N . The composite loss function L includes at least a data loss term L data and a physical constraint loss term L phys .

2. The method of claim 1, wherein, The composite loss optimization step further comprises: including but not limited to kernel width, smoothness parameter, anisotropy factor or regularization term coefficient) as trainable variables into a deep learning optimization framework with the parameters θ K ( the key parameters θ K including but not limited to kernel width, smoothness parameter, anisotropy factor or regularization term coefficient) as trainable variables into a deep learning optimization framework with the parameters θ N A joint optimization is implemented.

3. The method according to claim 1 or 2, characterized in that, The physical constraint loss term L phys is computed by applying a physical operator P(·) to the fine field data F fine The constraints imposed by P(·) include: (1) field gradient smoothness constraints, e.g., based on or regularization terms; (2) or a conservation of field constraint, e.g., a physical residual based on a convection-diffusion equation or a simplified version thereof. The physical residual R of the conservation of field constraint phys The computing step comprises: F fine Substituting into the predefined convection-diffusion equation Calculate residuals and with As a loss item.

4. The method of claim 1, wherein, The deep revision neural network N adopts a convolutional neural network CNN architecture, which can adopt a structure containing a coding-decoding structure and a skip connection (for example, a U-Net structure).

5. The method of claim 1, wherein, In the feature fusion and input step, the multi-channel fused feature map further comprises sparse point indication features M sparse , the M sparse Adopting a distance map representation is used to guide the neural network to perform correction in different confidence regions.

6. An apparatus for fine reconstruction of a sparse measurement environment parameter field, characterized in that, The device comprises: 1) Coarse field calculation unit: configured to receive sparse measurement point data and generate coarse field data F by a field reconstruction algorithm coarse ; 2) feature construction unit: configured to construct the F coarse Multi-channel fusion with spatial coordinate features to form a fused feature map; 3) a depth correction unit configured to receive the fused feature map and perform forward inference to output fine field data F fine , wherein the depth correction unit is internally provided with a model trained according to the method of claim 2, which receives the fused feature map and outputs the fine field data F fine , and the model parameters have been solidified through joint optimization; 4) An optimization training unit configured to perform the composite loss optimization and parameter joint optimization of any one of claims 2 to 3 in the model training phase.

7. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any one of claims 1 to 5.

Images