Airflow field sparse sampling high-precision reconstruction method based on motion estimation and compensation

Abstract

The invention discloses an airflow field sparse sampling high-precision reconstruction method based on motion estimation and compensation. The invention relates to the field of data completion, in particular to a gas concentration field distribution sparse sampling high-precision reconstruction method, which comprises the following steps of: constructing a data set required for training a network based on simulation and block sampling technologies, and dividing the data set into a training set, a verification set and a test set; constructing a gas concentration distribution reconstruction network based on motion estimation and compensation, wherein the gas concentration distribution reconstruction network comprises a gas motion estimation module, a gas motion compensation module, a feature extraction module, a feature fusion module and an up-sampling reconstruction module; and training, verifying and testing the reconstructed network by using the training set, the verification set and the test set to obtain a trained reconstructed network. The reconstruction method provided by the invention not only can extract spatial position information in a gas concentration distribution diagram of a single time step, but also considers a gas diffusion motion relation, can extract time related information from a gas concentration distribution diagram of an adjacent time step, improves the reconstruction precision, and effectively inhibits artifacts and fuzziness.

Description

Technical Field

[0001] The present invention relates to the field of data completion, and in particular to a high-precision reconstruction method for sparse sampling of gas concentration field distribution. Background Art

[0002] Gas concentration field distribution has wide application value in many fields. In the field of environmental protection, gas concentration distribution information helps to monitor the emission and diffusion of polluting gases (such as sulfur dioxide, nitrogen oxides, etc.), and provides a scientific basis for air quality assessment and environmental governance; in the industrial field, the leakage of harmful gases may lead to fire, explosion or poisoning risks. Accurate understanding of gas concentration distribution information can help detect the leakage of harmful gases and locate the leakage position, thereby preventing accidents and ensuring the safety of industrial production and workers.

[0003] However, due to the limitations of the principles of gas sensors, it is difficult to obtain dense gas concentration field distribution in practice. Commonly used gas chemical sensors are designed with active functional layers that easily react with target gases. The basic principle is to convert chemical reactions into fluctuations in electrical signals to obtain gas concentration values. Because these sensors can only measure the concentration of target gases through direct contact with gas molecules, the measurement space coverage of a single chemical sensor is limited, and it is impossible to obtain dense gas concentration distribution information over a large area. Especially in industrial plants, unless the sensor is installed near the leak point, it is almost impossible to track gas diffusion, and locating the source of the gas leak is challenging.

[0004] In order to solve the above inherent limitations, researchers have developed a chemical sensor grid system for indoor applications and industrial manufacturing sites, and began to study the corresponding gas concentration distribution reconstruction technology. The chemical sensor grid system alleviates the limitations of a single sensor's coverage range by using low-cost chemical sensors in batches and installing them in grids to form a sensor array. The grid system reconstructs the dense gas concentration distribution information in the area of ​​interest by monitoring the response data of each sensor in the array. The core technology used in this process is called gas concentration distribution reconstruction technology.

[0005] Gas concentration distribution reconstruction technology reconstructs a dense gas concentration distribution map of a target area by using sensors to perform sparse sampling in the target area. With the development of the Internet of Things, dense and real-time gas concentration distribution data can be generated by densely installing sensor arrays, but the high cost and maintenance challenges brought by this method are unacceptable in practice. Therefore, the key to gas concentration distribution reconstruction technology is to use as few sensors as possible to reconstruct a dense gas concentration distribution with the highest possible accuracy as quickly as possible.

[0006] For sensor grid systems, gas concentration distribution reconstruction techniques can be mainly divided into model-based methods, statistics-based methods, and learning-based methods. Model-based methods infer the parameters of the gas concentration distribution model from the measured data, and create gas concentration distribution maps by numerically solving the control equations through fluid mechanics simulation. However, the fluid mechanics simulation method has a large amount of calculation and is not suitable for complex practical scenarios that require high resolution and need to update the model in real time. Statistics-based methods use the statistical characteristics of the measured data to reconstruct the gas concentration distribution map, mainly including interpolation, Gaussian process regression, and Gaussian kernel extrapolation. The interpolation method is fast, but the interpolation based on sparse input cannot meet the reconstruction requirements of high accuracy; the reconstruction accuracy of Gaussian process regression and Gaussian kernel extrapolation is higher than that of interpolation, but their performance is significantly affected by multiple manual processing parameters. Recent studies have used learning-based methods, treating the gas concentration distribution reconstruction problem as a super-resolution problem, treating the sparse sampling data of the sensor array as a low-resolution image, and treating the reconstructed dense gas concentration distribution as a high-resolution image. The advanced deep convolutional neural network method in the field of super-resolution is used to directly learn the gas concentration distribution law and realize the mapping of sparse sampling data to dense gas concentration distribution. Research has shown that the learning-based super-resolution method has higher reconstruction accuracy than the other two methods, and its real-time performance can also meet actual requirements.

[0007] However, most of the super-resolution networks currently used for gas concentration distribution reconstruction are based on single-image super-resolution, which only utilizes the sensor sampling data of a single time step and the spatial information of the array distribution. If the time-related information of continuous time steps is introduced to realize the spatiotemporal mapping of multiple time steps, better reconstruction quality can be achieved. At present, some studies have tried to introduce sensor array sampling data of multiple continuous time steps as the input of the super-resolution network. One processing method is to treat the data of multiple time steps as multi-channel low-resolution images, and then input them into the convolutional neural network to extract features; another processing method is to treat the data of multiple time steps as a sequence of images, and then input them into the recurrent convolutional neural network to extract features. The features extracted by the two methods are reconstructed into corresponding high-resolution images, i.e., dense gas concentration distribution, through the upsampling decoder. However, these two methods do not explicitly or implicitly characterize the gas diffusion movement relationship between adjacent time steps, and the neural network cannot determine which regions of the adjacent time steps should be used to extract features to enhance the reconstruction effect of the current time step. Therefore, the existing gas concentration distribution reconstruction method based on super-resolution network does not fully exploit the spatiotemporal information between consecutive time steps, and the reconstruction effect is limited. Further improvements are needed for this. Summary of the invention

[0008] The technical problem to be solved by the present invention is to address the limitations of the above-mentioned prior art and, taking into account the relationship between the gas diffusion motion between consecutive time steps, propose a high-precision reconstruction method for sparse sampling of airflow field based on motion estimation and compensation to improve the quality of reconstruction. The method comprises the following steps:

[0009] The first step is to construct a data set required for training the network of the present invention based on simulation and block sampling technology, and divide it into a training set, a validation set and a test set;

[0010] The label when training the network of the present invention is a high-resolution gas concentration distribution map with continuous time steps The input for training the network of the present invention is a high-resolution gas concentration distribution map with continuous time steps. The corresponding time-step continuous low-resolution gas concentration distribution map obtained by block sampling Where T is the number of time steps, C, H and W are The number of channels, height and width, C, s×H and s×W are The number of channels, height and width, s is the downsampling multiple or magnification multiple, and the default value of s is 32;

[0011] The second step is to build a gas concentration distribution reconstruction network based on motion estimation and compensation;

[0012] The gas concentration distribution reconstruction network based on motion estimation and compensation includes:

[0013] (1) Gas motion estimation module, based on two low-resolution gas concentration distribution maps at adjacent time steps and Use motion estimation method to calculate from arrive and from arrive The positive displacement vector of the gas in and the reverse displacement vector The moving objects reflected in the distribution map can be blocks or pixels;

[0014] (2) Gas motion compensation module, based on the forward displacement vector and the reverse displacement vector The low-resolution gas concentration distribution map is resampled using resampling technology. The corresponding reverse feature and low-resolution gas concentration maps The corresponding positive feature Perform inverse warp mapping to obtain the reverse features after motion compensation and positive features

[0015] (3) Feature extraction module, using the forward extraction block and reverse extraction block of the bidirectional recurrent convolutional neural network to extract the low-resolution gas concentration distribution map with continuous time steps Perform forward feature extraction and reverse feature extraction; forward feature extraction starts from the first time step and proceeds in the direction of increasing time steps. The input of the forward extraction block is a low-resolution gas concentration distribution map of a single time step. and the forward feature after motion compensation at the previous time step Then output the positive feature of the current time step The reverse feature extraction starts from the Tth time step and proceeds in the direction of decreasing time steps. The input of the reverse extraction block is the low-resolution gas concentration distribution map of a single time step. and the reverse feature after motion compensation at the next time step Then output the reverse feature of the current time step

[0016] (4) Feature fusion module, which uses cross-attention mechanism and dilated convolution to transform the low-resolution gas concentration distribution map of a single time step Positive features and reverse features Perform implicit feature alignment and fusion to obtain the fusion feature f of the current time step t ;

[0017] (5) Upsampling reconstruction module, which uses a combination of transposed convolution and pixel reorganization to convert the low-resolution gas concentration distribution map of a single time step into The fusion feature f t Perform step-by-step upsampling to obtain the size and Same residual plot Will Compared with directly using the interpolation algorithm Interpolation diagram magnified s times Add together to get the output of the reconstructed network

[0018] The third step is to use the training set, validation set and test set to train, validate and test the gas concentration distribution reconstruction network based on motion estimation and compensation to obtain a trained reconstruction network; wherein the Charbonnier loss function is used to calculate the output of the reconstruction network in the training stage. High-resolution gas concentration distribution corresponding to the reconstruction network input In the validation and testing stages, the MSE and SSIM indicator functions are used to comprehensively evaluate the quality of the reconstruction method.

[0019] Furthermore, the steps of constructing the data set required for training the network of the present invention based on the simulation and block sampling technology described in the first step include:

[0020] 1) Use fluid simulation software to simulate the flow of gas in three-dimensional space, sample at a certain section at a certain period, and obtain a high-resolution gas concentration distribution map with continuous time steps

[0021] 2) Divide the high-resolution gas concentration distribution map of each time step into square grids, randomly or regularly select a point in each grid to simulate a gas sensor, and arrange the gas concentration values ​​corresponding to these points according to the position relationship of the grid to obtain a low-resolution gas concentration distribution map with continuous time steps

[0022] 3) High-resolution gas concentration distribution map with T time steps and low-resolution gas concentration maps As a pair of input and label when training the network of the present invention, a data set required for training the network of the present invention is constructed.

[0023] Furthermore, the motion estimation method described in the second step (1) is used to calculate the arrive and from arrive The positive displacement vector of the gas in and the reverse displacement vector The motion estimation method may be a block matching method, a pixel recursive method, an energy method, an optical flow method or a neural network optical flow estimation method.

[0024] Furthermore, the training phase described in the third step uses the Charbonnier loss function to calculate the reconstructed network output High-resolution gas concentration distribution corresponding to the reconstruction network input The loss value between , where the Charbonnier loss function calculation expression is as follows:

[0025]

[0026] Among them, C, H and W represent the reconstruction network output graphs respectively. High-resolution gas concentration distribution corresponding to the reconstruction network input The number of channels, height and width, and Respectively represent and The pixel value of the jth row and kth column of the i-th channel in , ε is a preset minimum constant value.

[0027] Furthermore, the verification and testing phase described in the third step uses the MSE and SSIM indicator functions to comprehensively evaluate the pros and cons of the reconstruction method, where the calculation expressions of the MSE and SSIM indicator functions are as follows:

[0028]

[0029] In the MSE expression, C, H, and W represent the network output graphs High-resolution gas concentration distribution map corresponding to network input The number of channels, height and width; in the SSIM expression, yes The average value of yes The variance of yes The average value of yes The variance of yes and The covariance of 1 , c 2 A preset minimum constant value.

[0030] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above method steps when executing the program.

[0031] Technical effects and advantages of the present invention:

[0032] The present invention takes into account the gas diffusion relationship between continuous time steps, and proposes a high-precision reconstruction method for sparse sampling of airflow field based on motion estimation and compensation, which can improve the quality of reconstruction. The gas concentration distribution reconstruction network based on motion estimation and compensation has the following advantages: 1) The gas motion estimation module and the gas motion compensation module can help the reconstruction network not only extract spatial information from the gas concentration distribution map of a single time step, but also fully mine time-related information from the gas concentration distribution map of adjacent time steps, thereby improving the accuracy of reconstruction and effectively suppressing artifacts and blur; 2) The feature extraction module and the feature fusion module perform bidirectional feature extraction and efficiently fuse the extracted forward features and reverse features, which can effectively alleviate the problem of unbalanced information transmission between different time steps caused by unidirectional feature extraction, and improve the overall quality of reconstruction; 3) The upsampling reconstruction module uses a mixture of transposed convolution and pixel reorganization to gradually upsample the residual between the reconstructed interpolation map and the target map, which can reduce the difficulty of network learning, accelerate the network convergence speed, and achieve high-magnification upsampling reconstruction.

[0033] Other features and advantages of the present invention will be described in the following description, and partly become obvious from the description, or be understood by implementing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A schematic diagram of a reconstruction network provided by an embodiment of the present invention;

[0035] Figure 2 A schematic diagram of visualizing motion estimation of gas concentration distribution diagrams at adjacent time steps provided by an embodiment of the present invention;

[0036] Figure 3 A schematic diagram of motion compensation visualization provided by an embodiment of the present invention;

[0037] Figure 4a Target gas concentration distribution diagram provided for the embodiment of the present invention And reference pictures Gas concentration distribution map after motion compensation Schematic diagram of the comparison;

[0038] Figure 4b Target gas concentration distribution diagram provided for the embodiment of the present invention And reference pictures Gas concentration distribution map after motion compensation Schematic diagram of the comparison;

[0039] Figure 5 A schematic diagram of a feature fusion module provided in an embodiment of the present invention;

[0040] FIG6 is a diagram showing the reconstruction effects of different reconstruction methods provided by an embodiment of the present invention; wherein: Figure 6a is the input low-resolution gas concentration distribution map, Figure 6b is the target high-resolution gas concentration distribution map, Figure 6c This is the bilinear interpolation reconstruction effect diagram. Figure 6d This is the bicubic interpolation reconstruction effect diagram. Figure 6e Reconstruct the renderings for SwinIR, Figure 6f This is a diagram of the reconstruction effect of the reconstruction method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In a specific embodiment of the present invention, a method for high-precision reconstruction of airflow field sparse sampling based on motion estimation and compensation comprises the following steps:

[0043] The first step is to construct a data set required for training the network of the present invention based on the fluid simulation software Fluent simulation and block sampling technology, and divide it into a training set, a validation set and a test set;

[0044] In this embodiment, the steps of constructing the data set required for training the network of the present invention based on the fluid simulation software Fluent simulation and block sampling technology include:

[0045] Step 1: Use the fluid simulation software Fluent to simulate the flow of gas in three-dimensional space, sample a certain section at a certain period, and obtain a two-dimensional high-resolution gas concentration distribution map with a size of 224×224 with continuous time steps.

[0046] Step 2: Divide the high-resolution gas concentration distribution map of each time step into a 7×7 square grid, randomly or regularly select a point in each grid to simulate a gas sensor, and arrange the gas concentration values ​​corresponding to these points according to the position relationship of the grid to obtain a low-resolution gas concentration distribution map of 7×7 time steps.

[0047] Step 3: Continuous high-resolution gas concentration distribution map at 3 time steps and low-resolution gas concentration maps As a pair of input and label when training the network of the present invention, a data set required for training the network of the present invention is constructed.

[0048] In the second step, the deep learning framework Pytorch is used to build a gas concentration distribution reconstruction network based on motion estimation and compensation. The structure of the reconstruction network is as follows: Figure 1 As shown, it includes a gas motion estimation module, a gas motion compensation module, a feature extraction module, a feature fusion module and an upsampling reconstruction module, and the modules share parameters between different time steps. The gas motion estimation module and the gas motion compensation module perform bidirectional motion estimation and compensation, and the feature extraction module outputs a low-resolution gas concentration distribution map The corresponding positive feature and reverse features The feature fusion module fuses the forward and reverse features to obtain the fused feature f t , and finally the upsampling reconstruction module is used to fusion feature f t Upsample to get size and Same residual plot Will Compared with directly using the interpolation algorithm Interpolation diagram with 32 times magnification Add together to get the output of the reconstructed network

[0049] In this embodiment, the gas movement estimation module is based on the low-resolution gas concentration distribution maps of two adjacent time steps. and Calculated using optical flow The blocks or pixels in the The displacement vectors including the horizontal and vertical It is used to explicitly characterize the diffusion relationship between two adjacent distribution maps. At the same time, in order to improve the final reconstruction quality, it will also calculate The blocks or pixels in the The displacement vectors including the horizontal and vertical Figure 2 Shows two low-resolution gas concentration distribution maps at adjacent time steps and The arrows represent the relative displacement of the pixel from the gas motion estimation module to another time step. The gas motion estimation characterizes the relationship between the gas diffusion motion in adjacent time steps, making it possible to extract relevant information from the relevant regions of interest in the adjacent time steps to enhance the reconstruction effect of the current time step.

[0050] The gas motion estimation module uses motion estimation methods to estimate gas motion and can be embedded in the network for end-to-end training and reasoning with the support of hardware parallel computing. There is no need to perform motion estimation in advance when making data sets, which simplifies the training and reasoning process and speeds up the training and reasoning process.

[0051] In this embodiment, the gas motion compensation module is based on the forward displacement vector and the reverse displacement vector The low-resolution gas concentration distribution map is resampled using resampling technology. The corresponding reverse feature and low-resolution gas concentration maps The corresponding positive feature Perform inverse warp mapping to obtain the reverse features after motion compensation and positive features Figure 3 Shows the positive displacement vector from time t to t+1 Low-resolution gas concentration distribution map The reverse feature The intersection of the straight lines in the figure represents a pixel. The left figure is the output of the gas motion compensation module. The picture on the right shows one of the module inputs. The solid straight arrow represents another input of the gas motion compensation module The displacement vector of a pixel in . In order to get The gas concentration value at the pixel location in the image needs to be determined under the guidance of the displacement vector. Find the pixel value at the corresponding position and place it in In. Because The position value of the corresponding pixel in is often not an integer, so it is necessary to use resampling technology to obtain the pixel value of the non-integer coordinate point. In this embodiment, the pixel values ​​of four adjacent integer coordinate points are used and the target value is obtained by interpolation technology, and then this value is placed on the left side of the image. For the pixel position The same is true for the pixel values ​​at other locations. In order to have an intuitive understanding of the role of the gas motion compensation module, Figure 4a Displays the target gas concentration distribution map And reference pictures Gas concentration distribution map obtained after inverse distortion mapping The comparison chart, Figure 4b Displays the target gas concentration distribution map And reference pictures Gas concentration distribution map obtained after inverse distortion mapping The gas motion compensation module performs reverse distortion mapping on the reference gas concentration distribution map according to the motion information, and places the pixels of the reference map at the position of the target map at that moment, so that the contents of the reference map and the target map are spatially aligned. When the two are stacked up in terms of the number of channels and input into the convolutional neural network, the part of the reference gas concentration distribution map after distortion mapping in the convolution area is highly correlated with the part of the target gas concentration distribution map, because the convolution kernel has the characteristic of a local fixed-size receptive field, and thus the reconstruction effect of the target gas concentration distribution map can be enhanced.

[0052] Gas motion compensation is not performed on the low-resolution gas concentration distribution map, but on the corresponding feature map. Moving motion compensation from the image level to the feature level will bring significant improvements, because the inaccuracy of motion estimation has a greater impact on the deviation of image-level motion compensation, which is more likely to lead to a decrease in reconstruction performance. In other words, the reconstruction network based on feature-level motion compensation is more robust to motion estimation.

[0053] In this embodiment, the feature extraction module is a bidirectional recurrent convolutional neural network, which includes a forward extraction block and a reverse extraction block. The feature extraction network is a convolutional neural network, specifically a stack of convolutional layers, normalization layers, and activation layers. The forward feature extraction part extracts and propagates features according to the forward timeline. Taking time t as an example, its input is a low-resolution gas concentration distribution map. as well as Motion compensated forward features The two are stacked in the channel dimension, sent to the forward feature extraction module, and then the forward feature is output The output is fed into the feature fusion module on the one hand, and into the forward feature extraction module at time t+1 after motion compensation on the other hand. In particular, at time t=1, the input of the forward feature extraction part is the low-resolution gas concentration distribution map And size and The same feature map whose values ​​are all 0 by default. In order to alleviate the problem that the initial time steps of forward feature extraction are too few and the information propagation path is too short, that is, the time receptive field is too small, resulting in poor reconstruction effect of the initial low-resolution gas concentration distribution map, the reverse feature extraction part is introduced, that is, the reverse of the above forward feature extraction operation, thus forming a bidirectional recurrent neural network. The bidirectional recurrent convolutional neural network greatly alleviates the problem of unbalanced information transmission between different time steps and can greatly improve the overall quality of reconstruction.

[0054] In this embodiment, the structure of the feature fusion module is as follows: Figure 5 As shown in Figure 1, it includes a cross attention module and a dilated convolution module. Taking time t as an example, the input of the feature fusion module comes from the output of the feature extraction module, that is, the forward feature and reverse features In the criss-cross attention module, one of the attention blocks reverses the features As the query Q, the positive feature As key-value pairs K and V, the other attention is the opposite. and reverse features There is no interference in the previous modules, and the contents of the two are not aligned in space, so it is necessary to perform motion estimation and compensation to improve the quality of the final reconstruction. However, feature maps do not have obvious physical meanings like distribution maps, so it is impossible to perform explicit motion estimation and explicit motion compensation. The cross-attention module can implicitly complete these two steps. When an attention is reversed, the feature map is reversed. As the query Q, the positive feature When viewed as key-value pairs K and V, the reverse feature Positive features For reference, from the positive feature Find similar feature representations in the , the higher the similarity, the more it will be used, and the dissimilar features will be suppressed, so as to use the positive features The relevant feature information compensates and enhances the reverse feature The feature representation of the other attention is the opposite. Therefore, the positive feature and reverse features The process of using each other's related information to compensate and enhance itself is similar to the process of motion estimation and motion compensation. Then, the two features after compensation and enhancement are stacked in the channel dimension and sent to the dilated convolution with different dilation factors for parallel feature aggregation to obtain the fused feature f t Compared with ordinary convolution, dilated convolution with different dilation factors can tolerate more spatial offsets of the related contents of two features, achieve better feature aggregation, and thus enhance the fusion feature f. t The representation ability of the image can improve the final reconstruction quality.

[0055] In this embodiment, the upsampling reconstruction module first uses two transposed convolution layers with an upsampling factor of 4, and then uses a pixel reorganization layer with an upsampling factor of 2 to transform the low-resolution gas concentration distribution map of a single time step into The fusion feature f t A total of 32 times upsampling is performed to obtain the size and Same residual plot Finally Compared with the direct use of bilinear interpolation algorithm Interpolation diagram with 32 times magnification Add together to get the output of the reconstructed network The upsampling reconstruction module uses a step-by-step upsampling method to improve the reconstruction accuracy, and a mixed use of transposed convolution and pixel reorganization to improve the reconstruction speed. In addition, by reconstructing the residual map between the target output and the interpolation map, the difficulty of network learning can be reduced, the network convergence speed can be accelerated, and the final reconstruction quality can be improved.

[0056] In the third step, the training set, the validation set and the test set are used to respectively train, validate and test the gas concentration distribution reconstruction network based on motion estimation and compensation to obtain a trained reconstruction network.

[0057] In the training phase of this embodiment, three time-step continuous low-resolution gas concentration distribution maps with a size of 7×7 are selected from the training set and input into the reconstruction network. The reconstruction network outputs three corresponding images with a size of 224×224. The loss value between the reconstruction network output map and the high-resolution gas concentration distribution map corresponding to the reconstruction network input is calculated using the Charbonnier loss function. The network parameters are updated by backpropagation according to the loss value to obtain a reconstruction network model after training and updating. Repeat the above training steps until a certain number of training times or a loss value threshold is reached, and the training is terminated; the Charbonnier loss function expression is as follows:

[0058]

[0059] Among them, C, H and W represent the reconstruction network output graphs respectively. High-resolution gas concentration distribution corresponding to the reconstruction network input The number of channels, height and width, and Respectively represent and The pixel value of the jth row and kth column of the ith channel in the image, ε is a preset minimum constant value, usually 10 -6 The smaller the Charbonnier loss value, the better the reconstruction performance. Compared with the commonly used pixel-level loss function MSE, the Charbonnier loss function is robust and can better handle outliers and improve the performance of network reconstruction.

[0060] In the verification stage of this embodiment, three time-step continuous low-resolution gas concentration distribution maps with a size of 7×7 are selected from the verification set and input into the reconstruction network. The reconstruction network outputs three corresponding images with a size of 224×224, and the evaluation value between the reconstruction network output map and the high-resolution gas concentration distribution map corresponding to the reconstruction network input is calculated using the MSE and SSIM indicator functions. After each training step, a verification step is performed, and the corresponding model with the best comprehensive evaluation value is selected as the final trained reconstruction network model; the expressions of the MSE and SSIM indicator functions are as follows:

[0061]

[0062] In the MSE expression, C, H, and W represent the network output graphs High-resolution gas concentration distribution map corresponding to network input The number of channels, height and width; in the SSIM expression, yes The average value of yes The variance of yes The average value of yes The variance of yes and The covariance of 1 , c 2 It is a preset minimum constant value to avoid the situation where the denominator is zero. The smaller the MSE evaluation value, the smaller the difference between the two images. However, the MSE evaluation index cannot measure the similarity of the two images according to the perception of the human eye. For this reason, the SSIM index is introduced as a supplement. The SSIM structural similarity index extracts the brightness, contrast and structural information in the image based on the assumption that the human eye will extract it, which is more in line with the visual perception of the human eye. The SSIM evaluation value range is -1 to 1. The closer it is to 1, the more similar the two images are. The comprehensive evaluation method using the MSE and SSIM indicator functions can reduce artifacts and blur in the reconstructed image while ensuring the reconstruction accuracy, thereby improving the overall quality of the reconstruction.

[0063] In the test phase of this embodiment, three time-step continuous low-resolution gas concentration distribution maps with a size of 7×7 are selected from the test set and input to the reconstruction network. The reconstruction network outputs three corresponding images with a size of 224×224, and the evaluation value between the reconstruction network output map and the high-resolution gas concentration distribution map corresponding to the reconstruction network input is calculated using the MSE and SSIM indicator functions. The test phase occurs after the training and validation phases, and only one test is performed to evaluate the pros and cons of different reconstruction methods.

[0064] Table 1 shows the comparison between this method and other reconstruction methods, and the evaluation indicators are MSE and SSIM. The following three methods are compared: bilinear interpolation, bicubic interpolation and image super-resolution network SwinIR. According to the experimental results, it can be seen that the method of the present invention has achieved the best results in both indicators.

[0065] Table 1 Comparison of this method and other reconstruction methods

[0066] Reconstruction methods MSE↓ SSIM↑ Bilinear interpolation 4.0657 0.9630 Bicubic interpolation 4.2124 0.9616 SwinIR 1.6735 0.9854 The present invention 0.5561 0.9938

[0067] Figure 6 shows the reconstruction effects of different reconstruction methods, where: Figure 6a is the input low-resolution gas concentration distribution map, Figure 6b is the target high-resolution gas concentration distribution map, Figure 6c This is the bilinear interpolation reconstruction effect diagram. Figure 6dThis is the bicubic interpolation reconstruction effect diagram. Figure 6e Reconstruct the renderings for SwinIR, Figure 6f This is a reconstruction effect diagram of the reconstruction method provided by the embodiment of the present invention. It can be seen that the method of the present invention not only has higher reconstruction accuracy, but also has fewer artifacts and blur in the reconstructed image, and higher reconstruction quality.

[0068] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A high-precision reconstruction method for sparse sampling of airflow field based on motion estimation and compensation, characterized in that: The following steps are involved: The first step is to construct a data set required for training the network of the present invention based on simulation and block sampling technology, and divide it into a training set, a validation set and a test set; The label when training the network of the present invention is a high-resolution gas concentration distribution map with continuous time steps The input for training the network of the present invention is a high-resolution gas concentration distribution map with continuous time steps. The corresponding time-step continuous low-resolution gas concentration distribution map obtained by block sampling Where T is the number of time steps, C, H and W are The number of channels, height and width, C, s×H and s×W are The number of channels, height and width, s is the downsampling multiple or magnification multiple, and the default value of s is 32; The second step is to build a gas concentration distribution reconstruction network based on motion estimation and compensation; The gas concentration distribution reconstruction network based on motion estimation and compensation includes: (1) Gas motion estimation module, based on two low-resolution gas concentration distribution maps at adjacent time steps and Use motion estimation method to calculate arrive and from arrive The positive displacement vector of the gas in and the reverse displacement vector The moving objects reflected in the distribution map can be blocks or pixels; (2) Gas motion compensation module, based on the forward displacement vector and the reverse displacement vector The low-resolution gas concentration distribution map is resampled using resampling technology. The corresponding reverse feature and low-resolution gas concentration maps The corresponding positive feature Perform inverse warp mapping to obtain the reverse features after motion compensation and positive features (3) Feature extraction module, using the forward extraction block and reverse extraction block of the bidirectional recurrent convolutional neural network to extract the low-resolution gas concentration distribution map with continuous time steps Perform forward feature extraction and reverse feature extraction; forward feature extraction starts from the first time step and proceeds in the direction of increasing time steps. The input of the forward extraction block is a low-resolution gas concentration distribution map of a single time step. and the forward feature after motion compensation at the previous time step Then output the positive feature of the current time step The reverse feature extraction starts from the Tth time step and proceeds in the direction of decreasing time steps. The input of the reverse extraction block is the low-resolution gas concentration distribution map of a single time step. and the reverse feature after motion compensation at the next time step Then output the reverse feature of the current time step (4) Feature fusion module, which uses cross-attention mechanism and dilated convolution to transform the low-resolution gas concentration distribution map of a single time step Positive features and reverse features Perform implicit feature alignment and fusion to obtain the fused feature ft of the current time step; (5) Upsampling reconstruction module, which uses a combination of transposed convolution and pixel reorganization to convert the low-resolution gas concentration distribution map of a single time step into The fusion feature ft is gradually upsampled to obtain the size and Same residual plot Will Compared with directly using the interpolation algorithm Interpolation diagram with s times magnification Add together to get the output of the reconstructed network The third step is to use the training set, validation set and test set to train, validate and test the gas concentration distribution reconstruction network based on motion estimation and compensation to obtain a trained reconstruction network; wherein the Charbonnier loss function is used to calculate the output of the reconstruction network in the training stage. High-resolution gas concentration distribution corresponding to the reconstruction network input In the validation and testing stages, the MSE and SSIM indicator functions are used to comprehensively evaluate the quality of the reconstruction method.

2. The method for high-precision reconstruction of airflow field sparse sampling based on motion estimation and compensation as claimed in claim 1, characterized in that: The first step of constructing a data set required for training the network of the present invention based on simulation and block sampling technology includes the following steps: 1) Use fluid simulation software to simulate the flow of gas in three-dimensional space, sample at a certain section at a certain period, and obtain a high-resolution gas concentration distribution map with continuous time steps 2) Divide the high-resolution gas concentration distribution map of each time step into square grids, randomly or regularly select a point in each grid to simulate a gas sensor, and arrange the gas concentration values ​​corresponding to these points according to the position relationship of the grid to obtain a low-resolution gas concentration distribution map with continuous time steps 3) High-resolution gas concentration distribution map with T time steps and low-resolution gas concentration maps As a pair of input and label when training the network of the present invention, a data set required for training the network of the present invention is constructed.

3. The method for high-precision reconstruction of airflow field sparse sampling based on motion estimation and compensation as claimed in claim 1, characterized in that: The motion estimation method described in the second step (1) is used to calculate the arrive and from arrive The positive displacement vector of the gas in and the reverse displacement vector The motion estimation method may be a block matching method, a pixel recursive method, an energy method, an optical flow method or a neural network optical flow estimation method.

4. The method for high-precision reconstruction of airflow field sparse sampling based on motion estimation and compensation as claimed in claim 1, characterized in that: The training phase described in step 3 uses the Charbonnier loss function to calculate the reconstructed network output High-resolution gas concentration distribution corresponding to the reconstruction network input The loss value between , where the Charbonnier loss function calculation expression is as follows: Among them, C, H and W represent the reconstruction network output graphs respectively. High-resolution gas concentration distribution corresponding to the reconstruction network input The number of channels, height and width, and Respectively represent and The pixel value of the jth row and kth column of the i-th channel in , ε is a preset minimum constant value.

5. The method for high-precision reconstruction of airflow field sparse sampling based on motion estimation and compensation as claimed in claim 1, characterized in that: The verification and testing phase described in the third step uses the MSE and SSIM indicator functions to comprehensively evaluate the quality of the reconstruction method, where the calculation expressions of the MSE and SSIM indicator functions are as follows: In the MSE expression, C, H, and W represent the network output graphs High-resolution gas concentration distribution map corresponding to network input The number of channels, height and width; in the SSIM expression, yes The average value of yes The variance of yes The average value of yes The variance of yes and The covariance of , c1, c2 are preset minimum constant values.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method steps described in any one of claims 1 to 5 are implemented.

Images