A method for spatio-temporal reconstruction of three-dimensional sea wave field based on dispersion propagation multi-frame fusion
By combining sparse convolutional neural networks and residual refinement networks with wave dispersion relations, the problem of lack of temporal consistency and physical self-consistency in the reconstruction results of ocean waves in 3D reconstruction was solved, and high-precision spatiotemporal reconstruction of ocean wave 3D field was achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- JIANGSU UNIV OF SCI & TECH
- Filing Date
- 2026-03-20
- Publication Date
- 2026-07-03
AI Technical Summary
Existing 3D wave reconstruction methods based on stereo vision do not fully utilize wave dispersion relations, resulting in a lack of temporal consistency and physical self-consistency in the reconstruction results, especially with insufficient reconstruction accuracy under high sparsity conditions.
Sparse-to-dense completion is performed using a sparse convolutional neural network, frequency domain phase propagation is combined with linear wave dispersion relation, a phase propagation operator is constructed for multi-frame fusion, incremental correction is learned through a residual refinement network, observation closed-loop dispersion consistency constraint is introduced, and the training loss function is optimized to ensure the temporal consistency and physical self-consistency of the reconstruction results.
It significantly improves the temporal consistency and physical self-consistency of ocean wave 3D field reconstruction, enhances reconstruction accuracy, reduces data acquisition costs, and increases the system's automation level.
Smart Images
Figure CN122336191A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of marine observation and 3D reconstruction technology, and relates to a spatiotemporal reconstruction method for ocean wave 3D field based on dispersion propagation multi-frame fusion. Specifically, it relates to a spatiotemporal reconstruction method for ocean wave 3D field based on observation closed-loop dispersion consistency constraints; more specifically, it relates to a spatiotemporal reconstruction method for ocean wave 3D field based on dispersion propagation multi-frame fusion and observation closed-loop consistency constraints. Background Technology
[0002] Real-time observation and reconstruction of the three-dimensional wave field of the sea surface has significant application value in marine engineering, ship safety, coastal protection, and marine energy development. Traditional wave observation methods, such as buoys, pressure sensors, and radar, typically only provide one-dimensional time-series information at a single point or along a line, making it difficult to obtain spatially distributed three-dimensional wave fields. Recently, three-dimensional reconstruction methods of the sea surface based on binocular stereo vision have attracted widespread attention. These methods use binocular cameras mounted on fixed platforms (such as marine observation towers or ship masts) to simultaneously photograph the sea surface, obtaining a three-dimensional sparse point cloud of the sea surface through stereo matching and triangulation. However, due to the uniformity of sea surface texture, drastic changes in illumination, and matching difficulties caused by wave motion, the point cloud obtained by stereo matching is usually sparse and unevenly distributed, with an effective pixel ratio typically only 5% to 50%, far from being directly usable for wave field analysis.
[0003] Existing sparse-to-dense reconstruction methods mainly include the following categories: (1) Traditional interpolation methods: such as inverse distance weighted interpolation, Kriging interpolation, etc.; these methods are simple to calculate, but only utilize spatial neighborhood information and cannot capture the temporal evolution of waves. When the sparsity is high, the reconstruction quality drops sharply; (2) Sparse completion methods based on convolutional neural networks: using sparse convolutional networks (SparseCNN) to complete sparse observations into dense fields; these methods can learn spatial features, but single-frame processing cannot utilize the temporal correlation of waves; (3) Physics-driven multi-frame fusion methods: such as the method proposed by Pistellato et al. (2021), which uses linear wave dispersion relations to propagate the completion results of adjacent frames (and) to the frequency domain phase through frequency domain phase propagation. At the current moment, the final result is obtained by weighted fusion with the observation of the current frame, followed by interpolation and convolution refinement. This method utilizes dispersion relation for unidirectional propagation fusion from previous and subsequent frames to the current frame during the inference phase, but it has the following shortcomings: First, dispersion relation is only used as an auxiliary fusion during inference and is not used as a constraint condition during training, so the network output does not guarantee that it meets the physical laws of waves; Second, it only utilizes unidirectional propagation from previous and subsequent frames to the current frame, without verifying the consistency of the reconstruction result propagating to previous and subsequent frames, and lacks closed-loop self-consistency test; Third, the refinement network directly outputs the final reconstructed value without adopting a residual learning strategy, which poses a risk of signal loss in deep networks; Fourth, the loss function only includes pixel-level reconstruction error and lacks physical constraint terms, making it difficult to guarantee the rationality of the reconstruction result in the frequency domain and gradient domain. Summary of the Invention
[0004] Purpose of the invention: The purpose of this invention is to provide a spatiotemporal reconstruction method for three-dimensional ocean waves that can solve the problems of existing three-dimensional ocean wave reconstruction methods based on stereo vision not fully utilizing the physical priors of wave dispersion relations during the training process, resulting in a lack of temporal consistency and physical self-consistency in the reconstruction results, especially insufficient reconstruction accuracy under high sparsity conditions.
[0005] The technical solution of this invention is as follows: A spatiotemporal reconstruction method for three-dimensional ocean wave fields based on multi-frame fusion of dispersion propagation, comprising the following steps: Step (1) Sparse Completion and Dispersion Propagation: Acquire three consecutive frames of sparse sea surface observation data In this process, pixels that were observed are represented by real values, while pixels that were not observed are marked as invalid values; and the main wave direction angle is obtained. The invalid values in the sparse data of each frame are set to zero and a binary mask is generated. Then, a sparse convolutional neural network with shared weights is used to process the previous frame. and the next frame Perform sparse-to-dense completion to obtain the completed result. and Construct a phase propagation operator based on the linear wave dispersion relation. Through frequency domain phase propagation In time increment spread, In time increment Propagation to the present moment The dispersion prediction value is obtained. and ; Step (2) Weighted fusion and physical coarse estimation: Based on the mask of the previous frame Current frame mask and the mask of the next frame and the current frame weight coefficient Construct normalized fusion weights for dispersion prediction values Sparse observations in the current frame and dispersion prediction value The element-wise weighted sum is then logically ORed with the element-wise result of the three frame masks to obtain the joint mask. Multiplying them together yields a fused sparse field. ; for the fused sparse field Perform inverse distance-weighted interpolation to obtain a coarse global physical estimate. ; Step (3) Residual refinement: using physical coarse estimation Using the base case, the residual between the fused sparse field and the coarse estimate is calculated and used as the input to the refinement network. The network correction is then learned through a multi-layer sparse convolutional network. ,Will The total incremental correction is obtained by adding the refined input through a skip connection. The final output is the reconstruction result. .
[0006] Furthermore, the three consecutive frames of sparse sea surface observation data mentioned in step (1) are acquired through a binocular stereo vision system, specifically including: Simultaneous imaging of the sea surface using binocular cameras, followed by image distortion correction, stereo matching, and triangulation, yields a three-dimensional sparse point cloud of the sea surface. This sparse point cloud is then projected onto... On the regular grid, in the formula, The value range is 128~512; inter-frame time interval Determined by the camera frame rate, the value ranges from 0.05 to 1.0 seconds.
[0007] Furthermore, the main wave direction angle mentioned in step (1) is pre-specified by the user; or it is automatically estimated through optical flow analysis of two adjacent frames. Specifically, this involves extracting sparse feature points from the first frame image, and then... The optical flow tracing algorithm calculates the displacement vectors of feature points in the second frame, and takes the average direction angle of all displacement vectors as the main wave direction angle. .
[0008] Furthermore, the phase propagation operator described in step (1) is constructed as follows: Based on the deep-water linear dispersion relation: In the formula, For wavenumber modulus, It is the acceleration due to gravity. The wave number vector in the main wave direction The projected components on, Wave components used to distinguish between propagation in the same direction as and against the waves; combined with time increments Construct the phase propagation operator: ;when In time, the wave field will be propagated from the present moment to the future. At time; when At that time, the wave field is propagated from the current moment to the past. At time; in the formula, .
[0009] Furthermore, when the water depth is finite, the dispersion relation is replaced with a finite water depth dispersion relation, at which point the angular frequency satisfies... In the formula, For water depth; when surface flow velocity exists At that time, Doppler correction is performed: ,in, The surface flow rate is respectively at direction and The directional component.
[0010] Furthermore, the specific process of frequency domain phase propagation in step (1) is as follows: For the completion result The spectrum is obtained by performing a two-dimensional fast Fourier transform. The spectrum and phase propagation operator After element-wise multiplication, a two-dimensional inverse fast Fourier transform is performed, and the real part is taken to obtain the dispersion prediction value. ; For the completion result Perform the same operation, combining the spectrum with the phase propagation operator. After element-wise multiplication, a two-dimensional inverse fast Fourier transform is performed, and the real part is taken to obtain the dispersion prediction value. .
[0011] Furthermore, the specific process of weighted fusion in step (2) is as follows: using the previous frame mask Current frame mask and the mask of the next frame Constructing a normalized denominator In the formula, the current frame weight coefficient The value range is 2 to 16. To prevent small constants from being divided by zero; the fusion weights for each channel are: The dispersion prediction and the current frame observation are summed element-wise according to their weights and then multiplied by the joint mask. , to obtain a fused sparse field In the formula, For element-wise multiplication; joint mask The result obtained by logically ORing the three frames of masks element by element is: .
[0012] Furthermore, the specific process of performing inverse distance weighted interpolation in step (2) is as follows: Construct inverse distance weight kernel The core size is Odd-numbered square matrices The value range is 11~41; for each pixel position within the kernel, let its Euclidean distance to the kernel center be... ,but: In the formula, The distance decay index ranges from 1.0 to 4.0; the weights at the kernel center are truncated to a finite value of 1 to avoid singular divergence at zero distance. A coarse physical estimate is obtained through convolution operations. In the formula, To prevent small constants from being divided by zero.
[0013] Furthermore, the specific process of residual refinement in step (3) is as follows: calculate the refinement input. ; The refined input is fed into a refined network consisting of multiple sparse convolutions. This refined network comprises four sequentially connected sparse convolutional layers with kernel sizes of 5×5, 3×3, 3×3, and 1×1, and channel numbers of 32, 16, 8, and 1, respectively. The first three layers use... The activation function is a linear activation function, used in the last layer. The output of the refined network is the network correction value. In the formula, This represents the refined network; The total incremental correction is obtained by adding the refined input through a skip connection. The final reconstruction result is: .
[0014] Furthermore, a training method for training the sparse convolutional neural network and the refinement network in the three-dimensional wave field spatiotemporal reconstruction method includes the following steps: (T1) Training data generation: Using the dense sea surface elevation field as the ground truth data, each training sample is generated using a random sampling rate. Bernoulli random sampling is used to generate sparse inputs to simulate sparse observations generated by a binocular stereo vision system under different sea conditions. The specific process is as follows: For each training sample, a random sampling rate is applied. Bernoulli random sampling is used to generate sparse inputs. From a preset range in each sample Uniformly and randomly sampled from within. The value range is 0.05 to 0.2. The value ranges from 0.3 to 0.8; the sparse input is fed into the network for reconstruction and then compared with the original dense ground truth. (T2) Forward reconstruction: Perform forward calculations on the generated sparse input according to steps (1) to (3) to obtain the reconstruction result. and intermediate quantity ; (T3) Consistency constraint of observation closed-loop dispersion: The reconstruction results After frequency domain transformation, multiply by the time increment to obtain: and The phase propagation operator is used to obtain the propagation prediction through inverse frequency domain transformation. and The observation degradation operator is defined as a sparse observation binary mask of adjacent frames. The element-wise product of the propagation prediction and the propagation prediction degenerates into a sparse observation form, which is then compared with the true sparse observations of adjacent frames. The sum of the observation consistency errors in the two directions is calculated as the observation closed-loop dispersion consistency loss. ; The calculation process is as follows: Reconstruction results The spectrum is obtained by performing a two-dimensional fast Fourier transform. ; Propagate to the next frame: ; Propagate to the previous frame: ; Define the observation degeneracy operator In the formula, For sparse observation binary masks of adjacent frames, This is element-wise multiplication; Forward observation error: , Backward observation error: In the formula, These are the true sparse observations from adjacent frames. Total number of valid pixels; Observational closed-loop dispersion uniformity loss: ; (T4) Joint optimization: Construct a total loss function, which is composed of a weighted sum of pixel-level reconstruction loss, observation closure dispersion consistency loss, residual sparsity constraint, residual energy regularization and gradient smoothing constraint; optimize the parameters of the sparse convolutional neural network and the refinement network based on the total loss function; The total loss function is: ; In the formula, ; For pixel-level reconstruction loss, The value range is 0.5 to 0.95; To observe the closed-loop dispersion uniformity loss; As a constraint on residual sparsity, the amount of correction required for the refined network is small. To constrain the overall energy of the refined residual as a residual energy regularization; For gradient smoothing constraints, in the formula, These are spatial difference operators in the horizontal and vertical directions, respectively, to ensure the spatial continuity of the reconstructed sea surface; The value range is 0.01 to 1.0; The value range is 0.001 to 0.1; The value range is 0.001 to 0.05; The value range is 0.001 to 0.1; Among them, the observation closed-loop dispersion consistency loss Employ a warm-up strategy: before training In the wheel, Setting it to 0 allows pixel-level loss to converge first; thereafter It grows linearly from 0 to the target value. The value range is 5 to 30 rounds.
[0015] Beneficial Effects: Compared with existing technologies, this invention has the following significant advantages: 1. Observation Closed-Loop Dispersion Consistency Constraint: By introducing the wave dispersion relation into the training loss function, the reconstruction result is required to propagate forward to the next frame and backward to the previous frame through the dispersion relation, and then be degraded into a sparse observation form by the observation degradation operator (sparse sampling mask), consistent with the real sparse observations of adjacent frames, forming a true closed-loop constraint of "reconstruction → dispersion propagation → observation degradation → comparison of sparse observations"; since sparse sampling is an irreversible operation, this closed-loop constraint provides a non-trivial physical consistency supervision signal, forcing the network to learn a wave field representation that conforms to physical laws, significantly improving the temporal consistency of the reconstruction result; 2. Residual Learning and Skip Connections: Using physical coarse estimation Based on this, the refinement network learns only incremental corrections and ensures gradient flow through skip connections, solving the signal loss problem of deep sparse convolutional networks and enabling the refinement network to function effectively; 3. Joint optimization of multiple physical constraints: pixel-level reconstruction loss ensures spatial accuracy, observation closed-loop dispersion consistency loss ensures temporal physical consistency, residual sparsity constraint prevents over-correction by the refinement network, residual energy regularization constrains the overall energy of the refinement residuals to prevent excessive residual deviation from physical coarse estimation, and gradient smoothing constraint ensures the continuity of sea surface space; the five losses work together to constrain reconstruction quality from multiple dimensions; 4. Supervised training strategy based on random degradation: using dense sea surface elevation field as the ground truth, sparse input is generated through random Bernoulli sampling for training, eliminating the need for additional manual point-by-point labeling and reducing data acquisition costs; 5. Automatic wave direction estimation: supports automatic estimation of the main wave direction through optical flow analysis without manual intervention, improving the automation and practicality of the system. Attached Figure Description
[0016] Figure 1 This is the overall flowchart of the present invention; Figure 2 This is a flowchart of the network joint optimization training process in this invention; Figure 3 This is a diagram illustrating the network architecture based on multi-frame dispersion propagation and residual refinement in this invention. Detailed Implementation
[0017] The specific technical solutions of the present invention will be further described in detail below with reference to specific examples and accompanying drawings.
[0018] Example 1: System Overall Architecture The ocean wave three-dimensional field spatiotemporal reconstruction system of the present invention includes a data acquisition module, a preprocessing module, a reconstruction network module, and a training and optimization module.
[0019] The data acquisition module employs a binocular stereo vision system, including a pair of calibrated industrial cameras mounted on a marine observation platform. The baseline length is 0.5–3.0 m, and the frame rate is 2–10 Hz. It obtains a three-dimensional sparse point cloud of the sea surface through image distortion correction, stereo matching, and triangulation, and then projects the point cloud onto… Sparse sea surface elevation observation data were obtained on a regular grid. The preprocessing module organizes three consecutive frames of sparse observation data into a three-channel input tensor. Pixels that were observed are represented by real elevation values, while pixels that were not observed are marked as... Simultaneously, a phase propagation matrix is constructed based on the wave dispersion relation and the inter-frame time interval.
[0020] Example 2: Construction of Phase Propagation Matrix Based on the deep-water linear dispersion relation: In the formula, For wavenumber modulus, It is the acceleration due to gravity; The wave number vector in the main wave direction The projected components on, The wave components used to distinguish between wave propagation in the direction of the wave and wave propagation in the direction of the wave are given positive frequencies and wave components in the direction of the wave propagation in the direction of the wave are given negative frequencies. Constructing phase propagation operators Defined as: In the formula, The time increment (can be positive or negative); when "Time" indicates communication to the future. Time indicates transmission to the past; due to Only a matrix in one direction needs to be stored; The phase propagation matrix is stored as a 4-channel real tensor: In the formula, This refers to the inter-frame time interval. For finite water depth conditions, the dispersion relation is modified as follows: In the formula, For water depth; when surface flow velocity exists At that time, Doppler correction is performed: , The main wave direction can be obtained in the following two ways: (a) The user pre-specifies the main wave direction angle based on prior knowledge. (b) Through two adjacent frames of images The optical flow tracing algorithm automatically estimates and takes the average direction angle of the displacement vectors of all feature points as the main wave direction angle. .
[0021] Example 3: Sparse Completion and Dispersion Propagation (Step 1) From each frame of sparse observation data The value is set to zero, and a corresponding binary mask is generated (the valid bits are 1). (Position 0), concatenate the data and mask into a dual-channel input; A sparse convolutional neural network with shared weights (SparseCNN) is used to process the previous frame. and the next frame Perform sparse-to-dense completion to obtain the completed result. and SparseCNN is a pre-trained sparse convolutional network whose weights can be further fine-tuned during training. The completion result is propagated to the current time using frequency domain phase propagation. : (from spread to Time increment ) (from spread to Time increment In the formula, and These represent the two-dimensional fast Fourier transform and inverse Fourier transform, respectively. The sign of the time increment directly corresponds to the propagation direction: a positive increment when propagating from a past frame to the current frame, and a negative increment when propagating from a future frame to the current frame.
[0022] Example 4: Weighted Fusion and Coarse Physical Estimation (Step 2) Dispersion prediction value , With the current frame sparse observation Weighted fusion by mask; defining normalized denominator In the formula, This is the weighting coefficient for the current frame, with a value ranging from 2 to 16. To prevent division by zero of small constants (such as 10⁻) 7 The current frame is given a higher weight because its observations do not require dispersion propagation, resulting in the highest accuracy. The fusion weights for each channel are: ; ; ; The sparse field is: In the formula, For element-wise multiplication; joint mask From the previous frame mask Current frame mask and the mask of the next frame Element-wise logical OR is used to obtain the result, that is, for each pixel position... ,when hour ,otherwise Ensure that at least one frame contains observed locations before they are included in the fusion process; Inverse distance weighting for fused sparse fields ( Interpolation yields a coarse global physical estimate: In the formula, The inverse distance weight kernel is defined as follows: , (The core is truncated to a finite value to avoid singular divergence at zero distance); The distance decay index is the kernel size range of 11 to 41, and it is an odd number. To prevent division by zero of small constants (such as 10⁻) 7 ); Among them, although the completion result of the sparse convolutional network in step (1) , Although it is already a dense field, its predictions at locations without observations rely entirely on network inference, resulting in significantly lower reliability compared to locations with observational support; joint masking. The purpose is to retain only locations with at least one real observation in the frame for weighted fusion, and to fuse the weights at these locations. The calculation based on the actual observation mask has a clear physical meaning; while in positions where there are no observations in all three frames, the normalized denominator... The weight values are unreliable; therefore, reliable sparse anchor points are first obtained through mask fusion, and then... The interpolation spreads from these anchor points to the unobserved region, which is more robust than using the inference values of the network directly at the unobserved location.
[0023] Example 5: Residual Refinement (Step 3) Calculate and refine input: ; This input represents the residual between the fused sparse field and the IDW coarse estimate, and has a value only at locations where observations are available. The refined input is fed into a refined network consisting of 4 layers of sparse convolutions: Layer 1: 5×5 convolution kernel, 32 output channels activation; Layer 2: 3×3 convolutional kernels, 16 output channels activation; Layer 3: 3×3 convolutional kernels, 8 output channels activation; Layer 4: 1×1 convolutional kernel, 1 output channel, linear activation; The output of the refined network is the network correction value. In the formula, Show refined network; will The total incremental correction is obtained by adding the refined input through a skip connection: ; The purpose of skip connections is to enable the network to refine itself when its weights are close to zero during the early stages of training. ,therefore This ensures that the signal does not disappear; as training progresses, the network gradually learns more refined corrections. Among them, although It is a sparse field (only when...) (where there is a value), but the refined network consists of multiple layers of convolutions, and its receptive field diffuses the information of the sparse input to the surrounding area, thus the network output... It is a dense field that is continuous in space; middle The sparsity introduces local discontinuities at the mask boundaries, but because It is inherently a fusion of sparse fields and The difference between the coarse estimates is relatively small; meanwhile, the gradient smoothing constraint in the total loss function... This will force the network to learn corresponding compensatory corrections, so that the final output... Maintain spatial continuity; The final reconstruction result is: .
[0024] Example 6: Observing Closed-Loop Dispersion Consistency Constraints (Step 4) like Figure 2 As shown, the reconstruction results The spectrum is obtained by performing a two-dimensional fast Fourier transform. After multiplying by the phase propagation operator with positive and negative time increments respectively, perform an inverse transform: Propagate to the next frame: ; Propagate to the previous frame: ; Define the observation degeneracy operator In the formula, It is a sparse observation binary mask for adjacent frames (1 for positions with observations and 0 for positions without observations). This is an element-wise multiplication; the operator degenerates dense propagation predictions into a sparse form that is identical to the actual observations. Calculate the observation closed-loop dispersion consistency loss: ; in: ; In the formula, These are the true sparse observations from adjacent frames (values exist only at valid pixels). Total number of valid pixels; Among them, if If the sea surface wave field is correct, then after it propagates to the adjacent time through the dispersion relation, it should be consistent with the actual observation at the effective observation position of the sensor. This constitutes a true closed loop of "reconstruction → dispersion propagation → observation degradation → comparison of sparse observations". Since the sparse sampling operator A is irreversible (information is lost from dense to sparse), this constraint provides a non-trivial supervision signal and will not degenerate into an identity transformation.
[0025] Example 7 Joint Loss Function and Training Strategy like Figure 2 As shown, the total loss function is: ; The definitions are as follows: (1) Pixel-level reconstruction loss: In the formula, The value range is 0.5 to 0.95. As a structural similarity index, Norms guarantee pixel-level precision; (2) Loss of uniformity of observed closed-loop dispersion See Example 6. The value range is 0.01 to 1.0; (3) Residual sparsity constraint: The amount of correction to the refined network is constrained to be small to prevent the network from deviating too far from the physical rough estimate; The value range is 0.001 to 0.1; (4) Residual energy regularization: Constraining the overall energy of the refined residuals and the sparsity constraint of the spatial domain residuals. They complement each other and work together to prevent the correction amount of the refined network from deviating too far from the physical rough estimate; The value range is 0.001 to 0.05; (5) Gradient smoothing constraint: In the formula, For spatial gradient operators (including) and (Two directions) to ensure the spatial continuity of the reconstructed sea surface; The value range is 0.001 to 0.1; Dispersion consistency loss employs a warm-up strategy: before training... In the wheel, Setting it to 0 allows pixel-level loss to converge first, establishing basic spatial reconstruction capabilities; subsequently... The value grows linearly from 0 to the target value, gradually introducing physical constraints. Training adopted The optimizer has a learning rate ranging from 1×10⁻ 5 ~1×10⁻³, gradient clipping threshold range 0.5~5.0, batch size range 4~64.
[0026] Example 8: Generation of Supervised Training Data Based on Random Degradation The training method of this invention adopts a supervised training approach based on the random degradation of a dense ground truth field to generate sparse inputs, which eliminates the need for additional manual point-by-point annotation; the specific process is as follows: The dense sea surface elevation field is used as the ground truth data (which can be derived from numerical simulations, high-density lidar scanning, or high-quality stereo matching results); for each training sample, a random sampling rate is used. Bernoulli random sampling is performed to generate sparse input, where From a preset range in each sample Uniformly and randomly sampled from within. The value range is 0.05 to 0.2. The value range is 0.3 to 0.8; Each training sample contains three consecutive frames of data, and the data generator outputs the ground truth value of the current frame. Used for pixel-level loss calculation while retaining three frames of sparse input. Used for calculating the closed-loop dispersion uniformity loss; Inputs and truth values are uniformly min-max normalized within each batch. Intervals, using the mean during inference. A normalization scheme is used to enhance robustness.
Claims
1. A spatiotemporal reconstruction method for three-dimensional ocean wave fields based on dispersion propagation multi-frame fusion, characterized in that, The operating steps are as follows: Step (1) Sparse Completion and Dispersion Propagation: Acquire three consecutive frames of sparse sea surface observation data In this process, pixels that were observed are represented by real values, while pixels that were not observed are marked as invalid values; and the main wave direction angle is obtained. The invalid values in the sparse data of each frame are set to zero and a binary mask is generated. Then, a sparse convolutional neural network with shared weights is used to process the previous frame. and the next frame Perform sparse-to-dense completion to obtain the completed result. and Construct a phase propagation operator based on the linear wave dispersion relation. Through frequency domain phase propagation In time increment spread, In time increment Propagation to the present moment The dispersion prediction value is obtained. and ; Step (2) Weighted fusion and physical coarse estimation: Based on the mask of the previous frame Current frame mask and the mask of the next frame and the current frame weight coefficient Construct normalized fusion weights for dispersion prediction values Sparse observations in the current frame and dispersion prediction value The element-wise weighted sum is then combined with the joint mask obtained by logically ORing the three frames of masks element-wise. Multiplying them together yields a fused sparse field. ; for fused sparse fields Perform inverse distance-weighted interpolation to obtain a coarse global physical estimate. ; Step (3) Residual refinement: using physical coarse estimation Using the base case, the residual between the fused sparse field and the coarse estimate is calculated and used as the input to the refinement network. The network correction is then learned through a multi-layer sparse convolutional network. ,Will The total incremental correction is obtained by adding the refined input through a skip connection. The final output is the reconstruction result. .
2. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The three consecutive frames of sparse sea surface observation data mentioned in step (1) are acquired through a binocular stereo vision system, specifically including: Simultaneous imaging of the sea surface using binocular cameras, followed by image distortion correction, stereo matching, and triangulation, yields a three-dimensional sparse point cloud of the sea surface. This sparse point cloud is then projected onto... On the regular grid, in the formula, The value range is 128~512; inter-frame time interval Determined by the camera frame rate, the value ranges from 0.05 to 1.0 seconds.
3. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The main wave direction angle mentioned in step (1) is specified in advance by the user; or it can be automatically estimated through optical flow analysis of two adjacent frames. Specifically, this involves extracting sparse feature points from the first frame image, and then... The optical flow tracing algorithm calculates the displacement vectors of feature points in the second frame, and takes the average direction angle of all displacement vectors as the main wave direction angle. .
4. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The phase propagation operator described in step (1) is constructed as follows: Based on the deep-water linear dispersion relation: In the formula, For wavenumber modulus, It is the acceleration due to gravity. The wave number vector in the main wave direction The projected components on, Wave components used to distinguish between propagation in the same direction as and against the waves; combined with time increments Construct the phase propagation operator: ,when In time, the wave field will be propagated from the present moment to the future. At time; when At that time, the wave field is propagated from the current moment to the past. At time; in the formula, .
5. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 4, characterized in that, When the water depth is finite, the dispersion relation is replaced with a dispersion relation for finite water depth, at which point the angular frequency satisfies In the formula, For water depth; when surface flow velocity exists At that time, Doppler correction is performed: ,in, The surface flow rate is respectively at direction and The directional component.
6. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 2, characterized in that, The specific process of frequency domain phase propagation in step (1) is as follows: For the completion result The spectrum is obtained by performing a two-dimensional fast Fourier transform. The spectrum and phase propagation operator After element-wise multiplication, a two-dimensional inverse fast Fourier transform is performed, and the real part is taken to obtain the dispersion prediction value. ; For the completion result Perform the same operation, combining the spectrum with the phase propagation operator. After element-wise multiplication, perform a two-dimensional inverse fast Fourier transform. The dispersion prediction value is obtained by taking the real part. .
7. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The specific process of weighted fusion described in step (2) is as follows: Previously a single frame mask Current frame mask and the mask of the next frame Constructing a normalized denominator In the formula, the current frame weight coefficient The value range is 2 to 16. To prevent small constants from being divided by zero; the fusion weights for each channel are: The dispersion prediction and the current frame observation are summed element-wise according to their weights and then multiplied by the joint mask. , to obtain a fused sparse field In the formula, This is element-wise multiplication; The joint mask is obtained by logically ORing the three frames of masks element by element as follows: .
8. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The specific process of performing inverse distance weighted interpolation in step (2) is as follows: Construct inverse distance weight kernel The core size is Odd-numbered square matrices The value range is 11~41; for each pixel position within the kernel, let its Euclidean distance to the kernel center be... ,but: In the formula, The distance decay index ranges from 1.0 to 4.0; the weights at the kernel center are truncated to a finite value of 1 to avoid singular divergence at zero distance. A coarse physical estimate is obtained through convolution operations. In the formula, To prevent small constants from being divided by zero.
9. The spatiotemporal reconstruction method for three-dimensional ocean wave field based on dispersion propagation multi-frame fusion according to claim 1, characterized in that, The specific process of residual refinement in step (3) is as follows: calculate the refinement input. ; The refined input is fed into a refined network consisting of multiple sparse convolutions. This refined network comprises four sequentially connected sparse convolutional layers with kernel sizes of 5×5, 3×3, 3×3, and 1×1, and channel numbers of 32, 16, 8, and 1, respectively. The first three layers use... The activation function is a linear activation function, used in the last layer. The output of the refined network is the network correction value. In the formula, This represents the refined network; The total incremental correction is obtained by adding the refined input through a skip connection. The final reconstruction result is .
10. A training method for training a sparse convolutional neural network and a refinement network in the spatiotemporal reconstruction method of three-dimensional wave fields according to any one of claims 1-9, characterized in that, Includes the following steps: (T1) Training data generation: Using the dense sea surface elevation field as the ground truth data, each training sample is generated at a random sampling rate. Bernoulli random sampling is used to generate sparse inputs to simulate sparse observations generated by a binocular stereo vision system under different sea conditions. (T2) Forward reconstruction steps: Perform forward calculations on the generated sparse input according to steps (1) to (3) to obtain the reconstruction results. and intermediate quantity ; (T3) Observation closed-loop dispersion consistency constraint: The reconstruction results After frequency domain transformation, multiply by the time increment to obtain: and The phase propagation operator is used to obtain the propagation prediction through inverse frequency domain transformation. and ; The observation degradation operator is defined as a sparse observation binary mask of adjacent frames. The element-wise product of the propagation prediction and the propagation prediction degenerates into a sparse observation form, which is then compared with the true sparse observations of adjacent frames. The sum of the observation consistency errors in the two directions is calculated as the observation closed-loop dispersion consistency loss. ; (T4) Joint optimization: Construct a total loss function, which is composed of a weighted sum of pixel-level reconstruction loss, observation closure dispersion consistency loss, residual sparsity constraint, residual energy regularization and gradient smoothing constraint; optimize the parameters of the sparse convolutional neural network and the refinement network based on the total loss function.