Visual intelligent reconstruction evaluation system for three-dimensional wave liquid level

Abstract

The invention discloses a three-dimensional wave liquid level visual intelligent reconstruction evaluation system, and belongs to the field of ocean engineering monitoring. The system comprises an image acquisition module, an image stereoscopic vision processing module, an attention-enhanced reconstruction neural network module, a camera attitude evaluation module, a visualization and output module and a hydrodynamic parameter analysis module. An image is collected through a fixed baseline binocular camera system, after preprocessing, a neural network fused with a multi-scale attention mechanism is utilized to reconstruct a three-dimensional wave structure, coordinate system conversion is achieved in combination with self-supervised attitude evaluation, and finally hydrodynamic parameters such as significant wave height and a three-dimensional velocity field are extracted and visualized. The system does not need explicit calibration and large-scale data, has high automation, real-time performance and strong environmental adaptability, can be deployed on various platforms, and significantly improves the precision and engineering applicability of non-contact wave observation.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of ocean engineering monitoring, and particularly relates to a three-dimensional wave liquid surface visual intelligent reconstruction evaluation system. BACKGROUND

[0002] With the continuous advancement of marine resource development and marine engineering construction, obtaining high-precision and time-efficient three-dimensional structure of sea waves and hydrodynamic parameters (such as wave height, main frequency, propagation direction and velocity field) thereof has become an important issue of marine environment monitoring. The current mainstream three-dimensional wave reconstruction and evaluation technology has significant deficiencies in practical application, specifically as follows:

[0003] 1. Traditional measurement method: contact sensors (such as buoys, pressure gauges) need to be laid on the sea surface, and have limited spatial coverage, low resolution, and are easily damaged in severe sea conditions; remote sensing technology (such as satellite images, radar) has wide-area observation capability, but the update frequency and resolution are limited, and it is difficult to realize real-time reconstruction of small-scale, high-dynamic three-dimensional wave structure, especially for the response of nonlinear characteristics such as broken waves.

[0004] 2. Computer stereo vision method: traditional stereo vision relies on explicit calibration and feature matching, and is prone to failure when facing dramatic changes in wave surface and light interference; traditional stereo algorithm relies on single-point matching, which has large computational load and is difficult to achieve real-time effect.

[0005] 3. Method based on machine learning: convolutional neural network (CNN), recurrent neural network (RNN) and other structures are used to model the wave field, but they have strong dependence on training data, and the accuracy decreases in small samples or new environments; to capture multi-scale dynamic characteristics, a deep network is often constructed, resulting in a long model, low running efficiency, difficulty in deployment, and weak generalization ability.

[0006] In summary, the existing technology has the problems of dependence on explicit calibration and complex modeling process, poor adaptability to complex sea conditions, inability to recover three-dimensional velocity field and dynamic spectral characteristics in real time, weak generalization ability in small samples, large computational load and poor real-time performance, and is difficult to meet engineering requirements. SUMMARY

[0007] In view of the above deficiencies, the present application provides a three-dimensional wave liquid surface visual intelligent reconstruction evaluation system to meet the needs of high precision, high timeliness and strong generalization ability in the fields of marine engineering, intelligent monitoring and environmental simulation.

[0008] The technical scheme adopted by the present application is as follows: a three-dimensional wave liquid surface visual intelligent reconstruction evaluation system, comprising an image acquisition module, an image stereo vision processing module, an attention-enhanced reconstruction neural network module, a camera pose evaluation module, a visualization and output module, and a hydrodynamic parameter analysis module,

[0009] The image acquisition module is used for acquiring visual information of a sea wave liquid surface region, adopts a fixed baseline binocular camera to collect image sequences of a target sea area in real time, ensures spatiotemporal consistency through synchronous triggering, and performs distortion correction and epipolar line correction preprocessing, and the collected data includes left and right view gray image sequences and time stamps, and the internal and external parameter information of the camera system is recorded synchronously;

[0010] The image stereo vision processing module processes the binocular images, generates a dense disparity map through SGBM stereo matching, calculates a three-dimensional point cloud in combination with the camera internal parameter, restores the depth of the image pixels and calculates the three-dimensional point cloud, and forms an original point cloud data set describing the geometric shape of the sea wave liquid surface.

[0011] The attention-enhanced reconstruction neural network module constructs a deep neural network fusing multi-scale optical flow estimation and spatial-channel attention mechanism, models the three-dimensional geometric structure of the wave, extracts multi-scale wave pattern features through an encoder, strengthens the feature response of the key areas of wave crests and troughs through the attention mechanism, and the decoder outputs a dense disparity map and a reconstructed point cloud, and an implicit geometry learning mechanism is introduced to realize high-fidelity three-dimensional reconstruction based on the disparity without external parameters.

[0012] The camera pose evaluation module constructs a self-supervised pose estimation network based on plane constraints to perform main plane fitting, separates the rotation and translation information of the camera in combination with affine transformation, and only relies on the geometric consistency of the wave surface to complete camera pose estimation, and realizes geometric transformation from the camera coordinate system to the liquid surface coordinate system.

[0013] The visualization and output module generates three-dimensional visualization results of the sea wave liquid surface based on the disparity map and the camera pose estimation results, converts the reconstructed point cloud into a real space wave pattern, and exhibits the wave evolution process in combination with color mapping and animation sequences, and visualizes and outputs the reconstructed wave shape.

[0014] The hydrodynamic parameter analysis module extracts key hydrodynamic parameters including significant wave height, wavelength, wave speed, main frequency, energy spectrum distribution and three-dimensional velocity field based on the reconstructed continuous three-dimensional wave liquid surface data, the three-dimensional velocity vector field is obtained through continuous frame surface displacement inversion, and the dynamic characteristics of nonlinear, non-stationary and multi-scale wave shapes are comprehensively evaluated.

[0015] Further, the method for generating a three-dimensional point cloud by the image stereo vision processing module is as follows: SGBM stereo matching is performed on binocular images, and the projection points of the three-dimensional wave points in the images are left image points and right image points respectively. And The projection points of the three-dimensional wave points in the images are left image points And right image points ​Then the horizontal disparity for the left image is defined as:

[0016]

[0017] where the disparity matrix is solved by SGBM algorithm for matching of gray-scale images,

[0018] The pixel coordinates and disparity value can form the following homogeneous vector:

[0019]

[0020] By the re-projection matrix The pixel is back-projected to the homogeneous coordinates of the three-dimensional point:

[0021]

[0022] where, is the homogeneous factor, and the re-projection matrix is in the form of:

[0023]

[0024] where, is the focal length of the camera; is the baseline length of the left and right cameras; and are the horizontal and vertical coordinates of the principal point of the left image; is the horizontal coordinate of the principal point of the right image;

[0025] Finally, by dividing the homogeneous factor the Euclidean three-dimensional coordinates are obtained:

[0026] .

[0027] Further, the structure of the attention-enhanced reconstruction neural network module follows the "encoding-matching-decoding" paradigm, including:

[0028] The encoding stage adopts multi-scale feature extraction: the left and right images are normalized to unify the brightness scale, and then the residual structure encoder is used to extract dense feature maps of 1 / 8, 1 / 16 and 1 / 32 scales, and the left image is encoded as a reference image to guide the feature alignment in the matching process;

[0029] The matching stage constructs a one-dimensional correlation pyramid: the similarity is calculated by point multiplication or normalized inner product to form a cost volume along the scanning line direction, and the matching similarity is normalized by the Softmax function to obtain sub-pixel level disparity estimation value for each pixel; the self-attention mechanism is introduced to model the global feature map, dynamically adjust the weight through context awareness, suppress noise and fill in the disparity holes;

[0030] The decoding stage adopts a multi-level upsampling fusion strategy to gradually restore the low-resolution disparity map to the original map scale, fuse the feature information at each level, and utilize a residual enhancement module to refine the edge structure, and finally output a floating-point disparity map with the same size as the original map.

[0031] Further, the attention-enhanced reconstruction neural network module adopts an L1 disparity loss and a position-aware weighted loss function during the training process, wherein the position-aware weighted loss function is as follows:

[0032]

[0033] wherein, is a set of valid pixels; is a position weighting function of the pixel in the vertical direction; is a predicted disparity value; is a real disparity value.

[0034] 5. The three-dimensional wave surface visual intelligent reconstruction evaluation system according to claim 1, wherein the camera pose evaluation module estimates the camera pose by constructing a self-supervised pose estimation network based on plane constraint and combining liquid surface plane constraint for main plane fitting; wherein the self-supervised loss based on point cloud geometric consistency constraint is defined as:

[0035]

[0036] wherein, is a wave plane normal vector; is a plane intercept;

[0037] the sum of the distances of each point in the point cloud to the fitted plane is calculated as a loss;

[0038] wherein, the wave horizontal plane equation in the camera coordinate system is represented as:

[0039]

[0040] The transformation process takes the wave free surface normal as the benchmark, constructs a right-handed coordinate system through Gram-Schmidt orthogonalization, determines the translation vector based on the projection of the camera origin on the wave plane, and finally outputs the transformation matrix from the camera coordinate system to the wave coordinate system.

[0041] Further, the visualization output module combines the reconstructed point cloud data with the camera pose information to generate a three-dimensional wave shape diagram in the real space, and supports wave animation and color mapping display.

[0042] ​Furthermore, the hydrodynamic parameter analysis module extracts a three-dimensional velocity vector field by analyzing the surface displacement between consecutive frames, and calculates the dynamic characteristics of wave energy spectrum, dominant frequency, and direction distribution by combining the Fourier transform method.

[0043] Furthermore, the hydrodynamic parameter analysis module uses a velocity field reconstruction model from the nonlinear shallow water wave theory to obtain the wave surface elevation field. Invert the three-dimensional velocity vector field, where the vertical velocity is reconstructed. With horizontal velocity component The calculation satisfies:

[0044]

[0045]

[0046]

[0047] Among them, vertical velocity The expression contains two key terms: one is the instantaneous rate of change of the wavefront. First, it reflects the rapid undulation behavior of the wave at that point; second, it is the tensor time integral term in the direction of the horizontal gradient of the wave surface. This expression is used to describe the fluid response caused by non-uniform propagation; the entire expression is... Weighting was applied to account for the correction of the vertical velocity by the wavefront slope; the horizontal velocity field... The calculation depends on The time integral of the tensor is then subtracted from the projection correction term of the vertical velocity onto the wavefront gradient. This reflects the local flow characteristics of water particles near the water surface; It is defined as a combination of gravitational potential energy and kinetic energy, with the first term being gravitational potential energy and the second term being the kinetic energy correction considering the wave surface velocity.

[0048] Using the theoretical model described above, numerical calculations were performed on the reconstructed time-series wavefront data. , , A three-dimensional velocity field for the time series was constructed, the spatial gradient was estimated using central difference, the time derivative was calculated using three-point backward difference, and the solution was obtained through time integration. The process ultimately reconstructs the complete three-dimensional velocity vector field for each frame. .

[0049] The present application has the advantages and beneficial effects that: the present application does not rely on external structured light or calibration board, and does not need large-scale manual annotation, has high automation and practicality, and is especially suitable for sea conditions where traditional buoys and underwater sensors are difficult to deploy. The system can be deployed on a shore-based observation platform, an unmanned ship system or a near-shore floating platform, and has real-time acquisition and online evaluation capabilities. Combined with multi-frame image input and GPU accelerated inference mechanism, the evaluation method can realize real-time wave three-dimensional structure and velocity field estimation at a speed of seconds, which meets the urgent needs of fast response and high spatial resolution wave field perception in current ocean engineering. The present application significantly improves the precision, efficiency and engineering adaptability of non-contact wave observation. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 is a flowchart of the intelligent wave field reconstruction method provided by the embodiment of the present application;

[0051] Figure 2 is a training label diagram provided by the embodiment of the present application;

[0052] Figure 3 is a three-dimensional neural network diagram under attention enhancement provided by the embodiment of the present application;

[0053] Figure 4 is a network output point cloud result diagram provided by the embodiment of the present application;

[0054] Figure 5 is a real wave three-dimensional reconstruction result diagram provided by the embodiment of the present application;

[0055] Figure 6 is a wave height parameter reconstruction error diagram provided by the embodiment of the present application;

[0056] Figure 7 is a wave height parameter time sequence evaluation effect diagram provided by the embodiment of the present application;

[0057] Figure 8 is a power spectrum evaluation effect diagram provided by the embodiment of the present application;

[0058] Figure 9 is a wave three-dimensional transient velocity evaluation effect diagram provided by the embodiment of the present application, wherein (a) is an angle proportion histogram of (Ux, Uy) vector, and (b) is a vertical velocity W accuracy comparison diagram. DETAILED DESCRIPTION

[0059] The application proposes a three-dimensional wave surface visual intelligent reconstruction and evaluation technology for real marine environment, the core of which is to realize accurate three-dimensional modeling of wave field and automatic extraction of hydrodynamic parameters through a deep neural network, which significantly improves the accuracy, efficiency and engineering adaptability of non-contact wave observation. To achieve the above goal, the specific implementation combines the disparity estimation and optical flow matching mechanism in deep learning to design and implement a wave field reconstruction network architecture that integrates multi-scale attention mechanism and implicit geometric modeling, and is supplemented by a self-supervised camera pose evaluation and point cloud conversion module, finally a complete three-dimensional wave surface reconstruction and hydrodynamic analysis system is built. As shown in Figure 1 The technical process of each module of the application is shown. The following will combine the technical process of the application to illustrate the overall architecture of the system, the function of the core module and its specific implementation in detail.

[0060] Embodiment 1

[0061] In order to obtain high-quality sea wave image data for model training and testing, a stereo image dataset for real complex sea conditions is constructed in the implementation process. The dataset is derived from a binocular camera actually deployed on the Black Sea coast, with a collection time spanning multiple day-night cycles, covering typical wave conditions from wind waves, small waves to more complex breaking waves. The binocular camera is installed on a fixed platform to ensure its stable baseline and the visual angle covers the wave propagation area of the nearshore sea area. The collected data includes left and right view gray image sequences and their timestamps, and the internal and external parameter information of the camera system is recorded synchronously to support subsequent epipolar correction and stereo matching. Each pair of images generates a dense disparity map through the SGBM algorithm, forming a training sample with true value, and generates a wave point cloud under stereo vision as a subsequent reference. Specifically:

[0062] Let the images taken by the left and right cameras be and The projection points of the three-dimensional wave points in the images are left image points and right image points , then the horizontal disparity of the left image is defined as:

[0063]

[0064] where the disparity matrix is solved by the SGBM algorithm for gray image matching.

[0065] The pixel coordinates and disparity values can form the following homogeneous vectors:

[0066]

[0067] The pixel is back-projected to the homogeneous coordinates of the three-dimensional point through the re-projection matrix .

[0068]

[0069] where is a homographic factor, the re-projection matrix is of the form:

[0070]

[0071] where, is the camera focal length; is the baseline length of left and right cameras; and are the horizontal and vertical coordinates of the principal point of the left image (usually the image center); is the horizontal coordinate of the principal point of the right image.

[0072] Finally, by dividing the homographic factor the Euclidean three-dimensional coordinates are obtained:

[0073]

[0074] As shown in Figure 2 , it is an instance of training label, on the basis of which different data sets are constructed by combining a plurality of light and angle changes, and are divided into training, verification and test subsets for network training and performance verification.

[0075] Compared with the traditional method which needs to perform cost calculation and global search in the two-dimensional image plane, the present application only performs correlation matching in the horizontal direction, greatly reduces the calculation complexity, and improves the training efficiency and convergence speed. The design limits the disparity between the left and right images to the offset along the scan line direction, and sets the vertical optical flow component to zero, thereby realizing the directional constraint of the matching task, and significantly improving the stability and accuracy of the disparity estimation in the sea wave dynamic scene.

[0076] The input image is a corrected binocular RGB image pair, and the uniform size is 1600*1200. The pixel value is normalized to the range of [0, 1] to adapt to the neural network training. The model training adopts the Adam optimizer, the initial learning rate is set to 0.0004, the cosine annealing scheduling strategy is adopted, the training is performed for a total of 200 epochs, each epoch contains 3600 image pair samples, and the batchsize is set to 8. The early stopping mechanism is set to terminate the training when the verification set EPE (end point error) continuously decreases by less than 1% for 15 rounds.

[0077] In the encoding stage, the left and right images are input into the residual feature extraction network sharing parameters, which is constructed based on the ResNet structure and outputs feature maps of three scales corresponding to 1 / 8, 1 / 16 and 1 / 32 resolutions respectively. The left image additionally generates a guide mask through a position encoding module, and the output dimensions of the guide mask are aligned with the feature maps of each scale, which are all stored in float32 precision.

[0078] As shown in Figure 3 , the application constructs an attention-enhanced reconstruction neural network module, and the overall structure follows the design paradigm of "encoding - matching - decoding". First, the input left and right images are normalized to unify the brightness scale and improve the robustness of the model to different lighting conditions. Then, the images are sent into the feature extractor with residual structure to extract dense feature maps at three scales of 1 / 8, 1 / 16 and 1 / 32. Meanwhile, the left image is encoded as a reference image to guide the feature alignment in the matching process.

[0079] The network then constructs a one-dimensional correlation pyramid at each scale, calculates the similarity using dot product or normalized inner product, forms a cost volume along the scanning line direction, and normalizes the matching similarity through the Softmax function to obtain sub-pixel level disparity estimation values for each pixel. To enhance the matching stability of weak texture and occluded areas, a self-attention mechanism is introduced to model the global feature map, which dynamically adjusts the weights through context awareness to suppress noise and fill in the disparity holes.

[0080] In the decoding stage, the network uses a multi-level upsampling fusion strategy to gradually restore the low-resolution disparity map to the original image size, while fusing feature information at each scale, and introduces a residual enhancement module to refine the edge structure, improving the clarity and continuity of the disparity map. The output disparity map is a floating-point tensor with the same size as the original image, with accurate sub-pixel resolution, ensuring complete restoration of the three-dimensional wave structure in complex sea conditions.

[0081] In the training process, the application not only uses the standard L1 disparity loss, but also designs a position-aware weighted loss function to explicitly depict the physical law of error tolerance at different vertical positions in the image. In the vertical direction of the image, waves close to the camera are more prone to errors due to larger disparity, so higher tolerance is given to the near area in error calculation, while stronger constraints are applied to distant waves, thereby improving the overall reconstruction stability and physical consistency.

[0082]

[0083] wherein, is the set of valid pixels; is the position weighting function of the pixel in the vertical direction, which reflects the physical law of the change of disparity value with the vertical coordinate.

[0084] The experimental results show that the average end point error (EPE) of the disparity estimation module on the test set is 1.12 pixels, and the average space error (MAE) of the three-dimensional point cloud reconstruction reaches millimeter level accuracy in most areas.

[0085] The disparity tensor data output by the preceding network processing module is taken as the main intermediate data format, and is sent to the camera pose evaluation module through a unified data interface. Specifically, after image preprocessing, feature matching and disparity calculation are completed, a dense disparity map is generated, and it is transmitted to the pose evaluation module through memory sharing or cache mechanism. The pose evaluation module combines the camera intrinsic parameters based on the input disparity map to further reconstruct the three-dimensional point cloud and estimate the pose change of the camera in the observation process, thereby providing spatial reference information for the subsequent reconstruction network and visualization module.

[0086] According to the re-projection formula in the stereo geometry model, the disparity value and the pixel position are jointly constructed into homogeneous coordinates, and are mapped into a three-dimensional point set in the camera coordinate system through the re-projection matrix. The point cloud data can intuitively reflect the wave fluctuation form, but due to its dependence on the camera coordinates, it lacks a unified marine reference framework and is difficult to use for engineering analysis. Therefore, the present application further introduces a self-supervised pose estimation network based on point cloud geometric consistency, which automatically fits the main plane equation of the wave free surface without manual calibration, and constructs the transformation matrix from the camera coordinate system to the wave coordinate system. The corresponding self-supervised loss based on the geometric consistency constraint of the point cloud is defined as:

[0087]

[0088] wherein, is the wave plane normal vector; is the plane intercept;

[0089] Therefore, the wave horizontal plane equation in the camera coordinate system can be expressed as:

[0090]

[0091] The transformation process takes the wave free surface normal as the benchmark, constructs a right-handed coordinate system through Gram-Schmidt orthogonalization, and determines the translation vector based on the projection of the camera origin on the wave plane. After completing the pose transformation, all three-dimensional points are projected into the wave coordinate system, constructing a relatively stable and physically meaningful expression of the three-dimensional form of the wave; the specific steps are:

[0092] Select the origin of the camera coordinate system Project onto the wave horizontal plane to determine the translation vector

[0093] ​

[0094] The axes of the wave coordinate system are defined by The axis and plane normal vectors are parallel, thus, The axis unit vectors are:

[0095]

[0096] The initial vectors are chosen to be not parallel with The initial vectors are chosen to be not parallel with The new coordinate system's axis unit vectors are determined by Gram-Schmidt orthogonalization:

[0097]

[0098] The new coordinate system's axis is determined by the right-hand rule:

[0099]

[0100] The final rotation matrix is composed of three orthogonal unit vectors:

[0101]

[0102] For any point in the original camera coordinate system, the point in the wave coordinate system is:

[0103]

[0104] Thus, the complete conversion of three-dimensional point cloud data from the camera coordinate system to the wave coordinate system is completed, and the effective extraction and analysis of wave surface information are realized. It is worth noting that the positive direction of the coordinate axis can be adjusted according to actual needs. By changing the positive and negative of the elements in the rotation matrix vector, the definition of the direction of each axis can be flexibly controlled to adapt to different measurement scenarios or analysis targets. For example, Figure 4 as shown, is a point cloud result diagram of the neural network output result, and Figure 5 as shown, is a three-dimensional reconstruction result diagram of the point cloud result mapped to the real wave.

[0105] As shown in Figure 6 , the wave height parameter reconstruction error diagram provided by the embodiment of the present application, the wave height sequence output by the neural network and the actual reference measurement data are highly consistent at most times.

[0106] Based on the completion of the three-dimensional point cloud construction, further use the velocity field reconstruction model in the nonlinear shallow water wave theory, the wave height program is converted into three-dimensional velocity vector field. The following formula provides the theoretical path from the wave height field to reconstruct the vertical velocity and the horizontal velocity component

[0107]

[0108]

[0109]

[0110] The vertical velocity contains two key items in the expression: one is the instantaneous change rate of the wave surface , reflecting the rapid fluctuation behavior of the wave at the point; the second is the tensor time integral term in the direction of the wave surface horizontal gradient , which is used to describe the fluid response caused by non-uniform propagation. The whole expression is weighted, so as to consider the correction of the wave surface slope to the vertical velocity. The calculation of the horizontal velocity field depends on the time integral of the tensor, and subtracts the projection correction term of the vertical velocity to the wave surface gradient , which embodies the local flow characteristics of the water body particles near the water surface. The horizontal velocity itself is defined as the combination of the gravitational potential energy term and the kinetic energy term, the first term is the gravitational potential energy, and the second term is the kinetic energy correction considering the wave surface velocity.

[0111] Using the above theoretical model, each item is numerically calculated from the reconstructed time series wave surface data , , , and the three-dimensional velocity field is constructed. The spatial gradient is estimated by central difference, the time derivative is calculated by three-point backward difference, and the process is solved by time integration. Finally, the complete three-dimensional velocity vector field of each frame is reconstructed.

[0112] The model first extracts the wave height change rate and horizontal gradient tensor based on the central difference method and time derivative calculation method, and restores the potential function through time integration, and then calculates the horizontal and vertical velocity vectors combined with the kinetic and potential energy components. This model strictly follows the physical constraints, can accurately restore the velocity structure near the surface of the wave without introducing fluid particle simulation, and is suitable for fluid coupling, vortex detection and wave-current interaction analysis and other application scenarios.

[0113] As Figure 7 ​Fig. 1 shows a wave height parameter evaluation effect diagram provided by an embodiment of the present application; as Figure 8 Fig. 4 shows a power spectrum evaluation effect diagram provided by an embodiment of the present application. Experiments show that the system can achieve millimeter-level three-dimensional reconstruction accuracy under multiple typical sea conditions, and the frequency main peak error is controlled within 0.01 Hz. As Figure 9 Fig. 5 shows a wave three-dimensional transient velocity evaluation effect diagram provided by an embodiment of the present application. The velocity vector field is highly consistent with the measured reference data. The network relying on the attention mechanism can still adaptively fill in the parallax missing area, avoiding the interference of the reconstruction cavity on the wave field modeling accuracy.

[0114] The implementation of the present application does not depend on external structured light or calibration board, and does not need large-scale manual annotation, has high automation and practicality, and is especially suitable for sea condition areas where traditional buoys and underwater sensors are difficult to deploy. The system can be deployed on a shore-based observation platform, an unmanned ship system or a near-sea floating platform, and has real-time acquisition and online evaluation capabilities. Combined with multi-frame image input and GPU accelerated inference mechanism, the method realizes real-time wave three-dimensional structure and velocity field estimation at a speed of seconds per image, and adapts to the urgent needs of fast response and high spatial resolution wave field perception in current ocean engineering.

Claims

1. A three-dimensional wave level visual intelligent reconstruction evaluation system, characterized in that, The image acquisition module, the image stereo vision processing module, the attention-enhanced reconstruction neural network module, the camera pose evaluation module, the visualization and output module, and the water dynamic parameter analysis module, The image acquisition module is used for acquiring visual information of a sea wave liquid surface region, adopts a fixed baseline binocular camera to collect image sequences of a target sea area in real time, ensures spatiotemporal consistency through synchronous triggering, and performs distortion correction and epipolar correction preprocessing, the collected data includes left and right view gray image sequences and time stamps, and the internal and external parameter information of the camera system is recorded synchronously; The image stereo vision processing module processes the binocular images, generates a dense disparity map through SGBM stereo matching, calculates three-dimensional point clouds in combination with the camera internal parameter, restores the depth of the image pixels and calculates three-dimensional point clouds, and forms an original point cloud data set describing the geometric shape of the sea wave liquid surface; The attention-enhanced reconstruction neural network module constructs a deep neural network integrating multi-scale optical flow estimation and spatial-channel attention mechanism, models the three-dimensional geometric structure of waves, extracts multi-scale wave pattern features through an encoder, strengthens the feature response of the key areas of wave crests and troughs through the attention mechanism, and outputs a dense disparity map and reconstructed point clouds through a decoder, and an implicit geometry learning mechanism is introduced to realize high-fidelity three-dimensional reconstruction based on disparity without external parameters; The camera pose evaluation module performs main plane fitting based on a plane constraint to construct a self-supervised pose estimation network, separates the rotation and translation information of the camera in combination with affine transformation, completes camera pose estimation only by relying on the geometric consistency of the wave surface, and realizes geometric transformation from the camera coordinate system to the liquid surface coordinate system; The visualization and output module generates three-dimensional visualization results of the sea wave liquid surface based on the disparity map and the camera pose estimation results, converts the reconstructed point clouds into real space wave pattern graphs, and exhibits the wave evolution process in combination with color mapping and animation sequences, and visualizes and outputs the reconstructed wave shape; The water dynamic parameter analysis module extracts key water dynamic parameters including significant wave height, wavelength, wave speed, main frequency, energy spectrum distribution and three-dimensional velocity field based on the reconstructed continuous three-dimensional wave liquid surface data, the three-dimensional velocity vector field is obtained through continuous frame surface displacement inversion, and the dynamic characteristics of nonlinear, non-stationary and multi-scale wave shapes are comprehensively evaluated.

2. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, wherein The method for generating a 3D point cloud by the image stereo vision processing module is as follows: SGBM stereo matching is performed on the binocular images, assuming the images captured by the left and right cameras are respectively... and Wave 3D Points The projection points in the image are respectively the left image point. With right image point Then the horizontal disparity of the left image is defined as: The disparity matrix is solved through the SGBM algorithm for gray image matching, The pixel coordinates and the disparity value can form the following homogeneous vector: By the re-projection matrix Homogeneous coordinates of the pixel back-projected to a three-dimensional point: wherein is a homogeneous factor, the re-projection matrix is of the form: wherein is the camera focal length; is the left-right camera baseline length; and is the horizontal coordinate of the left image principal point; is the horizontal coordinate of the right image principal point; Finally, by dividing by the homogeneous factor Euclidean three-dimensional coordinates are obtained: 。 3. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, wherein The structure of the attention-enhanced reconstruction neural network module follows the "encoding-matching-decoding" paradigm, including: The multi-scale feature extraction is adopted in the encoding stage: the left and right images are normalized to unify the brightness scale, and then the residual structure encoder is used to extract dense feature maps of 1 / 8, 1 / 16 and 1 / 32 scales, and the left image is encoded as a reference image to guide the feature alignment in the matching process. The matching stage constructs a one-dimensional correlation pyramid: the similarity is calculated by point multiplication or normalized inner product to form a cost volume along the scanning line direction, and the matching similarity is normalized by a Softmax function to obtain sub-pixel level disparity estimation value of each pixel; the self-attention mechanism is introduced to model the global feature map, and the weight is dynamically adjusted by context awareness to suppress noise and fill in the disparity holes; The decoding stage adopts a multi-level upsampling fusion strategy to gradually restore the low-resolution disparity map to the original image size, fuse the feature information at each level, and use a residual enhancement module to refine the edge structure, and finally output a floating-point disparity map with the same size as the original image.

4. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, wherein, The attention-enhanced reconstruction neural network module adopts an L1 disparity loss and a position-aware weighted loss function in the training process, and the position-aware weighted loss function is as follows: ， wherein, is a set of valid pixels; is a position weighting function of the pixels in the vertical direction; is a predicted disparity value; is a true disparity value.

5. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, characterized in that: The camera pose evaluation module estimates the camera pose by constructing a self-supervised pose estimation network based on plane constraints and fitting the main plane based on liquid surface plane constraints; wherein the self-supervised loss based on point cloud geometric consistency constraint is defined as: wherein is the wave plane normal vector; is the plane intercept; The sum of the distances of each point in the point cloud to the fitted plane is calculated as the loss; Wherein, the wave horizontal plane equation in the camera coordinate system is represented as: The transformation process takes the wave free surface normal as the reference, constructs a right-handed coordinate system through Gram-Schmidt orthogonalization, determines the translation vector based on the projection of the camera origin on the wave plane, and finally outputs the transformation matrix from the camera coordinate system to the wave coordinate system.

6. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, characterized in that: The visual output module combines the reconstructed point cloud data with the camera pose information to generate a three-dimensional wave shape graph in the real space, and supports wave animation and color mapping display.

7. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, characterized in that: The hydrodynamic parameter analysis module extracts the three-dimensional velocity vector field by analyzing the surface displacement between consecutive frames, and calculates the kinetic characteristics of the wave energy spectrum, main frequency and direction distribution by combining the Fourier transform method.

8. The three-dimensional wave-surface visual intelligent reconstruction evaluation system according to claim 1, characterized in that, The water power parameter analysis module reconstructs the velocity field from the wave surface elevation field inverts a three-dimensional velocity vector field, wherein the vertical velocity is reconstructed from the calculation of the horizontal velocity components satisfies: Among them, vertical velocity The expression contains two key terms: one is the instantaneous rate of change of the wavefront. First, it reflects the rapid undulation behavior of the wave at that point; second, it is the tensor time integral term in the direction of the horizontal gradient of the wave surface. This expression is used to describe the fluid response caused by non-uniform propagation; the entire expression is... Weighting was applied to account for the correction of the vertical velocity by the wavefront slope; the horizontal velocity field... The calculation depends on The time integral of the tensor is then subtracted from the projection correction term of the vertical velocity onto the wavefront gradient. This reflects the local flow characteristics of water particles near the water surface; It is defined as a combination of gravitational potential energy and kinetic energy, with the first term being gravitational potential energy and the second term being the kinetic energy correction considering the wave surface velocity. Further, the time series wave front data obtained from the reconstruction is calculated , , , the time series three-dimensional velocity field is constructed, the spatial gradient is estimated by central difference, the time derivative is solved by three-point backward difference, and the process is solved by time integration, and finally the complete three-dimensional velocity vector field of each frame is reconstructed .

Images