Sea surface wave field reconstruction method and device based on binocular stereo vision

Abstract

The invention provides a sea surface wave field reconstruction method and device based on binocular stereoscopic vision. The method comprises the following steps: acquiring a sea surface stereoscopic image of an observation area according to a preset frame rate based on a binocular image acquisition system; constructing an initial three-dimensional wave field by combining the sea surface three-dimensional image with a semi-global matching algorithm; filling the small-size cavity of the initial three-dimensional wave field in combination with morphological closed operation to obtain a preliminarily filled three-dimensional wave field; and reconstructing the large-size cavity of the preliminarily filled three-dimensional wave field in combination with a variational data assimilation physical model to obtain a complete sea surface wave field. Under a four-dimensional variation data assimilation framework, incomplete binocular vision observation data and a fluctuation physical model are fused, and the problem of large-scale data voids which are difficult to process by a traditional optical method is solved, so that a space-time continuous and physically consistent complete three-dimensional wave field is reconstructed.

Description

Technical Field

[0001] This invention relates to the field of sea surface wave fields, and more specifically to a method and apparatus for reconstructing sea surface wave fields based on binocular stereo vision. Background Technology

[0002] Existing wave observation technologies, such as single-point contact measurement methods like buoys and ADCP, cannot effectively reconstruct the spatial distribution information of wave fields. While satellite altimeters can observe over a wide area, they are limited by spatiotemporal resolution, cloud cover, trajectory path, and revisit period. Computer vision algorithms transform wavefront measurement into an image processing problem, reconstructing wavefront features through feature extraction and stereo matching.

[0003] Currently, wavefront reconstruction methods based on computer vision can be categorized into three types based on the number of cameras used: monocular, binocular, and multi-view vision. Monocular vision methods are limited in reconstruction accuracy due to the lack of accurate disparity information; while multi-view vision methods can provide richer visual information, they often face problems such as increased cumulative error from multiple cameras and decreased reconstruction efficiency. In contrast, binocular vision methods achieve a better balance between disparity information acquisition and reconstruction efficiency, and are therefore often considered a superior choice.

[0004] The main problems and challenges currently faced by binocular vision wavefront reconstruction methods include sparse features, weak texture, non-Lambertian reflections, illumination, rain, fog, and occlusion, which often result in numerous holes and noise in the reconstructed wavefield data, limiting subsequent accurate feature analysis. Existing techniques often employ simple interpolation or filtering methods to process this incomplete data, neglecting the inherent physical laws of wave motion and leading to distorted results. Summary of the Invention

[0005] Based on this, the present invention provides a method and apparatus for reconstructing sea surface wave field based on binocular stereo vision. By fusing incomplete binocular visual observation data with wave physics models within a four-dimensional variational data assimilation framework, the invention solves the problem of large-scale data gaps that are difficult to handle by traditional optical methods, thereby reconstructing a complete three-dimensional wave field that is spatiotemporally continuous and physically consistent.

[0006] In a first aspect, the present invention provides a method for reconstructing sea surface wave fields based on binocular stereo vision, comprising:

[0007] Acquire stereo images of the sea surface in the observation area by a binocular image acquisition system at a preset frame rate;

[0008] The initial three-dimensional wave field of the sea surface stereo image is constructed using a semi-global matching algorithm;

[0009] By combining morphological closing operations to fill small-sized holes in the initial three-dimensional wave field, a preliminary filled three-dimensional wave field is obtained;

[0010] The large-sized void in the initially filled three-dimensional wave field is reconstructed using a physical model based on variational data assimilation, resulting in a complete sea surface wave field.

[0011] Furthermore, the three-dimensional sea surface image includes a first sea surface image captured by the left camera and a second sea surface image captured by the right camera;

[0012] The step of constructing an initial three-dimensional wavefield by combining the three-dimensional sea surface image with a semi-global matching algorithm specifically includes the following steps:

[0013] By combining a semi-global matching algorithm, each pixel in the first sea surface image and the second sea surface image is matched to obtain the disparity of each pixel;

[0014] For each acquisition time, a disparity map is constructed based on the disparity of all pixels;

[0015] Using the calibration parameters of the binocular image acquisition system and the principle of triangulation, the disparity map is converted into a sea surface height field to obtain the initial three-dimensional wave field.

[0016] Furthermore, the step of filling small-sized holes in the initial three-dimensional wave field using morphological closing operations to obtain a preliminarily filled three-dimensional wave field specifically includes the following steps:

[0017] The initial three-dimensional wave field is expanded to fill the small-scale voids in the initial three-dimensional wave field, resulting in an expanded three-dimensional wave field.

[0018] The repeated portions of the expanded three-dimensional wave field are removed by erosion, restoring the size of the peaks and troughs to their original state, thus obtaining a partially filled three-dimensional wave field.

[0019] Furthermore, the reconstruction of the initially filled large-sized voids in the three-dimensional wavefield using a physical model based on variational data assimilation to obtain the complete sea surface wavefield specifically involves:

[0020] A cost function is constructed by combining the initial state of the sea surface wavefield and the initially filled three-dimensional wavefield;

[0021] The optimal initial state of the sea surface wave field is obtained based on the cost function and optimization algorithm.

[0022] Substitute the optimal initial state into the preset variational data assimilation physical model and perform positive integration to obtain the complete sea surface wave field.

[0023] The cost function for constructing the three-dimensional wavefield by combining the initial state of the sea surface wavefield and the preliminary filling is specifically expressed as follows:

[0024] ,

[0025] in, For the sea surface wave field in The initial state at time t, including the sea surface height field at the initial time. and initial velocity potential field ; For the background field, that is, for Prior estimates; Let be the error covariance matrix of the background field; for The initial filling of the three-dimensional wave field at time It is a fragmented wave field containing large-sized voids; For physical model operators, it represents the state from the initial state. Starting from this point, the model state at time t is obtained by integrating the physical equations; An observation operator, used to capture the complete model state. Mapped to the observation space; Let be the observation error covariance matrix, representing the variance of the observed data. The level of trust.

[0026] Furthermore, the sea surface wave field reconstruction method based on binocular stereo vision also includes:

[0027] The two-dimensional wave velocity field of the sea surface is obtained by inverting the complete sea surface wave field.

[0028] By combining the complete sea surface wave field, the mechanical characteristic field related to surface tension is obtained by calculating the surface curvature;

[0029] By combining the two-dimensional wave velocity field and the mechanical characteristic field related to surface tension, the dynamic parameters of the sea surface are obtained.

[0030] Furthermore, the sea surface wave field reconstruction method based on binocular stereo vision also includes:

[0031] Based on the complete sea surface wave field, the water depth parameters of the observation area are obtained by inversion using three-dimensional Fourier transform and dispersion relation model.

[0032] Secondly, the present invention also provides a sea surface wave field reconstruction device based on binocular stereo vision, comprising:

[0033] The image acquisition module is used to acquire stereo images of the sea surface in the observation area acquired by the binocular image acquisition system at a preset frame rate;

[0034] The initial wavefield construction module is used to construct the initial three-dimensional wavefield of the sea surface stereo image using a semi-global matching algorithm.

[0035] The small-size void filling module is used to fill small-size voids in the initial three-dimensional wave field by combining morphological closing operations, so as to obtain a preliminary filled three-dimensional wave field.

[0036] The large-size void reconstruction module is used to reconstruct the large-size voids in the initially filled three-dimensional wave field through a physical model assimilated by variational data, thereby obtaining the complete sea surface wave field.

[0037] Thirdly, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the sea surface wave field reconstruction methods based on binocular stereo vision in the first aspect.

[0038] Fourthly, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform any one of the sea surface wave field reconstruction methods based on binocular stereo vision in the first aspect.

[0039] The beneficial effects of adopting the above technical solution are as follows: This embodiment continuously acquires sea surface data through a synchronously calibrated binocular camera system, thereby obtaining high-frame-rate temporal stereo image pairs. Subsequently, a semi-global block matching algorithm is used to process these image pairs, calculating and generating an initial, but inevitably data-voiding, instantaneous three-dimensional wave field. To address these data gaps, morphological closing operations are first applied to quickly and effectively fill small-scale gaps. Then, for large-area missing regions, a novel physical model-driven reconstruction method based on variational data assimilation is introduced, fusing the incomplete observation data with the wave equation model to solve and generate a spatiotemporally complete, continuous, and physically consistent final wave field data. This invention not only solves the problem of data gaps in traditional optical measurement methods but also ensures the authenticity and accuracy of the reconstruction results by introducing physical model constraints, ultimately providing a systematic solution capable of generating high-precision, full-parameter (waveform, wave velocity, curvature, water depth) ocean dynamics information from original images. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0041] Figure 1 This is a schematic diagram of a sea surface wave field reconstruction method based on binocular stereo vision in one embodiment of this application;

[0042] Figure 2 This is a schematic diagram showing the position and orientation of a binocular image acquisition system in one embodiment of this application;

[0043] Figure 3 This is a disparity map in one embodiment of this application;

[0044] Figure 4This is a schematic diagram of a 3D point cloud for disparity map conversion in one embodiment of this application;

[0045] Figure 5 This is a schematic diagram of the sea surface height field on a regular grid for disparity map transformation in one embodiment of this application;

[0046] Figure 6 This is a schematic diagram of the complete optical texture field during the PIV method inversion of the sea surface wave velocity field in one embodiment of this application;

[0047] Figure 7 This is a schematic diagram of the difference grayscale and envelope contour lines at different times during the PIV method inversion of the sea surface wave velocity field in one embodiment of this application;

[0048] Figure 8 This is a schematic diagram of the velocity magnitude derived during the inversion of the sea surface wave velocity field using the PIV method in one embodiment of this application.

[0049] Figure 9 This is a schematic diagram of the three-dimensional sea surface shape during the inversion of the sea surface wave velocity field using the PIV method in one embodiment of this application;

[0050] Figure 10 This is a schematic diagram of a sea surface wave field reconstruction device based on binocular stereo vision in one embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention. To illustrate the present invention in more detail, the sea surface wave field reconstruction method and apparatus based on binocular stereo vision provided by the present invention will be specifically described below with reference to the accompanying drawings.

[0052] Unless otherwise defined, the technical or scientific terms used in this application shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Similarly, the terms "an," "a," or "the" do not indicate a quantity limitation, but rather indicate the presence of at least one. The terms "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. The terms "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up," "down," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0053] Existing binocular vision wavefront reconstruction methods mainly face problems such as sparse features, weak texture, non-Lambertian reflections, illumination, rain, fog, and occlusion, which often result in reconstructed wavefield data containing numerous holes and noise, limiting subsequent accurate feature analysis. However, current techniques often use simple interpolation or filtering to process this incomplete data, ignoring the inherent physical laws of wave motion and leading to distorted results.

[0054] Based on this, the present invention proposes a method and apparatus for reconstructing sea surface wave field based on binocular stereo vision. By fusing incomplete binocular visual observation data with wave physics models within the framework of four-dimensional variational data assimilation (4D-Var), the large-scale data gap problem that is difficult to handle by traditional optical methods is solved, thereby reconstructing a complete three-dimensional wave field that is spatiotemporally continuous and physically consistent.

[0055] This invention provides an application scenario for a sea surface wavefield reconstruction method based on binocular stereo vision. This application scenario includes the terminal device provided in the embodiment, which includes, but is not limited to, smartphones and computer devices. The computer device can be at least one of desktop computers, portable computers, laptop computers, mainframe computers, tablet computers, etc. The terminal device processes the sea surface image obtained by the binocular image acquisition system to obtain a complete three-dimensional wavefield that is spatiotemporally continuous and physically consistent. (See attached...) Figure 1 The diagram shows a sea surface wave field reconstruction method based on binocular stereo vision. For details, please refer to the embodiment of the sea surface wave field reconstruction method based on binocular stereo vision.

[0056] Step S100: Acquire stereo images of the sea surface in the observation area at a preset frame rate using a binocular image acquisition system.

[0057] Specifically, in this embodiment, a binocular image acquisition system is used to acquire stereoscopic images of the sea surface in the observation area for subsequent processing; as shown in the attached figure. Figure 2 As shown, the binocular image acquisition system in this embodiment consists of two high-performance industrial cameras (as shown in the attached diagram). Figure 2 The left camera (marked with a red dot) and the right camera (marked with a green dot) are fixed on a rigid base to ensure their relative position and attitude remain constant, and acquire images of various target points on the sea surface wavefield. The cameras in this binocular image acquisition system are rigorously calibrated through intrinsic parameter calibration (such as obtaining focal length, principal point, and distortion coefficients) and extrinsic parameter calibration (obtaining the rotation and translation relationships between the cameras). After calibration, the binocular image acquisition system continuously acquires stereoscopic images of the sea surface in the observation area at a preset frame rate using a synchronous triggering method. The stereoscopic images of the sea surface are a series of time-synchronized left and right view image pairs, which can be denoted as... ,in It refers to the left camera. The first image of the sea surface taken at any time. It refers to the right camera. The second image of the sea surface taken at that moment.

[0058] Step S200: Construct the initial three-dimensional wave field of the sea surface stereo image using a semi-global matching algorithm.

[0059] Specifically, step S200 aims to calculate the initial but incomplete three-dimensional wave field from the acquired three-dimensional sea surface image pairs. Its core is stereo matching, which means finding the corresponding point in the right image for each pixel in the left image.

[0060] In one specific implementation, the method described in this embodiment for constructing an initial three-dimensional wavefield by combining the three-dimensional sea surface image with a semi-global matching algorithm includes the following steps:

[0061] Step S201: The three-dimensional sea surface image includes a first sea surface image captured by the left camera and a second sea surface image captured by the right camera.

[0062] Step S202: The first sea surface image and the second sea surface image are matched with each pixel point by a semi-global matching algorithm to obtain the disparity of each matched pixel point.

[0063] Specifically, the goal of the semi-global block matching algorithm (SGBM) is to match each pixel... Find an optimal parallax This minimizes the matching cost function. For pixels in the first sea surface image... Parallax to be matched Its corresponding point in the second sea surface image is The matching cost between them is calculated using a cost function. .

[0064] Commonly used cost functions include SAD (Sum of Absolute Differences), BT (Birchfield-Tomasi), or Census transform. Among them, Census transform is not sensitive to changes in illumination and performs well in sea surface observations.

[0065] For pixels and parallax Its aggregation cost on path r Calculated using the following recursive formula:

[0066] in, It is a pixel At parallax The initial matching cost at that time, It is a path superior The previous pixel. The first penalty term is applied when the disparity change between adjacent pixels is 1. It allows the disparity to change smoothly along the inclined surface. The second penalty term is applied when the disparity change between adjacent pixels is greater than 1, and is used to handle discontinuous occlusion boundaries. The value is usually related to the gradient of the image. . It is the minimum aggregate cost of the previous pixel under all parallaxes, used for normalization so that the cost does not grow indefinitely.

[0067] The pixel is obtained by summing the aggregation costs of all paths r. In parallax Total aggregation cost The specific expression is .

[0068] Step S203: Construct a disparity map for each acquisition time based on the disparity of all pixels.

[0069] Specifically, a "winner-takes-all" (WTA) strategy is adopted, selecting the disparity value for each pixel that minimizes its total aggregation cost. The specific expression is: .

[0070] By following the steps above, the disparity map for each time step is obtained. As attached Figure 3As shown, due to factors such as weak textures, reflections, and occlusion, some pixels cannot find reliable matches, resulting in invalid values, or holes, in the disparity map.

[0071] Step S204: Using the calibration parameters of the binocular image acquisition system and the triangulation principle, the disparity map is converted into a sea surface height field to obtain an initial three-dimensional wave field.

[0072] Specifically, using camera calibration parameters and triangulation principles, the disparity map is... Convert to 3D point cloud (as attached) Figure 4 (as shown) or sea surface height field on a regular grid (as attached) Figure 5 As shown in the figure, the initial three-dimensional wave field still contains a large number of voids at this time.

[0073] Step S300: Combine morphological closing operation to fill the small-sized voids in the initial three-dimensional wave field to obtain a preliminarily filled three-dimensional wave field.

[0074] Specifically, for the small, isolated holes generated by the SGBM algorithm, we use morphological closing operations from image processing to fill them, which includes the following steps:

[0075] The initial three-dimensional wave field is expanded to fill the small-scale voids in the initial three-dimensional wave field, resulting in an expanded three-dimensional wave field.

[0076] The repeated portions of the expanded three-dimensional wave field are removed by erosion, restoring the size of the peaks and troughs to their original state, thus obtaining a partially filled three-dimensional wave field.

[0077] Specifically, in this embodiment, the morphological closing operation consists of two basic operations: dilation followed by erosion. This can fill small holes inside the object while basically maintaining the original outline of the object.

[0078] Assuming our wavefield data It is a two-dimensional grayscale image, i.e., a height field; It is a structuring element, that is, a small-sized kernel, such as a 3x3 or 5x5 matrix;

[0079] For the expansion operation: expansion operation Will The value of each pixel in the array is replaced with its neighborhood (by...). The maximum value within the defined range is expressed as follows: The expansion operation causes areas with higher elevation values ​​to expand outward, thereby filling in adjacent small cavities with lower elevation values.

[0080] For corrosion operations: corrosion operations Then, the value of each pixel is replaced with the minimum value in its neighborhood, as shown in the following expression: Corrosion operations can cause areas with higher elevation values ​​to shrink.

[0081] For morphological closing operations: closing operation Defined as to First, perform dilation, then erode the result. The specific expression is as follows: .

[0082] In this embodiment, small-scale voids in the initial three-dimensional wavefield are filled using morphological closing operations. First, dilation operations are used to effectively fill the small-scale voids and cracks in the wavefield. Then, erosion operations are used to "erode" the repetitive portions introduced by dilation, restoring the dimensions of key features such as crests and troughs to near their original state. After the closing operation, we obtain a wavefield where most of the small voids are reasonably filled. .

[0083] Step S400: The large-sized voids in the initially filled three-dimensional wave field are reconstructed using a physical model based on variational data assimilation to obtain a complete sea surface wave field.

[0084] For large-scale voids caused by large-area reflections or occlusions, simple interpolation is unreliable. In a specific embodiment of this invention, a physical model-driven reconstruction method based on four-dimensional variational data assimilation is further proposed to address this problem. The core idea of ​​this method is to find an optimal initial wavefield state based on the observed specific wavefield data, such that the time-series wavefield evolving from this initial wavefield state under the drive of the physical model (wave equation) best matches the incomplete wavefield we observe. Specifically, it includes the following steps:

[0085] First, before executing the specific steps, a pre-defined physical model for variational data assimilation is constructed based on a linear or weakly nonlinear water wave model using potential flow theory. The essence of variational data assimilation is to integrate scattered observation data into the physical model and solve for the initial field or parameters of the model that best fits the real situation. This physical model achieves a good balance between computational efficiency and accuracy.

[0086] In one specific implementation, the physical model constructed in this embodiment consists of the following set of equations:

[0087] (1) Laplace's equation: In the fluid domain, the velocity potential satisfy .

[0088] (2) Seabed boundary conditions: at a water depth of On the flat seabed, At .

[0089] (3) Kinematic boundary conditions for free surfaces: At .

[0090] (4) Dynamic boundary conditions of free surfaces (Bernoulli equation): At .

[0091] in This is the acceleration due to gravity. For a linear model, the above boundary conditions can be linearized at z=0.

[0092] Step S401: Construct a cost function by combining the initial state of the sea surface wavefield and the initially filled three-dimensional wavefield.

[0093] Specifically, the cost function constructed by combining the initial state of the sea surface wavefield and the initially filled three-dimensional wavefield is expressed as follows:

[0094] ,

[0095] in, For the sea surface wave field in The initial state at time t, including the sea surface height field at the initial time. and initial velocity potential field ; For the background field, that is, for A prior estimate of , if there is no prior estimate, can be set as zero field; Let be the error covariance matrix of the background field; for The initial filling of the three-dimensional wave field at time It is a fragmented wave field containing large-sized voids; For physical model operators, it represents the state from the initial state. Starting from this point, the model state at time t is obtained by integrating the physical equations; An observation operator, used to capture the complete model state. Mapped to the observation space, in this embodiment, Its functions include extracting the sea surface height field from the model state. The model grid points are interpolated to the observation grid points, and a "mask" is included to calculate the difference only at the locations where there is observation data, ignoring the hole regions; Let be the observation error covariance matrix, representing the variance of the observed data. The level of trust.

[0096] Step S402: Obtain the optimal initial state of the sea surface wave field according to the cost function and optimization algorithm.

[0097] Specifically, the expression for the optimal initial state is: ,in This is the optimal initial state.

[0098] In this embodiment, the optimization algorithm employs a gradient-based optimization algorithm, such as L-BFGS (a BFGS algorithm with limited memory). Therefore, it is necessary to efficiently compute the cost function. For control variables gradient This is achieved here using the adjoint model method. By solving the adjoint equation of the physical model (integrating backward from the end of the time window), complete gradient information can be obtained with only one forward integration and one backward integration computation.

[0099] Step S403: Substitute the optimal initial state into the preset variational data assimilation physical model and perform positive integration to obtain the complete sea surface wave field.

[0100] Find the optimal initial state using an optimization algorithm. Substituting this into the physical model and performing a positive integration, we can generate a model that spans the entire spacetime domain. A complete, continuous wave field that conforms to physical laws Thus, the wave field fills all the large-scale holes and performs physical-constrained smoothing and correction on the original data.

[0101] Furthermore, in constructing a complete temporal wave field Afterwards, in-depth ocean dynamic parameters can be inverted, as follows:

[0102] Step S501: Obtain the two-dimensional wave velocity field of the sea surface by inversion based on the complete sea surface wave field.

[0103] Specifically, Particle Image Velocimetry (PIV) is used to invert the two-dimensional sea surface wave velocity field. .

[0104] PIV combines two consecutive wavefield images. and This is viewed as a flow field image filled with "tracer particles" (i.e., brightness or height patterns on the wavefront). The first frame image... Divided into several small query windows, as shown in the attached image. Figure 6 As shown. For each query window, in the second frame image. Template matching is performed within a slightly larger search area to find the region most similar to the query window, as shown in the attached figure. Figure 7As shown. Similarity measures typically use a cross-correlation function: ,in This is the query window in the first frame. It is the window at the corresponding position in the second frame. It is a displacement vector. The displacement at which the cross-correlation function C reaches its peak. This represents the average displacement of the ripple pattern within the window. Therefore, the average velocity in this region is... Repeating this process for all query windows yields a two-dimensional velocity vector field covering the entire field of view. By processing consecutive frame pairs in the time-series wave field, a time-varying wave velocity field can be obtained. As attached Figure 8 As shown.

[0105] Step S502: Combine the complete sea surface wave field with the calculation of surface curvature to obtain the mechanical characteristic field related to surface tension.

[0106] Specifically, directly measuring the surface tension field is extremely difficult for sea surface wave fields. However, the surface tension effect in sea surface wave fields is mainly related to sea surface curvature and can be described by the Young-Laplace equation. Therefore, regions where surface tension plays a dominant role, such as the peaks of capillary waves, can be characterized by calculating the sea surface curvature field.

[0107] Treat the sea surface as a curved surface The main curvature measures are mean curvature and Gaussian curvature. First, the finite difference method is used in a complete wavefield grid. The calculation of the first and second partial derivatives includes:

[0108] First-order partial derivative:

[0109] ,

[0110] ,

[0111] Second-order partial derivative:

[0112] ,

[0113] ,

[0114] .

[0115] Then, the mean curvature and Gaussian curvature are calculated using differential geometry formulas; where the mean curvature... The expression is: Gaussian curvature The expression is: .

[0116] Through time-varying average curvature field and Gaussian curvature field The obtained curvature field provides a detailed description of the sea surface geometry, as shown in the attached figure. Figure 9 As shown, the high curvature region, especially the short-wavelength (high wavenumber) region, is where the surface tension effect is most significant.

[0117] Step S503: Combine the two-dimensional wave velocity field and the mechanical characteristic field related to surface tension of the sea surface to obtain the sea surface dynamic parameters.

[0118] Furthermore, this embodiment also achieves remote sensing inversion of the average water depth of the observation area by performing a three-dimensional Fourier transform on the complete time-series wavefield and fitting and aligning its energy spectrum with the theoretical dispersion relation. Specifically:

[0119] Step S600: Based on the complete sea surface wave field, the water depth parameters of the observation area are obtained by inversion using three-dimensional Fourier transform and dispersion relation model.

[0120] This step utilizes the dependence of water wave dispersion on water depth to achieve remote sensing detection of the average water depth in the observation area. Specifically, it employs a dispersion shell fitting method based on three-dimensional Fourier transform.

[0121] For the three-dimensional Fourier transform part: based on complete four-dimensional spatiotemporal wavefield data. Perform a three-dimensional Fourier transform, including spatial , Dimensions and Time Dimension, transforming it from the physical domain to the spectral domain, yields the complex spectrum. The specific expression for the complex spectrum is: ,in, For space wavenumber components, It is the angular frequency. Based on this, the energy spectral density of the sea surface wave field The specific expression is .

[0122] Furthermore, based on the dispersion relation theory, for finite water depths... The expression for the dispersion relation of a linear gravitational wave is as follows: ,in , This represents the magnitude of the wave number. Based on the above expression, for a given water depth... The energy of a wave in the spectral space In the middle, it will focus on a curve defined by this formula, in three-dimensional space The middle part is a surface of revolution, called a dispersive shell.

[0123] Based on this, the water depth of the sea surface wave field can be fitted, specifically to find an optimal water depth. This allows the theoretical dispersion shell to best penetrate the observed energy spectrum. The high-energy region specifically includes the following steps:

[0124] Step S601: Set a water depth range to be searched. .

[0125] Step S602, for each candidate water depth within this range Calculate its corresponding theoretical frequency: ,in Candidate water depth The corresponding theoretical frequency.

[0126] Step S603, construct the evaluation function For each The observed energy spectrum will be observed within a narrow band near the theoretical dispersion shell. Integrate points; The larger the corresponding integral value, the better the theoretical model matches the actual data. The evaluation function in this embodiment... The specific expression is: In actual calculations, the above evaluation function is... A small window nearby Inner integral.

[0127] Step S604, making the division The candidate water depth is reached at its maximum value. That is, the estimated optimal water depth Optimal water depth The specific expression is: .

[0128] It should be noted that the method for determining the optimal water depth in this embodiment is based on the assumption that the water depth within the observation area is essentially constant. If the water depth within the observation area varies significantly, the field of view can be divided into multiple sub-regions for separate analysis.

[0129] It should be understood that, although attached Figure 1 The steps in the flowchart are shown sequentially according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders. Furthermore, [the following is a list of steps]. Figure 1At least some of the steps in the process may include multiple sub-steps or sub-stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0130] The above-described embodiments of the present invention detail a method for reconstructing sea surface wave fields based on binocular stereo vision. Since this method can be implemented using various types of devices, the present invention also discloses a device for reconstructing sea surface wave fields based on binocular stereo vision, in conjunction with the appendix. Figure 10 The schematic diagram of the sea surface wave field reconstruction device based on binocular stereo vision is shown below. Specific embodiments are given in detail below.

[0131] Image acquisition module 701 is used to acquire stereo images of the sea surface in the observation area acquired by the binocular image acquisition system at a preset frame rate;

[0132] The initial wavefield construction module 702 is used to construct the initial three-dimensional wavefield of the sea surface stereo image using a semi-global matching algorithm;

[0133] The small-size void filling module 703 is used to fill small-size voids in the initial three-dimensional wave field by combining morphological closing operations, so as to obtain a preliminarily filled three-dimensional wave field.

[0134] The large-size void reconstruction module 704 is used to reconstruct the large-size voids in the initially filled three-dimensional wave field through a physical model assimilated by variational data, thereby obtaining the complete sea surface wave field.

[0135] For details regarding the binocular stereo vision-based sea surface wave field reconstruction device, please refer to the above description of the method's limitations; they will not be repeated here. Each module in the aforementioned device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the terminal device in hardware form or independently of it, or stored in the memory of the terminal device in software form, so that the processor can call and execute the corresponding operations of each module.

[0136] In one embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for reconstructing the sea surface wave field based on binocular stereo vision.

[0137] The computer-readable storage medium may be an electronic storage device such as flash memory, EEPROM (Electrically Erasable Programmable Read-Only Memory), EPROM (Erasable Programmable Read-Only Memory), hard disk, or ROM. Optionally, the computer-readable storage medium includes a non-transitory computer-readable storage medium. The computer-readable storage medium has storage space for program code that performs any of the method steps described above. This program code can be read from or written to one or more computer program products, and the program code may be compressed in an appropriate form.

[0138] In one embodiment, the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above-described method for reconstructing sea surface wave fields based on binocular stereo vision.

[0139] The computer device includes a memory, a processor, and one or more computer programs, wherein the one or more computer programs can be stored in the memory and configured to be executed by one or more processors, and the one or more application programs are configured to perform the above-described binocular stereo vision-based sea surface wave field reconstruction method.

[0140] A processor may include one or more processing cores. The processor connects to various parts of the computer device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory, and by calling data stored in memory. Optionally, the processor may be implemented using at least one hardware form of Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), or Programmable Logic Array (PLA). The processor may integrate one or a combination of several of the following: Central Processing Unit (CPU), Graphics Processing Unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also be implemented separately as a communication chip, without being integrated into the processor.

[0141] The memory may include random access memory (RAM) or read-only memory (ROM). The memory can be used to store instructions, programs, code, code sets, or instruction sets. The memory may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as touch functionality, sound playback functionality, image playback functionality, etc.), and instructions for implementing the various method embodiments described above. The data storage area may also store data created by the terminal device during use.

[0142] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reconstructing sea surface wave field based on binocular stereo vision, characterized in that, include: Acquire stereo images of the sea surface in the observation area by a binocular image acquisition system at a preset frame rate; The initial three-dimensional wave field of the sea surface stereo image is constructed using a semi-global matching algorithm; By combining morphological closing operations to fill small-sized holes in the initial three-dimensional wave field, a preliminary filled three-dimensional wave field is obtained; The large-sized void in the initially filled three-dimensional wave field is reconstructed using a physical model based on variational data assimilation, resulting in a complete sea surface wave field.

2. The sea surface wave field reconstruction method based on binocular stereo vision as described in claim 1, characterized in that, The three-dimensional image of the sea surface includes a first sea surface image taken by the left camera and a second sea surface image taken by the right camera; The process of constructing the initial three-dimensional wavefield of the sea surface stereo image using a semi-global matching algorithm specifically includes the following steps: A semi-global matching algorithm is used to match each pixel in the first and second sea surface images, and the disparity of each matched pixel is obtained. Based on the disparity of all pixels, construct a disparity map for each acquisition time. Using the calibration parameters and triangulation principle of the binocular image acquisition system, the disparity map is converted into a sea surface height field to obtain an initial three-dimensional wave field.

3. The sea surface wave field reconstruction method based on binocular stereo vision as described in claim 2, characterized in that, The process of filling small-sized holes in the initial three-dimensional wave field using morphological closing operations to obtain a preliminarily filled three-dimensional wave field specifically includes the following steps: The initial three-dimensional wave field is expanded to fill the small-scale voids in the initial three-dimensional wave field, resulting in an expanded three-dimensional wave field. The repeated portions of the expanded three-dimensional wave field are removed by erosion, restoring the size of the peaks and troughs to their original state, thus obtaining a partially filled three-dimensional wave field.

4. The sea surface wave field reconstruction method based on binocular stereo vision as described in claim 3, characterized in that, The large-sized void in the initially filled three-dimensional wavefield is reconstructed using a physical model based on variational data assimilation to obtain the complete sea surface wavefield, specifically as follows: A cost function is constructed by combining the initial state of the sea surface wavefield and the initially filled three-dimensional wavefield; The optimal initial state of the sea surface wave field is obtained based on the cost function and optimization algorithm. Substitute the optimal initial state into the preset variational data assimilation physical model and perform positive integration to obtain the complete sea surface wave field.

5. The sea surface wave field reconstruction method based on binocular stereo vision as described in claim 4, characterized in that, The cost function, which combines the initial state of the sea surface wavefield and the initially filled three-dimensional wavefield, is specifically expressed as follows: , in, For the sea surface wave field in The initial state at time t, including the sea surface height field at the initial time. and initial velocity potential field ; For the background field, that is, for Prior estimates; Let be the error covariance matrix of the background field; for The initial filling of the three-dimensional wave field at time It is a fragmented wave field containing large-sized voids; For physical model operators, it represents the state from the initial state. Starting from this point, the model state at time t is obtained by integrating the physical equations; An observation operator, used to capture the complete model state. Mapped to the observation space; Let be the observation error covariance matrix, representing the variance of the observed data. The level of trust.

6. The sea surface wave field reconstruction method based on binocular stereo vision as described in any one of claims 1-5, characterized in that, Also includes: The two-dimensional wave velocity field of the sea surface is obtained by inverting the complete sea surface wave field. By combining the complete sea surface wave field, the mechanical characteristic field related to surface tension is obtained by calculating the surface curvature; By combining the two-dimensional wave velocity field and the mechanical characteristic field related to surface tension, the dynamic parameters of the sea surface are obtained.

7. The sea surface wave field reconstruction method based on binocular stereo vision as described in any one of claims 1-5, characterized in that, Also includes: Based on the complete sea surface wave field, the water depth parameters of the observation area are obtained by inversion using three-dimensional Fourier transform and dispersion relation model.

8. A sea surface wave field reconstruction device based on binocular stereo vision, characterized in that, include: The image acquisition module is used to acquire stereo images of the sea surface in the observation area acquired by the binocular image acquisition system at a preset frame rate; The initial wavefield construction module is used to construct the initial three-dimensional wavefield of the sea surface stereo image using a semi-global matching algorithm. The small-size void filling module is used to fill small-size voids in the initial three-dimensional wave field by combining morphological closing operations, so as to obtain a preliminary filled three-dimensional wave field. The large-size void reconstruction module is used to reconstruct the large-size voids in the initially filled three-dimensional wave field through a physical model assimilated by variational data, thereby obtaining the complete sea surface wave field.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the sea surface wave field reconstruction method based on binocular stereo vision as described in any one of claims 1-7.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it performs the sea surface wave field reconstruction method based on binocular stereo vision as described in any one of claims 1-7.

Images