Sea surface wave field reconstruction method and device based on binocular stereo vision

Abstract

The application provides a sea surface wave field reconstruction method and device based on binocular stereo vision, comprising: collecting sea surface stereo images of an observation area based on a binocular image acquisition system at a preset frame rate; constructing an initial three-dimensional wave field by combining the sea surface stereo images with a semi-global matching algorithm; filling small-size holes in the initial three-dimensional wave field by combining a morphological closing operation to obtain a preliminarily filled three-dimensional wave field; and reconstructing large-size holes in the preliminarily filled three-dimensional wave field by combining a physical model of variational data assimilation to obtain a complete sea surface wave field. In a four-dimensional variational data assimilation framework, incomplete binocular vision observation data and a wave physical model are fused to solve the problem of large-scale data holes that cannot be handled by traditional optical methods, so that a complete three-dimensional wave field that is continuous in time and space and physically consistent is reconstructed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of sea surface wave field, in particular to a sea surface wave field reconstruction method and device based on binocular stereo vision. BACKGROUND

[0002] The existing sea wave observation technology, such as the single-point contact measurement method of the buoy and the ADCP, cannot effectively reconstruct the spatial distribution information of the wave field; although the satellite altimeter can observe a large range, it is limited by the conditions of spatial resolution, cloud and fog obstruction, trajectory path and revisit period. The computer vision algorithm converts the wave surface measurement into an image processing problem, and inversely reconstructs the wave surface features through feature extraction and stereo matching.

[0003] At present, the wave surface reconstruction method based on computer vision can be divided into monocular, binocular and multi-view vision according to the number of cameras. The monocular vision method has limitations in reconstruction accuracy due to the lack of accurate disparity information; although the multi-view vision method can provide more abundant visual information, it often faces the problems of increasing cumulative error of multiple cameras and decreasing reconstruction efficiency. In comparison, the binocular vision method achieves a good balance between disparity information acquisition and reconstruction efficiency, and is therefore often considered as a better choice.

[0004] The main problems and challenges of the current binocular vision wave surface reconstruction method include sparse features, weak textures, non-Lambertian reflection, light, rain, fog and occlusion, which will cause a large number of holes and noises in the reconstructed wave field data, limiting the subsequent accurate feature analysis. In the processing of these incomplete data, the existing technology often uses simple interpolation or filtering methods, which ignores the internal physical law of wave motion in the processing, resulting in distorted results. SUMMARY

[0005] Therefore, the present application provides a sea surface wave field reconstruction method and device based on binocular stereo vision, which solves the problem of large-scale data holes that cannot be handled by traditional optical methods by fusing incomplete binocular vision observation data with wave physical models in a four-dimensional variational data assimilation framework, to reconstruct a complete three-dimensional wave field that is continuous in time and space and physically consistent.

[0006] In a first aspect, the present application provides a sea surface wave field reconstruction method based on binocular stereo vision, comprising:

[0007] acquiring a sea surface stereo image of an observation area collected by a binocular image acquisition system at a preset frame rate;

[0008] constructing an initial three-dimensional wave field of the sea surface stereo image through a semi-global matching algorithm;

[0009] filling small-size holes of the initial three-dimensional wave field in combination with morphological closing operation to obtain a preliminary filled three-dimensional wave field;

[0010] The large-size voids of the preliminary filled three-dimensional wave field are reconstructed by a variational data assimilation physical model to obtain a complete sea surface wave field.

[0011] Further, the sea surface stereogram includes a first sea surface image taken by a left camera and a second sea surface image taken by a right camera.

[0012] The sea surface stereogram is combined with a semi-global matching algorithm to construct an initial three-dimensional wave field, specifically including the following steps:

[0013] Each pixel point in the first sea surface image and the second sea surface image is matched by combining a semi-global matching algorithm to obtain a disparity of each pixel point.

[0014] For each acquisition time, a disparity map is constructed according to the disparity of all pixel points.

[0015] The disparity map is converted into a sea surface height field by using calibration parameters of a binocular image acquisition system and a triangulation principle to obtain an initial three-dimensional wave field.

[0016] Further, the small-size voids of the initial three-dimensional wave field are filled by combining a morphological closing operation to obtain a preliminary filled three-dimensional wave field, specifically including the following steps:

[0017] The initial three-dimensional wave field is dilated to fill the small-size voids of the initial three-dimensional wave field to obtain a dilated three-dimensional wave field.

[0018] The repeated parts of the dilated three-dimensional wave field are removed by erosion processing to restore the size of wave crests and troughs to the original state to obtain a preliminary filled three-dimensional wave field.

[0019] Further, the large-size voids of the preliminary filled three-dimensional wave field are reconstructed by a variational data assimilation physical model to obtain a complete sea surface wave field, specifically:

[0020] A cost function is constructed by combining an initial state of the sea surface wave field and the preliminary filled three-dimensional wave field.

[0021] An optimal initial state of the sea surface wave field is obtained according to the cost function and an optimization algorithm.

[0022] The optimal initial state is substituted into a preset variational data assimilation physical model and is forward integrated to obtain a complete sea surface wave field.

[0023] The cost function is constructed by combining the initial state of the sea surface wave field and the preliminary filled three-dimensional wave field, and the specific expression is:

[0024] ,

[0025] wherein, For the sea surface wave field in The initial state at time t, including the sea surface height field at the initial time. and initial velocity potential field ; For the background field, that is, for Prior estimates; Let be the error covariance matrix of the background field; for The initial filling of the three-dimensional wave field at time It is a fragmented wave field containing large-sized voids; For physical model operators, it represents the state from the initial state. Starting from this point, the model state at time t is obtained by integrating the physical equations; The observation operator is used to capture the complete model state. Mapped to the observation space; Let be the observation error covariance matrix, representing the variance of the observed data. The level of trust.

[0026] Furthermore, the sea surface wave field reconstruction method based on binocular stereo vision also includes:

[0027] The two-dimensional wave velocity field of the sea surface is obtained by inverting the complete sea surface wave field.

[0028] By combining the complete sea surface wave field, the mechanical characteristic field related to surface tension is obtained by calculating the surface curvature;

[0029] By combining the two-dimensional wave velocity field and the mechanical characteristic field related to surface tension, the dynamic parameters of the sea surface are obtained.

[0030] Furthermore, the sea surface wave field reconstruction method based on binocular stereo vision also includes:

[0031] Based on the complete sea surface wave field, the water depth parameters of the observation area are obtained by inversion using three-dimensional Fourier transform and dispersion relation model.

[0032] Secondly, the present invention also provides a sea surface wave field reconstruction device based on binocular stereo vision, comprising:

[0033] The image acquisition module is used to acquire stereo images of the sea surface in the observation area acquired by the binocular image acquisition system at a preset frame rate;

[0034] The initial wavefield construction module is used to construct the initial three-dimensional wavefield of the sea surface stereo image using a semi-global matching algorithm.

[0035] The small-size void filling module is used to fill small-size voids in the initial three-dimensional wave field by combining morphological closing operations, so as to obtain a preliminary filled three-dimensional wave field.

[0036] a large-size cavity reconstruction module configured to reconstruct large-size cavities of the preliminary filled three-dimensional wave field by a physical model of variational data assimilation to obtain a complete sea surface wave field.

[0037] In a third aspect, the present application also provides a computer readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the sea surface wave field reconstruction methods based on binocular stereo vision in the first aspect.

[0038] In a fourth aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform any of the sea surface wave field reconstruction methods based on binocular stereo vision in the first aspect.

[0039] The beneficial effects of the above technical solution are as follows: in this embodiment, a binocular camera system is calibrated synchronously to continuously collect sea surface and thus obtain high-frame-rate time-series stereo image pairs. Subsequently, a semi-global block matching algorithm is used to process these image pairs to calculate an initial instantaneous three-dimensional wave field which inevitably contains data cavities due to the optical characteristics of sea surface. In view of these data losses, morphological closing operation is first applied to quickly and effectively fill small-scale cavities. Then, for large-area missing regions, a physical model driven reconstruction method based on variational data assimilation is innovatively introduced to fuse the incomplete observation data with a wave equation model, solve and generate a final wave field data which is complete, continuous and physically consistent in time and space. The present application not only solves the problem of data cavities in traditional optical measurement methods, but also ensures the authenticity and accuracy of the reconstruction results by introducing physical model constraints, and finally provides a systematic solution that can generate high-precision, full-parameter (waveform, wave speed, curvature, water depth) ocean dynamics information from original images. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows.

[0041] Figure 1 a sea surface wave field reconstruction method based on binocular stereo vision in an embodiment of the present application;

[0042] Figure 2 a position and posture of a binocular image acquisition system in an embodiment of the present application;

[0043] Figure 3 a disparity map in an embodiment of the present application;

[0044] Figure 4A schematic diagram of a three-dimensional point cloud converted from a parallax map in one embodiment of the present application;

[0045] Figure 5 A schematic diagram of a sea surface height field on a regular grid converted from a parallax map in one embodiment of the present application;

[0046] Figure 6 A schematic diagram of a complete optical texture field in the process of inverting a sea surface wave velocity field by a PIV method in one embodiment of the present application;

[0047] Figure 7 A schematic diagram of a difference gray scale and envelope contour between front and back time in the process of inverting a sea surface wave velocity field by a PIV method in one embodiment of the present application;

[0048] Figure 8 A schematic diagram of a velocity size heat map derived in the process of inverting a sea surface wave velocity field by a PIV method in one embodiment of the present application;

[0049] Figure 9 A schematic diagram of a three-dimensional sea surface shape in the process of inverting a sea surface wave velocity field by a PIV method in one embodiment of the present application;

[0050] Figure 10 A schematic diagram of a sea surface wave field reconstruction device based on binocular stereo vision in one embodiment of the present application. DETAILED DESCRIPTION

[0051] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application. In order to describe the present application in more detail, the binocular stereo vision based sea surface wave field reconstruction method and device provided by the present application will be described in detail below with reference to the accompanying drawings.

[0052] Unless otherwise defined, technical terms or scientific terms used in the present disclosure shall have the ordinary meaning as understood by a person having ordinary skill in the art to which the present disclosure pertains. The terms "first", "second", and similar terms are used herein to distinguish one element from another, but do not necessarily indicate an order of importance, quantity, or sequence. Similarly, the terms "one", "a", or "the" are not limited to one instance but encompass one or more instances. The terms "including", "containing", and similar terms are inclusive, meaning that the elements or objects listed after the term encompass the elements or objects listed before the term, equivalents thereof, and additional elements or objects. The terms "connected", "coupled", and similar terms are not limited to physical or mechanical connections or couplings, but also include electrical connections or couplings, whether direct or indirect. The terms "upper", "lower", "left", "right", and similar terms are used only to represent relative positions, and can change accordingly when the absolute positions of the described objects change.

[0053] The main problems of the existing binocular vision wave surface reconstruction method include sparse features, weak textures, non-Lambertian reflection, light, rain and fog, and occlusion, which will make the reconstructed wave field data often have a large number of holes and noises, limiting the subsequent accurate feature analysis. However, the existing technology mainly uses simple interpolation or filtering to process these incomplete data, ignoring the internal physical law of wave motion, resulting in distorted results.

[0054] Based on this, the embodiment of the present application proposes a sea surface wave field reconstruction method and device based on binocular stereo vision, which fuses incomplete binocular vision observation data and wave physical model in a four-dimensional variational data assimilation (4D-Var) framework, solves the problem of large-scale data holes that traditional optical methods cannot handle, and reconstructs a complete three-dimensional wave field that is continuous in time and space and physically consistent.

[0055] The embodiment of the present application provides an application scenario of the sea surface wave field reconstruction method based on binocular stereo vision, which includes a terminal device provided by the embodiment. The terminal device includes but is not limited to a smart phone and a computer device. The computer device can be at least one of a desktop computer, a portable computer, a laptop computer, a mainframe computer, a tablet computer, etc. The terminal device obtains a sea surface image from a binocular image acquisition system, and after processing, obtains a complete three-dimensional wave field that is continuous in time and space and physically consistent. The sea surface wave field reconstruction method based on binocular stereo vision is shown in the accompanying drawings. Figure 1 The embodiment of the sea surface wave field reconstruction method based on binocular stereo vision is shown in the accompanying drawings, and the specific process is described in the embodiment of the sea surface wave field reconstruction method based on binocular stereo vision.

[0056] In step S100, the sea surface stereo image of the observation area is acquired by the binocular image acquisition system at a preset frame rate.

[0057] Specifically, in this embodiment, a binocular image acquisition system is used to acquire the sea surface stereo images of the observation area for subsequent processing; as shown in the accompanying drawings Figure 2 The binocular image acquisition system of this embodiment is composed of two high-performance industrial cameras (as shown in the accompanying drawings Figure 2 The left camera marked with a red dot and the right camera marked with a green dot are fixed on a rigid base to ensure that their relative positions and attitudes are constant, and each sea surface wave field target point is acquired. The cameras of the above-mentioned binocular image acquisition system are strictly calibrated through internal parameter calibration (such as obtaining focal length, principal point and distortion coefficient) and external parameter calibration (obtaining the rotation and translation relationship between the cameras). The calibrated binocular image acquisition system continuously acquires the sea surface stereo images of the observation area at a preset frame rate in a synchronous triggering manner. The sea surface stereo images are a series of time-synchronized left and right view image pairs, which can be denoted as wherein is the first sea surface image taken by the left camera at is the second sea surface image taken by the right camera at . is the second sea surface image taken by the right camera at

[0058] Step S200, constructing an initial three-dimensional wave field of the sea surface stereo images by a semi-global matching algorithm.

[0059] Specifically, step S200 aims to calculate an initial but incomplete three-dimensional wave field from the acquired sea surface stereo image pairs, and the core is stereo matching, that is, finding the corresponding point of each pixel point in the left image in the right image.

[0060] In one specific embodiment, the construction of the initial three-dimensional wave field by the sea surface stereo images and the semi-global matching algorithm according to the embodiment includes the following steps:

[0061] Step S201, the sea surface stereo images include the first sea surface image taken by the left camera and the second sea surface image taken by the right camera.

[0062] Step S202, matching each pixel point in the first sea surface image and the second sea surface image by a semi-global matching algorithm to obtain the disparity of each matched pixel point.

[0063] Specifically, the goal of the semi-global matching algorithm (Semi-Global Block Matching, SGBM for short) is to find a best disparity for each pixel so as to minimize the matching cost function. For a pixel in the first sea surface image and a matching disparity , the corresponding point in the second sea surface image is​ The matching cost between them is calculated using a cost function. .

[0064] Commonly used cost functions include SAD (Sum of Absolute Differences), BT (Birchfield-Tomasi), or Census transform. Among them, Census transform is not sensitive to changes in illumination and performs well in sea surface observations.

[0065] For pixels and parallax Its aggregation cost on path r Calculated using the following recursive formula:

[0066] in, It is a pixel At parallax The initial matching cost at that time, It is a path superior The previous pixel. The first penalty term is applied when the disparity change between adjacent pixels is 1. It allows the disparity to change smoothly along the inclined surface. The second penalty term is applied when the disparity change between adjacent pixels is greater than 1, and is used to handle discontinuous occlusion boundaries. The value is usually related to the gradient of the image. . It is the minimum aggregate cost of the previous pixel under all parallaxes, used for normalization so that the cost does not grow indefinitely.

[0067] The pixel is obtained by summing the aggregation costs of all paths r. In parallax Total aggregation cost The specific expression is .

[0068] Step S203: Construct a disparity map for each acquisition time based on the disparity of all pixels.

[0069] Specifically, a "winner-takes-all" (WTA) strategy is adopted, selecting the disparity value for each pixel that minimizes its total aggregation cost. The specific expression is: .

[0070] By following the steps above, the disparity map for each time step is obtained. As attached Figure 3As shown in the figure. Due to weak texture, reflection, occlusion and other reasons, some pixels cannot find reliable matches, forming invalid values, i.e. holes, in the disparity map.

[0071] Step S204, using the calibration parameters of the binocular image acquisition system and the triangulation principle, the sea surface height field is converted from the disparity map, and the initial three-dimensional wave field is obtained.

[0072] Specifically, using the camera calibration parameters and the triangulation principle, the disparity map is converted into a sea surface height field on a regular grid (as shown in the accompanying or three-dimensional point cloud (as shown in the accompanying Figure 4 At this time, the initial three-dimensional wave field still contains a large number of holes. Figure 5

[0073] Step S300, combining morphological closing operation to fill small size holes of the initial three-dimensional wave field, and obtaining the preliminary filled three-dimensional wave field.

[0074] Specifically, for the small and isolated holes generated by the SGBM algorithm, morphological closing operation in image processing is used for filling, which includes the following steps:

[0075] The initial three-dimensional wave field is dilated to fill the small scale holes of the initial three-dimensional wave field, and the dilated three-dimensional wave field is obtained.

[0076] The repeated part of the dilated three-dimensional wave field is removed through erosion processing, so that the size of the wave crest and trough is restored to the original state, and the preliminary filled three-dimensional wave field is obtained.

[0077] Specifically, the morphological closing operation in this embodiment is composed of two basic operations of dilation and erosion, which can fill small holes in the object while basically maintaining the original outline of the object.

[0078] Suppose our wave field data is a two-dimensional gray image, i.e. height field; is a structure element, i.e. a small size kernel, such as a 3x3 or 5x5 matrix;

[0079] For dilation operation: the dilation operation will replace the value of each pixel in with the maximum value in its neighborhood (defined by ), and the specific expression is: Through the dilation operation, the area with high elevation value will expand outward, thereby filling the adjacent small low value holes.

[0080] For erosion operation: the erosion operation ​​Then the value of each pixel is replaced by the minimum value in its neighborhood, which is expressed as: By the erosion operation, the area with higher elevation value is shrunk.

[0081] For the morphological closing operation: closing operation is defined as the dilation operation on Then the result is eroded, which is expressed as .

[0082] In this embodiment, the small size holes in the initial three-dimensional wave field are filled by the morphological closing operation. In this process, first, the small scale holes and cracks in the wave field are effectively filled by the dilation operation; then, the repeated parts introduced by the dilation are "eroded" by the erosion operation, so that the size of the main features such as wave peaks and wave troughs is restored to near the original state. After the closing operation, we get a wave field in which most of the small holes are reasonably filled .

[0083] Step S400, the large size holes in the preliminary filled three-dimensional wave field are reconstructed by combining the physical model of variational data assimilation, and a complete sea surface wave field is obtained.

[0084] For large-scale holes caused by large-area reflection or shielding, simple interpolation is unreliable. In one specific embodiment of the present application, a physical model driven reconstruction method based on four-dimensional variational data assimilation is further proposed to solve this problem. The core idea of this method is: based on the observed specific wave field data, find an optimal initial wave field state, so that the time series wave field evolved from the initial wave field state under the driving of the physical model (wave equation) is most consistent with the observed incomplete wave field, which includes the following steps:

[0085] First, before the specific steps are performed, a pre-set physical model of variational data assimilation is constructed based on the linear or weakly nonlinear water wave model of potential flow theory, wherein the essence of variational data assimilation is to fuse the scattered observation data into the physical model to solve the model initial field or parameter that best fits the real situation, and the physical model has achieved a good balance between calculation efficiency and accuracy.

[0086] In one specific embodiment, the physical model constructed in this embodiment is composed of the following equation sets:

[0087] (1) Laplace equation: in the fluid domain, the velocity potential satisfies .

[0088] (2) Seabed boundary condition: on the flat seabed with water depth , . .

[0089] (3) Kinematic boundary condition on free surface: at .

[0090] (4) Dynamic boundary condition on free surface (Bernoulli equation): at .

[0091] where g is the acceleration of gravity. For linear models, the above boundary conditions can be linearized at z = 0.

[0092] Step S401, a cost function is constructed in combination with an initial state of a sea surface wave field and a three-dimensional wave field preliminarily filled in.

[0093] Specifically, the cost function is constructed in combination with the initial state of the sea surface wave field and the three-dimensional wave field preliminarily filled in, and a specific expression is as follows:

[0094] ,

[0095] wherein, is an initial state of the sea surface wave field at a time t, including a sea surface height field and an initial velocity potential field at an initial time; is a background field, i.e., a priori estimation of , if there is no a priori estimation, it can be set as a zero field; is an error covariance matrix of the background field; is a three-dimensional wave field preliminarily filled in at the time t , which is a defective wave field containing a large-size cavity; is a physical model operator, indicating that a model state at the time t is obtained by integrating a physical equation from the initial state ; is an observation operator, used for mapping a complete model state to an observation space, in the embodiment, the role of includes extracting a sea surface height field from the model state, interpolating the model grid to the observation grid, and containing a "mask" to calculate the difference only at the position with observation data and ignore the cavity area; is an observation error covariance matrix, indicating a degree of confidence in the observation data . Step S402, an optimal initial state of the sea surface wave field is obtained according to the cost function and an optimization algorithm.

[0096] Step S402, an optimal initial state of the sea surface wave field is obtained according to the cost function and an optimization algorithm.

[0097] ​​Specifically, the expression for the optimal initial state is: ,in This is the optimal initial state.

[0098] In this embodiment, the optimization algorithm employs a gradient-based optimization algorithm, such as L-BFGS (a BFGS algorithm with limited memory). Therefore, it is necessary to efficiently compute the cost function. For control variables gradient This is achieved here using the adjoint model method. By solving the adjoint equation of the physical model (integrating backward from the end of the time window), complete gradient information can be obtained with only one forward integration and one backward integration computation.

[0099] Step S403: Substitute the optimal initial state into the preset variational data assimilation physical model and perform positive integration to obtain the complete sea surface wave field.

[0100] Find the optimal initial state using an optimization algorithm. Substituting this into the physical model and performing a positive integration, we can generate a model that spans the entire spacetime domain. A complete, continuous wave field that conforms to physical laws Thus, the wave field fills all the large-scale holes and performs physical-constrained smoothing and correction on the original data.

[0101] Furthermore, in constructing a complete temporal wave field Afterwards, in-depth ocean dynamic parameters can be inverted, as follows:

[0102] Step S501: Obtain the two-dimensional wave velocity field of the sea surface by inversion based on the complete sea surface wave field.

[0103] Specifically, Particle Image Velocimetry (PIV) is used to invert the two-dimensional sea surface wave velocity field. .

[0104] PIV combines two consecutive wavefield images. and This is viewed as a flow field image filled with "tracer particles" (i.e., brightness or height patterns on the wavefront). The first frame image... Divided into several small query windows, as shown in the attached image. Figure 6 As shown. For each query window, in the second frame image. Template matching is performed within a slightly larger search area to find the region most similar to the query window, as shown in the attached figure. Figure 7The similarity measure is usually the Cross-Correlation: where is the query window of the first frame, is the corresponding window of the second frame, is the displacement vector. The displacement at which the Cross-Correlation C reaches its peak is the average displacement of the ripple pattern within the window. Thus, the average velocity of the region is . Repeating this process for all query windows yields a two-dimensional velocity vector field covering the entire field of view. By processing consecutive frame pairs in the time series wave field, a time-varying wave speed field is obtained, as shown in FIG. 4B. Figure 8

[0105] At step S502, the surface tension related mechanical feature field is obtained by calculating the surface curvature in combination with the complete sea surface wave field.

[0106] Specifically, for the sea surface wave field, it is extremely difficult to directly measure the surface tension field. However, the surface tension effect on the sea surface wave field is mainly related to the sea surface curvature, which can be described by the Young-Laplace equation. Therefore, the sea surface curvature field can be calculated to represent the region where the surface tension plays a dominant role, such as the wave crest of capillary wave.

[0107] The sea surface is regarded as a curved surface , and the main curvature measures are the Mean Curvature and the Gaussian Curvature. First, the first-order and second-order partial derivatives are calculated on the complete wave field grid using the finite difference method, including:

[0108] First-order partial derivative:

[0109] ,

[0110] ,

[0111] Second-order partial derivative:

[0112] ,

[0113] ,

[0114] .

[0115] Then, the Mean Curvature and the Gaussian Curvature are calculated according to the differential geometry formula; the expression of the Mean Curvature is: ; and the expression of the Gaussian Curvature​ The expression of the formula is: .

[0116] By the average curvature field and the Gaussian curvature field , the curvature field obtained can finely describe the geometry of the sea surface, as shown in the accompanying drawings. Among them, the high curvature area, especially the short wave (high wave number) area, is the most significant place for surface tension effect. Figure 9

[0117] Step S503, the sea surface two-dimensional wave velocity field and the surface tension related mechanical characteristic field are combined to obtain the sea surface dynamics parameters.

[0118] Further, the embodiment also realizes remote sensing inversion of the average water depth of the observation area by performing three-dimensional Fourier transform on the complete time sequence wave field and aligning the energy spectrum with the theoretical dispersion relation, specifically:

[0119] Step S600, according to the complete sea surface wave field, the water depth parameters of the observation area are obtained by inversion combining three-dimensional Fourier transform and dispersion relation model.

[0120] This step utilizes the dependence of water wave dispersion relation on water depth to realize remote sensing detection of the average water depth of the observation area, and specifically adopts the dispersion shell fitting method based on three-dimensional Fourier transform:

[0121] For the part of three-dimensional Fourier transform: based on the complete wave field data in space-time four dimensions Perform three-dimensional Fourier transform, including space , dimension and time dimension, convert it from the physical domain to the spectral domain to obtain a complex spectrum . The specific expression of the complex spectrum is: , wherein is the spatial wave number component, is the angular frequency. Based on this, the specific expression of the energy spectrum density of the sea surface wave field is .

[0122] Further, based on the dispersion relation theory, for the linear gravity wave of the limited water depth , the expression of the dispersion relation is: , wherein , represents the magnitude of the wave number. Based on the above expression, for a given water depth , the energy of the wave will be concentrated on a curve defined by the formula in the spectral space , in three-dimensional space ​The middle part is a surface of revolution, called a dispersive shell.

[0123] Based on this, the water depth of the sea surface wave field can be fitted, specifically to find an optimal water depth. This allows the theoretical dispersion shell to best penetrate the observed energy spectrum. The high-energy region specifically includes the following steps:

[0124] Step S601: Set a water depth range to be searched. .

[0125] Step S602, for each candidate water depth within this range Calculate its corresponding theoretical frequency: ,in Candidate water depth The corresponding theoretical frequency.

[0126] Step S603, construct the evaluation function For each The observed energy spectrum will be observed within a narrow band near the theoretical dispersion shell. Integrate points; The larger the corresponding integral value, the better the theoretical model matches the actual data. The evaluation function in this embodiment... The specific expression is: In actual calculations, the above evaluation function is... A small window nearby Inner integral.

[0127] Step S604, making the division The candidate water depth is reached at its maximum value. That is, the estimated optimal water depth Optimal water depth The specific expression is: .

[0128] It should be noted that the method for determining the optimal water depth in this embodiment is based on the assumption that the water depth within the observation area is essentially constant. If the water depth within the observation area varies significantly, the field of view can be divided into multiple sub-regions for separate analysis.

[0129] It should be understood that, although attached Figure 1 The steps in the flowchart are shown sequentially according to the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders. Furthermore, [the flowchart is attached]. Figure 1At least one of the steps in the above method can comprise a plurality of sub-steps or sub-phases, which are not necessarily performed at the same time, but can be performed at different times, and the order of the execution of the sub-steps or sub-phases is not necessarily sequential, but can be performed alternately or alternately with at least one part of other steps or sub-steps or sub-phases of other steps.

[0130] The method for reconstructing a sea surface wave field based on binocular stereo vision is described in detail in the above embodiments of the present application, and the above method of the present application can be implemented in various forms of devices, and therefore the present application further discloses a device for reconstructing a sea surface wave field based on binocular stereo vision, which is described in detail below with reference to the accompanying drawings. Figure 10 The device for reconstructing a sea surface wave field based on binocular stereo vision is shown in the schematic diagram, and specific embodiments are given below for detailed description.

[0131] The image acquisition module 701 is configured to acquire a sea surface stereo image of an observation area collected by a binocular image acquisition system at a preset frame rate.

[0132] The initial wave field construction module 702 is configured to construct an initial three-dimensional wave field of the sea surface stereo image by using a semi-global matching algorithm.

[0133] The small-size cavity filling module 703 is configured to fill a small-size cavity of the initial three-dimensional wave field by using a morphological closing operation to obtain a preliminary filled three-dimensional wave field.

[0134] The large-size cavity reconstruction module 704 is configured to reconstruct a large-size cavity of the preliminary filled three-dimensional wave field by using a physical model of variational data assimilation to obtain a complete sea surface wave field.

[0135] The device for reconstructing a sea surface wave field based on binocular stereo vision can refer to the above description of the method, and will not be described here. Each module in the above device can be implemented by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor of the terminal device in hardware form, or stored in the memory of the terminal device in software form, so that the processor executes the operations corresponding to each module.

[0136] In one embodiment, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the above method for reconstructing a sea surface wave field based on binocular stereo vision.

[0137] The computer readable storage medium can be an electronic storage, such as a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), an EPROM (Erasable Programmable Read-Only Memory), a hard disk, or a ROM. Alternatively, the computer readable storage medium includes a non-transitory computer readable medium. The computer readable storage medium has a storage space for storing program codes for performing any of the method steps described above. The program codes can be read from or written to one or more computer program products, which can be compressed in a suitable form.

[0138] In one embodiment, the present application provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the above method for reconstructing sea surface wave field based on binocular stereo vision.

[0139] The computer device comprises a memory, a processor, and one or more computer programs, wherein the one or more computer programs can be stored in the memory and configured to be executed by the one or more processors, and the one or more application programs are configured to perform the above method for reconstructing sea surface wave field based on binocular stereo vision.

[0140] The processor can include one or more processing cores. The processor connects various parts within the entire computer device through various interfaces and lines, performs various functions of the computer device and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Alternatively, the processor can be implemented in at least one of a hardware form of a Digital Signal Processing (DSP), a Field-Programmable Gate Array (FPGA), and a Programmable Logic Array (PLA). The processor can integrate a combination of one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU) for reporting and verifying embedded data, and a modem. Among them, the CPU mainly processes operating systems, user interfaces, and application programs, etc.; the GPU is responsible for rendering and drawing display content; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but can be realized by a separate communication chip.

[0141] The memory can include a random access memory (RAM) and can also include a read-only memory (ROM). The memory can be used to store instructions, programs, codes, code sets, or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The data storage area can also store data created by the terminal device in use, etc.

[0142] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for reconstructing sea surface wave field based on binocular stereo vision, characterized in that, The method comprises the following steps: acquiring a sea surface stereo image of an observation area collected by a binocular image acquisition system at a preset frame rate, wherein the sea surface stereo image comprises a first sea surface image captured by a left camera and a second sea surface image captured by a right camera; constructing an initial three-dimensional wave field of the sea surface stereo image by using a semi-global matching algorithm, specifically as follows: matching each pixel point in the first sea surface image and the second sea surface image by using the semi-global matching algorithm, and obtaining a disparity of each matched pixel point; constructing a disparity map of each collection time according to the disparities of all pixel points; converting the disparity map into a sea surface height field by using a calibration parameter of the binocular image acquisition system and a triangulation principle, and obtaining the initial three-dimensional wave field; filling small-size holes in the initial three-dimensional wave field by using a morphological closing operation, and obtaining a preliminarily filled three-dimensional wave field; reconstructing large-size holes in the preliminarily filled three-dimensional wave field by using a physical model of variational data assimilation, and obtaining a complete sea surface wave field, specifically as follows: constructing a cost function according to an initial state of the sea surface wave field and the preliminarily filled three-dimensional wave field; obtaining an optimal initial state of the sea surface wave field according to the cost function and an optimization algorithm; substituting the optimal initial state into a preset physical model of variational data assimilation, and performing forward integration to obtain the complete sea surface wave field.

2. The binocular stereo vision based sea surface wave field reconstruction method of claim 1, wherein, The method for filling the small-size holes in the initial three-dimensional wave field by using the morphological closing operation, specifically comprises the following steps: performing inflation processing on the initial three-dimensional wave field to fill small-size holes in the initial three-dimensional wave field, and obtaining an inflated three-dimensional wave field; removing repeated parts of the inflated three-dimensional wave field by using erosion processing, so that the sizes of wave crests and troughs are restored to original states, and a preliminarily filled three-dimensional wave field is obtained.

3. The binocular stereo vision based sea surface wave field reconstruction method of claim 1, wherein, The cost function is constructed according to the initial state of the sea surface wave field and the preliminarily filled three-dimensional wave field, and a specific expression is as follows: , where, is the initial state of the sea surface wave field at time, including the initial sea surface height field and the initial velocity potential field ; is the background field, i.e. the a priori estimate of ; is the error covariance matrix of the background field; is the preliminary filled 3D wave field at time is the incomplete wave field containing large size holes; is the physical model operator, representing the derivation of the model state at time from the initial state t by integrating the physical equations; is the observation operator, used to map the complete model state to the observation space; is the observation error covariance matrix, representing the degree of trust in the observation data .

4. The binocular stereo vision based sea surface wave field reconstruction method according to any one of claims 1-3, characterized in that, The method further comprises the following steps: inverting a sea surface two-dimensional wave velocity field according to the complete sea surface wave field; obtaining a mechanical characteristic field related to surface tension by calculating surface curvature in combination with the complete sea surface wave field; obtaining sea surface dynamic parameters in combination with the sea surface two-dimensional wave velocity field and the mechanical characteristic field related to surface tension.

5. The binocular stereo vision based sea surface wave field reconstruction method according to any one of claims 1-3, characterized in that, The method further comprises the following steps: inverting a water depth parameter of the observation area in combination with a three-dimensional Fourier transform and a dispersion relation model according to the complete sea surface wave field.

6. A sea surface wave field reconstruction device based on binocular stereo vision, characterized in that, The method comprises the following steps: an image acquisition module is configured to acquire a sea surface stereo image of an observation area collected by a binocular image acquisition system at a preset frame rate, wherein the sea surface stereo image comprises a first sea surface image captured by a left camera and a second sea surface image captured by a right camera; an initial wave field construction module is configured to construct an initial three-dimensional wave field of the sea surface stereo image by using a semi-global matching algorithm, specifically as follows: matching each pixel point in the first sea surface image and the second sea surface image by using the semi-global matching algorithm, and obtaining a disparity of each matched pixel point; constructing a disparity map of each collection time according to the disparities of all pixel points; converting the disparity map into a sea surface height field by using a calibration parameter of the binocular image acquisition system and a triangulation principle, and obtaining the initial three-dimensional wave field; The small-size cavity filling module is configured to fill small-size cavities of an initial three-dimensional wave field by combining a morphological closing operation to obtain a preliminary filled three-dimensional wave field. The large-size cavity reconstruction module is configured to reconstruct large-size cavities of the preliminary filled three-dimensional wave field by a variational data assimilation physical model to obtain a complete sea surface wave field, specifically: constructing a cost function in combination with an initial state of the sea surface wave field and the preliminary filled three-dimensional wave field; obtaining an optimal initial state of the sea surface wave field according to the cost function and an optimization algorithm; substituting the optimal initial state into a preset variational data assimilation physical model and performing forward integration to obtain the complete sea surface wave field.

7. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the sea surface wave field reconstruction method based on binocular stereo vision in any one of claims 1-5.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the sea surface wave field reconstruction method based on binocular stereo vision in any one of claims 1-5.

Images