A mobile platform anti-swing sea wave measurement method and system based on binocular vision

Abstract

The application discloses a mobile platform anti-swing sea wave measuring method and system based on binocular vision, and the method comprises the following steps: acquiring a binocular image sequence, acquiring and verifying camera calibration parameters through physical calibration board imaging, carrying out de-distortion and stereo correction transformation on the binocular image sequence based on the calibration parameters, synchronously acquiring the attitude change information of a mobile platform at sea, constructing a dynamic coordinate mapping relationship which is dynamically updated with the collection time by using the attitude change information, generating a three-dimensional wave surface point cloud in a camera coordinate system, transforming the three-dimensional wave surface point cloud to a world coordinate system based on the dynamic coordinate mapping relationship to obtain an anti-swing stable wave surface point cloud and extract wave feature parameters, and triggering a closed-loop feedback mechanism for adaptive correction and adjustment based on point cloud quality evaluation.

Description

Technical Field

[0001] This invention relates to the field of wave monitoring technology in marine engineering, and more specifically, to a method and system for measuring anti-swaying ocean waves on a mobile platform based on binocular vision. Background Technology

[0002] Real-time monitoring of wave parameters (such as wave height and period) is a crucial foundation for ensuring the safety of offshore construction operations. Traditional wave measurement methods, such as contact measurements using wave buoys and pressure sensors, suffer from sparse measurement points and high deployment and maintenance costs. Non-contact measurements, such as radar remote sensing and satellite remote sensing, have limited spatial resolution and struggle to capture small-scale wave characteristics. Binocular stereo vision-based 3D wave field reconstruction technology is gaining increasing attention due to its advantages, including lower cost, flexible deployment, and the ability to acquire dense 3D point clouds.

[0003] Chinese invention patent application CN121788722A discloses a method and system for high-precision 3D reconstruction of wave fields in a net cage based on binocular stereo vision. This method involves calibrating a binocular vision camera, acquiring left and right image data of the wave surface in the area to be measured, obtaining the 3D spatial coordinates in the camera coordinate system through stereo matching and triangulation, then transforming the 3D spatial coordinates to the water surface coordinate system through coordinate transformation, and finally completing the 3D reconstruction of the wave field in the net cage through post-processing. This method can achieve high-precision 3D real-time reconstruction of wave fields under fixed installation conditions in a laboratory water tank.

[0004] However, in actual marine engineering monitoring, binocular stereo vision systems often need to be deployed on mobile offshore platforms such as construction vessels. Unlike fixed laboratory platforms, mobile offshore platforms experience continuous attitude changes such as roll, pitch, and bow under the influence of waves. Existing technical solutions rely on the premise that the camera remains relatively stationary with respect to the water surface, and their coordinate transformation relationship remains constant during the measurement process. When this solution is directly applied to a mobile offshore platform, the platform's swaying causes equivalent changes in the camera's extrinsic parameters, resulting in a systematic drift in the world coordinate point cloud obtained through fixed coordinate transformation. The camera's perspective change caused by the platform's swaying is incorrectly mapped to wave surface undulations, leading to severe distortion in wave height measurement results. Furthermore, existing technical solutions lack online quality assessment and feedback correction mechanisms for measurement results, making it difficult to address the degradation of measurement accuracy caused by factors such as structural micro-deformation and time synchronization drift during long-term operation at sea.

[0005] Therefore, there is an urgent need for a binocular stereo vision wave measurement method that can adapt to the swaying conditions of offshore mobile platforms, effectively eliminate the interference of platform swaying on three-dimensional wave measurement while ensuring measurement accuracy, and have the ability to evaluate and correct measurement quality online. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for measuring anti-swaying waves on a mobile platform based on binocular vision, so as to solve the above-mentioned problems existing in the prior art.

[0007] The application is as follows: This invention provides a method for measuring the anti-swaying wave motion of a mobile platform based on binocular vision, comprising the following steps: S1. Acquire binocular image sequence: Adjust the baseline distance, installation height, and tilt angle relative to the horizontal plane of the binocular stereo vision acquisition unit so that its overlapping field of view covers the target sea surface area; synchronously acquire video images of the target sea surface area through the binocular stereo vision acquisition unit to obtain left eye image sequence and right eye image sequence with aligned timestamps; S2. Obtain and verify camera calibration parameters through imaging with a physical calibration board: In the field of view of the binocular stereo vision acquisition unit, a physical plane calibration board of known size is presented in various spatial poses and the physical plane calibration board is photographed; feature point coordinates are extracted from the acquired image pairs of the physical plane calibration board, and the intrinsic parameter matrix and distortion coefficient of each in the binocular stereo vision acquisition unit, as well as the rotation matrix and translation vector between them, are calculated; the extracted feature point coordinates are reprojected onto the image plane, the reprojection error is calculated to verify the parameter accuracy, and the verified intrinsic parameter matrix, distortion coefficient, rotation matrix, and translation vector are saved as calibration parameters; S3. Image distortion correction and stereo correction transformation based on calibration parameters: Read the calibration parameters and perform distortion correction and stereo correction transformation on each frame of the left eye image sequence and the right eye image sequence. After the transformation, the projection points of the same spatial point in the left eye image and the right eye image satisfy the epipolar constraint, that is, they are located in the same pixel row. S4. Synchronous acquisition of attitude change information: Acquire the attitude change information of the offshore mobile platform in three-dimensional space that is precisely time-aligned with the left and right eye images of each frame at the acquisition time; S5. Construct a dynamic world coordinate mapping relationship using attitude change information: Construct a static coordinate transformation link from the platform coordinate system, through the camera coordinate system, to the world coordinate system; convert the attitude change information acquired at each frame acquisition time into a dynamic rotation matrix and dynamic translation vector of the platform coordinate system relative to the world coordinate system corresponding to each frame acquisition time, and superimpose the dynamic rotation matrix and dynamic translation vector into the static coordinate transformation link to construct a dynamic coordinate mapping relationship that is dynamically updated with the acquisition time and used to map points in the camera coordinate system to the world coordinate system; S6. Generation of 3D wavefront point cloud in camera coordinate system: For each frame of the left and right eye images processed by S3, disparity estimation is performed to generate a dense disparity map and the disparity confidence of each pixel in the dense disparity map; Based on the principle of triangulation, using the intrinsic focal length in the calibration parameters and the magnitude of the translation vector, the disparity value of each pixel in the dense disparity map is converted into the corresponding depth value to generate a 3D wavefront point cloud in camera coordinate system for each frame; S7. Generate anti-sway wavefront point cloud and extract wave parameters based on dynamic coordinate mapping relationship: call the dynamic coordinate mapping relationship to transform the three-dimensional wavefront point cloud in the camera coordinate system to the world coordinate system point by point to obtain the anti-sway stable wavefront point cloud in the world coordinate system; and extract wave feature parameters based on the anti-sway stable wavefront point cloud. S8. Closed-loop feedback correction based on point cloud quality evaluation: Online quality evaluation is performed on the anti-sway stable wavefront point cloud. When the evaluation result exceeds the preset quality threshold, a closed-loop feedback mechanism is triggered to adaptively correct and adjust the parameters in the disparity estimation process in S6, the time alignment relationship between the attitude change information and the image acquisition time in S4, and / or the parameters in the static coordinate transformation link in S5.

[0008] Furthermore, the attitude change information mentioned in S4 includes at least the roll angle, pitch angle, and bow angle of the offshore mobile platform; The time alignment specifically involves using a hardware synchronization signal line or software timestamp interpolation based on a unified time source to ensure that the acquisition time of each frame of the left and right eye images corresponds to a set of determined attitude change information.

[0009] Furthermore, step S5 specifically includes: Define a world coordinate system W fixed to the Earth, a platform coordinate system B fixed to the offshore mobile platform, and a camera coordinate system C fixed to the binocular stereo vision acquisition unit. The fixed installation pose of the camera coordinate system C relative to the platform coordinate system B is determined using a static rotation matrix R. bc and static translation vector t bc express; The attitude change information acquired at the t-th frame acquisition time is converted into a dynamic rotation matrix R of the platform coordinate system B relative to the world coordinate system W at that time. wb (t) and the dynamic translation vector t wb (t); Then the dynamic rotation matrix R of the camera coordinate system C relative to the world coordinate system W at the acquisition time of the t-th frame is... wc (t) and the dynamic translation vector t wc (t) is constructed as follows: R wc (t)=R wb (t)·R bc t wc (t)=R wb (t)·t bc +t wb (t) The dynamic coordinate mapping relationship is determined by the dynamic rotation matrix R. wc (t) and the dynamic translation vector t wc (t) characterization.

[0010] Furthermore, as described in S7, the anti-sway stable wavefront point cloud in the world coordinate system is obtained as follows: For any point P in the camera coordinate system C c Its corresponding point P in the world coordinate system W w (t) by P w (t)=R wc (t)·P c +t wc (t) is determined.

[0011] Furthermore, the extraction of wave feature parameters described in S7 specifically includes: For a continuous anti-rolling stable wave surface point cloud within a time window, a spatial reference plane representing the mean sea level is fitted; the normal distance from each point in the anti-rolling stable wave surface point cloud to the spatial reference plane is calculated, and the normal distance is used as the instantaneous wave surface undulation or wave height value; and based on the time series of the instantaneous wave surface undulation or wave height value, wave period parameters are extracted.

[0012] Furthermore, the online quality assessment described in step S8 includes at least one of the following verification actions: Reprojection verification: Take a point in the anti-sway stable wavefront point cloud, inversely transform it back to the camera coordinate system, and further project it back to the image plane. Calculate the reprojection error between the projected pixel coordinates and the original pixel coordinates corresponding to the point in the disparity estimation. Stability verification: Within a set short time window, the variance or cumulative drift of the coordinate values ​​of the anti-sway stable wavefront point cloud corresponding to the same spatial location in the world coordinate system in the vertical direction is statistically analyzed.

[0013] Furthermore, the adaptive correction and adjustment of the closed-loop feedback mechanism described in step S8 specifically includes at least one of the following operations: When at least one of the reprojection error, the variance, or the cumulative drift exceeds the corresponding index threshold in the preset quality threshold, pixels with disparity confidence lower than the preset confidence threshold in the dense disparity map are removed, and the disparity estimation in S6 is re-executed. Correct the time alignment between the attitude change information described in S4 and the acquisition time; Alternatively, the reprojection error, the variance, or the cumulative drift can be used as observations to fine-tune and update the parameters in the static coordinate transformation link in S5 online.

[0014] On the other hand, the present invention also provides a mobile platform anti-sway wave measurement system based on binocular vision, comprising: The binocular stereo vision acquisition unit is used to simultaneously acquire video images of the target sea surface area and obtain left and right eye image sequences with aligned timestamps. The calibration and stereo correction module is used to calculate and verify the camera calibration parameters through imaging using a physical calibration plate, and to perform stereo correction transformation on the left and right eye image sequences based on the calibration parameters. The attitude acquisition and time synchronization module is used to acquire attitude change information of the maritime mobile platform in three-dimensional space that is precisely time-aligned with the acquisition time of each frame of the left and right eye images. The 3D solution and world coordinate mapping module is used to generate a 3D wavefront point cloud in the camera coordinate system and construct a dynamic coordinate mapping relationship based on the attitude change information to transform the 3D wavefront point cloud to the world coordinate system to generate an anti-sway stable wavefront point cloud. The wave measurement output and quality assessment module is used to extract wave feature parameters from the anti-sway stable wave surface point cloud, perform online quality assessment, and trigger a closed-loop feedback mechanism for adaptive correction and adjustment.

[0015] Compared with the prior art, the embodiments of the present invention achieve the following beneficial effects: This invention synchronously acquires real-time attitude change information of a mobile offshore platform and explicitly incorporates it into a coordinate transformation link from the platform coordinate system, through the camera coordinate system, to the world coordinate system. It constructs a dynamic coordinate mapping relationship that updates dynamically with each frame acquisition, transforming platform sway from an uncontrollable source of measurement error into a known compensation quantity in the coordinate mapping. During each frame's transformation of the 3D wavefront point cloud from the camera coordinate system to the world coordinate system, this dynamic coordinate mapping relationship compensates in real time for changes in camera viewpoint caused by platform roll, pitch, and bow roll. This ensures that the wavefront point cloud output to the world coordinate system is unaffected by platform sway, fundamentally solving the technical problem of 3D point cloud drift and wave height measurement distortion caused by platform sway when existing fixed-platform binocular wave measurement schemes are directly applied to mobile offshore platforms. Meanwhile, the present invention continuously monitors the measurement results through online quality assessment and closed-loop feedback correction mechanisms. When the reprojection error or wavefront point cloud stability exceeds the preset threshold, it automatically performs adaptive correction on the parallax estimation process, the time alignment relationship between attitude and image, or the static coordinate transformation link parameters, ensuring the measurement stability and data reliability of the system under the influence of gradual factors such as structural micro-deformation and time synchronization drift during long-term operation at sea. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for measuring the anti-swaying waves of a mobile platform based on binocular vision, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a mobile platform anti-sway wave measurement system based on binocular vision provided in an embodiment of the present invention. Detailed Implementation

[0017] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments given herein are for illustration and explanation only and are not intended to limit the present invention.

[0018] It should be noted that many specific details are set forth in the following description in order to provide a full understanding of the present invention. However, the present invention may have other embodiments, and therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0019] Reference Figure 1 As shown in the figure, this embodiment provides a method for measuring the anti-swaying waves of a mobile platform based on binocular vision, and the specific steps are as follows.

[0020] S1. Obtain the stereo image sequence: On a mobile offshore platform, a binocular stereo vision acquisition unit consisting of a first image acquisition device and a second image acquisition device is installed. The installation of this binocular stereo vision acquisition unit must meet the following conditions: the baseline distance between the first and second image acquisition devices is determined based on the distance to the target sea surface area and the required measurement accuracy; the installation height of the binocular stereo vision acquisition unit is determined based on the average height of the platform deck above sea level; and the tilt angle of the optical axis of the binocular stereo vision acquisition unit relative to the horizontal plane must ensure that its overlapping field of view can cover the target sea surface area.

[0021] With the installation method described above, the overlapping field of view of the binocular stereo vision acquisition unit covers the target sea surface area. During wave monitoring, the first and second image acquisition devices are controlled to start exposure simultaneously via hardware trigger signals or software synchronization control, synchronously acquiring video images of the target sea surface area. The exposure time center stamps of each frame of the left and right eye images are precisely aligned, thereby obtaining a sequence of left and right eye images with aligned time stamps. The image sampling frame rate can be set according to the wave change rate to ensure that rapidly changing wave surface features can be captured.

[0022] S2. Obtain and verify camera calibration parameters through imaging using a physical calibration plate: After installing the binocular stereo vision acquisition unit, it needs to be calibrated to obtain camera calibration parameters. This embodiment uses a calibration method based on a physical plane calibration plate.

[0023] The specific operation process is as follows: Within the field of view of the binocular stereo vision acquisition unit, a physical plane calibration board of known size is held by hand or clamped by a robotic arm, and presented in various spatial poses. These various spatial poses refer to placing the physical plane calibration board relative to the binocular stereo vision acquisition unit at different tilt angles, rotation directions, and distances. This includes tilting the calibration board forward, backward, left, and right, as well as rotating it around its normal direction by different angles, while simultaneously changing the vertical distance between the calibration board and the camera within the field of view of the binocular stereo vision acquisition unit. In each spatial pose, the binocular stereo vision acquisition unit captures a pair of images of the physical plane calibration board; that is, the first image acquisition device captures the left calibration image, and the second image acquisition device captures the right calibration image.

[0024] After acquiring multiple sets of image pairs of the physical plane calibration plate, the coordinates of feature points on the physical plane calibration plate are extracted from each set of image pairs. In this embodiment, the physical plane calibration plate adopts a checkerboard pattern, and the feature points are the corner points of the checkerboard. Based on the pixel coordinates of the feature points extracted from each set of image pairs in the left-eye image and the right-eye image, combined with the known world coordinates of the feature points on the physical plane calibration plate, the respective internal parameter matrices and distortion coefficients of the first image acquisition device and the second image acquisition device in the binocular stereo vision acquisition unit, as well as the rotation matrix and translation vector between the two, are solved.

[0025] The internal parameter matrix includes the equivalent focal length f along the horizontal axis of the image x , the equivalent focal length f along the vertical axis of the image y and the coordinates of the optical center in the image (c x ,c y ). The distortion coefficients include radial distortion coefficients and tangential distortion coefficients, which are used to describe the deviation of actual imaging from the ideal perspective projection. The rotation matrix between the first image acquisition device and the second image acquisition device describes the relative rotation transformation relationship between the two camera coordinate systems, and the translation vector describes the spatial offset relationship between the origins of the two camera coordinate systems. The modulus length of the translation vector is the baseline length of the binocular stereo vision acquisition unit.

[0026] To verify the accuracy of the calibration result, the known world coordinates of the feature points on the physical plane calibration plate are re-projected to the image plane by using the solved internal parameter matrix, distortion coefficients, rotation matrix and translation vector, and the Euclidean distance between the re-projected pixel coordinates and the pixel coordinates of the feature points actually extracted from the image is calculated, which is the re-projection error. When the statistical value (such as the average value or root mean square value) of the re-projection errors of all feature points in all image pairs is less than the preset pixel-level threshold, the calibration result is determined to be qualified. In this embodiment, the preset pixel-level threshold is set to 0.2 pixels to 1.0 pixels. The verified internal parameter matrix, distortion coefficients, rotation matrix and translation vector are saved as calibration parameters for subsequent steps to call.

[0027] S3. Image undistortion and stereo rectification transformation based on calibration parameters: The calibration parameters saved in S2 are read, and each frame of image in the left-eye image sequence and the right-eye image sequence acquired in S1 is processed. First, the distortion coefficients are used to perform undistortion processing on the left-eye image and the right-eye image respectively, so as to eliminate the radial distortion and tangential distortion introduced by the optical system and restore the real geometric projection relationship of the scene. Then, the internal parameter matrix and the rotation matrix and translation vector between the first image acquisition device and the second image acquisition device are used to perform stereo rectification transformation on the undistorted left-eye image and right-eye image.

[0028] Stereo correction transformation involves reprojecting two images so that the optical axes of the two virtual cameras are parallel and their image planes are coplanar. After distortion correction and stereo correction transformation, the projection points of the same spatial point in the left and right eye images satisfy epipolar constraints, meaning they lie in the same pixel row. This means that for any physical point in the world coordinate system, its projection position in the transformed left eye image has the same row coordinate (vertical coordinate) as its projection position in the right eye image, differing only in its column coordinate (horizontal coordinate). This characteristic reduces the search space for subsequent stereo matching from two dimensions to one dimension, significantly improving the efficiency and accuracy of disparity estimation.

[0029] S4. Synchronous acquisition of attitude change information: During the acquisition of wave video images by the binocular stereo vision acquisition unit, the attitude change information of the offshore mobile platform is acquired simultaneously. The acquisition of attitude change information and video images are synchronized in time to ensure that they have the same acquisition time reference. In this embodiment, an inertial measurement unit is installed on the offshore mobile platform. This inertial measurement unit is rigidly connected to or fixed to the same mounting base as the binocular stereo vision acquisition unit to measure the motion state of the offshore mobile platform.

[0030] To achieve precise time alignment between binocular image acquisition and attitude information acquisition, this embodiment employs the following method: A hardware synchronization signal line is used, where the computing unit sends a synchronization pulse signal to the inertial measurement unit (IMU) simultaneously with triggering synchronous exposure of the first and second image acquisition devices. This ensures that each frame's acquisition time is associated with corresponding IMU data recording. Alternatively, data from the camera and IMU's respective hardware timestamps are recorded separately, and then, based on a unified time source, a software timestamp interpolation algorithm is used to align the IMU data to the acquisition time of each frame. Thus, the acquisition time of each frame of the left and right binocular images corresponds to a defined set of attitude change information.

[0031] In this embodiment, the attitude change information includes at least the roll angle, pitch angle, and bow angle of the offshore mobile platform. The roll angle represents the rotation angle of the offshore mobile platform about its bow and stern lines, the pitch angle represents the rotation angle of the offshore mobile platform about its port and starboard sides, and the bow angle represents the rotation angle of the offshore mobile platform about its vertical axis.

[0032] S5. Construct a dynamic world coordinate mapping relationship using attitude change information: Specifically, three coordinate systems are first defined to clarify the reference datum for subsequent coordinate transformations: The world coordinate system W is fixed to the Earth and remains unchanged regardless of the movement of the offshore platform. In this embodiment, the XY plane of the world coordinate system W is parallel to the mean sea level, and the Z-axis points upwards vertically. The world coordinate system W serves as the final reference for wave measurements, and all anti-roll stable wave surface point clouds are represented in this coordinate system.

[0033] Platform coordinate system B is fixed to the mobile offshore platform. Its origin can be defined at the geometric center or center of gravity of the mobile offshore platform, and the coordinate axes are aligned with the bow and stern lines, port and starboard directions, and vertical direction of the mobile offshore platform. Platform coordinate system B moves with the mobile offshore platform in six degrees of freedom, including three rotational degrees of freedom (roll, pitch, and bow) and three translational degrees of freedom (sway, roll, and heave).

[0034] The camera coordinate system C is fixed to the binocular stereo vision acquisition unit, with the optical center of the first image acquisition device (left eye camera) as its origin, the optical axis direction as the positive Z-axis, and the X and Y axes parallel to the row and column directions of the image sensor, respectively. The camera coordinate system C moves together with the binocular stereo vision acquisition unit.

[0035] Specifically, the static coordinate transformation link is constructed as follows: After the binocular stereo vision acquisition unit is installed and fixed on the offshore mobile platform, the installation pose of the camera coordinate system C relative to the platform coordinate system B is determined and remains fixed during the normal operation of the offshore mobile platform. This fixed installation pose is determined by the static rotation matrix R. bc and static translation vector t bc express.

[0036] R bc Rbc is a 3×3 orthogonal matrix describing the rotational relationship of the three axes of the camera coordinate system C relative to the three axes of the platform coordinate system B. Since the binocular stereo vision acquisition unit is fixedly installed on the mobile offshore platform, the elements of Rbc are determined by the mechanical positioning and orientation during installation. bc R is a 3×1 column vector describing the three-dimensional spatial position of the origin of the camera coordinate system C in the platform coordinate system B. bc and t bc The theoretical values ​​can be obtained from the structural design drawings, or they can be obtained through on-site measurement and calibration after installation using hand-eye calibration methods.

[0037] Therefore, a static coordinate transformation link is constructed from the camera coordinate system C, through the platform coordinate system B, to the world coordinate system W. The mathematical expression of this link is: for any spatial point in the camera coordinate system C, first through R... bc and t bc Transform to platform coordinate system B, and then transform the pose of platform coordinate system B in world coordinate system W to world coordinate system W.

[0038] Specifically, dynamic pose acquisition and transformation: The attitude change information acquired at each frame acquisition time in S4 contains data that can describe the instantaneous motion state of the offshore mobile platform. In this embodiment, the attitude change information includes at least the roll angle α(t), pitch angle β(t), and bow angle γ(t), where t represents the acquisition time of the t-th frame image. The roll angle α(t) represents the rotation angle of the offshore mobile platform about its bow and stern line direction (X-axis direction of platform coordinate system B), the pitch angle β(t) represents the rotation angle of the offshore mobile platform about its port and starboard sides direction (Y-axis direction of platform coordinate system B), and the bow angle γ(t) represents the rotation angle of the offshore mobile platform about its vertical axis direction (Z-axis direction of platform coordinate system B).

[0039] The roll angle α(t), pitch angle β(t), and yaw angle γ(t) acquired at time t are converted into a dynamic rotation matrix R of the platform coordinate system B relative to the world coordinate system W at that time. wb (t). In this embodiment, the standard conversion method from Euler angles to rotation matrices is adopted: the rotation about the Z-axis corresponding to the pitch angle γ(t), the rotation about the Y-axis corresponding to the pitch angle β(t), and the rotation about the X-axis corresponding to the roll angle α(t) are multiplied sequentially in a specific rotation order (e.g., ZYX order) to obtain a 3×3 dynamic rotation matrix R. wb (t). R wb (t) fully describes the rotational changes of the three coordinate axes of the platform coordinate system B relative to the three coordinate axes of the world coordinate system W caused by the platform's swaying.

[0040] Simultaneously, based on the real-time positional changes of the offshore mobile platform in the world coordinate system W (which can be obtained through the Global Positioning System or Differential Positioning System), the dynamic translation vector t of the platform coordinate system B relative to the world coordinate system W at the acquisition time of frame t is obtained. wb (t). t wb (t) is a 3×1 column vector describing the instantaneous three-dimensional spatial position of the origin of the platform coordinate system B in the world coordinate system W. In this embodiment, if wave measurement mainly focuses on parameters such as wave height and period, and is not sensitive to the slow translation of the offshore mobile platform, t can be... wb (t) is approximately taken as the zero vector or set to a constant value; if it is necessary to obtain the absolute position of the wavefront point cloud in the world coordinate system W, then t is updated in real time based on the actual position measurement data. wb (t).

[0041] Specifically, the construction of the dynamic coordinate mapping relationship: The above dynamic rotation matrix R... wb (t) and the dynamic translation vector t wb (t) is superimposed onto the established static coordinate transformation link to obtain the dynamic rotation matrix R of the camera coordinate system C relative to the world coordinate system W at the acquisition time of frame t.wc (t) and the dynamic translation vector t wc (t), the specific construction method is as follows: R wc (t)=R wb (t)·R bc t wc (t)=R wb (t)·t bc +t wb (t) The first formula states that the rotation transformation from the camera coordinate system C to the world coordinate system W consists of two rotation steps. The first step involves rotating through R... bc The orientation of the camera coordinate system C is converted to an orientation relative to the platform coordinate system B—this is a fixed installation relationship. The second step is to use R... wb (t) The orientation of platform coordinate system B is transformed into its orientation relative to world coordinate system W—a dynamic relationship that changes in real time with platform sway. Multiplying the two rotation matrices yields the dynamic rotation matrix of camera coordinate system C relative to world coordinate system W at frame t. Since R... wb (t) varies with the time of data collection, R wc (t) also updates dynamically with the acquisition time, reflecting the real-time orientation change of the camera in the world coordinate system.

[0042] The second formula states that the translation transformation from the origin of the camera coordinate system C to the origin of the world coordinate system W consists of two components. The first term R... wb (t)·t bc The vector t represents the translation of the origin of the camera coordinate system C relative to the origin of the platform coordinate system B. bc After the platform's current rotation R wb The expression of (t) in the world coordinate system W after transformation. The second term t wb (t) represents the instantaneous position of the origin of the platform coordinate system B in the world coordinate system W. Adding the two terms yields the dynamic translation vector of the origin of the camera coordinate system C in the world coordinate system W at the acquisition time of frame t.

[0043] By dynamic rotation matrix R wc (t) and the dynamic translation vector t wc The coordinate mapping relationship jointly represented by (t) is the dynamic coordinate mapping relationship constructed in this invention. This dynamic coordinate mapping relationship is updated in real time according to the attitude change of the mobile platform at each frame acquisition time.

[0044] S6. Generation of 3D wavefront point cloud in camera coordinate system: For each frame of the left and right eye images after S3 distortion correction and stereo correction transformation, stereo matching and 3D calculation are performed.

[0045] Specifically, disparity estimation is performed first. For each pixel in the left-eye image, its corresponding point (i.e., the projection point of the same spatial point in the right-eye image) is searched in the same pixel row of the right-eye image. The difference between the column coordinates of the pixel in the left-eye image and the column coordinates of the corresponding point in the right-eye image is calculated, which is the disparity value of the pixel. Disparity estimation is performed on all pixels or pixels within a selected region of interest to generate a dense disparity map. Simultaneously, the disparity estimation algorithm outputs the disparity confidence score for each pixel in the dense disparity map during the calculation process, which is used to measure the reliability of the disparity value of that pixel. The disparity confidence score can be obtained based on matching cost, left-right consistency detection, or other quality assessment metrics. In this embodiment, disparity estimation can be implemented using an end-to-end stereo matching algorithm based on a convolutional neural network.

[0046] Then, based on the principle of triangulation, the intrinsic focal length (i.e., the equivalent focal length f) in the calibration parameters obtained from S2 is used. x and f y The disparity value d of each pixel in the dense disparity map is converted into the corresponding depth value Z by the magnitude of the translation vector between the first and second image acquisition devices (i.e., the baseline length B). Z=(f·B) / d Then, the X and Y coordinates of the corresponding 3D spatial point in the camera coordinate system are calculated according to the following relationship: X=((uc x )·Z) / f x Y=((vc y )·Z) / f y Where u and v are the column and row coordinates of the pixel in the image, and c x and c y These are the column and row coordinates of the optical center in the image.

[0047] Thus, the pixels in each frame of the left-eye image are converted into three-dimensional spatial points in the camera coordinate system C, and the three-dimensional spatial points corresponding to all pixels constitute the three-dimensional wavefront point cloud of that frame in the camera coordinate system.

[0048] S7. Generate anti-sway wavefront point clouds and extract wave parameters based on dynamic coordinate mapping relationships: The dynamic coordinate mapping relationship corresponding to each frame acquisition time is constructed by calling S5, which is determined by the dynamic rotation matrix R. wc (t) and the dynamic translation vector t wc The mapping relationship represented by (t) transforms the 3D wavefront point cloud generated by S6 in the camera coordinate system C point by point to the world coordinate system W.

[0049] For any point P in the camera coordinate system C c =(X c ,Y c Z c ) T Its corresponding point P in the world coordinate system W w (t) is determined by the following transformation: P w (t)=R wc (t)·P c +t wc (t) The physical meaning of this transformation is to "rotate and translate" the 3D point cloud acquired from the camera's perspective (which is tilted due to platform sway) in the current frame to a fixed world coordinate system unaffected by platform sway. Since the dynamic coordinate mapping relationship is updated in real time with the platform's attitude at each frame acquisition moment, even if the mobile platform at sea continuously experiences roll, pitch, and bow movements during acquisition, the wavefront point cloud transformed to the world coordinate system W still maintains spatial stability, thus obtaining a sway-resistant and stable wavefront point cloud.

[0050] Specifically, wave feature parameters are extracted based on the anti-rolling stable wavefront point cloud. In one implementation, multiple frames of the anti-rolling stable wavefront point cloud within a time window (e.g., 30 seconds to 5 minutes) are processed. Spatial analysis is performed on all three-dimensional points within this time window to fit a spatial reference plane representing the mean sea level. This spatial reference plane can be obtained by plane fitting the point cloud using the least squares method or a random sample consensus algorithm, and the fitted spatial reference plane is represented by the plane equation ax + by + cz + d = 0.

[0051] Specifically, the normal distance from each point in the anti-rolling stable wavefront point cloud to the spatial reference plane is calculated. This normal distance is used as the instantaneous wavefront undulation (i.e., wave height) at that spatial location. A positive value indicates that the wavefront is above the mean sea level, while a negative value indicates that the wavefront is below the mean sea level. Based on the time series of instantaneous wavefront undulation or wave height values ​​calculated from multiple consecutive frames of images, wave characteristic parameters such as wave period, significant wave height, and maximum wave height are extracted using the zero-point crossing method or spectral analysis.

[0052] S8. Closed-loop feedback correction based on point cloud quality evaluation: Online quality assessment is performed on the anti-sway stable wave surface point cloud generated by S7. When the assessment result exceeds the preset quality threshold, a closed-loop feedback mechanism is triggered to perform adaptive correction and adjustment to ensure the measurement stability and data reliability of the system during long-term operation under complex sea conditions.

[0053] Online quality assessment includes at least one of the following verification actions: Reprojection verification: A 3D point in the anti-sway stable wavefront point cloud is inversely transformed from the world coordinate system W back to the camera coordinate system C. Then, using camera calibration parameters, the point is projected back from the camera coordinate system C to the image plane, obtaining the projected pixel coordinates. These projected pixel coordinates are compared with the original pixel coordinates corresponding to the point during S6 disparity estimation, and the Euclidean distance between them is calculated to obtain the reprojection error of that point. After calculating the reprojection error for all 3D points used for verification, their average, maximum, or root mean square value can be used as the reprojection error evaluation index for the current frame.

[0054] Stability verification: Within a set short time window (e.g., 5 to 10 consecutive frames), for the same spatial location (i.e., the same XY coordinates) in the world coordinate system W, calculate the variance or cumulative drift of the anti-sway stable wavefront point cloud in the vertical direction (Z-axis direction). Variance reflects the degree of fluctuation in wave height measurements at the same spatial location over a short period, while cumulative drift reflects whether there is a systematic unidirectional shift trend in the measurements. Ideally, the measurement results should have small variance and no significant cumulative drift.

[0055] When at least one of the reprojection error, variance, or cumulative drift calculated above exceeds the corresponding index threshold in the preset quality threshold, a closed-loop feedback mechanism is triggered, and at least one of the following adaptive correction and adjustment operations is performed: First, pixels with disparity confidence levels below a preset confidence threshold are removed from the dense disparity map, and disparity estimation in step S6 is re-executed to generate a higher-quality 3D wavefront point cloud. Pixels with low disparity confidence levels typically correspond to weak texture regions, repetitive texture regions, or occluded regions in the image. Removing these unreliable pixels helps improve the overall accuracy of wavefront reconstruction.

[0056] Second, the time alignment between attitude change information and acquisition time in S4 is corrected. In practice, the optimal time offset between the attitude change information sequence and the image sequence can be found through correlation analysis or mutual information maximization methods. The time alignment parameters are then adjusted based on this offset to eliminate time alignment deviations caused by hardware synchronization errors or software timestamp processing delays.

[0057] Third, using reprojection error, variance, or cumulative drift as observations, an error-state Kalman filter or recursive least squares estimator is constructed to fine-tune and update the parameters in the static coordinate transformation link of S5 online. These parameters include, but are not limited to, the parameter components related to the installation angle error in the rotation matrix and translation vector describing the fixed installation pose of the binocular stereo vision acquisition unit on the offshore mobile platform, and the translation parameter components describing the spatial offset relationship between the origin of the platform coordinate system and the origin of the camera coordinate system. Through online fine-tuning and updating, the system coordinate mapping deviation caused by factors such as structural micro-deformation and vibration loosening during the long-term operation of the offshore mobile platform can be eliminated.

[0058] Reference Figure 2 As shown, this embodiment provides a mobile platform anti-sway wave measurement system based on binocular vision, used to implement the methods described in the above embodiments. The system includes the following components: A binocular stereo vision acquisition unit is used to simultaneously acquire video images of a target sea surface area, obtaining left-eye image sequences and right-eye image sequences with aligned timestamps. In this embodiment, the binocular stereo vision acquisition unit includes two industrial cameras equipped with fixed-focus lenses, which are mounted on the deck or mast of a mobile offshore platform via rigid brackets.

[0059] The calibration and stereo correction module is used to calculate and verify camera calibration parameters through imaging using a physical calibration board, and to perform stereo correction transformation on the left and right eye image sequences based on the calibration parameters. During the calibration phase, this module performs functions such as calibration image acquisition control, feature point extraction, parameter calculation, and reprojection error verification; during the runtime phase, it performs distortion correction and stereo correction transformation functions.

[0060] The attitude acquisition and time synchronization module is used to acquire attitude change information of the offshore mobile platform in three-dimensional space, precisely time-aligned with the acquisition time of each frame of the left and right eye images. This module includes an inertial measurement unit (IMU) and a synchronization control unit. The IMU measures the three-axis angular velocity and acceleration information of the offshore mobile platform, and obtains the roll, pitch, and yaw angles through integration and attitude calculation algorithms. The synchronization control unit is responsible for generating a synchronization trigger signal to keep the hardware timestamps of image acquisition and attitude measurement synchronized.

[0061] The 3D solution and world coordinate mapping module generates 3D wavefront point clouds in the camera coordinate system and constructs a dynamic coordinate mapping relationship based on attitude change information to transform the 3D wavefront point clouds to the world coordinate system, generating anti-sway stable wavefront point clouds. This module includes a disparity estimation submodule, a triangulation submodule, a dynamic extrinsic parameter construction submodule, and a coordinate transformation submodule. The dynamic extrinsic parameter construction submodule receives attitude change information in real time, calculates the dynamic rotation matrix and dynamic translation vector at each frame acquisition time, and fuses the dynamic pose with the static installation pose to construct a coordinate mapping relationship that updates in real time with the acquisition time.

[0062] The wave measurement output and quality assessment module is used to extract wave characteristic parameters from the anti-rolling stable wavefront point cloud, perform online quality assessment, and trigger a closed-loop feedback mechanism for adaptive correction and adjustment. This module includes a wavefront analysis submodule, a wave parameter calculation submodule, a quality assessment submodule, and a feedback control submodule. The wavefront analysis submodule performs mean sea level fitting and wave height calculation on multiple frames of stable wavefront point clouds. The wave parameter calculation submodule extracts parameters such as wave height, period, and wave direction from the wave height time series. The quality assessment submodule performs reprojection verification and stability verification. The feedback control submodule triggers closed-loop feedback when the assessment results exceed a preset quality threshold, adaptively correcting the system parameters.

[0063] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0064] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. Any of the claimed embodiments can be used in any combination.

Claims

1. A method for measuring the anti-swaying waves of a mobile platform based on binocular vision, characterized in that, Includes the following steps: S1. Acquire binocular image sequence: Adjust the baseline distance, installation height, and tilt angle relative to the horizontal plane of the binocular stereo vision acquisition unit so that its overlapping field of view covers the target sea surface area; synchronously acquire video images of the target sea surface area through the binocular stereo vision acquisition unit to obtain left eye image sequence and right eye image sequence with aligned timestamps; S2. Obtain and verify camera calibration parameters through imaging with a physical calibration board: In the field of view of the binocular stereo vision acquisition unit, a physical plane calibration board of known size is presented in various spatial poses and the physical plane calibration board is photographed; feature point coordinates are extracted from the acquired image pairs of the physical plane calibration board, and the intrinsic parameter matrix and distortion coefficient of each in the binocular stereo vision acquisition unit, as well as the rotation matrix and translation vector between them, are calculated; the extracted feature point coordinates are reprojected onto the image plane, the reprojection error is calculated to verify the parameter accuracy, and the verified intrinsic parameter matrix, distortion coefficient, rotation matrix, and translation vector are saved as calibration parameters; S3. Image distortion correction and stereo correction transformation based on calibration parameters: Read the calibration parameters and perform distortion correction and stereo correction transformation on each frame of the left eye image sequence and the right eye image sequence. After the transformation, the projection points of the same spatial point in the left eye image and the right eye image satisfy the epipolar constraint, that is, they are located in the same pixel row. S4. Synchronous acquisition of attitude change information: Acquire the attitude change information of the offshore mobile platform in three-dimensional space that is precisely time-aligned with the left and right eye images of each frame at the acquisition time; S5. Construct a dynamic world coordinate mapping relationship using attitude change information: Construct a static coordinate transformation link from the platform coordinate system, through the camera coordinate system, to the world coordinate system; convert the attitude change information acquired at each frame acquisition time into a dynamic rotation matrix and dynamic translation vector of the platform coordinate system relative to the world coordinate system corresponding to each frame acquisition time, and superimpose the dynamic rotation matrix and dynamic translation vector into the static coordinate transformation link to construct a dynamic coordinate mapping relationship that is dynamically updated with the acquisition time and used to map points in the camera coordinate system to the world coordinate system; S6. Generation of 3D wavefront point cloud in camera coordinate system: For each frame of the left and right eye images processed by S3, disparity estimation is performed to generate a dense disparity map and the disparity confidence of each pixel in the dense disparity map; Based on the principle of triangulation, using the intrinsic focal length in the calibration parameters and the magnitude of the translation vector, the disparity value of each pixel in the dense disparity map is converted into the corresponding depth value to generate a 3D wavefront point cloud in camera coordinate system for each frame; S7. Generate anti-sway wavefront point cloud and extract wave parameters based on dynamic coordinate mapping relationship: call the dynamic coordinate mapping relationship to transform the three-dimensional wavefront point cloud in the camera coordinate system to the world coordinate system point by point to obtain the anti-sway stable wavefront point cloud in the world coordinate system; and extract wave feature parameters based on the anti-sway stable wavefront point cloud. S8. Closed-loop feedback correction based on point cloud quality evaluation: Online quality evaluation is performed on the anti-sway stable wavefront point cloud. When the evaluation result exceeds the preset quality threshold, a closed-loop feedback mechanism is triggered to adaptively correct and adjust the parameters in the disparity estimation process in S6, the time alignment relationship between the attitude change information and the image acquisition time in S4, and / or the parameters in the static coordinate transformation link in S5.

2. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 1, characterized in that, The attitude change information mentioned in S4 includes at least the roll angle, pitch angle, and bow angle of the offshore mobile platform; The time alignment specifically involves using a hardware synchronization signal line or software timestamp interpolation based on a unified time source to ensure that the acquisition time of each frame of the left and right eye images corresponds to a set of determined attitude change information.

3. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 1, characterized in that, Step S5 specifically includes: Define a world coordinate system W fixed to the Earth, a platform coordinate system B fixed to the offshore mobile platform, and a camera coordinate system C fixed to the binocular stereo vision acquisition unit. The fixed installation pose of the camera coordinate system C relative to the platform coordinate system B is determined using a static rotation matrix R. bc and static translation vector t bc express; The attitude change information acquired at the t-th frame acquisition time is converted into a dynamic rotation matrix R of the platform coordinate system B relative to the world coordinate system W at that time. wb (t) and the dynamic translation vector t wb (t); Then the dynamic rotation matrix R of the camera coordinate system C relative to the world coordinate system W at the acquisition time of the t-th frame is... wc (t) and the dynamic translation vector t wc (t) is constructed as follows: R wc (t)=R wb (t)·R bc t wc (t)=R wb (t)·t bc +t wb (t) The dynamic coordinate mapping relationship is determined by the dynamic rotation matrix R. wc (t) and the dynamic translation vector t wc (t) characterization.

4. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 3, characterized in that, The anti-sway stable wavefront point cloud obtained in the world coordinate system as described in S7 is as follows: For any point P in the camera coordinate system C c Its corresponding point P in the world coordinate system W w (t) by P w (t)=R wc (t)·P c +t wc (t) is determined.

5. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 1, characterized in that, The extraction of wave feature parameters described in S7 specifically includes: For a continuous anti-rolling stable wave surface point cloud within a time window, a spatial reference plane representing the mean sea level is fitted; the normal distance from each point in the anti-rolling stable wave surface point cloud to the spatial reference plane is calculated, and the normal distance is used as the instantaneous wave surface undulation or wave height value; and based on the time series of the instantaneous wave surface undulation or wave height value, wave period parameters are extracted.

6. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 1, characterized in that, The online quality assessment described in step S8 includes at least one of the following verification actions: Reprojection verification: Take a point in the anti-sway stable wavefront point cloud, inversely transform it back to the camera coordinate system, and further project it back to the image plane. Calculate the reprojection error between the projected pixel coordinates and the original pixel coordinates corresponding to the point in the disparity estimation. Stability verification: Within a set short time window, the variance or cumulative drift of the coordinate values ​​of the anti-sway stable wavefront point cloud corresponding to the same spatial location in the world coordinate system in the vertical direction is statistically analyzed.

7. The method for measuring the anti-swaying waves of a mobile platform based on binocular vision according to claim 1 or 6, characterized in that, The adaptive correction and adjustment of the closed-loop feedback mechanism described in step S8 specifically includes at least one of the following operations: When at least one of the reprojection error, the variance, or the cumulative drift exceeds the corresponding index threshold in the preset quality threshold, pixels with disparity confidence lower than the preset confidence threshold in the dense disparity map are removed, and the disparity estimation in S6 is re-executed. Correct the time alignment between the attitude change information described in S4 and the acquisition time; Alternatively, the reprojection error, the variance, or the cumulative drift can be used as observations to fine-tune and update the parameters in the static coordinate transformation link in S5 online.

8. A mobile platform anti-sway wave measurement system based on binocular vision, characterized in that, For implementing the binocular vision-based mobile platform anti-sway wave measurement method as described in any one of claims 1 to 7, the system comprises: The binocular stereo vision acquisition unit is used to simultaneously acquire video images of the target sea surface area and obtain left and right eye image sequences with aligned timestamps. The calibration and stereo correction module is used to calculate and verify the camera calibration parameters through imaging using a physical calibration plate, and to perform stereo correction transformation on the left and right eye image sequences based on the calibration parameters. The attitude acquisition and time synchronization module is used to acquire attitude change information of the maritime mobile platform in three-dimensional space that is precisely time-aligned with the acquisition time of each frame of the left and right eye images. The 3D solution and world coordinate mapping module is used to generate a 3D wavefront point cloud in the camera coordinate system and construct a dynamic coordinate mapping relationship based on the attitude change information to transform the 3D wavefront point cloud to the world coordinate system to generate an anti-sway stable wavefront point cloud. The wave measurement output and quality assessment module is used to extract wave feature parameters from the anti-sway stable wave surface point cloud, perform online quality assessment, and trigger a closed-loop feedback mechanism for adaptive correction and adjustment.

Images