Method for measuring the motion of a floating body

Abstract

The application relates to the field of computer vision and relates to a measurement method for the movement of a marine floating body. The measurement method comprises the following steps: acquiring a video of the movement of the marine floating body on a water body; determining a feature tracking point set corresponding to each frame of image in the video, the feature tracking point set being related to the marine floating body; and determining six-degree-of-freedom movement parameters of the marine floating body according to the feature tracking point set. The application realizes the measurement of three-dimensional complete movement information, i.e. the measurement of six-degree-of-freedom movement parameters, and can improve the robustness and completeness of the measurement of the movement information of the marine floating body.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and more specifically to a method for measuring the motion of floating bodies at sea. Background Technology

[0002] In the research and development testing of marine floating bodies, it is crucial to accurately measure their motion parameters in a real marine environment.

[0003] In related technologies, cameras are often used to achieve non-contact measurement, that is, to collect images of floating bodies at sea, and to calculate the translation information in a two-dimensional plane by analyzing the displacement of the floating bodies in the images.

[0004] However, the motion parameters obtained by related technologies have limited dimensions, and the motion information of floating bodies at sea is incomplete. Summary of the Invention

[0005] To overcome the problems existing in related technologies, the present invention provides a method for measuring the motion of a floating body at sea, the method comprising:

[0006] To obtain video footage of floating objects moving on the water.

[0007] Determine a set of feature tracking points corresponding to each frame of the video, the set of feature tracking points being associated with the floating body at sea;

[0008] Based on the feature tracking point set corresponding to adjacent frame images, the covariance matrix corresponding to adjacent frame images is determined. The covariance matrix corresponding to adjacent frame images represents the spatial distribution change of the feature tracking point set corresponding to adjacent frame images.

[0009] The roll angle and pitch angle of the floating body at sea are determined based on the relative changes in the main diagonal elements of the covariance matrix corresponding to adjacent frame images.

[0010] The roll angle of the floating body is determined based on the relative changes in the first diagonal elements of the covariance matrix corresponding to adjacent frames, and the pitch angle is determined based on the relative changes in the second diagonal elements of the covariance matrix corresponding to adjacent frames. The first diagonal elements characterize the dispersion of the feature tracking point set in the vertical direction of the image, and the second diagonal elements characterize the dispersion of the feature tracking point set in the horizontal direction of the image. The directions of the roll and pitch angles of the floating body are determined based on the signs of the off-diagonal elements of the covariance matrix corresponding to adjacent frames. The off-diagonal elements characterize the distribution correlation of the feature tracking point set in the horizontal and vertical directions of the image.

[0011] Based on the correspondence between the feature tracking point sets corresponding to adjacent frame images, the similarity transformation model corresponding to adjacent frame images is determined;

[0012] The similarity transformation model is analyzed to obtain the translation component, rotation component, and scaling component.

[0013] The sway displacement and the roll displacement are determined based on the translation component, the pitch angle is determined based on the rotation component, and the heave displacement is determined based on the scaling component.

[0014] In some exemplary embodiments of this disclosure, the scaling component includes a scaling factor; the step of parsing the similarity transformation model to obtain the scaling component includes:

[0015] The scaling factor is obtained by analyzing the similarity transformation model.

[0016] Determining the heave displacement based on the scaling component includes:

[0017] The heave displacement is determined based on the scaling factor and the preset mapping function.

[0018] In some exemplary embodiments of this disclosure, determining the set of feature tracking points corresponding to each frame of the video includes:

[0019] Determine the set of feature tracking points corresponding to the first frame of the video;

[0020] The feature tracking point set corresponding to the first frame image is tracked, and the feature tracking point set corresponding to each frame image after the first frame image is determined frame by frame.

[0021] In some exemplary embodiments of this disclosure, determining the feature tracking point set corresponding to the first frame of the video includes:

[0022] In the floating body feature detection region of the first frame of the video, the set of feature tracking points corresponding to the first frame image is detected.

[0023] The technical solutions provided by the embodiments of the present invention can include the following beneficial effects: the data acquisition method for measuring the motion of a floating body at sea is through video of the floating body's motion on the water, achieving non-contact measurement to improve measurement robustness. By determining the set of feature tracking points related to the floating body for each frame of the image, a data foundation is provided for determining the six degrees of freedom motion parameters. Based on this set of feature tracking points, the six degrees of freedom motion parameters are determined. Compared with related technologies that can only acquire two-dimensional motion parameters, the present invention achieves the measurement of complete three-dimensional motion information, that is, the measurement of six degrees of freedom motion parameters. Thus, the robustness and completeness of the motion information measurement of the floating body at sea can be improved.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0026] Figure 1 This is a schematic diagram illustrating the application environment of a method for measuring the motion of a floating body at sea, according to an exemplary embodiment.

[0027] Figure 2 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a first exemplary embodiment.

[0028] Figure 3 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a second exemplary embodiment.

[0029] Figure 4 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a third exemplary embodiment.

[0030] Figure 5 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a fourth exemplary embodiment.

[0031] Figure 6 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a fifth exemplary embodiment.

[0032] Figure 7 This is a schematic diagram illustrating a method for measuring the motion of a floating body at sea, according to a sixth exemplary embodiment.

[0033] Figure 8 This is a block diagram of a computer device according to an exemplary embodiment.

[0034] In the picture:

[0035] 1-Floating hull at sea; 2-Monocular camera;

[0036] 100 - Computer equipment; 101 - Computing unit; 102 - ROM; 103 - RAM; 104 - Bus; 105 - Input / output interface; 106 - Input unit; 107 - Output unit; 108 - Storage unit; 109 - Communication unit. Detailed Implementation

[0037] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. It should also be understood that the term "and / or" as used in the present invention refers to any or all possible combinations comprising one or more of the associated listed items.

[0038] In the research and development testing of marine floating bodies, it is crucial to accurately measure their motion parameters in a real marine environment.

[0039] In related technologies, cameras are often used to achieve non-contact measurement, that is, to collect images of floating bodies at sea, and to calculate the translation information in a two-dimensional plane by analyzing the displacement of the floating bodies in the images.

[0040] However, the motion parameters obtained by related technologies have limited dimensions, and the motion information of floating bodies at sea is incomplete.

[0041] Based on this, this disclosure provides a method for measuring the motion of a floating body at sea. The data acquisition method for measuring the motion of the floating body is through video of its movement on the water, achieving non-contact measurement to improve measurement robustness. By determining the set of feature tracking points related to the floating body for each frame of the image, a data foundation is provided for determining the six degrees of freedom motion parameters. Based on this set of feature tracking points, the six degrees of freedom motion parameters are determined. Compared to related technologies that can only acquire two-dimensional motion parameters, this invention achieves the measurement of complete three-dimensional motion information, i.e., the measurement of six degrees of freedom motion parameters. Thus, the robustness and completeness of the motion information measurement of the floating body at sea can be improved.

[0042] This embodiment provides a method for measuring the motion of a floating body at sea, applicable to computer equipment. The computer equipment includes an image acquisition device, a display device, a memory, and a processor that are interconnected. The image acquisition device includes a monocular camera for acquiring video of the floating body's motion on the water. The display device includes a monitor for displaying a virtual motion model of the floating body.

[0043] like Figure 1 As shown in the diagram, this embodiment illustrates an application environment for a method of measuring the motion of a floating body at sea. The floating body 1 moves on a body of water (e.g., seawater). A monocular camera 2 of a computer device is positioned at a stable observation point, such as a dock or observation tower capable of capturing video of the floating body's motion. After the monocular camera 2 captures the video of the floating body's motion, it is directly input into the computing unit of the computer device, causing the computing unit to execute the method for measuring the motion of the floating body. Alternatively, the video can be transmitted to other computer devices via a communication module so that those other computer devices can execute the method for measuring the motion of the floating body.

[0044] like Figure 2 As shown, in one exemplary embodiment, a method for measuring the motion of a floating body at sea is provided, the method comprising:

[0045] S210. Obtain video footage of floating objects moving on the water.

[0046] S220. Determine the set of feature tracking points corresponding to each frame of the video, the set of feature tracking points being related to the floating body at sea.

[0047] S230. Determine the six-degree-of-freedom motion parameters of the floating body at sea based on the feature tracking point set.

[0048] The technical solutions provided by the embodiments of the present invention can include the following beneficial effects: The data acquisition method for measuring the motion of a floating body at sea is through video of the floating body's motion on the water, achieving non-contact measurement to improve measurement robustness. By determining the set of feature tracking points related to the floating body for each frame of the image, a data foundation is provided for determining the six degrees of freedom motion parameters. Based on this set of feature tracking points, the six degrees of freedom motion parameters are determined. Compared to related technologies that can only acquire two-dimensional motion parameters, the present invention achieves the measurement of complete three-dimensional motion information, i.e., the measurement of six degrees of freedom motion parameters. Thus, the robustness and completeness of the motion information measurement of the floating body at sea can be improved.

[0049] For example, acquiring video of the movement of the floating body on the water in step S210 above includes:

[0050] Videos of floating objects moving on the water are captured using a monocular camera.

[0051] In this embodiment, compared to the related technologies that rely on multi-view stereo vision systems to obtain raw data for determining the motion parameters of floating bodies at sea, this embodiment only uses a monocular camera to obtain raw data for determining the motion parameters of floating bodies at sea, which can reduce hardware threshold, reduce costs and improve the adaptability of application scenarios.

[0052] In one embodiment, such as Figure 3 As shown, the determination of the feature tracking point set corresponding to each frame of the video in step S220 above includes:

[0053] S310. Determine the set of feature tracking points corresponding to the first frame of the video.

[0054] S320. Track the feature tracking point set corresponding to the first frame image, and determine the feature tracking point set corresponding to each frame image after the first frame image.

[0055] In this embodiment, a corresponding feature tracking point set is established from the first frame of the video, and the feature tracking point set is tracked frame by frame to ensure the temporal continuity and information integrity of the data used for subsequent calculations, thereby improving the accuracy and completeness of the measurement of the motion parameters of the floating body at sea.

[0056] In one embodiment, determining the feature tracking point set corresponding to the first frame image of the video in step S310 above includes:

[0057] In the floating body feature detection area of ​​the first frame of the video, the set of feature tracking points corresponding to the first frame image is detected.

[0058] In this embodiment, the detection area of ​​the feature tracking point set is limited to the floating body feature detection area of ​​the first frame image, rather than the entire area of ​​the first frame image. This can eliminate the interference of irrelevant background (such as waves and spray), improve the robustness and accuracy of the feature tracking point set corresponding to the first frame image, and provide a high-quality initial feature tracking point set for subsequent frame-by-frame tracking, thereby improving the robustness and accuracy of subsequent motion parameter calculation.

[0059] For example, the step of detecting the floating body feature detection region in the first frame of the video, and detecting the feature tracking point set corresponding to the first frame image, includes:

[0060] In the floating body feature detection area of ​​the first frame of the video, the corner detection method is used to detect the feature tracking point set corresponding to the first frame image.

[0061] In this embodiment, the corner detection method has stronger robustness and accuracy compared to the traditional edge detection, thereby improving the robustness and accuracy of the feature tracking point set, and further improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0062] For example, in the above steps, the floating body feature detection region in the first frame of the video is used to detect the feature tracking point set corresponding to the first frame image using a corner detection method, including at least one of the following:

[0063] In the floating body feature detection area of ​​the first frame of the video, the minimum feature value of grayscale change of each pixel in the first frame image when it moves in all directions within a preset window is calculated; the set of pixels whose minimum feature value is greater than or equal to the preset feature value is determined as the feature tracking point set corresponding to the first frame image.

[0064] In the floating body feature detection area of ​​the first frame of the video, the gray value of each pixel in the first frame image is compared with the gray value of the pixels in the surrounding circular neighborhood; the pixels whose absolute value of the difference between their gray value and the gray value of the pixels in the surrounding circular neighborhood is greater than or equal to the preset gray value are determined as the feature tracking point set corresponding to the first frame image.

[0065] In the floating body feature detection area of ​​the first frame of the video, a scale space is constructed and the approximate value of the Hessian matrix determinant of the pixel is calculated; the pixel points whose scale and determinant approximate values ​​meet the preset conditions are determined as the feature tracking point set corresponding to the first frame image.

[0066] In the floating body feature detection area of ​​the first frame of the video, the second-order moment matrix of the image gradient of each pixel in the first frame image within a preset window is calculated; the corner response value corresponding to the pixel is calculated based on the second-order moment matrix; and the pixels with corner response values ​​greater than or equal to a preset response threshold are determined as the feature tracking point set.

[0067] In this embodiment, based on four different corner detection methods, pixels with rich texture (i.e., pixels with drastic grayscale or gradient changes and high distinguishability in local areas of the image) and easy tracking (i.e., pixels with high local contrast and anisotropic gradient distribution) can be selected. Such pixels are determined as the feature tracking point set corresponding to the first frame image, which helps to improve the robustness and accuracy of the feature tracking point set, thereby improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0068] For example, in the above steps, the minimum feature value of grayscale change for each pixel in the first frame image when it moves in all directions within a preset window can be calculated. For instance, the preset window can be a 3×3 neighborhood. By calculating the covariance matrix of the 3×3 neighborhood, the minimum feature value of grayscale change for the pixel when it moves in all directions within the preset window can be determined. The set of pixels whose minimum feature value is greater than or equal to the preset feature value is determined as the feature tracking point set corresponding to the first frame image. For example, within a 5×5 neighborhood, pixels whose minimum feature value is less than the preset feature value are removed, and pixels whose minimum feature value is greater than or equal to the preset feature value are retained. These pixels are then determined as the feature tracking point set.

[0069] In this embodiment, it is beneficial to ensure the uniqueness and trackability of each feature tracking point in the feature tracking point set, thereby improving the robustness and accuracy of the feature tracking point set.

[0070] For example, calculating the corner response value corresponding to the pixel based on the second-order moment matrix includes:

[0071]

[0072]

[0073] in, Corner response value, The second-order moment matrix of the image gradient of this pixel within a preset window. for The determinant, It is an empirical constant. for traces, For the default window, This represents the spatial gradient of the pixel in the horizontal direction. This represents the spatial gradient of the pixel in the vertical direction. The value range is [0.04, 0.06]. The value can be 0.04, 0.05, or 0.06.

[0074] In this embodiment, it is beneficial to improve the robustness of corner response value calculation, thereby improving the robustness and accuracy of feature tracking point set.

[0075] For example, the method for determining the floating body feature detection region in the first frame in the above steps includes:

[0076] Receive the user's bounding box instruction in the first frame. The bounding box instruction represents the floating body feature detection area that the user has bounded in the first frame image.

[0077] Based on the box selection command, the floating body feature detection area in the first frame is determined.

[0078] In this embodiment, the floating body feature detection area can be implemented through user interaction, which helps meet user needs. The feature tracking point set determined based on the floating body feature detection area determined by the user's box selection command eliminates background interference in the image, resulting in higher robustness and accuracy, thereby improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0079] For example, the step S320 above, which tracks the feature tracking point set corresponding to the first frame image and determines the feature tracking point set corresponding to each frame image after the first frame image, includes:

[0080] The pyramid Lucas-Cannard optical flow method is used to track the feature tracking point set corresponding to the previous frame image forward to the current frame image, thereby obtaining the predicted position of the feature tracking point set in the current frame image.

[0081] The Lucas-Cannard optical flow method is used to reverse track the predicted position and obtain the backtracking position of the feature tracking point set in the previous frame image.

[0082] The feature tracking point set corresponding to the current frame image is determined based on the predicted position of the feature tracking point set in the current frame image and the backtracking position of the feature tracking point set in the previous frame image.

[0083] For example, the backtracking position includes backtracking coordinates, and the predicted position includes predicted coordinates. Using the pyramid Lucas-Cannard optical flow method, the feature tracking point set corresponding to the previous frame image is traced forward from the previous frame image to the current frame image. The resulting feature tracking point set is used to track a certain feature tracking point at the [missing information - likely a specific position or location]. The predicted coordinates of the frame (i.e., the current frame, where t is a natural number) are ( , ), and perform reverse tracking on the predicted coordinates to obtain the feature tracking point at the , The backtracking coordinates of the previous frame image are ( , Based on predicted coordinates ( , ) and backtracking coordinates ( , ), determine the feature tracking point corresponding to the (t+1)th frame, and multiple feature tracking points determine the feature tracking point set.

[0084] In this embodiment, the pyramid Lucas-Cannard optical flow method is used for forward tracking, and then reverse tracking is used for reverse verification, thus achieving bidirectional consistency verification. This tracking method has strong robustness and avoids tracking failures caused by excessive motion amplitude of the floating body at sea or by image noise and local occlusion. It improves the robustness of the feature tracking point set, thereby improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0085] In this embodiment, the implementation of the pyramid Lucas-Cannard optical flow method includes:

[0086] The constant brightness is assumed to be:

[0087]

[0088] in, To track the direction, the frame number of the previous frame, For feature tracking points at the th The x-coordinate of the frame image, For feature tracking points at the th The vertical coordinate of the frame image, This is the inter-frame time interval. To track the frame number of the next frame in the direction of travel, For feature tracking points at the th Frame relative to the first The change in the horizontal coordinate of a frame image. For feature tracking points at the th Frame relative to the first The change in the vertical coordinate of a frame image. For feature tracking points at the th The x-coordinate of the frame image, For feature tracking points at the th The vertical coordinate of the frame image, For feature tracking points at the th The grayscale function of a frame image. For feature tracking points at the th The grayscale function of a frame image.

[0089] Expanding the above equations using Taylor and ignoring higher-order terms, we obtain the constraint equations:

[0090]

[0091]

[0092] in, It is the first Frame relative to the first The gradient of the frame image in the horizontal, vertical, and temporal directions. For the first Frame relative to the first The optical flow velocity of a frame image in the horizontal direction. For the first Frame relative to the first The optical flow velocity of a frame image along the vertical axis.

[0093] Based on the above formula, the predicted coordinates obtained from forward tracking or the backtracking coordinates obtained from backward tracking can be calculated as follows: , ).

[0094] In this embodiment, the mathematical model based on the pyramid Lucas-Cannard optical flow method can ensure the robustness and accuracy of feature tracking.

[0095] For example, determining the feature tracking point set corresponding to the current frame image based on the predicted position of the feature tracking point set in the current frame image and the backtracking position of the feature tracking point set in the previous frame image includes:

[0096] Calculate the Euclidean distance between the predicted position of the feature tracking point set in the current frame image and the backtracking position of the feature tracking point set in the previous frame image;

[0097] Based on Euclidean distance, the corresponding feature tracking points are determined to be either abnormal or normal points.

[0098] Remove the predicted positions of outliers in the feature tracking point set in the current frame image, and determine the predicted positions of normal points in the feature tracking point set in the current frame image as the feature tracking point set corresponding to the current frame image.

[0099] For example, a certain feature tracking point in the feature tracking point set is at the th... The predicted coordinates of the current frame image are ( , The feature tracking point is at the 1st The backtracking coordinates of the previous frame image are ( , The Euclidean distance between the two can be calculated using the following formula:

[0100]

[0101] In this embodiment, the robustness of feature tracking is quantified by calculating the Euclidean distance between the predicted position and the backtracking position. The accuracy of abnormal or normal points determined based on the Euclidean distance is high. By removing abnormal points and retaining normal points, the robustness and accuracy of the feature tracking point set corresponding to the current frame image are improved, thereby improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0102] For example, based on the Euclidean distance of each feature tracking point in the feature tracking point set corresponding to the previous frame image, determining whether the corresponding feature tracking point is an anomaly or a normal point includes:

[0103] When the Euclidean distance is greater than or equal to the preset distance, the corresponding feature tracking point is identified as an anomaly.

[0104] When the Euclidean distance is less than the preset distance, the corresponding feature tracking point is determined to be a normal point.

[0105] In actual engineering, the preset distance can be adjusted according to actual needs. For example, the preset distance can be 2 pixels, 3 pixels or 4 pixels, and so on.

[0106] In this embodiment, the criteria for judging normal or abnormal points are further refined, and the robustness of feature tracking is quantified, thereby improving the robustness and accuracy of the feature tracking point set corresponding to the current frame image, and further improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0107] For example, the step S320 above, which tracks the feature tracking point set corresponding to the first frame image and determines the feature tracking point set corresponding to each frame image after the first frame image, further includes:

[0108] Determine the number of feature tracking points in the feature tracking point set corresponding to the current frame image.

[0109] When the number of feature tracking points is less than a preset threshold, the feature tracking point set corresponding to the current frame image is re-detected in the floating body feature detection area of ​​the current frame image.

[0110] In practical engineering, the preset quantity threshold can be determined based on the minimum required number of feature points estimated by the motion of the floating body at sea. For example, the preset quantity threshold can be set to 20, 25 or 30, which will not be listed here.

[0111] In this embodiment, the number of feature tracking points in the feature tracking point set is monitored in real time. When the number is lower than the preset threshold, it indicates that problems such as target occlusion, target rotation, and sudden changes in illumination may have been encountered. At this time, the robustness and accuracy of tracking cannot be guaranteed. The feature tracking point set is re-detected, and new feature tracking points are injected into the original feature tracking point set to improve the robustness and accuracy of tracking, thereby improving the robustness and accuracy of the calculation of the six degrees of freedom parameters of the floating body at sea.

[0112] In the floating body feature detection region of the current frame image, re-detect the corresponding feature tracking point set of the current frame image, including:

[0113] The first priority detection method is executed, namely: in the floating body feature detection region of the current frame of the video, the minimum feature value of grayscale change of each pixel in the current frame image when moving in all directions within a preset window is calculated; the set of pixels with minimum feature values ​​greater than or equal to preset feature values ​​is determined as the feature tracking point set corresponding to the current frame image; or, in the floating body feature detection region of the current frame of the video, the second-order moment matrix of the image gradient of each pixel in the current frame image within a preset window is calculated; the corner response value corresponding to the pixel is calculated based on the second-order moment matrix; and the pixels with corner response values ​​greater than or equal to a preset response threshold are determined as the feature tracking point set corresponding to the current frame image.

[0114] If the number of feature tracking points in the feature tracking point set corresponding to the current frame image is still less than the preset threshold after the first priority detection method is executed, the second priority detection method is executed. That is, in the floating body feature detection area in the current frame of the video, the scale space is constructed and the approximate value of the Hessian matrix determinant of the pixel is calculated; the pixel points whose scale and determinant approximate values ​​meet the preset conditions are determined as the feature tracking point set corresponding to the current frame image.

[0115] If, after executing the second-priority detection method, the number of feature tracking points in the feature tracking point set corresponding to the current frame image is still less than a preset threshold, the third-priority detection method is executed. That is, in the floating body feature detection area of ​​the current frame of the video, the gray value of each pixel in the current frame image is compared with the gray value of the pixels in the surrounding circular neighborhood; the pixels whose absolute value of the difference between their gray value and the gray value of the pixels in the surrounding circular neighborhood is greater than or equal to the preset gray value are determined as the feature tracking point set corresponding to the current frame image.

[0116] If, after executing the third priority detection method, the number of feature tracking points in the feature tracking point set corresponding to the current frame image is still less than the preset number threshold, the preset number threshold is lowered, and the first priority detection method is executed again.

[0117] In this embodiment, the first priority corner detection method, which minimizes gray-level variance, or the method based on calculating corner response values ​​using the gradient second-order moment matrix, is preferred for feature tracking. This helps to select feature tracking points with uniform distribution and higher robustness. If the first priority detection method cannot meet the preset number threshold, a second priority method based on scale-invariant feature transformation is used for supplementary detection, detecting more discriminative feature tracking points and exhibiting better detection performance for rotation and scaling of floating bodies at sea. If the second priority detection method still cannot meet the preset number threshold, a third priority method based on circular neighborhood gray-level comparison is used for supplementary detection. This method can quickly detect high-contrast points in areas with less texture, serving as a backup detection method. When none of the first, second, or third priority detection methods can meet the preset number threshold, it indicates that the preset number threshold is not suitable for the current scenario, and the preset number threshold needs to be reduced to improve the success rate of feature tracking. In this way, it can adapt to situations with rich surface textures on floating bodies at sea, as well as situations with relatively smooth surface textures, automatically selecting the most effective detection method in the current scenario, thus improving the robustness and adaptability of feature tracking.

[0118] In one embodiment, the six degrees of freedom motion parameters include heel displacement, pitch displacement, sway displacement, roll angle, pitch angle, and bow angle; the determination of the six degrees of freedom motion parameters of the floating body based on the feature tracking point set in step S230 above includes at least one of the following:

[0119] Based on the spatial distribution changes of the feature tracking point set corresponding to adjacent frame images, the roll and pitch angles of the floating body at sea are determined.

[0120] Based on the correspondence of feature tracking point sets corresponding to adjacent frame images, the sway displacement, roll displacement, heave displacement, and bow angle of the floating body at sea are determined.

[0121] It should be noted that a right-handed Cartesian coordinate system is established, fixed to the image plane and the camera, to describe the motion of the floating object at sea. From the camera's perspective, the horizontal axis ( The positive direction of the vertical axis is horizontal to the right. The sway displacement represents the translation of the floating object along the horizontal plane; positive values ​​indicate movement to the right, and negative values ​​indicate movement to the left. From the camera's perspective, the vertical axis (…) The positive direction of the optical axis is vertically downwards. Sway displacement represents the vertical translation of a floating object along the horizontal plane; positive values ​​indicate downward movement, and negative values ​​indicate upward movement. (Camera optical axis) The direction of the x-axis points to the distance between the scene and the camera; a positive direction indicates movement away from the camera. Heave displacement represents the movement of the floating object along the camera's optical axis; a positive value represents movement away from the camera, and a negative value represents movement towards the camera. The definitions of roll, pitch, and yaw angles follow the right-hand rule: the roll angle represents the movement of the floating object around the x-axis (the horizontal axis of the camera). The rotation angle of the horizontal axis, holding the horizontal axis ( The positive direction of the roll angle is indicated by the vertical axis (the direction in which the other four fingers bend), and the opposite direction is indicated by the vertical axis (the direction in which the other four fingers bend). The pitch angle represents the direction of the floating body around the vertical axis (the direction in which the other four fingers bend). The rotation angle of the vertical axis (axis), hold the vertical axis (axis) The positive direction of the pitch angle is indicated by the optical axis of the camera, and the direction in which the other four fingers bend is the positive direction of the pitch angle. The opposite direction of the bending of the other four fingers is the negative direction of the pitch angle. The pitch angle represents the direction of the floating body around the camera's optical axis (the direction of the camera's optical axis). The rotation angle of the axis, hold the camera optical axis ( The positive direction of the yaw angle is the direction in which the other four fingers bend, and the opposite direction is the opposite direction of the yaw angle.

[0122] In this embodiment, by analyzing the spatial distribution changes of the feature tracking point sets corresponding to adjacent frame images, the tilt attitude of the floating body at sea can be perceived, thereby determining the roll and pitch angles of the floating body. By analyzing the correspondence between the feature tracking point sets corresponding to adjacent frame images, the planar motion, axial near and far motion, and horizontal rotation of the floating body at sea can be perceived, thereby determining the sway displacement, heave displacement, and bow angle of the floating body at sea. In this way, the motion parameters are expanded from two-dimensional two-degree-of-freedom motion parameters (sway displacement and heave displacement) to three-dimensional six-degree-of-freedom motion parameters, improving the completeness of the motion information measurement of the floating body at sea.

[0123] In one embodiment, such as Figure 4 As shown, the step above, determining the roll and pitch angles of the floating body at sea based on the spatial distribution changes of the feature tracking point set corresponding to adjacent frame images, includes:

[0124] S410. Based on the feature tracking point set corresponding to adjacent frame images, determine the covariance matrix corresponding to adjacent frame images. The covariance matrix corresponding to adjacent frame images represents the spatial distribution change of the feature tracking point set corresponding to adjacent frame images.

[0125] S420. Determine the roll and pitch angles of the floating body at sea based on the covariance matrix corresponding to adjacent frame images.

[0126] In this embodiment, the covariance matrix determined by the feature tracking point set corresponding to adjacent frame images can accurately quantify the spatial distribution change of the feature tracking point set corresponding to adjacent frame images, and extract the feature evolution related to the three-dimensional tilt of the floating body from the two-dimensional feature tracking point set. The roll and pitch angles of the floating body obtained in this way are more accurate.

[0127] For example, in step S410 above, determining the covariance matrix corresponding to adjacent frame images based on the feature tracking point set corresponding to adjacent frame images, where the covariance matrix represents the spatial distribution change of the feature tracking point set corresponding to adjacent frame images, includes:

[0128] The feature tracking point set corresponding to each frame of the image is:

[0129]

[0130]

[0131] in, Assign frame numbers to the image. For the first The set of feature tracking points corresponding to the frame image The feature tracking point number is the feature tracking point number in the feature tracking point set. The number of feature tracking points in the feature tracking point set. For the first The first frame of the feature tracking point set corresponding to the frame image One feature tracking point, For the first The feature tracking point at the th ... The x-coordinate of the position in the frame image. For the first The feature tracking point at the th ... The vertical coordinate of the position in the frame image.

[0132] Calculate the two-dimensional mean of the feature tracking point set corresponding to each frame of the image:

[0133]

[0134]

[0135]

[0136] in, For the first Feature tracking point set corresponding to frame image The two-dimensional mean, For the first Feature tracking point set corresponding to frame image The mean of the horizontal axis, For the first Feature tracking point set corresponding to frame image The mean of the vertical axis.

[0137] Calculate the covariance matrix for each frame of the image:

[0138]

[0139]

[0140]

[0141]

[0142] in, For the first The covariance matrix corresponding to the frame image For the first Feature tracking point set corresponding to frame image The variance of the x-axis, For the first Feature tracking point set corresponding to frame image The variance of the ordinate, For the first Feature tracking point set corresponding to frame image The covariance of the x-coordinate and y-coordinate.

[0143] In this embodiment, the covariance matrix corresponding to each frame of the image is quantized. Characterizes the degree of dispersion of the feature tracking point set in the vertical direction of the image. Characterizes the degree of dispersion of the feature tracking point set in the horizontal direction of the image. The covariance matrix of each frame image represents the spatial distribution of the corresponding feature tracking point set in the horizontal and vertical directions. Therefore, the covariance matrix of each frame image can represent the spatial distribution of the corresponding feature tracking point set. This allows the covariance matrix of adjacent frames image to represent the spatial distribution changes of the feature tracking point set in adjacent frames image, thus improving the accuracy and reliability of the roll and pitch angles of the floating body at sea.

[0144] In one embodiment, such as Figure 5 As shown, step S420 above, determining the roll and pitch angles of the floating body based on the covariance matrix corresponding to adjacent frame images, includes:

[0145] S510. Determine the roll and pitch angles of the floating body at sea based on the relative changes in the diagonal elements of the covariance matrix corresponding to adjacent frames.

[0146] S520. Determine the direction of the roll and pitch angles of the floating body at sea based on the signs of the non-main diagonal elements of the covariance matrix corresponding to adjacent frame images.

[0147] In this embodiment, the main diagonal elements of the covariance matrix represent the dispersion of the feature tracking point set in the horizontal or vertical direction. When the roll or pitch angle of the floating body changes, the main diagonal elements change accordingly. Therefore, the relative change in the main diagonal elements of the covariance matrix corresponding to adjacent frames can reflect the magnitude of the roll and pitch angles. The off-diagonal elements of the covariance matrix represent the correlation of the feature tracking point set in the horizontal and vertical directions. When the direction of the roll or pitch angle of the floating body changes, the sign of the off-diagonal elements changes accordingly. Therefore, the sign of the off-diagonal elements of the covariance matrix corresponding to adjacent frames can reflect the direction of the roll and pitch angles. Thus, the complex attitude perception problem is transformed into the calculation of matrix elements, thereby making the acquired roll and pitch angles of the floating body more accurate.

[0148] In one embodiment, the step S510 above, which determines the roll and pitch angles of the floating body based on the relative changes in the main diagonal elements of the covariance matrix corresponding to adjacent frame images, includes:

[0149] The roll angle of the floating body at sea is determined by the relative change of the first diagonal element of the covariance matrix corresponding to adjacent frames, and the pitch angle of the floating body at sea is determined by the relative change of the second diagonal element of the covariance matrix corresponding to adjacent frames. The first diagonal element represents the degree of dispersion of the feature tracking point set in the vertical direction of the image, and the second diagonal element represents the degree of dispersion of the feature tracking point set in the horizontal direction of the image.

[0150] The step S520 above, which determines the direction of the roll and pitch angles of the floating body based on the signs of the non-main diagonal elements of the covariance matrix corresponding to adjacent frame images, includes:

[0151] Based on the signs of the non-main diagonal elements of the covariance matrix corresponding to adjacent frames, the directions of the roll and pitch angles of the floating body at sea are determined. The non-main diagonal elements represent the distribution correlation of the feature tracking point set in the horizontal and vertical directions of the image.

[0152] In this embodiment, the first main diagonal element is explicitly designated as the main diagonal element for determining the roll angle magnitude. This is because the first main diagonal element represents the dispersion of the feature tracking point set in the vertical direction of the image. The second main diagonal element is explicitly designated as the main diagonal element for determining the pitch angle magnitude. This is because the second main diagonal element represents the dispersion of the feature tracking point set in the vertical direction of the image. The directions of the roll and pitch angles are determined based on the signs of the non-main diagonal elements. This is because the non-main diagonal elements represent the correlation of the feature tracking point set's distribution in the horizontal and vertical directions of the image. Thus, quantitative calculation of the roll and pitch angles is achieved, which helps to make the calculation of the roll and pitch angles more accurate.

[0153] For example, the determination of the roll angle of the floating body at sea based on the relative change of the first main diagonal elements of the covariance matrix corresponding to adjacent frame images in the above steps is made using the following formula:

[0154]

[0155] in, The size of the roll angle, For the first The variance of the ordinate of the feature tracking point set corresponding to the frame image. For the first The variance of the ordinate of the feature tracking point set corresponding to the frame image.

[0156] The pitch angle of the floating body at sea is determined in the above steps based on the relative changes in the second main diagonal elements of the covariance matrix corresponding to adjacent frames, using the following formula:

[0157]

[0158] in, The size of the pitch angle For the first The variance of the x-coordinates of the feature tracking point set corresponding to the frame image. For the first The variance of the x-coordinates of the feature tracking point set corresponding to the frame image.

[0159] The step described above, determining the direction of the roll and pitch angles of the floating body based on the signs of the non-main diagonal elements of the covariance matrix corresponding to adjacent frames, includes:

[0160] when At that time, the roll angle of the floating body at sea is determined as the first direction, and the pitch angle as the second direction.

[0161] when At that time, the roll angle of the floating body at sea is determined as the third direction, and the pitch angle as the fourth direction.

[0162] in, The symbols represent positive and negative signs; For the first The covariance of the x and y coordinates of the feature tracking point set corresponding to the frame image. The first direction is opposite to the third direction, and the second direction is opposite to the fourth direction.

[0163] In practical engineering, the first direction, second direction, third direction, and fourth direction can be defined according to actual needs. For example, the first direction is the positive direction of rotation around the horizontal axis, the third direction is the negative direction of rotation around the horizontal axis, the second direction is the positive direction of rotation around the vertical axis, and the fourth direction is the negative direction of rotation around the vertical axis.

[0164] In this embodiment, the covariance matrix of the feature tracking point set corresponding to adjacent frame images is mapped to the roll angle and pitch angle in the above manner, realizing the quantitative calculation of the roll angle and pitch angle, which is beneficial to make the calculation of the roll angle and pitch angle more accurate.

[0165] In one embodiment, such as Figure 6 As shown, the steps described above, which determine the pitch displacement, sway displacement, heave displacement, and bow angle of the floating body based on the correspondence of feature tracking point sets corresponding to adjacent frame images, include:

[0166] S610. Based on the correspondence between the feature tracking point sets corresponding to adjacent frame images, determine the similarity transformation model corresponding to adjacent frame images.

[0167] S620. Based on the similarity transformation model, determine the longitudinal sway displacement, transverse sway displacement, heave displacement, and bow angle of the floating body at sea.

[0168] In this embodiment, based on the correspondence between feature tracking point sets corresponding to adjacent frame images, a similarity transformation model containing translation, rotation, and scaling information is determined, thereby enabling the determination of sway displacement, heave displacement, and yaw angle. Furthermore, the fitting process of this similarity transformation model can suppress noise interference, improving the robustness and accuracy of the calculation of sway displacement, heave displacement, and yaw angle.

[0169] For example, the correspondence in step S610 above includes a positional correspondence. The step S610 of determining the similarity transformation model corresponding to adjacent frame images based on the correspondence of feature tracking point sets corresponding to adjacent frame images includes:

[0170] Using the random sampling consensus algorithm, a robust fitting of the positional correspondence of feature tracking point sets corresponding to adjacent frames is performed using the similarity transformation model as the geometric model, and the fitted similarity transformation model is output.

[0171] In this embodiment, a random sampling consensus algorithm is introduced. Its core mechanism is to generate hypotheses through random sampling and then verify them using all the data. This avoids feature tracking point matching errors caused by wave obstruction, droplet interference, or calculation errors. Based on this algorithm, a similarity transformation model is used as the geometric model to robustly fit the positional correspondence of feature tracking point sets in adjacent frames. This ensures that the final similarity transformation model can reflect the true motion of the floating body at sea, improving the accuracy and robustness of the calculation of pitch displacement, sway displacement, heave displacement, and bow angle.

[0172] For example, the above steps, using the random sampling consensus algorithm to robustly fit the positional correspondence of feature tracking point sets corresponding to adjacent frame images with the similarity transformation model as the geometric model, and outputting the fitted similarity transformation model, include:

[0173] The positional correspondence of feature tracking point sets corresponding to adjacent frames is used as matching point pairs. Iterative operations are performed as follows: From all matching point pairs, a first number of matching point pairs are randomly selected as the minimum sample set; a candidate similarity transformation model is calculated based on the minimum sample set; the candidate similarity transformation model is used to test all matching point pairs to obtain the transformation error; matching point pairs with transformation errors less than or equal to a preset error threshold are determined as interior point pairs. The iteration operation ends when the ratio of the number of interior point pairs to all matching point pairs is greater than or equal to a preset ratio, or when the number of iterations equals a preset threshold.

[0174] When the iteration operation ends, the candidate similarity transformation model with the most interior point pairs is directly used as the fitted similarity transformation model, or the candidate similarity transformation model with the most interior point pairs is used as the fitted similarity transformation model.

[0175] In practical engineering, the number of all matching point pairs, the initial quantity, the preset error threshold, and the preset ratio can be adjusted according to actual needs. For example, the number of all matching point pairs can be 1000, 1500, or 2000, and the initial quantity can be 2, 3, or 4. The preset error threshold can be 1.5 pixels, 2 pixels, or 2.5 pixels, and the preset ratio can be 50%, 55%, or 60%.

[0176] The above steps involve testing all matching point pairs using a candidate similarity transformation model to obtain the transformation error, including:

[0177] The actual coordinates of the feature tracking points corresponding to the previous frame image in the matching point pair are transformed to the coordinate system of the current frame image through the candidate similarity transformation model to obtain the transformed coordinates.

[0178] The Euclidean distance between the transformed coordinates and the actual coordinates corresponding to the current frame image is used as the transformation error.

[0179] In this embodiment, candidate similarity transformation models are generated by iteratively sampling matching point pairs, and then all matching point pairs are used to verify these candidate similarity transformation models. This method finds the candidate similarity transformation model with the largest number of interior point pairs, which helps improve the robustness and accuracy of the similarity transformation model. By setting an upper limit on the number of iterations and setting the ratio of the number of interior point pairs to the total number of matching point pairs, the iteration process can be terminated, thus balancing the computational efficiency and robustness of the random sampling consensus algorithm.

[0180] In one embodiment, such as Figure 7 As shown, step S620 above, which determines the pitch displacement, sway displacement, heave displacement, and bow angle of the floating body based on the similarity transformation model, includes:

[0181] S710. Analyze the similarity transformation model to obtain the translation component, rotation component, and scaling component.

[0182] S720. Determine the sway displacement and transverse displacement based on the translation component, determine the pitch angle based on the rotation component, and determine the heave displacement based on the scaling component.

[0183] In this embodiment, the translation, rotation, and scaling information contained in the similarity transformation model are analyzed into translation components, rotation components, and scaling components, respectively. The translation components are then mapped to sway displacement and roll displacement, the rotation components to pitch angle, and the scaling components to heave displacement. This achieves accurate calculation of the four motion parameters—sway displacement, roll displacement, heave displacement, and pitch angle—and improves the dimensionality of the motion parameters.

[0184] For example, in step S710 above, the similarity transformation model is analyzed to obtain translation components, rotation components, and scaling components, including:

[0185] The similarity transformation model is as follows:

[0186]

[0187] in, The transformation matrix is... For feature tracking points at the th The x-coordinate of the frame image, For feature tracking points at the th The vertical coordinate of the frame image, For feature tracking points at the th The x-coordinate of a frame image after transformation using a similarity transformation model. For feature tracking points at the th The ordinate of a frame image after transformation using a similarity transformation model. For rotational components, To scale the components, For translation components, The translation component is along the horizontal axis. This represents the translation component along the vertical axis.

[0188] In practical engineering, rotational component Scaled components Translation components Both can be derived from the transformation matrix. The result is extracted as shown below:

[0189]

[0190]

[0191]

[0192]

[0193] In this embodiment, rotational components, scaling components, and translational components can be accurately extracted from the transformation matrix in the similar transformation model, thereby more accurately determining the sway displacement, lateral displacement, heave displacement, and pitch angle.

[0194] For example, the step S720 above, which involves determining the sway displacement and the transverse displacement based on the translation components, includes:

[0195] The translation component along the horizontal axis is taken as the sway displacement, and the translation component along the vertical axis is taken as the sway displacement. That is, the translation component along the horizontal axis is taken as the sway displacement. As a longitudinal displacement, As a sway displacement.

[0196] In this embodiment, the translation component along the horizontal axis extracted from the transformation matrix is ​​directly used as the sway displacement, and the translation component along the vertical axis is used as the sway displacement, which helps to improve the accuracy of the sway displacement and the sway displacement.

[0197] For example, determining the bow roll angle based on the rotation component in step S720 above includes:

[0198] The rotational component is used as the pitch angle. That is, the rotational component is used as the pitch angle. As the bow angle.

[0199] In this embodiment, the rotation component extracted from the transformation matrix is ​​directly used as the pitch angle, which helps to improve the accuracy of the pitch angle.

[0200] In one embodiment, the scaling component includes a scaling factor; the analytical similarity transformation model in step S710 above, to obtain the scaling component, includes:

[0201] The scaling factor is obtained by analyzing the similarity transformation model.

[0202] Determining the heave displacement based on the scaling component in step S720 above includes:

[0203] The heave displacement is determined based on the scaling factor and the preset mapping function.

[0204] In this embodiment, the scaling component is limited to a scaling factor, and a quantitative relationship between the scaling component and the heave displacement is established based on the scaling factor and a preset mapping function. This enables the accurate calculation of four motion parameters: sway displacement, roll displacement, heave displacement, and pitch angle, thereby improving the dimensionality of the motion parameters.

[0205] For example, the analytical similarity transformation model in the above steps to obtain the scaling factor includes: using the scaling component of the similarity transformation model as the scaling factor. That is, using s as the scaling factor.

[0206] The step described above, determining the heave displacement based on the scaling factor and the preset mapping function, includes:

[0207] Input the scaling factor into the following preset mapping function to obtain the heave displacement:

[0208]

[0209] in, For heave displacement, For mapping coefficients, For scaling components.

[0210] In this embodiment, the accuracy of the output heave displacement can be improved by using the scaling component extracted from the transformation matrix as a scaling factor and inputting a preset mapping function.

[0211] For example, after determining the six-degree-of-freedom motion parameters of the floating body based on the feature tracking point set in step S230 above, the method further includes:

[0212] The six-degree-of-freedom motion parameters are denoised to eliminate high-frequency noise signals.

[0213] In this embodiment, high-frequency noise caused by factors such as water splash interference is removed from the six degrees of freedom parameters, making the overall motion trend formed by the noise-reduced six degrees of freedom motion parameters more continuous and smooth, better representing the real motion trend of the floating body at sea, and improving the output robustness under complex sea conditions.

[0214] For example, the denoising process for the six-degree-of-freedom motion parameters in the above steps includes:

[0215] A first-order infinite impulse response low-pass filter is used to smooth the original motion sequence of each degree of freedom motion parameter in real time, so as to obtain the denoised six-degree-of-freedom motion parameters.

[0216] The filter coefficients of the first-order infinite impulse response low-pass filter are determined based on the motion frequency of the ocean waves.

[0217] In practical engineering, the transfer function of a first-order infinite impulse response low-pass filter includes:

[0218]

[0219] in, The number of frames in the video. For the first The six-DOF motion parameters after frame denoising. For the first The six degrees of freedom motion parameters before frame denoising. For the first The six-DOF motion parameters after frame denoising. These are the filter coefficients. The value range is [0, 1].

[0220] The filter coefficients are determined based on the motion frequency of ocean waves, including:

[0221]

[0222] in, This is the equivalent continuous-time cutoff frequency of a first-order infinite impulse response low-pass filter. The sampling frequency (i.e., video frame rate). Based on the frequency of ocean wave movement Determined, that is, Higher than , The value range is [0.5Hz, 1Hz].

[0223] For example, when =0.5Hz =30Hz ≈0.094.

[0224] In this embodiment, a first-order infinite impulse response (IRR) low-pass filter is used for smoothing, ensuring robustness for long-term continuous measurements. Determining the filter coefficients of the IRR low-pass filter by the motion frequency of ocean waves allows the filter to adapt to different sea conditions, thereby improving the method's versatility.

[0225] For example, after determining the six-degree-of-freedom motion parameters of the floating body based on the feature tracking point set in step S230 above, the method further includes:

[0226] The six-degree-of-freedom parameters are input into the digital twin model of the floating body at sea, so that the digital twin model can reproduce the actual motion attitude of the floating body at sea.

[0227] For example, the step of inputting the six-degree-of-freedom parameters into the digital twin model of the floating body at sea includes:

[0228] Import the 3D model of the floating body at sea as a digital twin model of the floating body.

[0229] Align and register the local coordinate system in the 3D model with the coordinate system defined by the camera.

[0230] The six-degree-of-freedom motion parameters, acquired in real time or after smoothing, are converted into translation vectors and rotation matrices required to drive the 3D model, so that the translation vectors and rotation matrices drive the motion of the digital twin model. The translation vectors include sway, pitch, and heave displacements, and the rotation matrices include roll, pitch, and yaw angles.

[0231] In this embodiment, the abstract six-degree-of-freedom motion parameter data of the floating body at sea is visualized, which can intuitively reproduce the motion attitude of the floating body at sea.

[0232] For example, after determining the six-degree-of-freedom motion parameters of the floating body based on the feature tracking point set in step S230 above, at least one of the following is also included:

[0233] Visualize the video and feature tracking point set.

[0234] Visualize the digital twin model of a floating body at sea.

[0235] Visualize the motion timeline curves of the six-degree-of-freedom parameters.

[0236] Visualize the curve showing how the number of effective feature points changes over time.

[0237] Visualize the frequency domain analysis plot of the heave displacement after Fast Fourier Transform.

[0238] In this embodiment, visualizing the video and feature tracking point set provides a clear overview of the feature tracking process. Visualizing the digital twin model of the floating body allows for a direct reproduction of its motion attitude. Visualizing the motion time-series curves of the six degrees of freedom parameters and the curves showing the number of effective feature points changing over time can be used to evaluate the quality of the measurement method. Visualizing the frequency domain analysis graph of the heave displacement after Fast Fourier Transform allows for the analysis of the dominant frequency of the heave displacement, which can be used for wave characteristic analysis and the study of device resonance. This enhances the user experience and enriches the application scenarios of the measurement method.

[0239] In one exemplary embodiment, a computer device is provided, including a processor and a memory, the memory storing a computer program, the processor executing the computer program to implement the steps of any of the above-described methods for measuring the motion of a floating body at sea.

[0240] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described methods for measuring the motion of a floating body at sea. The computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage device, etc.

[0241] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of any of the above-described methods for measuring the motion of a floating body at sea.

[0242] refer to Figure 8 The following is a structural block diagram of a computer device that can serve as the camera 2 or terminal 1 of the present invention. The computer device 100 includes a computing unit 101, which can perform various appropriate actions and processes according to a computer program stored in a ROM 102 or a computer program loaded from a storage unit 108 into a RAM 103. The RAM 103 may also store various programs and data required for the operation of the computer device 100. The computing unit 101, ROM 102, and RAM 103 are interconnected via a bus 104. An input / output (I / O) interface 105 is also connected to the bus 104.

[0243] Multiple components in computer device 100 are connected to I / O interface 105, including: input unit 106, output unit 107, storage unit 108, and communication unit 109. Input unit 106 can be any type of device capable of inputting information into computer device 100. Input unit 106 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of computer device 100, and may include, but is not limited to, a mouse, keyboard, touchscreen, trackpad, trackball, joystick, microphone, and / or remote control. Output unit 107 can be any type of device capable of presenting information, and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 108 may include, but is not limited to, a hard disk and an optical disk. Communication unit 109 allows computer device 100 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0244] The computing unit 101 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 101 performs the various methods and processes described above, such as the method for measuring the motion of a floating body at sea. For example, in some embodiments, the method for measuring the motion of a floating body at sea may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 108. In some embodiments, part or all of the computer program may be loaded and / or installed on the computer device 100 via ROM 102 and / or communication unit 109. When the computer program is loaded into RAM 103 and executed by the computing unit 101, one or more steps of the method for measuring the motion of a floating body at sea described above may be performed. Alternatively, in other embodiments, the computing unit 101 may be configured by any other suitable means (e.g., by means of firmware) to perform a method for measuring the motion of a floating body at sea.

[0245] The computer device 100 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method for measuring the motion of floating bodies at sea.

[0246] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein.

[0247] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for measuring the motion of a floating body at sea, characterized in that, The measurement method includes: To obtain video footage of floating objects moving on the water. Determine a set of feature tracking points corresponding to each frame of the video, the set of feature tracking points being associated with the floating body at sea; Based on the feature tracking point set corresponding to adjacent frame images, the covariance matrix corresponding to adjacent frame images is determined. The covariance matrix corresponding to adjacent frame images represents the spatial distribution change of the feature tracking point set corresponding to adjacent frame images. The roll and pitch angles of the floating body at sea are determined based on the relative changes in the main diagonal elements of the covariance matrix corresponding to adjacent frames. The roll angle of the floating body is determined based on the relative changes in the first diagonal elements of the covariance matrix corresponding to adjacent frames, and the pitch angle is determined based on the relative changes in the second diagonal elements of the covariance matrix corresponding to adjacent frames. The first diagonal elements characterize the dispersion of the feature tracking point set in the vertical direction of the image, and the second diagonal elements characterize the dispersion of the feature tracking point set in the horizontal direction of the image. The directions of the roll and pitch angles of the floating body are determined based on the signs of the off-diagonal elements of the covariance matrix corresponding to adjacent frames. The off-diagonal elements characterize the distribution correlation of the feature tracking point set in the horizontal and vertical directions of the image. Based on the correspondence between the feature tracking point sets corresponding to adjacent frame images, the similarity transformation model corresponding to adjacent frame images is determined; The similarity transformation model is analyzed to obtain the translation component, rotation component, and scaling component. The sway displacement and lateral sway displacement are determined based on the translation component, the pitch angle is determined based on the rotation component, and the heave displacement is determined based on the scaling component.

2. The measurement method according to claim 1, characterized in that, The scaling component includes a scaling factor; by analyzing the similarity transformation model, the scaling component is obtained, including: The scaling factor is obtained by analyzing the similarity transformation model. Determining the heave displacement based on the scaling component includes: The heave displacement is determined based on the scaling factor and the preset mapping function.

3. The measurement method according to claim 1 or 2, characterized in that, The determination of the feature tracking point set corresponding to each frame of the video includes: Determine the set of feature tracking points corresponding to the first frame of the video; The feature tracking point set corresponding to the first frame image is tracked, and the feature tracking point set corresponding to each frame image after the first frame image is determined frame by frame.

4. The measurement method according to claim 3, characterized in that, Determining the feature tracking point set corresponding to the first frame of the video includes: In the floating body feature detection region of the first frame of the video, the set of feature tracking points corresponding to the first frame image is detected.

Images