Coplanar target pose measurement method fusing image and point cloud

By fusing image and point cloud data, defining nonlinear projection model and error function, solving Euler angle and coupling position, the problem of degradation of coplanar target pose measurement accuracy is solved, and high-precision pose estimation is achieved, suitable for navigation field.

CN119006597APending Publication Date: 2024-11-22BEIHANG UNIV

Patent Information

Application Number
CN202411109140.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing coplanar target pose measurement methods based on point clouds are easily trapped in local optimization, resulting in a decrease in measurement accuracy and making it difficult to effectively use image and point cloud data to improve the coplanar target pose measurement accuracy.

Method used

A coplanar target pose measurement method that combines images and point clouds is designed. By receiving the coplanar target grayscale image and point cloud data obtained by the depth camera, the nonlinear projection model and error function are defined, and the Euler angle and coupling are solved. position, and then calculate the target final position.

Benefits of technology

The target position estimation accuracy is improved and the measurement accuracy in the navigation field is enhanced. The simulation results show that the posture estimation accuracy is less than 0.02 degrees and the relative position proportion error is less than 0.02%.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006597A_ABST
    Figure CN119006597A_ABST
Patent Text Reader

Abstract

The invention relates to a coplanar target pose measurement method fusing an image and a point cloud, and the method comprises the following steps: receiving a coplanar target grayscale image and feature point cloud data obtained by a depth camera at the same time, and defining a nonlinear projection model of the image and point cloud data from a three-dimensional world coordinate to a two-dimensional image coordinate, defining a grayscale image feature point error function and a point cloud error function; solving a point cloud error function according to the two feature point cloud data, obtaining a normal vector of a plane where the coplanar target is located, and obtaining an Euler angle from the normal vector; solving an image error function according to the image feature points to obtain a target coupling position; and according to the Euler angle and the target coupling position, the final pose of the target is solved. According to the method, the pose measurement precision of the coplanar target is improved by fusing the image and the point cloud data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the field of target positioning system design in the field of vision, and in particular relates to a coplanar target posture measurement method by fusing images and point clouds. Background Art

[0002] At present, vision-based relative measurement includes determining the attitude and position of the target. Considering the application of visual pose measurement technology in many fields such as robots and spacecraft navigation, many pose solution methods have been proposed. Usually, there are many feature points on coplanar targets, and different measurement sensors can obtain different data such as images and point clouds. The existing point cloud-based pose measurement usually assumes the non-coplanar characteristics of the target point cloud data, so that the iterative pose measurement method based on point cloud can achieve high-precision pose measurement, but the pose measurement of coplanar target point cloud is easy to fall into local optimality, resulting in a decrease in measurement accuracy. Therefore, how to use different types of data to improve the pose measurement accuracy of coplanar targets and serve the navigation field is an urgent problem to be solved in the current field of visual relative measurement engineering. Summary of the invention

[0003] In the actual visual target positioning task, since different types of sensor data will bring complexity to the algorithm, the present invention provides a coplanar target pose measurement method by fusing images and point clouds, comprising the following steps: S1: Receive the coplanar target grayscale image and feature point cloud data simultaneously acquired by the depth camera, define the nonlinear projection model of the image and point cloud data from the three-dimensional world coordinates to the two-dimensional image coordinates, and define the grayscale image feature point error function and the point cloud error function; S2: Obtain the point cloud error function based on the feature point cloud data and solve the Euler angle; obtain the grayscale image feature point error function based on the grayscale image feature points and obtain the target coupling position; S3: Obtain the final target pose based on the two Euler angles and the target coupling position.

[0004] The beneficial effects of the present invention compared with the prior art are: (1) The present invention takes into account different types of data, image features and point cloud features, and designs a pose measurement framework that integrates the two features.

[0005] (2) The present invention sets a fusion of part of the point cloud data and image data, which improves the accuracy of target pose estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 A flow chart of a method for measuring coplanar target pose by fusion of images and point clouds according to an embodiment of the present invention; Figure 2 A schematic diagram of a coplanar target depth camera projection model according to an embodiment of the present invention; Figure 3 4 is a simulation result of the target pose estimation method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0007] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms, and the present disclosure should not be limited by the embodiments described herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0008] Figure 1 FIG. 1 is a flow chart of a method for measuring coplanar target pose by fusion of image and point cloud according to an embodiment of the present invention. Figure 1 As shown, the target posture measurement method includes the following steps: S1: Receive the coplanar target grayscale image and point cloud data obtained simultaneously by the depth camera, and define the nonlinear projection model of the depth camera as shown in the following equations (1) and (2), and define the grayscale image feature point error function and the point cloud error function. The nonlinear projection model of the depth camera refers to the nonlinear projection model of the image and point cloud data from the three-dimensional world coordinates to the two-dimensional image coordinates, where the three-dimensional world coordinates are the coordinates of the feature points in the world coordinate system, and the two-dimensional image coordinates are the coordinates of the feature points projected in the camera plane. An example of the coplanar target grayscale image and point cloud data obtained is as follows Figure 2 As shown, the coplanar target refers to the target feature point In the same plane, Figure 2 As shown in is the coplanar target reference coordinate system, is the depth camera reference coordinate system, Represents the attitude matrix and position vector between the two coordinate systems, and the four feature points The grayscale image of the coplanar target refers to the two-dimensional image coordinates obtained by projecting the four feature points through the nonlinear model. ,Point cloud data refers to the three-dimensional coordinates of all feature points in the plane where the coplanar target is located in the depth camera.

[0009] (1) (2) in, Represents the two-dimensional pixel coordinates of the feature point obtained through the nonlinear projection model, Represents the depth value, Indicates image intensity. Focal length and imaging center coordinates are the internal known parameters of the depth camera. and Represent the three-dimensional coordinates of the feature point in the camera coordinate system and the world coordinate system respectively. Since the target is a coplanar target, define Indicates that the target points are all in the XY plane. According to the depth value and the pixel coordinates of the feature points, the feature point cloud data can be easily obtained. , where the depth value refers to the parameter . In formula (2) Indicates the relative position of the target. represents the attitude rotation matrix and is defined as: (3) in, Indicates winding Axis rotation Euler angle The obtained rotation matrix is Indicates winding Axis rotation Euler angle The obtained rotation matrix is Indicates winding Axis rotation Euler angle The obtained rotation matrix.

[0010] (4) (5) (6) Then the grayscale image feature points and point cloud error function are defined as: (7) in, represents the point cloud error function, Represents the error function of grayscale image feature points. First, consider the coplanar target point cloud error function as: (8) The Z-axis direction vector is defined as , so define the direction vector , where R refers to the posture matrix defined in the above formula (3).

[0011] For grayscale image feature point error function , the image point Reproject onto a sphere of unit radius with the camera origin as the center to obtain a spherical point: (9) Here, t refers to the position vector.

[0012] Calculate the square of the spatial distance between the feature points on two unit virtual spheres: (10) According to the above formula, we can get Size and Related to the coupling position Related, we get the following formula: (11) , Among them, g ( ) refers to the grayscale image feature point error function, Refers to two point features and Distance error after coupled position transformation.

[0013] S2: Obtain the point cloud error function based on the feature point cloud data and solve the Euler angle; obtain the grayscale image feature point error function based on the grayscale image feature points and obtain the target coupling position. The process is as follows: To solve the error function in formula (7), first, as shown in formula (8), the vector Posture Matrix To rotate: (12) because Indicates winding Axis rotation angle , we can get Therefore, the error function in equation (8) is derived as: (13) in, They refer to the two Euler angles respectively.

[0014] According to the equation Can analyze: (14) because The least squares method and vector So we can get two Euler angles for: (15) In addition, solve the error function , through the function For variables Find the partial derivative: (16) Next, we use nonlinear optimization methods such as Levenberg-Marquardt and combine them with Iteration, solving for coupled position terms .

[0015] S3: According to the coupling position and two Euler angles Obtain the final target pose, including the definition of the spherical point in equation (8) and the coupling position , the pose can be determined by the least squares problem, the process is as follows: First, solve the rotation matrix corresponding to another Euler angle . Define the following function: (17) Since the error function Solve to get two Euler angles and , formula (17) is: (18) consider The third line is the vector , so the following matrix is ​​defined: (19) in, Representation Matrix From row 1 to row 2 and column 1 to column 2, Denotes the Euler angle. Similarly, the following definitions are given: (20) in, The vector consisting of the first and second elements representing the coupling position, Indicates the coupling position.

[0016] Therefore, through the matrix dimensionality reduction method, we can get: (twenty one) Performing singular value decomposition on the above formula, we can get the attitude matrix Therefore, we finally get The attitude matrix of the axis rotation is: (twenty two) Finally, the target pose matrix and relative position are as follows: (twenty three) in, , Represents two-dimensional pixel coordinates The number of

[0017] According to the target pose measurement method of the fusion image and point cloud data of the embodiment of the present invention, the simulation results are as follows: Figure 3 As shown, under different degrees of image noise characteristics, the pose estimation accuracy of the coplanar target is less than 0.02 degrees, and the relative position ratio error is less than 0.02%.

[0018] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0019] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments may be envisioned within the scope of the invention thus described. In addition, it should be noted that the language used in this specification is primarily selected for readability and instructional purposes, rather than for the purpose of explaining or limiting the subject matter of the present invention.

Claims

1. A coplanar target pose measurement method by fusing images and point clouds, characterized in that: The steps include: S1: Receive the coplanar target grayscale image and feature point cloud data simultaneously acquired by the depth camera, define the nonlinear projection model of the image and point cloud data from the three-dimensional world coordinates to the two-dimensional image coordinates, and define the grayscale image feature point error function and the point cloud error function; S2: Obtain the point cloud error function based on the feature point cloud data and solve the Euler angle; obtain the grayscale image feature point error function based on the grayscale image feature points and obtain the target coupling position; S3: Obtain the final target pose based on the two Euler angles and the target coupling position.

2. The method for measuring coplanar target pose by fusion image and point cloud according to claim 1, wherein: The nonlinear projection model described in step S1 is as follows: , , in, Represents the two-dimensional pixel coordinates of the feature point obtained through the nonlinear projection model, Represents the depth value, represents the image intensity, and are the internal known parameters of the depth camera, focal length and imaging center coordinates, and Respectively represent the three-dimensional coordinates of the feature point in the camera coordinate system and the world coordinate system, where It is defined to mean that all target points are in the XY plane. Indicates the relative position of the target. represents the attitude rotation matrix, where Indicates winding Axis rotation Euler angle The obtained rotation matrix is Indicates winding Axis rotation Euler angle The obtained rotation matrix is Indicates winding Axis rotation Euler angle The obtained rotation matrix; Defining the grayscale image feature point error function and the point cloud error function includes: , in, represents the point cloud error function, Represents the grayscale image feature point error function.

3. The method for measuring coplanar target pose by fusion image and point cloud according to claim 2, wherein: The step S2 of obtaining the point cloud error function according to the feature point cloud data includes the following derivation of the point cloud error function: , in, and , Representing 3D points The number of points is solved to obtain two Euler angles. ; Solving the target coupling position by nonlinear optimization method ,Nonlinear optimization methods include the Levenberg-Marquardt method.

4. The method for measuring coplanar target pose by fusion image and point cloud according to claim 3, wherein: Obtaining the final position and posture of the target according to the Euler angle and the target coupling position in step S3 includes: defining the following error function: , in, , From the above formula, we can get the angle by singular value decomposition. , so the target posture matrix and relative position are obtained as follows: , in, , Represents two-dimensional pixel coordinates The number of Represents the 3×3 identity matrix.

Citation Information

Patent Citations

  • Monocular vision pose estimation method based on coplanar target

    CN113379840A

  • Point cloud and image fused synchronous positioning and mapping method

    CN115601422A

  • Binocular vision and IMU-based underwater scene three-dimensional reconstruction method, and device

    WO2024045632A1

Cited By

  • Infrared image enhancement method and device, equipment and storage medium

    CN121504738A