Camera Pose Estimation Optimization Method, Device, Equipment and Storage Medium

Abstract

The present invention relates to the field of machine vision, and discloses an optimization method, device, equipment and storage medium for camera pose estimation. The method includes: acquiring a first image and a second image captured by a camera at two different poses and converting them into HSV images to obtain HSV image data; calculating a first projection point of a spatial point in the first image, and acquiring first color data of the first projection point according to the first projection point and the HSV image data of the first image; re-projecting the spatial point according to a preset camera pose change amount to obtain a second projection point of the spatial point in the second image, and acquiring second color data of the second projection point according to the second projection point and the HSV image data of the second image; constructing a non-linear optimization function about the camera pose change amount according to the first color data and the second color data, and optimizing the camera pose change amount according to the non-linear optimization function. The present invention can improve the accuracy and robustness of camera pose estimation.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision, and particularly to a method, device, equipment and computer-readable storage medium for optimizing camera pose estimation. Background Art

[0002] In recent years, the visual SLAM (Simultaneous Localization and Mapping) technology has been widely applied in practical application scenarios such as AGV (Automated Guided Vehicle) autonomous driving, unmanned delivery vehicles and mobile patrol monitoring vehicles, providing necessary support for path planning, obstacle avoidance and autonomous control.

[0003] The direct method is an important branch of the visual SLAM algorithm, which has the advantages of low computational complexity and relatively dense scene reconstruction. The direct method is based on the gray-scale invariance hypothesis, that is, it is assumed that the pixel gray-scale of the same spatial point is fixed in each image. On this premise, the direct method needs to first convert the image captured by the camera into a gray-scale image and then select feature points, and then construct a non-linear optimization model for minimizing the gray-scale error of the camera pose based on the feature points, and finally solve the camera pose. In actual applications, there may be a large number of situations where the colors are different but the gray-scales are almost the same in the image. For example, the white wall and the yellow railing in the same area are almost merged after being converted into a gray-scale image. If only optimized according to the gray-scale, the number of feature points may be too small and the optimization scale may be too small, resulting in unreliable or even unusable optimization results. In addition, the brightness changes introduced by the environment or the camera imaging will also greatly affect the optimization process and the determination of the final solution, resulting in poor robustness of the algorithm. Summary of the Invention

[0004] The main purpose of the present invention is to propose a method, device, equipment and computer-readable storage medium for optimizing camera pose estimation, aiming to improve the accuracy and robustness of camera pose estimation.

[0005] The first aspect of the present invention provides a method for optimizing camera pose estimation, and the method for optimizing camera pose estimation includes:

[0006] Obtain a first image and a second image captured by a camera at two different poses, convert the first image and the second image into HSV images to obtain HSV image data;

[0007] Calculate the first projection points of a plurality of known spatial points in the first image, and obtain the first color data of the first projection points according to the first projection points and the HSV image data of the first image;

[0008] Reproject the spatial point according to a preset camera pose change amount to obtain a second projection point of the spatial point in the second image, and obtain second color data of the second projection point according to the second projection point and the HSV image data of the second image;

[0009] Construct a non - linear optimization function about the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non - linear optimization function.

[0010] Optionally, the step of calculating first projection points of multiple known spatial points in the first image and obtaining first color data of the first projection points according to the first projection points and the HSV image data of the first image includes:

[0011] Obtain the camera internal parameters and the three - dimensional coordinates of multiple known spatial points;

[0012] Calculate according to the camera internal parameters and the three - dimensional coordinates of the spatial points to obtain the pixel coordinates of the first projection points of the spatial points in the first image;

[0013] Determine the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and obtain the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

[0014] Optionally, the step of re - projecting the spatial point according to a preset camera pose change amount to obtain a second projection point of the spatial point in the second image, and obtaining second color data of the second projection point according to the second projection point and the HSV image data of the second image includes:

[0015] Calculate the pixel coordinates of the second projection point of the spatial point in the second image according to the projection equation P’ = TP, p’ = KP’;

[0016] where T represents the preset camera pose change amount, P represents the three - dimensional coordinates of the spatial point, P’ represents the three - dimensional coordinates of the spatial point in the camera coordinate system after camera movement, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection point of the spatial point in the second image;

[0017] Determine the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and obtain the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

[0018] Optionally, the color components of the first color data and the second color data are the same, and the color components include at least two of hue H, saturation S, and brightness V.

[0019] Optionally, the step of constructing a non-linear optimization function for the camera pose change amount according to the first color data and the second color data, and optimizing the camera pose change amount according to the non-linear optimization function includes:

[0020] When the color components of the first color data and the second color data are saturation S and brightness V, construct a non-linear optimization function E(T),

[0021]

[0022] where i represents the serial number of the spatial point, n represents the number of spatial points, w represents a preset kernel function, a and b represent preset weight coefficients, S(p i ) and V(p i ) respectively represent the saturation and brightness of the first projection point p i of the i-th spatial point P i , and S'(p' i ) and V'(p' i ) respectively represent the saturation and brightness of the second projection point p i ' of the i-th spatial point P i ;

[0023] Obtain the target pose change amount corresponding to the minimum value of E(T), and use the target pose change amount as the optimization result of the camera pose change amount.

[0024] The second aspect of the present invention provides a camera pose estimation optimization device, and the camera pose estimation optimization device includes:

[0025] A conversion module, configured to obtain a first image and a second image captured by a camera at two different poses, and convert the first image and the second image into HSV images to obtain HSV image data;

[0026] A first acquisition module, configured to calculate first projection points of multiple known spatial points in the first image, and obtain first color data of the first projection points according to the first projection points and the HSV image data of the first image;

[0027] A second acquisition module, configured to re-project the spatial points according to a preset camera pose change amount to obtain second projection points of the spatial points in the second image, and obtain second color data of the second projection points according to the second projection points and the HSV image data of the second image;

[0028] An optimization module, configured to construct a non - linear optimization function for the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non - linear optimization function.

[0029] Optionally, the first acquisition module is further configured to:

[0030] Acquire the camera internal parameters and the three - dimensional coordinates of multiple known spatial points;

[0031] Calculate according to the camera internal parameters and the three - dimensional coordinates of the spatial points to obtain the pixel coordinates of the first projection point of the spatial point in the first image;

[0032] Determine the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and acquire the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

[0033] Optionally, the second acquisition module is further configured to:

[0034] Calculate the pixel coordinates of the second projection point of the spatial point in the second image according to the projection equation P’ = TP, p’ = KP’;

[0035] where T represents a preset camera pose change amount, P represents the three - dimensional coordinates of the spatial point, P’ represents the three - dimensional coordinates of the spatial point in the camera coordinate system after camera movement, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection point of the spatial point in the second image;

[0036] Determine the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and acquire the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

[0037] Optionally, the color components of the first color data and the second color data are the same, and the color components include at least two of hue H, saturation S, and brightness V.

[0038] Optionally, the optimization module is further configured to:

[0039] When the color components of the first color data and the second color data are saturation S and brightness V, construct a non - linear optimization function E(T),

[0040]

[0041] where i represents the serial number of the spatial point, n represents the number of the spatial points, w represents a preset kernel function, a and b represent preset weight coefficients, S(p i ) and V(p i ) respectively represent the saturation and brightness of the first projection point p i of the i-th spatial point P i , and S'(p' i ) and V'(p' i ) respectively represent the saturation and brightness of the second projection point p i ' of the i-th spatial point P i ;

[0042] Obtain the target pose change amount corresponding to the minimum value of E(T), and use the target pose change amount as the optimization result of the camera pose change amount.

[0043] The third aspect of the present invention provides a camera pose estimation optimization device, which includes: a memory and at least one processor, and instructions are stored in the memory; the at least one processor calls the instructions in the memory to enable the camera pose estimation optimization device to execute the above camera pose estimation optimization method.

[0044] The fourth aspect of the present invention provides a computer-readable storage medium, in which instructions are stored, and when it runs on a computer, it enables the computer to execute the above camera pose estimation optimization method.

[0045] The present invention obtains a first image and a second image respectively captured by a camera at two different poses, converts the first image and the second image into HSV images to obtain HSV image data; calculates first projection points of a plurality of known spatial points in the first image, and obtains first color data of the first projection points according to the first projection points and the HSV image data of the first image; reprojects the spatial points according to a preset camera pose change amount to obtain second projection points of the spatial points in the second image, and obtains second color data of the second projection points according to the second projection points and the HSV image data of the second image; constructs a non-linear optimization function about the camera pose change amount according to the first color data and the second color data, and optimizes the camera pose change amount according to the non-linear optimization function. The present invention optimizes camera pose estimation by introducing color data in the HSV color space, weakens the dependence on gray invariance in the prior art direct method, and enables accurate determination of the camera pose even when the image gray gradient is small; at the same time, in practical applications, factors such as camera exposure parameters and ambient brightness changes will cause changes in image brightness. Compared with gray data, color data is less affected by image brightness changes, so the robustness of the optimization algorithm is greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic flowchart of an embodiment of the method for optimizing camera pose estimation according to the present invention;

[0047] Figure 2 It is a schematic diagram of the projection of spatial points in the first image and the second image in the embodiment of the method for optimizing camera pose estimation according to the present invention;

[0048] Figure 3 It is a schematic diagram of modules of an embodiment of the device for optimizing camera pose estimation according to the present invention;

[0049] Figure 4 It is a schematic structural diagram of the device for optimizing camera pose estimation provided by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] The embodiments of the present invention provide a method, a device, a device and a computer-readable storage medium for optimizing camera pose estimation, which can quickly find a maximum heartbeat interval that can realize the optimization of camera pose estimation, so that the terminal reduces power consumption and network resource consumption while maintaining the network connection.

[0051] In the description, claims and the above drawings of the present invention, the terms "first", "second", "third", "fourth", etc. (if any) are used to distinguish similar objects and do not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments described herein can be implemented in an order other than those illustrated or described herein. In addition, the term "comprising" or "having" and any variation thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device comprising a series of steps or units does not necessarily limit to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0052] For ease of understanding, the specific process of the embodiment of the camera pose estimation optimization method of the present invention will be described below.

[0053] Referring to Figure 1 , Figure 1 which is a schematic flowchart of an embodiment of the camera pose estimation optimization method of the present invention. The method includes:

[0054] Step 101, obtain a first image and a second image respectively captured by a camera in two different poses, and convert the first image and the second image into HSV images to obtain HSV image data;

[0055] The camera pose estimation optimization method of this embodiment can be applied to devices with visual SLAM functions such as robots, unmanned vehicles, and unmanned aerial vehicles. A camera is mounted on the device to capture scenes. At the same time, an embedded system can be mounted to process the captured images, or the captured images can be processed by a background server communicating with the device to realize functions such as autonomous positioning, mapping, and path planning of the device.

[0056] As the camera moves, the poses of the camera at different times and the captured images are different. Therefore, camera pose estimation can be performed through the images captured by the camera in different poses. Specifically, first, obtain a first image and a second image respectively captured by the camera in two different poses, and convert the first image and the second image into HSV images to obtain HSV image data. The HSV image data includes color data of three channels: H, S, and V, where H represents hue, S represents saturation, and V represents brightness. The original image output format of the camera is generally RGB image or YUV image. The conversion of RGB image or YUV image into HSV image can refer to existing image format conversion algorithms, which will not be elaborated here.

[0057] Step 102: Calculate the first projection points of multiple known spatial points in the first image, and obtain the first color data of the first projection points according to the first projection points and the HSV image data of the first image;

[0058] In this step, the number and three-dimensional coordinates of the spatial points can be obtained in advance by the direct method. For example, sparse feature points can be selected as spatial points by the sparse direct method, or some feature points can be selected as spatial points by the semi-dense direct method, or all feature points can be selected as spatial points by the dense direct method. This embodiment does not make any limitations in this regard.

[0059] Refer to Figure 2 , Figure 2 which is a schematic diagram of the projection of spatial points in the first image and the second image in the embodiment of the camera pose estimation optimization method of the present invention. For any known spatial point P, let its first projection point in the first image I1 be p, and its second projection point in the second image I2 be p'. According to the three-dimensional coordinates of the known spatial point P and the camera internal parameters, the position of the first projection point p in the first image I1 can be calculated, and then combined with the HSV image data of the first image I1, the first color data of the first projection point p can be obtained. The color components of the first color data can be flexibly set in advance, and it can include at least one of the three channels of hue H, saturation S, and brightness V. For example, when the color components are set to saturation S and brightness V, the first color data obtained is the saturation and brightness of the first projection point p.

[0060] Step 103: Reproject the spatial points according to the preset camera pose change amount to obtain the second projection points of the spatial points in the second image, and obtain the second color data of the second projection points according to the second projection points and the HSV image data of the second image;

[0061] In this step, let the camera pose change amount from the first image I1 to the second image I2 be T, and T is the variable to be optimized. Specifically, T can be an Euclidean transformation matrix, which represents the rotation transformation and translation transformation of the camera pose from I1 to I2. The rotation transformation can be represented by a 3*3 matrix, and the translation transformation can be represented by a 1*3 matrix. Combined, it can be written as an Euclidean transformation matrix, and an initial value can be given during optimization.

[0062] Reproject the spatial point P according to T. By reprojecting, it means that when the value of the current optimization variable T is fixed, calculate the spatial point coordinates P' of the spatial point P in the camera coordinate system after the camera movement, and then project P' onto the camera plane to obtain the projection point. It can be intuitively understood that as the camera movement varies, the position of the stationary spatial point changes in the camera's field of view. Since T here is the value to be optimized (solved), it does not represent the actual camera movement. It only assumes where the spatial point should appear in the photo under the current rotation and translation, and then compares it with the actually obtained photo. When the error is relatively small, it is considered that T is close to the actual camera movement. This imaginary projection process is called the reprojection process.

[0063] After reprojecting the spatial point P, obtain the position of the second projection point p' in the second image I2, and then combine the HSV image data of the second image I2 to obtain the second color data of the second projection point p'. It should be noted that similar to the first color data, the color components of the second color data can also be flexibly set in advance. It can include at least one of the three channels of hue H, saturation S, and brightness V. At the same time, for the convenience of subsequent calculations, the color components of the first color data and the second color data can be set to be the same, such as both being saturation S and brightness V.

[0064] Step 104, construct a non - linear optimization function regarding the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non - linear optimization function.

[0065] After obtaining the first color data and the second color data, construct a non - linear optimization function E(T) regarding the camera pose change amount T according to the first color data and the second color data. Then, use a preset optimization algorithm such as the Gauss - Newton iteration method or the LM optimization algorithm to optimize E(T), and obtain the target pose change amount corresponding to the minimum value of E(T). This target pose change amount represents the movement trajectory of the camera from the first image I1 to the second image I2. According to this target pose change amount, combined with the pose of the camera at the previous moment (the shooting moment of the first image), the pose of the camera at the current moment (the shooting moment of the second image) can be determined.

[0066] In this embodiment, by introducing the color data in the HSV color space to optimize the camera pose estimation, the dependence on gray - scale invariance in the prior art direct method is weakened, enabling accurate determination of the camera pose even in the case of small image gray - scale gradients. At the same time, in practical applications, factors such as camera exposure parameters and ambient brightness changes will cause changes in image brightness. Compared with gray - scale data, color data is less affected by image brightness changes. Therefore, the robustness of the optimization algorithm is greatly improved.

[0067] Furthermore, based on the first embodiment of the camera pose estimation optimization method in the present invention, a second embodiment of the camera pose estimation optimization method in the present invention is proposed.

[0068] In this embodiment, step 102 above may further include: obtaining the camera internal parameters and the three-dimensional coordinates of a plurality of known spatial points; calculating according to the camera internal parameters and the three-dimensional coordinates of the spatial points to obtain the pixel coordinates of the first projection point of the spatial point in the first image; determining the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and obtaining the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

[0069] Specifically, both the camera internal parameters and the three-dimensional coordinates of the spatial point P are known parameters. According to the camera internal parameters and the three-dimensional coordinates of the spatial point P, the pixel coordinates (u, v) of the first projection point p in the first image I1 can be calculated. According to the pixel coordinates (u, v), the position of the first projection point p in I1 can be determined, and the color data corresponding to this position is the first color data. For example, when the color components of the preset first color data are two types, namely saturation S and brightness V, the first color data can be represented by saturation S(p) and brightness V(p).

[0070] Furthermore, step 103 above may include: calculating the pixel coordinates of the second projection point of the spatial point in the second image according to the projection equations P’ = TP, p’ = KP’; where T represents the preset camera pose change amount, P represents the three-dimensional coordinates of the spatial point, P’ represents the three-dimensional coordinates of the spatial point in the camera coordinate system after camera movement, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection point of the spatial point in the second image; determining the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and obtaining the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

[0071] Specifically, the reprojection process is as follows:

[0072] 1. Calculate P’ = TP. The three-dimensional coordinates P have a column vector form of {x, y, z, 1} in homogeneous coordinates. Multiply P on the left by T to obtain the three-dimensional coordinates P’ of the spatial point P in the camera coordinate system after camera movement. T reflects the camera movement. Thus, P’ is a function of the camera movement T and changes with the change of T.

[0073] 2. Obtain the pixel coordinates of the second projection point p’ corresponding to P’ in the camera field of view through the perspective transformation K, p’ = KP’, where K is the camera internal parameter and can be represented by a 3*3 matrix.

[0074] According to the pixel coordinates of the second projection point p' obtained by reprojection, the position of the second projection point p' in I2 can be determined, and the color data corresponding to this position is the second color data. For example, when the color components of the preset second color data are two types, namely saturation S and brightness V, the second color data can be represented by saturation S'(p') and brightness V'(p').

[0075] In this embodiment, by calculating the first color data of the first projection point of the spatial point in the first image and the second color data of the second projection point in the second image, it provides a prerequisite guarantee for constructing a non-linear optimization function subsequently.

[0076] Furthermore, based on the first and second embodiments of the camera pose estimation optimization method in the present invention, a third embodiment of the camera pose estimation optimization method in the present invention is proposed.

[0077] In this embodiment, the color components of the first color data and the second color data are the same, and the color components include at least two of hue H, saturation S, and brightness V.

[0078] In this embodiment, to ensure the accuracy of camera pose estimation and facilitate error calculation at the same time, the first color data and the second color data contain the same color components, and the color components include at least two of hue H, saturation S, and brightness V, that is, two types of H and S, or two types of S and V, or two types of H and V, or three types of H, S, and V. Furthermore, considering that there is relatively large noise in the H channel in practice, the first color data and the second color data can only select to include two channels of S and V.

[0079] Furthermore, the above step 104 may include: when the color components of the first color data and the second color data are saturation S and brightness V, constructing a non-linear optimization function E(T).

[0080]

[0081] where i represents the serial number of the spatial point, n represents the number of spatial points, w represents a preset kernel function. The kernel function is a commonly used function in non-linear optimization, which is used to suppress outliers (the errors of outliers are very large) and reduce the influence of outliers on the optimization process; a and b represent preset weight coefficients, and a and b can be flexibly set according to camera imaging parameters and the environment. For example, if the camera color saturation is too high or too low, a can be appropriately adjusted, or if the ambient light changes, b can be reduced; in particular, the sum of the squares of a and b can be set equal to 1, so as to facilitate comparing the effects obtained by different values of a and b and facilitate parameter adjustment; S(p i )、V(p i) respectively represent the first projection point p of the i-th spatial point P i of the saturation and brightness of S'(p' i ), and V'(p' i ) respectively represent the second projection point p of the i-th spatial point P i ; |S(p i ) - S'(p' i )| i can be regarded as the color error, and |V(p i ) - V'(p' 2 )| i can be regarded as the grayscale error.

[0082] Obtain the target pose change amount corresponding to when E(T) obtains the minimum value, and use the target pose change amount as the optimization result of the camera pose change amount.

[0083] During optimization, an initial value can be given to the optimization variable T, and then the Gauss-Newton iteration method, the LM optimization algorithm, etc. can be used to make E(T) reach the minimum value, that is, the grayscale error and the color error reach the minimum value. At this time, the corresponding pose change amount T is the motion trajectory of the camera.

[0084] In this embodiment, the camera pose change amount is optimized based on two channels of saturation S and brightness V, avoiding the influence of the noise of the hue H channel on the optimization result, and further improving the accuracy of camera pose estimation.

[0085] The embodiment of the present invention also provides a camera pose estimation optimization device.

[0086] Refer to Figure 3 Figure 3

[0087] is a schematic diagram of modules of an embodiment of the camera pose estimation optimization device of the present invention. In this embodiment, the camera pose estimation optimization device includes:

[0088] A conversion module 10, configured to obtain a first image and a second image captured by the camera in two different poses, and convert the first image and the second image into HSV images to obtain HSV image data;

[0089] A first acquisition module 20, configured to calculate the first projection points of multiple known spatial points in the first image, and obtain the first color data of the first projection points according to the first projection points and the HSV image data of the first image;

[0090] ​​​The second acquisition module 30 is configured to re-project the spatial point according to a preset camera pose change amount to obtain a second projection point of the spatial point in the second image, and acquire second color data of the second projection point according to the second projection point and HSV image data of the second image;

[0090] The optimization module 40 is configured to construct a non-linear optimization function regarding the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non-linear optimization function.

[0091] Optionally, the first acquisition module 20 is further configured to:

[0092] Acquire the camera internal parameters and the three-dimensional coordinates of multiple known spatial points;

[0093] Calculate according to the camera internal parameters and the three-dimensional coordinates of the spatial point to obtain the pixel coordinates of the first projection point of the spatial point in the first image;

[0094] Determine the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and acquire the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

[0095] Optionally, the second acquisition module 30 is further configured to:

[0096] Calculate the pixel coordinates of the second projection point of the spatial point in the second image according to the projection equations P’ = TP and p’ = KP’;

[0097] where T represents the preset camera pose change amount, P represents the three-dimensional coordinates of the spatial point, P’ represents the three-dimensional coordinates of the spatial point in the camera coordinate system after camera movement, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection point of the spatial point in the second image;

[0098] Determine the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and acquire the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

[0099] Optionally, the color components of the first color data and the second color data are the same, and the color components include at least two of hue H, saturation S, and brightness V.

[0100] Optionally, the optimization module 40 is further configured to:

[0101] When the color components of the first color data and the second color data are saturation S and brightness V, a non - linear optimization function E(T) is constructed.

[0102]

[0103] Where i represents the serial number of the spatial point, n represents the number of spatial points, w represents a preset kernel function, a and b represent preset weight coefficients, S(p i ), V(p i ) respectively represent the saturation and brightness of the first projection point p i of the i - th spatial point P i , and S'(p' i ) and V'(p' i ) respectively represent the saturation and brightness of the second projection point p i ' of the i - th spatial point P i ;

[0104] Obtain the target pose change amount corresponding to the minimum value of E(T), and use the target pose change amount as the optimization result of the camera pose change amount.

[0105] The function implementation and beneficial effects of each module in the above camera pose estimation optimization device correspond to the steps in the above camera pose estimation optimization method embodiment, and will not be elaborated here.

[0106] The above - mentioned camera pose estimation optimization device in the embodiment of the present invention has been described in detail from the perspective of modular functional entities. Next, the camera pose estimation optimization device in the embodiment of the present invention will be described in detail from the perspective of hardware processing.

[0107] Referring to Figure 4 , Figure 4 is a schematic structural diagram of a camera pose estimation optimization device provided by an embodiment of the present invention. The camera pose estimation optimization device 300 may vary greatly due to configuration or performance differences, and may include one or more processors (central processing units, CPU) 310 and a memory 320, and one or more storage media 330 (such as one or more mass storage devices) for storing application programs 333 or data 332. Among them, the memory 320 and the storage media 330 can be short - term storage or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the camera pose estimation optimization device 300. Further, the processor 310 can be set to communicate with the storage media 330 and execute a series of instruction operations in the storage media 330 on the camera pose estimation optimization device 300.

[0108] The camera pose estimation optimization device 300 may further include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 4 The structure of the camera pose estimation optimization device shown does not limit the camera pose estimation optimization device, and it may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0109] The present invention also provides a camera pose estimation optimization device, which includes a memory and a processor. Instructions are stored in the memory, and when the instructions are executed by the processor, the processor is caused to execute the steps of the camera pose estimation optimization method in the above-mentioned various embodiments.

[0110] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium, and when the instructions run on a computer, the computer is caused to execute the steps of the camera pose estimation optimization method in the above-mentioned various embodiments.

[0111] Those skilled in the art can understand that if the above-mentioned integrated module or unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in various embodiments of the present invention. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0112] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An optimized method for camera pose estimation, characterized in that, The camera pose estimation optimization method includes the following steps: Obtain a first image and a second image captured by the camera at two different poses, convert the first image and the second image into HSV images to obtain HSV image data; Calculate the first projection points of multiple known spatial points in the first image, and obtain the first color data of the first projection points according to the first projection points and the HSV image data of the first image; According to a preset camera pose change amount, re-project the spatial points to obtain second projection points of the spatial points in the second image, and obtain the second color data of the second projection points according to the second projection points and the HSV image data of the second image; Construct a non-linear optimization function regarding the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non-linear optimization function.

2. The camera pose estimation optimization method according to claim 1, characterized in that, The step of calculating the first projection points of multiple known spatial points in the first image and obtaining the first color data of the first projection points according to the first projection points and the HSV image data of the first image includes: Obtain the camera internal parameters and the three-dimensional coordinates of multiple known spatial points; Calculate according to the camera internal parameters and the three-dimensional coordinates of the spatial points to obtain the pixel coordinates of the first projection points of the spatial points in the first image; Determine the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and obtain the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

3. The camera pose estimation optimization method according to claim 2, wherein, The step of re-projecting the spatial points according to a preset camera pose change amount to obtain second projection points of the spatial points in the second image and obtaining the second color data of the second projection points according to the second projection points and the HSV image data of the second image includes: Calculate the pixel coordinates of the second projection points of the spatial points in the second image according to the projection equation P’ = TP, p’ = KP’; wherein, T represents the preset camera pose change amount, P represents the three-dimensional coordinates of the spatial points, P’ represents the three-dimensional coordinates of the spatial points in the camera coordinate system after camera movement, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection points of the spatial points in the second image; Determine the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and obtain the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

4. The camera pose estimation optimization method according to any one of claims 1-3, characterized in that The color components of the first color data and the second color data are the same, and the color components include at least two of hue H, saturation S, and brightness V.

5. The camera pose estimation optimization method according to claim 4, wherein The step of constructing a non-linear optimization function regarding the camera pose change amount according to the first color data and the second color data and optimizing the camera pose change amount according to the non-linear optimization function includes: When the color components of the first color data and the second color data are saturation S and brightness V, a non-linear optimization function E(T) is constructed. where, i represents the serial number of the spatial point, n represents the number of the spatial points, w represents a preset kernel function, a and b represent preset weight coefficients, S(p i ), V(p i ) respectively represent the saturation and brightness of the first projection point p i of the i-th spatial point P i , and S'(p' i ) and V'(p' i ) respectively represent the saturation and brightness of the second projection point p i ' of the i-th spatial point P i ; The target pose change amount corresponding to the minimum value of E(T) is obtained, and the target pose change amount is used as the optimization result of the camera pose change amount.

6. An apparatus for optimizing camera pose estimation, characterized in that, The camera pose estimation optimization device includes: A conversion module, configured to obtain a first image and a second image captured by a camera at two different poses, convert the first image and the second image into HSV images, and obtain HSV image data. A first acquisition module, configured to calculate first projection points of multiple known spatial points in the first image, and obtain first color data of the first projection points according to the first projection points and the HSV image data of the first image. A second acquisition module, configured to re-project the spatial points according to a preset camera pose change amount to obtain second projection points of the spatial points in the second image, and obtain second color data of the second projection points according to the second projection points and the HSV image data of the second image. An optimization module, configured to construct a non-linear optimization function regarding the camera pose change amount according to the first color data and the second color data, and optimize the camera pose change amount according to the non-linear optimization function.

7. The camera pose estimation optimization device according to claim 6, wherein The first acquisition module is further configured to: Obtain the camera internal parameters and the three-dimensional coordinates of multiple known spatial points. Calculate according to the camera internal parameters and the three-dimensional coordinates of the spatial points to obtain the pixel coordinates of the first projection points of the spatial points in the first image. Determine the first position of the first projection point in the first image according to the pixel coordinates of the first projection point, and obtain the color data corresponding to the first position from the HSV image data of the first image as the first color data of the first projection point.

8. The camera pose estimation optimization device according to claim 7, wherein, The second acquisition module is further configured to: Calculate the pixel coordinates of the second projection points of the spatial points in the second image according to the projection equation P’ = TP, p’ = KP’; where T represents the preset camera pose change amount, P represents the three-dimensional coordinates of the spatial points, P’ represents the three-dimensional coordinates of the spatial points in the camera coordinate system after the camera moves, K represents the camera internal parameters, and p’ represents the pixel coordinates of the second projection points of the spatial points in the second image. Determine the second position of the second projection point in the second image according to the pixel coordinates of the second projection point, and obtain the color data corresponding to the second position from the HSV image data of the second image as the second color data of the second projection point.

9. An apparatus for optimizing camera pose estimation, characterized in that, The camera pose estimation optimization device includes: a memory and at least one processor, and instructions are stored in the memory. The at least one processor calls the instructions in the memory so that the camera pose estimation optimization device executes the camera pose estimation optimization method according to any one of claims 1-5.

10. A computer-readable storage medium, on which instructions are stored, characterized in that, When the instructions are executed by the processor, the camera pose estimation optimization method according to any one of claims 1-5 is implemented.

Images