Camera pose determination method and device, electronic equipment and storage medium
By obtaining the target image captured by the camera and the 2D-3D point pair in the target map, determining the camera's center of light projection area and voting, the problem of mismatch in less texture or repeated texture scenes is solved, and a more accurate camera is achieved Position positioning.
Patent Information
- Application Number
- CN202310566612.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-18
- Publication Date
- 2024-11-22
AI Technical Summary
In scenes with few textures or repeated textures, camera pose positioning methods based on 2D-3D feature point matching are prone to mismatch, resulting in low internal points proportions, which in turn leads to failure of camera pose positioning.
By acquiring the target image captured by the camera, multiple 2D-3D point pairs are acquired, and the projection area of the camera's light center on the reference two-dimensional plane is obtained based on each 2D-3D point pair, and the reference two-dimensional plane is matched by the projection area. Voting to determine the target area, and use the 2D-3D point pair corresponding to the target area as the inner point to more accurately determine the camera's position.
It improves the accuracy and robustness of camera positioning, reduces the impact of mismatch, and ensures that pose calculations can be effectively performed in complex scenarios.
Smart Images

Figure CN119006587A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer vision technology, and in particular to a camera posture determination method, device, electronic device, and storage medium. Background Art
[0002] In scenarios such as SLAM (Simultaneous Localization And Mapping) and visual navigation, the camera pose is usually tracked when capturing scene images to facilitate matching of scene images with three-dimensional maps and achieve further mapping or navigation functions.
[0003] Related visual positioning solutions usually adopt a positioning method based on 2D-3D feature point matching. In the part of finding matches and estimating poses, the most common solution is to perform nearest neighbor matching of feature points between the query frame and the reference frame retrieved in the coarse positioning, and use a threshold-based similarity point filtering strategy to perform inlier screening of 2D-3D matching point pairs to realize pose calculation.
[0004] However, in scenes with little texture or repeated texture, the 2D-3D feature point matching obtained by nearest neighbor search usually contains a large number of false matches, resulting in a low ratio of inliers, which can easily cause the camera's pose positioning to fail. Summary of the invention
[0005] The present invention provides a camera posture determination method, device, electronic device and storage medium.
[0006] According to one aspect of the present disclosure, a method for determining a camera pose is provided, the method comprising:
[0007] Acquire a target image captured by a camera, wherein the target image includes a plurality of 2D feature points;
[0008] Acquire 3D feature points matching the multiple 2D feature points from the target map to obtain multiple 2D-3D point pairs;
[0009] Based on each 2D-3D point pair, a projection area of the optical center of the camera on a reference two-dimensional plane is obtained, and the target image is aligned with the target map in the gravity direction;
[0010] Determine a target area in the reference two-dimensional plane through the projection areas corresponding to the multiple 2D-3D point pairs;
[0011] Inliers are determined based on 2D-3D point pairs corresponding to the target area, and the position and posture of the camera are determined according to the inliers.
[0012] According to a second aspect of the present disclosure, a camera posture determination device is provided, characterized in that the device comprises:
[0013] An image acquisition module is used to acquire a target image captured by a camera, wherein the target image includes a plurality of 2D feature points;
[0014] A feature point acquisition module, used to acquire 3D feature points matching the multiple 2D feature points from the target map to obtain multiple 2D-3D point pairs;
[0015] A projection area determination module, for obtaining a projection area of the optical center of the camera on a reference two-dimensional plane based on each 2D-3D point pair, wherein the target image is aligned with the target map in a gravity direction;
[0016] A target area determination module, configured to determine a target area in the two-dimensional plane through the projection areas corresponding to the multiple 2D-3D point pairs;
[0017] The pose determination module is used to determine the inner points based on the 2D-3D point pairs corresponding to the target area, and determine the pose of the camera according to the inner points.
[0018] According to a third aspect of the present disclosure, an electronic device is provided, which includes a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.
[0019] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the program is executed by a processor, the above method of the present disclosure is implemented.
[0020] The camera posture determination method, device, electronic device and storage medium provided by the embodiments of the present disclosure obtain a target image captured by a camera, obtain multiple 2D-3D point pairs from the target map and the target image, obtain the projection area of the optical center of the camera on the reference two-dimensional plane based on each 2D-3D point pair, vote on the reference two-dimensional plane through the projection area to determine the target area in the reference two-dimensional plane, use the 2D-3D point pairs corresponding to the target area as inner points, and determine the camera posture based on the inner points. In this way, by determining the inner points from the multiple 2D-3D point pairs obtained, the camera posture can be determined more accurately through these determined inner points. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Further details, features and advantages of the present disclosure are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which:
[0022] Figure 1 A schematic diagram of a scenario provided for an exemplary embodiment of the present disclosure;
[0023] Figure 2 A flowchart of a camera pose determination method provided for an exemplary embodiment of the present disclosure;
[0024] Figure 3 A schematic block diagram of functional modules of a camera pose determination device provided for an exemplary embodiment of the present disclosure;
[0025] Figure 4 A structural block diagram of an electronic device provided for an exemplary embodiment of the present disclosure;
[0026] Figure 5 A structural block diagram of a computer system provided for an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.
[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.
[0029] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.
[0030] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".
[0031] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.
[0032] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.
[0033] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
[0034] As an optional but non-limiting implementation, in response to receiving an active request from the user, the method of sending a prompt message to the user may be, for example, a pop-up window, in which the prompt message may be presented in text. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understandable that the above notification and the process of obtaining user authorization are only illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure.
[0035] In scenarios such as visual relocalization, SLAM or visual navigation, AR (Augmented Reality) and other related applications are often involved. For example, if a user wants to display some special effects in a scenic spot, or the scenic spot operator sets up AR special effects in a specific location in advance, the user can use a mobile phone or other terminal device to shoot the environment in which he is located, and on this basis, he can determine whether he can see the special effects, such as stickers, at the current location. If the user wants to get such a special effect directly through the mobile phone, it is necessary to be able to locate the global posture of the user's mobile phone camera, which is the overall goal of the visual relocalization service.
[0036] If you need to locate the global position of the user's mobile phone camera, you first need to build a sparse point cloud map for the scene involved in advance. When locating the position of the user's mobile phone camera, local feature points will be extracted from the image taken by the camera on the user's mobile phone.
[0037] In the process of feature point extraction, relevant feature detection algorithms need to be involved. Traditional feature detection algorithms are generally based on the grayscale information of pixels, such as SIFT (Scale-invariant feature transform), SURF (Speeded Up Robust Features), ORB (Oriented Fast and Rotated Brief) algorithms, etc. The extracted feature points are usually in the edges, corners, textures and other places where the pixel grayscale values change significantly. However, in the face of complex environmental scenes, traditional algorithms usually have unstable feature extraction. With the development and application of computer vision, methods based on deep learning have been applied to the field of feature extraction and have achieved good results. For example, deep learning convolutional neural networks.
[0038] After extracting some local feature points on the captured image, a match can be made with the three-dimensional map points in the pre-built point cloud map. After the feature points on the captured image are extracted, for example, there will be a multi-dimensional descriptor, wherein the descriptor can be 128-dimensional, which is only an exemplary description in the embodiment and is not limited to this. At the same time, when the map points in the point cloud map are constructed, there will also be a corresponding descriptor. This requires the two-dimensional (2D for short) feature points on the image to find the most similar three-dimensional (3D for short) feature points in the point cloud map with a corresponding relationship of similar descriptors. For example, the 3D feature points in the point cloud map that match the 2D feature points of the image are found by the nearest neighbor method. After the above feature points are matched, there will often be some errors or some wrong matches. For example, if the scene is large, there will be many similar 3D feature points in the point cloud map, so the 3D feature points found are often not real matching points, that is, there is a problem of mismatching. For example, in large scenes such as cities and large shopping malls, or scenes with few or repeated textures, or scenes with weather, seasons, and lighting changes, the 2D-3D point pairs found by the nearest neighbor will contain a large number of mismatches, resulting in a low ratio of internal points in the 2D-3D point pairs with correct matches, and the RANSAC algorithm is also prone to failure, and thus the image cannot be effectively positioned, that is, the user's mobile phone cannot be effectively positioned. It should be noted that the embodiments are described by taking the image taken by the camera on the mobile phone as an example, and the image taken can also be obtained directly by the camera, etc.
[0039] In order to effectively locate the posture of the user's mobile phone, it is necessary to detect and eliminate the mismatches in the matched 2D-3D point pairs, so that a more accurate posture can be obtained by calculating the posture through the remaining more accurate 2D-3D points. That is, a global posture of the current 6dof (degree of freedom) of the device is obtained, including the freedom of rotation in three directions and the freedom of translation in three directions in the three-dimensional coordinate system. In implementation, the device can be a terminal including a mobile phone or a camera, etc.
[0040] In an embodiment, the gravity information of the image captured by the mobile phone can be obtained, and the image and the point cloud map can be aligned in the direction of gravity using the gravity information, wherein the gravity information can be obtained through the IMU (Inertial Measurement Unit) sensor on the terminal such as the mobile phone, and the gravity information generally obtained has a high accuracy. For example, the gravity direction of the image can be provided by the gravity sensor on the mobile phone, and the point cloud map will also include the gravity direction, so that the gravity direction of the image and the gravity direction of the map are aligned to obtain the aligned image. After aligning the captured image with the point cloud map in the gravity direction, the 6dof pose estimation problem of the camera can be converted into a 4-degree-of-freedom estimation. Before aligning the gravity direction, it is necessary to solve the 6dof pose of the image in the point cloud map, that is, including 3 degrees of freedom translation (3D coordinates of the camera optical center) and 3 degrees of freedom rotation. After aligning the gravity direction, the solved image corresponding camera pose can be converted into 3 degrees of freedom translation and 1 degree of freedom rotation in the embodiment.
[0041] In the embodiment, in order to determine the inner point from the matched 2D-3D point pairs, it is necessary to perform geometric relationship modeling on each 2D-3D point pair, such as Figure 1 As shown in the figure, if the position of the camera on the mobile phone is known, and the coordinates of a 2D feature point on the image are known, the position of the corresponding 3D feature point in the point cloud map will exist within the cone 11 emitted from the optical center of the camera. If the height range of the 3D feature point is further constrained, we can actually use two planes to cut the cone, and get the part of the cone between the two planes, that is, Figure 1 The truncated cone 12 shown in FIG. 1 is a 3D feature point that exists at a certain position in the space defined by the truncated cone 12. The projection area is obtained by projecting the truncated cone 12 onto a reference two-dimensional plane. In the embodiment, the projection area obtained after projection can be approximately a geometrically shaped area as required, for example, a polygonal area, such as Figure 1 As shown, in some examples, the projection area may be a quadrilateral area 13. The reference two-dimensional plane may be a plane perpendicular to the height direction.
[0042] Therefore, if the position and coordinates of the camera and the 2D feature points are known, the range of the projection of the 3D feature points on the reference two-dimensional plane will be limited to a projection area. Similarly, if the rotation angle and height of the camera are known, and the coordinates of the 3D feature points and the coordinates of the 2D feature points on the image are known, the projection of the optical center coordinates of the camera on the reference two-dimensional plane is equivalent to a rigid body transformation of the above-mentioned projection area. In this way, based on each 2D-3D point pair, the range of the projection area is obtained by traversing multiple height values of the camera in a possible height interval, and traversing multiple rotation angle values of the camera in a possible rotation angle interval. Among them, the rotation angle can be the angle of rotation with the height direction as the rotation axis. In this way, by limiting the height interval and rotation angle interval of the camera, the above-mentioned 4dof estimation can be converted into a 2dof estimation, and the solution of the position and posture of the optical center of the desired camera is limited to a two-dimensional plane, that is, limited to the above-mentioned projection area, and the 2Dof position range of the camera on a two-dimensional plane perpendicular to the height direction is estimated.
[0043] In the embodiment, the target area in the reference two-dimensional plane can be determined by the projection area of multiple 2D-3D point pairs projected on the reference two-dimensional plane, and the inner point can be determined based on the 2D-3D point pairs corresponding to the target area, so as to determine the position and posture of the camera according to the inner point. Among them, the coordinates of the 2D-3D point pair are the coordinates corresponding to any point in the pixel coordinate system and the coordinates corresponding to the point in the world coordinate system. In the embodiment, the reference two-dimensional plane can be voted by the projection area, and the target area in the reference two-dimensional plane can be determined based on the number of votes obtained, so that each 2D-3D point pair can vote for the position of the camera in the two-dimensional space by traversing the height interval and rotation angle interval of the camera. After voting based on all 2D-3D point pairs, it can be considered that the position of the camera will be located in the area where the number of votes reaches the preset condition, such as the area with the most votes, and this area can be called the target area. In the embodiment, the target area can be considered as an area with the highest probability of camera position distribution on the two-dimensional horizontal plane. Then, the 2D-3D point pair that has voted for the target area is also most likely to be determined as a correctly matched 2D-3D point pair, that is, the 2D-3D point pair corresponding to the target area can be used as an inner point, and the camera posture can be determined more accurately by determining these inner points.
[0044] In an embodiment, the above-mentioned rotation angle interval can obtain the rotation angle of the camera by obtaining the information collected by the magnetometer on the device, for example, from 100° to 200°, avoiding direct traversal from 0° to 200°, which is equivalent to a screening and filtering of the camera rotation angle, and can reduce the complexity of the calculation.
[0045] In the embodiment, the height interval and rotation angle interval of the camera traversed above can be pre-set, for example, the height interval can be traversed from -5m to 5m, and the rotation angle interval can be traversed from 100° to 200°. Exemplarily, the camera obtains multiple projection areas obtained by the above projections at (-5m, 100°), multiple projection areas at (-5m, 105°), multiple projection areas at (5m, 200°), and so on, until the loop ends.
[0046] In an embodiment, the user's authorization consent for positioning information can also be obtained through a pop-up window or the like. With the user's consent, the positioning information will be obtained when the target image is captured by the camera. When the user captures the target image through the camera on the mobile phone, the captured target image will carry positioning information. For example, the positioning information can be the location information of the GPS (Global Positioning System), which can be used to constrain the position of the camera on the reference two-dimensional horizontal plane. For example, a 100*100 (meter) area can be divided according to the GPS location information. If the voting position of the 2D-3D point pair exceeds the area range, the voting result can be directly excluded because the 2D-3D point pair is a wrong match. In addition, the voting results of the 2D-3D point pairs that do not meet the common view relationship will also be excluded in the embodiment. For example, the camera rotation angle corresponding to some 3D points can be constrained according to the known common view point relationship of the target map, and the voting results that do not meet the angle constraint can be excluded.
[0047] In the embodiment, gridding can be performed in the reference two-dimensional plane area based on the point cloud map, and the grid shape in the embodiment is not limited to triangles or quadrilaterals. By traversing the height interval and rotation angle interval of the camera based on each 2D-3D point pair, multiple projection areas will be projected on the reference two-dimensional plane on the point cloud map. Because the projection area obtained by projection may be large or small. In the embodiment, the reference two-dimensional plane area will be gridded at a certain resolution. For example, in a reference two-dimensional plane of 100m*100m, when the resolution is 20*20, if the grid is of equal size, the size of each grid is 5m*5m; when the resolution is 10*10, if the grid is of equal size, the size of each grid is 10m*10m. For each projection area, its voting range will cover the already divided grids, which is equivalent to voting for the covered grids. Since the size of each projection area may be inconsistent, such as a relatively large projection area, a low-resolution large grid can be used for the convenience of calculation, so that the number of grids to be counted will be reduced. If the projection area is small, a high-resolution small grid can be used, which is convenient for counting the number of grids covered by the projection area. This can reduce the complexity of the calculation and avoid the problem of using all high-resolution grids, which leads to a large number of grids covering a large projection area and resulting in high calculation complexity.
[0048] Based on the above embodiments, the present disclosure provides a method for determining a camera posture, such as Figure 2 As shown, the method may include the following steps:
[0049] In step S210, a target image captured by a camera is acquired.
[0050] The target image includes multiple 2D feature points.
[0051] In the embodiment, the camera may be a camera on a terminal or an independent camera device, and the terminal may be a mobile phone. The target image may be a two-dimensional image captured by a user through a camera on a mobile phone, and the 2D feature points in the target image may be obtained by the feature detection algorithm in the above embodiment.
[0052] In step S220, 3D feature points matching the multiple 2D feature points are acquired from the target map to obtain multiple 2D-3D point pairs.
[0053] In an embodiment, the target map may be a pre-built point cloud map, which includes the shooting scene of the camera. 3D feature points matching the 2D feature points of the image in the point cloud map may be searched by the nearest neighbor method to obtain multiple 2D-3D point pairs.
[0054] In step S230, based on each 2D-3D point pair, a projection area of the optical center of the camera on the reference two-dimensional plane is obtained, wherein the target image is aligned with the target map in the gravity direction.
[0055] In an embodiment, the gravity information of the target image can also be obtained, and the target image and the target map are aligned in the direction of gravity using the gravity information. For example, the gravity direction of the target image can be provided by a gravity sensor on a camera or a mobile phone, and the target map will also include the gravity direction, so that the gravity direction of the target image and the gravity direction of the target map are aligned to obtain the aligned target image. After the captured target image is aligned with the point cloud map in the direction of gravity, the 6dof pose estimation problem can be converted into a 4-degree-of-freedom estimation. Before the gravity direction is aligned, it is necessary to solve the 6dof pose in the point cloud map when the camera captures the target image, that is, including 3 degrees of freedom translation and 3 degrees of freedom rotation. After aligning the gravity direction, the pose of the camera corresponding to the target image to be solved can be converted into 3 degrees of freedom translation and 1 degree of freedom rotation (i.e., the angle at which the camera rotates with the gravity direction as the rotation axis) in an embodiment.
[0056] In the embodiment, multiple 2D-3D point pairs may be first obtained through the target image, and then the target image and the target map may be aligned in the gravity direction. Alternatively, after obtaining the target image, the target image may be aligned in the gravity direction, and then multiple 2D-3D point pairs may be determined from the target image aligned in the gravity direction, but the embodiment is not limited thereto.
[0057] In an embodiment, by obtaining the position and posture of the camera and the coordinates of the 2D feature points in the target image, the range of the projection of the 3D feature points in the target image on the reference two-dimensional plane can be determined, and the range of the projection will be limited to an area. Similarly, if the rotation angle and height of the camera are known, by obtaining the coordinates of the 3D feature points in the target image and the coordinates of the 2D feature points in the target image, the range of the projection of the optical center coordinates of the camera on the reference two-dimensional plane will also be limited to a corresponding area. In this way, based on each 2D-3D point pair, by traversing multiple height values of the camera in a possible height interval, and traversing multiple rotation angle values of the camera in a possible rotation angle interval, the range of the projection area is obtained. Among them, the rotation angle can be the angle of rotation with the height direction as the rotation axis. In this way, by limiting the height interval and rotation angle interval of the camera, the above-mentioned 4dof estimation can be converted into a 2dof estimation, and the solution of the position and posture of the optical center of the desired camera is limited to the reference two-dimensional plane, that is, limited to the above-mentioned projection area, and the 2Dof position range of the camera on the reference two-dimensional plane perpendicular to the height direction is estimated.
[0058] In step S240, a target area in a reference two-dimensional plane is determined through projection areas corresponding to a plurality of 2D-3D point pairs.
[0059] In an embodiment, the target area in the reference two-dimensional plane can be determined by projecting multiple 2D-3D point pairs onto the reference two-dimensional plane to obtain the projection area, and the inner points can be determined based on the 2D-3D point pairs corresponding to the target area, so as to determine the camera's posture based on the inner points.
[0060] For example, after obtaining n projection areas, a threshold m may be set, and if there are more than or equal to m projection areas with a common intersection area, the intersection area may be used as the target area, where m is less than n, and both m and n are positive integers.
[0061] Exemplarily, when m is equal to 3, if there are three projection areas, namely a1, a2 and a3, and the three projection areas have an intersection area a, then the intersection area a can be used as one of the target areas; in addition, if there are five projection areas (b1, b2, b3, b4, b5), and the five projection areas have an intersection area b, then the intersection area b can be used as one of the target areas.
[0062] In step S250, the interior points are determined based on the 2D-3D point pairs corresponding to the target area, and the position and posture of the camera are determined according to the interior points.
[0063] In the embodiment, the target area can be considered as an area where the camera's position distribution on the reference two-dimensional horizontal plane is most likely. Then, the 2D-3D point pairs that have voted for the target area are also most likely to be determined as correctly matched 2D-3D point pairs, that is, the corresponding 2D-3D point pairs in the target area can be used as internal points. By determining these internal points, the camera's posture can be determined more accurately, effectively improving the camera's posture determination accuracy and robustness.
[0064] The camera posture determination method provided by the embodiment of the present disclosure obtains a target image captured by a camera, obtains multiple 2D-3D point pairs from the target map and the target image, obtains the projection area of the optical center of the camera on the reference two-dimensional plane based on each 2D-3D point pair, and determines the target area in the reference two-dimensional plane by voting on the reference two-dimensional plane through the projection area, takes the 2D-3D point pairs corresponding to the target area as inner points, and determines the camera posture according to the inner points. In this way, by determining the inner points from the multiple 2D-3D point pairs obtained, the camera posture can be determined more accurately through these determined inner points.
[0065] Based on the above embodiment, in another embodiment provided by the present disclosure, the above step S230 may further include the following steps:
[0066] S231, obtaining the rotation angle range and height range of the camera.
[0067] In an embodiment, the rotation angle interval of the camera can be determined by obtaining magnetometer information provided by a magnetometer on a mobile phone, such as 100° to 200°, so as to avoid traversing the camera from 0° to 360°, thereby reducing the complexity of the calculation. In addition, the height interval of the camera can be set between -5m and 5m.
[0068] S232, traverse the rotation angle interval and the height interval respectively, obtain the projection range of the optical center of the camera on the reference two-dimensional plane, and determine the projection area based on the projection range.
[0069] It can be known from the above embodiments that, based on each 2D-3D point pair, by traversing the possible height values within the camera's height range and traversing the possible rotation angles of the camera in the rotation angle range, the range of the projection area of the camera's optical center on the reference two-dimensional plane is obtained. In the embodiment, in order to reduce the amount of calculation, the traversal can be performed at a preset traversal interval. For example, when traversing in the height range, the preset traversal interval of the traversed height value can be 1m, and when traversing in the rotation angle range, the preset traversal interval of the rotation angle is 5°. It can also be set according to actual needs, and the embodiment is not limited to this. In the embodiment, by traversing in the rotation angle range and the height range respectively, the traversal range can be narrowed, the projection area can be determined more quickly, the calculation efficiency is improved, and the accuracy of the camera's posture determination can also be further improved.
[0070] Based on the above embodiment, in another embodiment provided by the present disclosure, the reference two-dimensional plane may include multiple grids, and the above step S240 may further include the following steps:
[0071] S241, determining overlapping information between projection areas corresponding to a plurality of 2D-3D point pairs and a plurality of grids respectively.
[0072] S242, determining the number of votes corresponding to each grid based on the overlap information, and determining the target area in the reference plane based on the number of votes. If a grid and a projection area meet a preset overlap condition, the number of votes for the grid increases once.
[0073] In an embodiment, the preset overlap condition may be whether the overlap rate of a grid and the projection area reaches a preset overlap rate threshold. If the overlap rate of a grid and the projection area reaches the preset overlap rate threshold, it can be determined that the preset overlap condition is met, otherwise the preset overlap condition is not met. When the overlap rate of a grid and the projection area reaches the preset overlap rate threshold, a vote will be cast for the grid. The overlap rate in the embodiment may be the ratio of the overlap area of the corresponding grid to the grid area. The preset overlap rate may be 80% or 100%, etc., and the embodiment is not limited thereto.
[0074] In an embodiment, a target grid whose number of votes is greater than a first preset number can be obtained, and by counting the number of votes of each grid, grids whose number of votes is greater than the first preset number are determined, and these grids are determined as target grids. The first preset number can be set according to actual needs. Alternatively, multiple grids are sorted in descending order based on the number of votes, and a second preset number of target grids with higher rankings are obtained. In an embodiment, multiple grids can be sorted in descending order based on the number of votes, and a second preset number of grids with higher rankings are selected, and these grids are determined as target grids. The second preset number can be set according to actual needs.
[0075] In the embodiment, since the projection area may be an irregularly shaped area, and the shape of the grid is generally a triangle or a quadrilateral, when the projection area overlaps with multiple grids, the projection area may completely overlap with one grid, or may partially overlap. Therefore, in order to more accurately determine the number of votes corresponding to each grid, it can be determined based on the overlapping information such as the overlap rate between the projection area and the grid. In this way, it can be quickly and accurately determined whether each grid meets the preset overlapping conditions with the projection area, which can greatly improve the processing efficiency.
[0076] In an embodiment, the reference two-dimensional plane may include a plurality of first grids having a first resolution and a plurality of second grids having a second resolution, wherein the first resolution is greater than the second resolution, and the method may further include the following steps:
[0077] S260, obtaining the area size of the projection area.
[0078] S270, based on the area size, determine a target resolution corresponding to the projection area from at least the first resolution and the second resolution, and obtain the number of votes of the grids corresponding to the target resolution in the projection area.
[0079] S280, determining a target grid based on the number of votes of the projection area on the first grid and the second grid, and determining a target area based on the target grid.
[0080] In an embodiment, the shape and size of the grids in the first grid are consistent, and the shape and size of the grids in the second grid are consistent. In an embodiment, the reference two-dimensional plane area can be grid-divided at a certain resolution. For example, in a two-dimensional plane of 100m*100m, when the resolution is 20*20, if the grids are of equal size, the size of each grid is 5m*5m; when the resolution is 10*10, if the grids are of equal size, the size of each grid is 10m*10m. For each projection area, its voting range will cover the already divided grids, which is equivalent to voting for the covered grids. Since the size of each projection area may be inconsistent, such as a relatively large projection area, in order to facilitate calculation, a large grid with low resolution can be used, such as a second grid with a second resolution, so that the number of grids to be counted will be reduced. If the projection area is small, a small grid with high resolution can be used, such as a first grid with a first resolution, so that it is convenient to count the number of grids covered by the projection area. Therefore, the target grid corresponding to the first grid can be determined in the first grid, and the target grid corresponding to the second grid can be determined in the second grid. Different overlapping condition judgment criteria and target grid determination criteria can be used for grids of different resolutions. For example, the overlapping condition judgment criteria of high-resolution grids can be lowered, and the overlapping condition judgment criteria of low-resolution grids can be increased; for example, since the first resolution is greater than the second resolution, the overlapping condition judgment criteria of the second grid can be higher than the overlapping condition judgment criteria of the first grid. For example, the overlapping condition judgment criteria of the second grid can be an overlapping rate greater than 80%, and the overlapping condition judgment criteria of the first grid can be an overlapping rate greater than 50%. The embodiment is not limited to this. In the embodiment, the grid of corresponding resolution is used according to the size of the projection area, which can reduce the complexity of calculation and avoid the problem of high calculation complexity caused by using all high-resolution grids, resulting in too many grids covering a large projection area.
[0081] In an embodiment, the first grid and the second grid have a first preset corresponding relationship, and the first preset corresponding relationship may include one second grid corresponding to multiple first grids, so that the target grid in the first grid can be determined based on the number of votes of the first grid, the number of votes of the second grid and the first preset corresponding relationship, and the target grid in the second grid can also be determined. For example, in a reference two-dimensional plane, including a first grid with a resolution of 20*20 and a second grid with a resolution of 10*10, one second grid can correspond to 4 first grids, so that it can correspond to the first grid according to the number of votes on the second grid, that is, in the second grid, the votes on one grid correspond to the votes on 4 grids in the first grid. If the projection area is large, a grid with a small resolution can be used for voting statistics, and when the projection area is small, a grid with a large resolution can be used for voting statistics, so that the target grid can be finally determined uniformly in the grids with the same resolution. For example, when the target grid is determined in the first grid, if the second grid obtains one vote, the votes of the multiple first grids corresponding to the second grid are increased once, thereby improving the calculation efficiency during voting statistics.
[0082] In another embodiment provided by the present disclosure, the reference two-dimensional plane also includes a third grid with a third resolution, the third resolution is less than the first resolution, and is different from the second resolution. For example, the reference two-dimensional plane may also include a third grid with a third resolution, the third resolution is less than the second resolution, the first grid and the second grid respectively have a second corresponding preset relationship with the third grid rate, and the second preset relationship may include one second grid corresponding to multiple first grids, and one third grid corresponding to multiple second grids, so that the target grid in the first grid can be determined based on the number of votes of the first grid, the number of votes of the second votes, the number of votes of the third grid, and the second preset corresponding relationship, and the target grid in the second grid or the third grid can also be determined. For example, in the reference two-dimensional plane, a first grid with a resolution of 20*20 may be included, a second grid with a resolution of 10*10 may be included, and a third grid with a resolution of 5*5 may be included, that is, three layers of grids with different resolutions are covered. In this way, the number of votes on the third grid can be corresponded to the first grid, that is, the vote on one grid in the third grid corresponds to the vote on 16 grids in the first grid. Similarly, the number of votes on the second grid can be mapped to the first grid, that is, the votes on one grid in the second grid can be mapped to the votes on four grids in the first grid, so as to determine the target grid in the first grid. In the embodiment, if the projection area is large, a grid with a small resolution can be used for voting statistics, and if the projection area is small, a grid with a large resolution can be used for voting statistics, thereby improving the calculation efficiency.
[0083] In an embodiment, by dividing the reference two-dimensional plane into grids of a certain resolution, the distribution area of the corresponding projection area on the grid generated based on the 2D-3D point pair can be determined, that is, which grids can be covered by a projection area. Since there are multiple projection areas, the same grid may be covered by multiple projection areas at the same time, and each time a grid is covered by a projection area can be recorded as a vote, so that the number of votes corresponding to each grid can be obtained. In an embodiment, the target grid can be determined based on the number of votes counted, and the area where the target grid is located is used as the target area in the reference two-dimensional plane.
[0084] In the embodiments provided in the present disclosure, the projection area of each 2D-3D point pair projected on the two-dimensional plane can realize the voting of each 2D-3D point pair for the camera position on the reference two-dimensional plane. After voting based on all 2D-3D point pairs, the position of the camera is likely to be located in an area where the number of votes reaches a preset condition, and this area can be called the target area. Among them, the preset condition can be an area where the number of votes exceeds a certain threshold or one or several areas that are ranked at the front in the positive order of the votes, etc. Therefore, the 2D-3D point pair corresponding to the target area is most likely to be an inner point, so that the position and posture of the camera can be determined more accurately through the inner point.
[0085] Based on the above embodiment, in another embodiment provided by the present disclosure, the above step S240 may further include the following steps:
[0086] S243, obtaining the positioning information of the camera when shooting the target image.
[0087] S244, determining a positioning constraint condition of the reference two-dimensional plane based on the positioning information, and determining a target area in the reference two-dimensional plane based on the positioning constraint condition.
[0088] In an embodiment, after obtaining the user's consent through a pop-up window or the like, the positioning information of the camera when shooting the target image can be obtained. For example, the constraint area can be determined based on the positioning information, and the grids outside the constraint area can be excluded from the candidate range of the target grid, so that the target area can be determined from the grids located in the constraint area. For example, for any grid, when the overlap rate between the grid and the constraint area is greater than a threshold, the grid can be considered to be located in the constraint area. Among them, the threshold can be 100%, 80% or 50%, etc., and the embodiment is not limited to this. In an embodiment, it is also possible to first determine the initial target grid, and then use the positioning constraint condition to determine the final target grid from the initial target grid, or before voting for the grid, first use the positioning constraint condition to determine the candidate grid to be voted, and then determine the target grid from the candidate grid.
[0089] In the embodiments provided by the present disclosure, the first area range of the reference two-dimensional plane can also be determined based on the positioning information, and the votes outside the first area range can be excluded, so that the target area in the reference two-dimensional plane can be determined based on the number of votes after the votes are excluded. Specifically, the positioning information provided by GPS when shooting the target image can also be used to divide a 100m*100m area range. If the voting position of the 2D-3D point pair exceeds the area range, since the 2D-3D point pair is a wrong match, the voting result can be directly excluded, which can improve the efficiency of determining the inner point.
[0090] The camera pose determination method provided by the embodiments of the present disclosure can assist in determining the camera pose through at least one sensor information, including obtaining the gravity information of the image through a gravity sensor, and using the gravity information to align the image and the point cloud map in the gravity direction, and can also obtain constraint information determined by using a magnetometer and positioning information, and finally convert the matching of 2D-3D point pairs into constraints on the camera pose on a reference two-dimensional plane, which can effectively exclude mismatched 2D-3D point pairs, and then determine the inner points from multiple 2D-3D point pairs, which can effectively improve the camera pose determination accuracy and robustness.
[0091] In the case of dividing each functional module according to each function, an embodiment of the present disclosure provides a camera posture determination device, which can be a server or a chip applied to a server. Figure 3 A schematic block diagram of the functional modules of a camera posture determination device provided by an exemplary embodiment of the present disclosure. Figure 3 As shown, the camera posture determination device includes:
[0092] An image acquisition module 10 is used to acquire a target image captured by a camera, wherein the target image includes a plurality of 2D feature points;
[0093] A feature point acquisition module 20 is used to acquire 3D feature points matching the plurality of 2D feature points from the target map to obtain a plurality of 2D-3D point pairs;
[0094] A projection area determination module 30, for obtaining a projection area of the optical center of the camera on a reference two-dimensional plane based on each 2D-3D point pair, wherein the target image is aligned with the target map in the direction of gravity;
[0095] A target area determination module 40, configured to determine a target area in the two-dimensional plane through the projection areas corresponding to the multiple 2D-3D point pairs;
[0096] The pose determination module 50 is used to determine the inner points based on the 2D-3D point pairs corresponding to the target area, and determine the pose of the camera according to the inner points.
[0097] In another embodiment provided by the present disclosure, the projection area determination module is further configured to:
[0098] Obtaining the rotation angle range and height range of the camera;
[0099] The rotation angle interval and the height interval are traversed respectively to obtain a projection range of the optical center of the camera on a reference two-dimensional plane, and a projection area is determined based on the projection range.
[0100] In another embodiment provided by the present disclosure, the reference two-dimensional plane includes a plurality of grids, and the target area determination module is further configured to:
[0101] Determine overlapping information between the projection areas corresponding to the multiple 2D-3D point pairs and the multiple grids respectively;
[0102] The number of votes corresponding to each grid is determined based on the overlapping information, and the target area in the reference plane is determined based on the voting number; wherein, if one of the grids and one of the projection areas reaches a preset overlapping condition, the number of votes of the grid is increased once.
[0103] In another embodiment provided by the present disclosure, the target area determination module is further configured to:
[0104] Obtaining a target grid whose number of votes is greater than a first preset number;
[0105] Alternatively, the plurality of grids are sorted in descending order based on the number of votes, and a second preset number of target grids with top rankings are obtained;
[0106] A target area in the reference two-dimensional plane is determined based on the area where the target grid is located.
[0107] In another embodiment provided by the present disclosure, the reference two-dimensional plane includes a plurality of first grids having a first resolution and a plurality of second grids having a second resolution, the first resolution being greater than the second resolution, and the apparatus further includes:
[0108] An area acquisition module, used to acquire the area size of the projection area;
[0109] A grid acquisition module, configured to determine a target resolution corresponding to the projection area at least from the first resolution and the second resolution based on the area size, and acquire a number of votes for a grid corresponding to the target resolution in the projection area;
[0110] The area determination module is used to determine a target grid based on the number of votes of the projection area on the first grid and the second grid, and determine the target area based on the target grid.
[0111] In another embodiment provided by the present disclosure, the first grid and the second grid have a preset corresponding relationship, and the preset corresponding relationship includes that one second grid corresponds to multiple first grids; the device further includes:
[0112] The target grid determination module is used to determine the target grid in the first grid based on the voting data of the first grid, the number of votes of the second grid and the preset corresponding relationship.
[0113] In yet another embodiment provided by the present disclosure, the reference two-dimensional plane further includes a third grid having a third resolution, where the third resolution is smaller than the first resolution and different from the second resolution.
[0114] In another embodiment provided by the present disclosure, the target area determination module is further configured to:
[0115] Acquiring positioning information of the camera when taking the target image;
[0116] Determine a first area range of the two-dimensional plane based on the positioning information;
[0117] Votes outside the first area are excluded, and a target area in the two-dimensional plane is determined based on the number of votes after the excluded votes.
[0118] In another embodiment provided by the present disclosure, the device further includes:
[0119] An information acquisition module, used to acquire magnetometer information when the camera captures the target image;
[0120] The interval determination module is used to determine the rotation angle interval of the camera based on the magnetometer information.
[0121] Since the device part corresponds to the above method, please refer to the description of the method part for details, which will not be repeated here.
[0122] The camera posture determination device provided by the embodiment of the present disclosure obtains a target image captured by a camera, obtains multiple 2D-3D point pairs from the target map and the target image, obtains the projection area of the optical center of the camera on the reference two-dimensional plane based on each 2D-3D point pair, and determines the target area in the reference two-dimensional plane by voting on the reference two-dimensional plane through the projection area, takes the 2D-3D point pairs corresponding to the target area as inner points, and determines the camera posture according to the inner points. In this way, by determining the inner points from the obtained multiple 2D-3D point pairs, the camera posture can be determined more accurately through these determined inner points.
[0123] An embodiment of the present disclosure also provides an electronic device, comprising: at least one processor; and a memory for storing instructions executable by the at least one processor; wherein the at least one processor is configured to execute the instructions to implement the above method disclosed in the embodiment of the present disclosure.
[0124] Figure 4 The structure diagram of an electronic device provided by an exemplary embodiment of the present disclosure is shown in FIG. Figure 4 As shown, the electronic device 1800 includes at least one processor 1801 and a memory 1802 coupled to the processor 1801 , and the processor 1801 can execute corresponding steps in the above method disclosed in the embodiment of the present disclosure.
[0125] The processor 1801 may also be referred to as a central processing unit (CPU), which may be an integrated circuit chip having signal processing capabilities. Each step in the method disclosed in the embodiment of the present disclosure may be completed by an integrated logic circuit of hardware in the processor 1801 or by instructions in the form of software. The processor 1801 may be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present disclosure may be directly embodied as being executed by a hardware decoding processor, or may be executed by a combination of hardware and software modules in a decoding processor. The software module may be located in a memory 1802, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, or other mature storage media in the art. The processor 1801 reads the information in the memory 1802 and completes the steps of the method in combination with its hardware.
[0126] In addition, when various operations / processes according to the present disclosure are implemented by software and / or firmware, they can be transmitted from a storage medium or a network to a computer system having a dedicated hardware structure, such as Figure 5 The computer system 1900 shown is installed with the programs constituting the software. When the various programs are installed, the computer system can perform various functions, including the functions described above. Figure 5 A structural block diagram of a computer system provided for an exemplary embodiment of the present disclosure.
[0127] Computer system 1900 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.
[0128] like Figure 5 As shown, the computer system 1900 includes a computing unit 1901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1902 or a computer program loaded from a storage unit 1908 to a random access memory (RAM) 1903. In the RAM 1903, various programs and data required for the operation of the computer system 1900 can also be stored. The computing unit 1901, the ROM 1902, and the RAM 1903 are connected to each other via a bus 1904. An input / output (I / O) interface 1905 is also connected to the bus 1904.
[0129] A plurality of components in the computer system 1900 are connected to the I / O interface 1905, including: an input unit 1906, an output unit 1907, a storage unit 1908, and a communication unit 1909. The input unit 1906 may be any type of device capable of inputting information to the computer system 1900, and the input unit 1906 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1907 may be any type of device capable of presenting information, and may include, but is not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1908 may include, but is not limited to, a disk, an optical disk. The communication unit 1909 allows the computer system 1900 to exchange information / data with other devices over a network such as the Internet, and may include, but is not limited to, a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth™ device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.
[0130] The computing unit 1901 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1901 performs the various methods and processes described above. For example, in some embodiments, the above methods disclosed in the embodiments of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1908. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1900 via the ROM 1902 and / or the communication unit 1909. In some embodiments, the computing unit 1901 may be configured to perform the above methods disclosed in the embodiments of the present disclosure in any other appropriate manner (e.g., by means of firmware).
[0131] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the above method disclosed in the embodiment of the present disclosure.
[0132] The computer-readable storage medium in the disclosed embodiments may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. The computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specifically, the computer-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0133] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.
[0134] The embodiments of the present disclosure further provide a computer program product, including a computer program, wherein when the computer program is executed by a processor, the above method disclosed in the embodiments of the present disclosure is implemented.
[0135] In embodiments of the present disclosure, computer program codes for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof, including but not limited to object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer.
[0136] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.
[0137] The modules, components or units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the names of the modules, components or units do not, in some cases, limit the modules, components or units themselves.
[0138] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.
[0139] The above descriptions are only some embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, a technical solution formed by replacing the above features with the technical features with similar functions disclosed in the present disclosure (but not limited to).
[0140] Although some specific embodiments of the present disclosure have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present disclosure. It should be understood by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.
Claims
1. A camera pose determination method, characterized in that: The method comprises: Acquire a target image captured by a camera, wherein the target image includes a plurality of 2D feature points; Acquire 3D feature points matching the multiple 2D feature points from the target map to obtain multiple 2D-3D point pairs; Based on each 2D-3D point pair, a projection area of the optical center of the camera on a reference two-dimensional plane is obtained; the target image is aligned with the target map in the gravity direction; Determine a target area in the reference two-dimensional plane through the projection areas corresponding to the multiple 2D-3D point pairs; Inliers are determined based on 2D-3D point pairs corresponding to the target area, and the position and posture of the camera are determined according to the inliers.
2. The method according to claim 1, characterized in that The obtaining of a projection area of the optical center of the camera on a reference two-dimensional plane includes: Obtaining the rotation angle range and height range of the camera; The rotation angle interval and the height interval are traversed respectively to obtain a projection range of the optical center of the camera on a reference two-dimensional plane, and a projection area is determined based on the projection range.
3. The method according to claim 1, characterized in that: The reference two-dimensional plane includes a plurality of grids, and determining a target area in the reference two-dimensional plane includes: Determine overlapping information between the projection areas corresponding to the multiple 2D-3D point pairs and the multiple grids respectively; The number of votes corresponding to each grid is determined based on the overlapping information, and the target area in the reference plane is determined based on the voting number; wherein, if one of the grids and one of the projection areas reaches a preset overlapping condition, the number of votes of the grid is increased once.
4. The method according to claim 3, characterized in that The step of determining the target area in the reference two-dimensional plane based on the number of votes obtained includes: Obtaining a target grid whose number of votes is greater than a first preset number; Alternatively, the plurality of grids are sorted in descending order based on the number of votes, and a second preset number of target grids with top rankings are obtained; A target area in the reference two-dimensional plane is determined based on the area where the target grid is located.
5. The method according to claim 3, characterized in that: The reference two-dimensional plane includes a plurality of first grids having a first resolution and a plurality of second grids having a second resolution, the first resolution being greater than the second resolution, and the method further includes: Obtaining the area size of the projection area; Based on the size of the area, determine a target resolution corresponding to the projection area at least from the first resolution and the second resolution, and obtain the number of votes for the grids corresponding to the target resolution in the projection area; A target grid is determined based on the number of votes of the projection area on the first grid and the second grid, and the target area is determined based on the target grid.
6. The method according to claim 5, characterized in that The first grid and the second grid have a preset corresponding relationship, and the preset corresponding relationship includes that one second grid corresponds to multiple first grids; the method further includes: Based on the voting data of the first grid, the number of votes of the second grid and the preset corresponding relationship, a target grid in the first grid is determined.
7. The method according to claim 5, characterized in that The reference two-dimensional plane also includes a third grid having a third resolution, the third resolution being smaller than the first resolution and different from the second resolution.
8. The method according to claim 1, characterized in that The determining of the target area in the reference two-dimensional plane comprises: Acquiring positioning information of the camera when taking the target image; A positioning constraint condition of the reference two-dimensional plane is determined based on the positioning information, and a target area in the reference two-dimensional plane is determined based on the positioning constraint condition.
9. A camera posture determination device, characterized in that: The device comprises: An image acquisition module is used to acquire a target image captured by a camera, wherein the target image includes a plurality of 2D feature points; A feature point acquisition module, used to acquire 3D feature points matching the multiple 2D feature points from the target map to obtain multiple 2D-3D point pairs; A projection area determination module, for obtaining a projection area of the optical center of the camera on a reference two-dimensional plane based on each 2D-3D point pair, wherein the target image is aligned with the target map in a gravity direction; A target area determination module, configured to determine a target area in the two-dimensional plane through the projection areas corresponding to the multiple 2D-3D point pairs; The pose determination module is used to determine the inner points based on the 2D-3D point pairs corresponding to the target area, and determine the pose of the camera according to the inner points.
10. An electronic device, characterized in that: include: at least one processor; a memory for storing the at least one processor-executable instruction; The at least one processor is configured to execute the instructions to implement the method according to any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the method as claimed in any one of claims 1 to 8.
Citation Information
Patent Citations
Camera pose determination method and device, electronic equipment and storage medium
CN115205383A
Target pose estimation method and system based on attention mechanism and hough voting
WO2022178952A1
