A monocular camera horizontal orientation angle estimation method and system

Abstract

The application provides a monocular camera horizontal orientation angle estimation method and system, a depth model is used to process a single RGB image to obtain a depth map, back projection is performed to obtain three-dimensional coordinates in a camera coordinate system, a pitch angle and a roll angle are obtained, and the three-dimensional coordinates are converted to a world coordinate system; a building binary mask is output according to an image segmentation model and is aligned with the depth image pixels, the pixels are filtered and fitting is performed to obtain a fitting wall line; GPS latitude and longitude coordinates of the camera are converted to projection coordinates, corresponding unmanned aerial vehicle DOM tiles are selected, and a building edge ring binary mask is generated; all candidate angles are traversed, the wall surface 3D point cloud or the fitting wall line is rotated to the world coordinate system, and the number of the two falling on the building edge ring is counted as a basic score and is corrected; the wall surface 3D point cloud is divided into several segments, the optimal yaw estimation value in the segment is obtained through traversal, and the corresponding target yaw angle is obtained according to the circular dispersion degree. The application can significantly reduce matching noise and improve the estimation accuracy of the orientation angle.

Description

Technical Field

[0001] This invention relates to the field of camera attitude estimation technology, and in particular to a method and system for estimating the horizontal orientation angle of a monocular camera. Background Technology

[0002] For spatial orientation estimation using monocular cameras or street view images, the main technical approaches typically fall into the following categories: The first category is orientation estimation methods based on inertial devices or satellite navigation assistance. For example, these methods directly provide the camera's orientation prior using sensors such as GPS, electronic compasses, and IMUs, or first obtain a rough position and direction using consumer-grade GPS and compasses, then perform visual correction. The second category is monocular visual localization / orientation methods based on street view databases or prior map reconstructions. These methods generally perform coarse-to-fine matching between the monocular image to be estimated and street view databases such as Google Street View, then combine local bundle adjustment or 2D-3D geometric relationships to solve for the camera pose. The third category is cross-view matching methods between ground images and overhead images, which match street view images with satellite images, aerial images, or overhead view databases to estimate the position and orientation of the ground camera. While the aforementioned methods have demonstrated the feasibility of using ground image and overhead image matching to estimate camera orientation, they still have the following limitations: First, most methods use satellite images or publicly available aerial photographs as overhead references, which are limited in resolution, shooting time, and update frequency, making it difficult to flexibly adapt to the latest road conditions, temporary facilities, tree obstructions, and building changes in the target area. In contrast, UAV projection images have the characteristics of high resolution, strong timeliness, and controllable acquisition angle and range. Existing technologies still do not make sufficient use of "UAV projection images as orientation estimation reference images".

[0003] Second, many methods emphasize cross-view retrieval or location estimation. Although orientation estimation is involved, they are still not specific enough for the single problem of "monocular camera horizontal orientation angle". Especially under limited field of view, ordinary street view photos, and non-panoramic conditions, orientation estimation is still easily affected by matching ambiguity.

[0004] Third, while some existing technologies incorporate semantic information, they typically use semantic segmentation as an auxiliary feature or training label for deep networks. They rarely start from interpretable semantic entity correspondences and directly utilize stable semantic structures such as roads, building facades, intersections, green belts, and wall lines to establish explicit matching relationships between UAV projection images and street view images to solve for orientation angles. Therefore, when textures are repetitive, lighting changes drastically, seasonal changes are significant, or there is interference from dynamic targets, matching methods relying solely on appearance features remain insufficiently robust.

[0005] Fourth, many existing solutions rely on large-scale labeled datasets, complex deep network training, or pre-built databases for the target region, resulting in high training costs, limited cross-regional generalization capabilities, complex engineering deployment, and insufficient interpretability of results. Regarding the scenario addressed in the patent title—"using UAV projection images and street scene semantic matching to achieve monocular camera horizontal orientation angle estimation"—existing technologies have not yet fully resolved how to achieve highly robust and interpretable orientation angle estimation under conditions of limited prior knowledge, limited field of view, complex urban environments, and changing scenes.

[0006] Therefore, there is an urgent need for a new method for estimating the horizontal orientation angle of a monocular camera, which can fully utilize the high resolution and timeliness of UAV projection images and combine them with stable semantic structures in street view images for cross-view matching. Summary of the Invention

[0007] Therefore, it is necessary to provide a method and system for estimating the horizontal orientation angle of a monocular camera to address the aforementioned technical problems.

[0008] A method for estimating the horizontal orientation angle of a monocular camera includes the following steps: receiving a single RGB image and inputting it into a depth model to output a pixel-level depth map; back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters; obtaining the pitch and roll angles and converting the three-dimensional coordinates into point cloud coordinates in the world coordinate system; segmenting the building exterior wall region pixels from the single RGB image using a promptable image segmentation model, outputting a building binary mask, aligning the building binary mask with the depth map at the pixel level, and projecting the filtered pixels into a 3D point cloud of the wall in the BEV coordinate system based on joint filtering conditions; performing near-weighted RANSAC line fitting on the 3D point cloud of the wall to obtain a fitted wall line; receiving the camera's GPS latitude and longitude coordinates and converting them into projected coordinates; selecting UAV DOM tiles covering the corresponding locations based on the projected coordinates; and adding... Using the building binary mask, BEV frames are cropped centered on the projected coordinates. Morphological dilation-erosion difference operation is performed on the cropped BEV frames to generate a building edge ring binary mask. All candidate angles are traversed with a preset step size. The wall 3D point cloud or fitted wall line is rotated from the camera coordinate system to the world coordinate system. The number of hits of the rotated wall 3D point cloud or fitted wall line on the building edge ring binary mask is counted as a base score, and the base score is corrected through a consistent prior. The principal axis is obtained based on the principal component direction of the wall 3D point cloud. The wall 3D point cloud is divided into several segments along the arc length of the principal axis. The optimal yaw estimate within each segment is obtained based on the corrected base score, and its circular dispersion is calculated. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is taken as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line.

[0009] In one embodiment, the process of backprojecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters is expressed by the following formula: Z c (u,v)=D(u,v) ; X c (u,v)=[(uc x ) / f x ]*Z c ; Y c (u,v)= [(vc y ) / f y ]* Z c (u,v) ; In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Z c ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、f y This refers to the camera's focal length.

[0010] In one embodiment, obtaining the pitch and roll angles and converting the three-dimensional coordinates into point cloud coordinates in the world coordinate system includes: ; ; ; ; In the formula, P cam Three-dimensional coordinates in the camera coordinate system ( x c ,y c ,z c ), R roll This is the roll angle rotation matrix. ϕ This is the roll angle. Ppitch This is the pitch angle rotation matrix. i The pitch angle, P world These are the coordinates of the point cloud in the world coordinate system.

[0011] In one embodiment, the joint filtering condition is to retain pixels located within the building binary mask, with a depth within the minimum and maximum depth range, and a pixel-aligned height within the minimum and maximum height range.

[0012] In one embodiment, the step of traversing all candidate angles with a preset step size, rotating the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, and counting the number of hits of the rotated 3D point cloud of the wall or the fitted wall line falling on the binary mask of the building edge ring as a base score, and correcting the base score through a consistent prior, includes: traversing with a preset step size i yaw All candidate angles ∈ [0°, 360°); for each candidate angle, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, using the formula: ; ; In the formula, i yaw As candidate angles, ( x c , z c ) is the camera coordinate system for the 3D point cloud of the wall or the fitted wall line. x world , z world (The coordinate system is denoted as ). The number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring is counted to obtain a base score. The pixel proportions of the building binary mask in the left and right halves of the image are compared with the edge proportions of the building edge ring binary mask on the left and right sides of the camera at the current candidate angle. A consistency coefficient is calculated, and the base score is corrected based on the consistency coefficient to obtain a corrected score, using the following formula: ; In the formula, S ( i yaw (This is to correct the score.) H ( i yaw )for i yaw The number of hit points on the building edge ring of the 3D point cloud or fitted wall line on the lower wall surface. H ( iyaw )∈[0,1] is the consistency coefficient between the left-right ratio of the SAM segment and the left-right ratio of the DOM edge. w s These are lateral weights used to control the prior strength.

[0013] In one embodiment, the step of obtaining the principal axis based on the principal component direction of the wall 3D point cloud, dividing the wall 3D point cloud into several segments along the arc length of the principal axis, obtaining the optimal yaw estimate within each segment based on the corrected base score, and calculating its circular dispersion, where the optimal yaw estimate is taken as the target yaw angle when the circular dispersion is lower than a preset threshold; otherwise, obtaining the corresponding fitted wall line and obtaining the corresponding target yaw angle based on the fitted wall line includes: extracting the first principal component direction as the wall principal axis direction from the principal component direction of the wall 3D point cloud, projecting the wall 3D point cloud onto the principal axis, calculating the cumulative arc length of each point on the principal axis, and establishing a one-dimensional parameterized coordinate system; uniformly dividing the wall 3D point cloud into several segments along the arc length of the principal axis, independently performing a yaw search on each segment, obtaining the optimal yaw estimate within each segment by traversing the corrected base score; and calculating the circular dispersion of the optimal yaw estimate for each segment using the following formula: ; ; In the formula, S s Indicates the circular dispersion. α k Let K be the optimal yaw angle obtained by independent search of the k-th segment, where K is the total number of segments and R∈[0,1] is the average sum vector length. When the circular dispersion is lower than or equal to a preset threshold, it indicates that the orientation of each segment of the wall is the same, and the optimal yaw estimated angle within the segment is taken as the target yaw angle. When the circular dispersion is equal to or higher than the preset threshold, it indicates that there is a non-target building. The corresponding fitted wall line is obtained, and the optimal yaw estimated value is obtained based on the base score after correction of the fitted wall line, which is taken as the target yaw angle.

[0014] A monocular camera horizontal orientation angle estimation system is provided to implement the monocular camera horizontal orientation angle estimation method described above. The system includes: a depth map acquisition module for receiving a single RGB image and inputting it into a depth model, outputting a pixel-level depth map, and back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters; a coordinate transformation module for acquiring pitch and roll angles, converting the three-dimensional coordinates into point cloud coordinates in the world coordinate system; a wall segmentation module for segmenting building exterior wall pixels from the single RGB image based on a promptable image segmentation model, outputting a building binary mask, pixel-level aligning the building binary mask with the depth map, and projecting the filtered pixels into a 3D point cloud of the wall in the BEV coordinate system based on joint filtering conditions; a wall line fitting module for performing near-weighted RANSAC straight line fitting on the 3D point cloud of the wall to obtain a fitted wall line; and an edge loop generation module for receiving camera GPS latitude and longitude coordinates and converting them into projected coordinates, based on the... The projected coordinates select the drone DOM tiles covering the corresponding location, load the building binary mask, and crop the BEV frame centered on the projected coordinates. Morphological dilation-erosion difference operation is performed on the cropped BEV frame to generate a building edge ring binary mask. A scoring module iterates through all candidate angles with a preset step size, rotates the wall 3D point cloud or fitted wall line from the camera coordinate system to the world coordinate system, and counts the number of hits of the rotated wall 3D point cloud or fitted wall line on the building edge ring binary mask as the base score. The base score is then corrected using a consistent prior. A target yaw angle acquisition module obtains the principal axis based on the principal component direction of the wall 3D point cloud, divides the wall 3D point cloud into several segments along the arc length of the principal axis, obtains the optimal yaw estimate within each segment based on the corrected base score, and calculates its circular dispersion. When the circular dispersion is below a preset threshold, the optimal yaw estimate is used as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained from the fitted wall line.

[0015] Compared to existing technologies, the advantages and beneficial effects of this invention are as follows: By inputting a single RGB image into a depth model, a pixel-level depth map is obtained; based on camera intrinsic parameters, the depth map is back-projected into three-dimensional coordinates in the camera coordinate system; the pitch and roll angles are obtained, and the three-dimensional coordinates are converted into point cloud coordinates in the world coordinate system; based on a suggestible image segmentation model, pixels of the building exterior wall region are segmented from a single RGB image, accurately distinguishing between building exterior walls and non-wall targets, significantly reducing matching noise, outputting a building binary mask, and performing pixel-level alignment with the depth map; based on joint filtering conditions, the filtered and retained pixels are projected into a 3D point cloud in the BEV coordinate system; a near-weighted RANSAC straight-line fitting is performed on the wall 3D point cloud to obtain the fitted wall line; the camera's GPS latitude and longitude coordinates are received and converted into projected coordinates; based on the projected coordinates, UAV DOM tiles covering the corresponding locations are selected, the building binary mask is loaded, BEV frames are cropped with the projected coordinates as the center, and morphological dilation-erosion difference operations are performed on them to generate building edge loops. Binary masks improve spatial resolution and expand coverage. By replacing the fill area mask with a morphological edge loop, a refined ability to distinguish between interior and exterior buildings is achieved. All candidate angles are traversed with a preset step size. The 3D point cloud of the wall or the fitted wall line is rotated from the camera coordinate system to the world coordinate system. The number of hits of the rotated 3D point cloud or fitted wall line on the binary mask of the building edge loop is used as the base score. The base score is corrected through a consistent prior. Simultaneously, two matching primitives, wall line and point cloud, are maintained to achieve adaptive selection for different wall shapes (straight / curved / complex facades). The principal axis is obtained from the principal component direction of the 3D point cloud of the wall. The 3D point cloud of the wall is divided into several segments along the arc length of the principal axis. The optimal yaw estimate within each segment is obtained based on the corrected base score, and its circular dispersion is calculated. When the circular dispersion is below a preset threshold, the optimal yaw estimate is used as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained from the fitted wall line. This prevents catastrophic mismatches and improves the estimation accuracy of the orientation angle. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a method for estimating the horizontal orientation angle of a monocular camera in one embodiment.

[0017] Figure 2 This is a flowchart of a method for estimating the horizontal orientation angle of a monocular camera in one embodiment.

[0018] Figure 3 This is a schematic diagram of the structure of a monocular camera horizontal orientation angle estimation system in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In one embodiment, such as Figure 1-2 As shown, a method for estimating the horizontal orientation angle of a monocular camera is provided, including the following steps: Step S110: Receive a single RGB image and input it into the depth model, output a pixel-level depth map, and back-project the depth map into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters.

[0021] Specifically, a single street scene RGB image is captured by a fixed surveillance camera, such as a traffic monitoring camera or a security monitoring camera. The single RGB image is input into a depth model, and the depth model outputs a pixel-level depth map. Based on the camera intrinsic parameters, the depth map is back-projected into three-dimensional coordinates in the camera coordinate system. The real-scale 3D geometric point cloud of the scene is recovered from the monocular image, providing a foundation for subsequent BEV projection and map matching.

[0022] The formula for backprojecting the depth map into three-dimensional coordinates in the camera coordinate system is as follows: Z c (u,v)=D(u,v) ; X c (u,v)=[(uc x ) / f x ]*Z c ; Y c (u,v)= [(vc y ) / f y ]* Z c (u,v) ; In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Z c ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、fy This refers to the camera's focal length.

[0023] Step S120: Obtain the pitch and roll angles, and convert the 3D coordinates into point cloud coordinates in the world coordinate system.

[0024] Specifically, the pitch and roll angles provided by the sensor are obtained, and the three-dimensional coordinates in the camera coordinate system are converted into point cloud coordinates in the world coordinate system to facilitate subsequent BEV (Bird's Eye View Projection) projection.

[0025] The formula for converting 3D coordinates to point cloud coordinates in the world coordinate system is as follows: ; ; ; ; In the formula, P cam Three-dimensional coordinates in the camera coordinate system ( x c ,y c ,z c ), R roll This is the roll angle rotation matrix. ϕ This is the roll angle. P pitch This is the pitch angle rotation matrix. i The pitch angle, P world These are the coordinates of the point cloud in the world coordinate system.

[0026] Specifically, the 3D point cloud in the camera coordinate system is transformed into three-dimensional coordinates in the world coordinate system to eliminate perspective distortion, so as to facilitate subsequent BEV projection.

[0027] Step S130: Based on the image segmentation model with prompts, the pixels of the building exterior wall area are segmented from a single RGB image, and a building binary mask is output. The building binary mask is aligned with the depth map at the pixel level. Based on the joint filtering conditions, the filtered and retained pixels are projected into a 3D point cloud of the wall in the BEV coordinate system.

[0028] Specifically, a prompting-enabled image segmentation model (such as SAM3.1) is invoked, using text prompts such as descriptions of building exterior walls, above-ground building facades, and other sets of building exterior wall descriptions to guide the model in segmenting the pixel regions of building exterior walls from a single RGB image. The model outputs a binary mask of the building and aligns the SAM semantic mask with the depth map at the pixel level, achieving calibration and synchronization between the depth camera and the RGB camera. Based on joint filtering conditions, the aligned binary mask of the building is filtered to obtain the retained pixels, which are then projected as a 3D point cloud of the wall in the BEV coordinate system. By using text-guided semantic segmentation instead of traditional height interval geometric filtering, the model can accurately distinguish between building exterior walls and non-wall targets such as pedestrians, vehicles, road signs, and roadside trees, significantly reducing matching noise and retaining only the pixels that truly belong to the building exterior walls for subsequent matching.

[0029] The joint filtering condition is to retain pixels that are located within the building's binary mask, have a depth within the minimum and maximum depth range, and whose height after pixel alignment is within the minimum and maximum height range.

[0030] Step S140: Perform near-weighted RANSAC line fitting on the 3D point cloud of the wall to obtain the fitted wall line.

[0031] Specifically, a near-weighted RANSAC straight line fitting is performed on the 3D point cloud of the wall. The sampling probability is set to be inversely proportional to the point depth, so that the weight of near points is higher, making the fitted wall line anchored on the near main wall surface. At the same time, principal component analysis of interior points can be used to adjust the orientation parameters.

[0032] Based on the standard RANSAC, a distance decay weight function is introduced, which gives a higher weight to the interior points that are close to the current fitted line in the model evaluation and final parameter optimization, thereby improving the fitting accuracy for uniformly distributed noise and non-uniformly distributed data.

[0033] Step S150: Receive the camera's GPS latitude and longitude coordinates and convert them into projected coordinates. Select the UAV DOM tile covering the corresponding location based on the projected coordinates, load the building binary mask, crop out the BEV frame centered on the projected coordinates, perform morphological dilation-erosion difference operation on the cropped BEV frame, and generate a building edge ring binary mask.

[0034] Specifically, the camera's GPS latitude and longitude coordinates (WGS84 / CGCS2000 coordinate system) are received and converted into projected coordinates (E, N) through Gauss-Kruger projection (EPSG:4544, central meridian 105°E). Based on the projected coordinates, the UAV DOM tiles covering the location are automatically selected, and a pre-generated building binary mask is loaded. BEV frames are cropped with the camera's projected coordinates as the center. Morphological dilation-erosion difference operation is performed on the cropped BEV frames. The dilation is used to expand the boundary of the foreground region, and the top view is used to shrink the boundary of the foreground region. The difference between the two is used to obtain the thickness of the object edge. This method has stronger robustness to noise and can maintain the connectivity of the edges, generating a building edge loop binary mask.

[0035] The physical meaning of the edge ring is the outer boundary ring of the building outline. Only wall points that truly fit the outline of the building's exterior wall can hit the edge ring and score points, while points that fall inside the building do not score points, thus achieving a precise distinction between the interior and exterior of the building.

[0036] Step S160: Traverse all candidate angles with a preset step size, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, count the number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring as the base score, and correct the base score through consistent prior.

[0037] Specifically, based on a preset step size, such as 1°, all candidate angles are traversed. At each candidate angle, the 3D point cloud of the wall or the fitted wall line is rotated from the camera coordinate system to the world coordinate system. The number of points or lines that fall on the building edge ring pixels after rotation is counted to obtain a basic score. Spatial consistency and DOM edge ring are used as constraints to perform consistency priors and correct the basic score to improve mask accuracy.

[0038] In addition, this embodiment retains two matching primitives: the fitted wall line and the 3D point cloud of the wall surface, to adapt to different wall surface shapes. The fitted wall line has strong directional constraints and is resistant to interference from distant buildings in straight wall or long straight street wall scenes; the 3D point cloud of the wall surface can maintain the integrity of shape information in curved or curved surface scenes; the two primitives can be adaptively switched according to the scene.

[0039] Step S160 includes: traversing at a preset step size i yaw All candidate angles ∈ [0°, 360°); for each candidate angle, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, using the formula: ; ; In the formula, i yawAs candidate angles, ( x c , z c ) is the camera coordinate system for the 3D point cloud of the wall or the fitted wall line. x world , z world (The coordinate system is denoted as ). The number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring is counted to obtain the base score. The pixel proportions of the building binary mask in the left and right halves of the image are compared with the edge proportions of the building edge ring binary mask on the left and right sides of the camera at the current candidate angle. A consistency coefficient is calculated, and the base score is corrected based on the consistency coefficient to obtain the corrected score. The formula is: ; In the formula, S ( i yaw (This is to correct the score.) H ( i yaw )for i yaw The number of hit points on the building edge ring of the 3D point cloud or fitted wall line on the lower wall surface. H ( i yaw )∈[0,1] is the consistency coefficient between the left-right ratio of the SAM segment and the left-right ratio of the DOM edge. w s The lateral weight is used to control the prior strength and is set to 0.75.

[0040] Specifically, when performing consistency prior, the consistency coefficient is calculated by comparing the pixel proportions of the SAM semantic mask in the left and right halves of the image with the edge proportions of the DOM edge ring on the left and right sides of the camera under the current yaw. The base score is then corrected in a weighted manner to ensure that the segmentation result is robust under geometric transformation.

[0041] Step S170: Obtain the principal axis based on the principal component direction of the wall 3D point cloud computing. Divide the wall 3D point cloud into several segments along the arc length of the principal axis. Obtain the optimal yaw estimate within the segment based on the corrected base score and calculate its circular dispersion. When the circular dispersion is lower than a preset threshold, take the optimal yaw estimate as the target yaw angle. Otherwise, obtain the corresponding fitted wall line and obtain the corresponding target yaw angle based on the fitted wall line.

[0042] Specifically, the principal axis is obtained based on the principal component direction of the 3D point cloud of the wall. The 3D point cloud of the wall is uniformly divided into several segments along the arc length of the principal axis. A yaw search is performed independently on each segment based on the correction score. The angle with the highest score is taken as the optimal yaw estimate within the segment. The circular dispersion of the optimal yaw estimate within the segment is calculated. The circular dispersion is used to determine whether the optimal yaw estimate within the segment is the target yaw angle. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is taken as the target yaw angle. Otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line. This effectively prevents the jump error of point cloud matching under the interference of curved walls or distant buildings, and can maintain the matching accuracy in normal scenes.

[0043] Step S170 includes: extracting the principal component direction of the wall 3D point cloud as the principal axis direction, projecting the wall 3D point cloud onto the principal axis, calculating the cumulative arc length of each point on the principal axis, and establishing a one-dimensional parameterized coordinate system; uniformly dividing the wall 3D point cloud into several segments along the principal axis arc length, independently performing a yaw search on each segment, and obtaining the optimal yaw estimate within each segment based on the corrected base score; calculating the circular dispersion of the optimal yaw estimate for each segment, using the following formula: ; ; In the formula, S s Indicates the circular dispersion. α k Let K be the optimal yaw angle obtained by independent search of the k-th segment, where K is the total number of segments and R∈[0,1] is the average resultant vector length. When the circular dispersion is lower than the preset threshold, it means that the orientation of each segment of the wall is the same, and the optimal yaw estimated angle in the segment is taken as the target yaw angle. When the circular dispersion is equal to or higher than the preset threshold, it means that there is a non-target building, and the corresponding fitted wall line is obtained. The optimal yaw estimated value is obtained based on the base score after the fitted wall line is corrected, and it is taken as the target yaw angle. Specifically, principal component direction calculations are performed on the 3D point cloud of the wall surface. The first principal component, i.e., the direction of maximum variance, is extracted as the principal axis direction of the wall surface. The 3D point cloud of the wall surface is projected onto this principal axis, and the cumulative arc length of each point on the principal axis is calculated to establish one-dimensional parameterized coordinates. The 3D point cloud of the wall surface is evenly divided into several segments along the arc length of the principal axis. A yaw search is performed independently on each segment, and the optimal yaw estimate within the segment is obtained by iterating through the corrected scores. Since the yaw angle has periodicity, the global directional stability of the wall surface point cloud can be judged based on the circular dispersion of the calculated optimal yaw estimate of each segment.

[0044] When the average resultant vector length approaches 1, the circular dispersion approaches 0, indicating that the height of each yaw segment is consistent and the global orientation of the wall is stable. When the average resultant vector length approaches 0, the circular dispersion approaches infinity, indicating that the height of each yaw segment is uniformly distributed. The preset threshold for circular dispersion can be set to 35°. When the circular dispersion is less than 35°, it means that the segments are oriented in the same direction. Conversely, when the circular dispersion is equal to or greater than 35°, it means that points from distant non-target buildings may have been mixed in. In this case, we fall back to the fitted wall line branch with stronger directional constraints, traverse the optimal yaw estimate based on the fitted wall line, and use it as the target yaw angle. This allows us to automatically fall back to a more robust matching method when distant non-target buildings are mixed in, preventing catastrophic mismatches.

[0045] In this embodiment, a pixel-level depth map is obtained by inputting a single RGB image into a depth model. Based on camera intrinsic parameters, the depth map is back-projected into 3D coordinates in the camera coordinate system. The pitch and roll angles are obtained, and the 3D coordinates are converted into point cloud coordinates in the world coordinate system. According to the suggestible image segmentation model, the pixels of the building exterior wall area are segmented from the single RGB image, accurately distinguishing the building exterior wall from non-wall targets, significantly reducing matching noise, outputting a building binary mask, and aligning it pixel-level with the depth map. Based on joint filtering conditions, the filtered and retained pixels are projected into a 3D point cloud in the BEV coordinate system. Near-weighted RANSAC line fitting is performed on the wall 3D point cloud to obtain the fitted wall line. The camera's GPS latitude and longitude coordinates are received and converted into projected coordinates. Based on the projected coordinates, the UAV DOM tiles covering the corresponding positions are selected, the building binary mask is loaded, and the BEV frame is cropped with the projected coordinates as the center. Morphological dilation-erosion difference operation is performed on it to generate a building edge ring binary mask, improving the accuracy of the image. Spatial resolution is improved, expanding the coverage area. By replacing the fill area mask with a morphological edge ring, a refined ability to distinguish between interior and exterior buildings is achieved. All candidate angles are traversed with a preset step size. The 3D point cloud of the wall or the fitted wall line is rotated from the camera coordinate system to the world coordinate system. The number of hits of the rotated 3D point cloud or fitted wall line on the binary mask of the building edge ring is used as the base score. The base score is corrected through a consistent prior. Simultaneously, two matching primitives, wall line and point cloud, are maintained to achieve adaptive selection for different wall shapes (straight / curved / complex facades). The principal axis is obtained from the principal component direction of the 3D point cloud of the wall. The 3D point cloud of the wall is divided into several segments along the arc length of the principal axis. The optimal yaw estimate within each segment is obtained based on the corrected base score, and its circular dispersion is calculated. When the circular dispersion is below a preset threshold, the optimal yaw estimate is used as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained from the fitted wall line. This prevents catastrophic mismatches and improves the estimation accuracy of the orientation angle.

[0046] like Figure 3 As shown, a monocular camera horizontal orientation angle estimation system 20 is provided to implement the monocular camera horizontal orientation angle estimation method described above, including: The depth map acquisition module 21 is used to receive a single RGB image and input it into the depth model, outputting a pixel-level depth map, and back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on the camera intrinsic parameters. The coordinate transformation module 22 is used to obtain the pitch angle and roll angle, and convert the three-dimensional coordinates into point cloud coordinates in the world coordinate system. The wall segmentation module 23 is used to segment a large model based on the promptable image, segment the pixels of the building exterior wall area from a single RGB image, output a building binary mask, align the building binary mask with the depth map at the pixel level, and project the filtered and retained pixels into a 3D point cloud of the wall in the BEV coordinate system based on joint filtering conditions. The wall line fitting module 24 is used to perform near-weighted RANSAC line fitting on the 3D point cloud of the wall to obtain the fitted wall line. The edge ring generation module 25 is used to receive the GPS latitude and longitude coordinates of the camera and convert them into projected coordinates. Based on the projected coordinates, it selects the UAV DOM tiles covering the corresponding location, loads the building binary mask, clips out the BEV frame with the projected coordinates as the center, performs morphological dilation-erosion difference operation on the clipped BEV frame, and generates the building edge ring binary mask. The scoring module 26 is used to traverse all candidate angles with a preset step size, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, count the number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring as the basic score, and correct the basic score through consistent prior. The target yaw angle acquisition module 27 is used to obtain the principal axis based on the principal component direction of the wall 3D point cloud computing, divide the wall 3D point cloud into several segments along the arc length of the principal axis, obtain the optimal yaw estimate within the segment based on the corrected base score, and calculate its circular dispersion. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is taken as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line.

[0047] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0048] It is obvious to those skilled in the art that the modules or steps of the present invention described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a computer storage medium (ROM / RAM, magnetic disk, optical disk) for execution by the computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Therefore, the present invention is not limited to any particular hardware and software combination.

[0049] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered within the scope of protection of the present invention.

Claims

1. A method for estimating the horizontal orientation angle of a monocular camera, characterized in that, Includes the following steps: A single RGB image is received and input into a depth model, and a pixel-level depth map is output. The depth map is then back-projected into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters. Obtain the pitch and roll angles, and convert the three-dimensional coordinates into point cloud coordinates in the world coordinate system; Based on the suggestible image segmentation model, the pixels of the building exterior wall area are segmented from the single RGB image, and a building binary mask is output. The building binary mask is then aligned with the depth map at the pixel level. Based on joint filtering conditions, the filtered and retained pixels are projected into a 3D point cloud of the wall in the BEV coordinate system. Perform a near-weighted RANSAC line fitting on the 3D point cloud of the wall to obtain the fitted wall line; Receive the GPS latitude and longitude coordinates of the camera and convert them into projected coordinates. Select the UAV DOM tile covering the corresponding position according to the projected coordinates, load the building binary mask, crop out the BEV frame with the projected coordinates as the center, perform morphological dilation-erosion difference operation on the cropped BEV frame, and generate the building edge ring binary mask. Traverse all candidate angles with a preset step size, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, count the number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring as the base score, and correct the base score through consistent prior. The principal axis is obtained from the principal component direction of the 3D point cloud of the wall. The 3D point cloud of the wall is divided into several segments along the arc length of the principal axis. The optimal yaw estimate within the segment is obtained based on the corrected base score, and its circular dispersion is calculated. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is taken as the target yaw angle. Otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line.

2. The method for estimating the horizontal orientation angle of a monocular camera according to claim 1, characterized in that, The formula for backprojecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters is as follows: Z c (u,v)=D(u,v) ； X c (u,v)=[(uc x ) / f x ]*Z c ； Y c (u,v)= [(vc y ) / f y ]* Z c (u,v) ； In the formula, (u,v) For pixels, D(u,v) The depth of a pixel in the depth map is ( X c ,Y c ,Z c ( ) represents three-dimensional coordinates. c x 、c y As the camera's main point, f x 、f y This refers to the camera's focal length.

3. The method for estimating the horizontal orientation angle of a monocular camera according to claim 2, characterized in that, The process of obtaining the pitch and roll angles and converting the three-dimensional coordinates into point cloud coordinates in the world coordinate system includes: ； ； ； ； In the formula, P cam Three-dimensional coordinates in the camera coordinate system ( x c ,y c ,z c ), R roll This is the roll angle rotation matrix. ϕ This is the roll angle. P pitch This is the pitch angle rotation matrix. θ The pitch angle, P world These are the coordinates of the point cloud in the world coordinate system.

4. The method for estimating the horizontal orientation angle of a monocular camera according to claim 1, characterized in that, The joint filtering condition is to retain pixels located within the building's binary mask, with a depth within the minimum and maximum depth range, and a height within the minimum and maximum height range after pixel alignment.

5. The method for estimating the horizontal orientation angle of a monocular camera according to claim 1, characterized in that, The process involves traversing all candidate angles with a preset step size, rotating the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, and counting the number of hits where the rotated 3D point cloud of the wall or the fitted wall line falls on the binary mask of the building edge ring as a base score. This base score is then corrected through a consistent prior, including: Traverse with preset step size θ yaw All candidate angles ∈ [0°, 360°); For each candidate angle, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, using the following formula: ； ； In the formula, θ yaw As candidate angles, ( x c , z c ) is the camera coordinate system for the 3D point cloud of the wall or the fitted wall line. x world , z world () represents the corresponding world coordinate system; The base score is obtained by counting the number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring. Compare the pixel proportions of the building binary mask in the left and right halves of the image with the edge proportions of the building edge ring binary mask on the left and right sides of the camera at the current candidate angle. Calculate the consistency coefficient, and adjust the base score based on the consistency coefficient to obtain the adjusted score. The formula is as follows: ； In the formula, S ( θ yaw (This is to correct the score.) H ( θ yaw )for θ yaw The number of hit points on the building edge ring of the 3D point cloud or fitted wall line on the lower wall surface. H ( θ yaw )∈[0,1] is the consistency coefficient between the left-right ratio of the SAM segment and the left-right ratio of the DOM edge. w s These are lateral weights used to control prior strength.

6. The method for estimating the horizontal orientation angle of a monocular camera according to claim 1, characterized in that, The process involves obtaining the principal axis based on the principal component direction of the 3D point cloud of the wall surface, dividing the 3D point cloud of the wall surface into several equal segments along the arc length of the principal axis, obtaining the optimal yaw estimate within each segment based on the corrected base score, and calculating its circular dispersion. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is used as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line, including: For the principal component direction of the 3D point cloud of the wall, extract the first principal component direction as the principal axis direction of the wall, project the 3D point cloud of the wall onto the principal axis, calculate the cumulative arc length of each point on the principal axis, and establish a one-dimensional parametric coordinate. The wall 3D point cloud is evenly divided into several segments along the main axis arc length. A yaw search is performed independently on each segment of the point cloud. The optimal yaw estimate within the segment is obtained by iterating through the corrected base score. The circular dispersion of the optimal yaw estimate for each segment is calculated using the following formula: ； ； In the formula, S s Indicates the circular dispersion. α k Let be the optimal yaw angle obtained by independent search of the kth segment, where K is the total number of segments and R∈[0,1] is the average length of the composite vector. When the circular dispersion is lower than or equal to a preset threshold, it indicates that all segments of the wall face are oriented in the same direction, and the optimal yaw estimated angle within the segment is taken as the target yaw angle. When the circular dispersion is equal to or higher than a preset threshold, it indicates the existence of a non-target building. The corresponding fitted wall line is obtained, and the optimal yaw estimate is obtained based on the base score after correction of the fitted wall line, which is used as the target yaw angle.

7. A monocular camera horizontal orientation angle estimation system, characterized in that, A method for estimating the horizontal orientation angle of a monocular camera as described in any one of claims 1-6 includes: The depth map acquisition module is used to receive a single RGB image and input it into the depth model, outputting a pixel-level depth map, and back-projecting the depth map into three-dimensional coordinates in the camera coordinate system based on camera intrinsic parameters. The coordinate transformation module is used to obtain the pitch angle and roll angle, and convert the three-dimensional coordinates into point cloud coordinates in the world coordinate system; The wall segmentation module is used to segment a large model based on the promptable image, segment the pixels of the building exterior wall area from the single RGB image, output the building binary mask, align the building binary mask with the depth map at the pixel level, and project the filtered and retained pixels into a 3D point cloud of the wall in the BEV coordinate system based on joint filtering conditions. The wall line fitting module is used to perform near-weighted RANSAC straight line fitting on the 3D point cloud of the wall to obtain the fitted wall line. The edge ring generation module is used to receive the GPS latitude and longitude coordinates of the camera and convert them into projected coordinates. Based on the projected coordinates, it selects the UAV DOM tiles covering the corresponding location, loads the building binary mask, clips out BEV frames with the projected coordinates as the center, performs morphological dilation-erosion difference operation on the clipped BEV frames, and generates a building edge ring binary mask. The scoring module is used to traverse all candidate angles with a preset step size, rotate the 3D point cloud of the wall or the fitted wall line from the camera coordinate system to the world coordinate system, count the number of hits of the rotated 3D point cloud of the wall or the fitted wall line on the binary mask of the building edge ring as the base score, and correct the base score through consistent prior. The target yaw angle acquisition module is used to obtain the principal axis based on the principal component direction of the wall 3D point cloud, divide the wall 3D point cloud into several segments along the arc length of the principal axis, obtain the optimal yaw estimate within the segment based on the corrected base score, and calculate its circular dispersion. When the circular dispersion is lower than a preset threshold, the optimal yaw estimate is taken as the target yaw angle; otherwise, the corresponding fitted wall line is obtained, and the corresponding target yaw angle is obtained based on the fitted wall line.

Images