Method and apparatus for calculating true azimuth of a carrier using the sun

Abstract

The application relates to a method and device for calculating a true azimuth angle of a carrier by using the sun, wherein the method comprises the following steps: S1. calculating a solar azimuth angle A s by using a solar altitude angle H s , a geographical latitude phi and a solar declination delta; S2. acquiring a target image with the sun by using a camera, detecting and identifying a solar contour from the target image, and calculating pixel plane coordinates [u, v] T of a solar centroid according to the solar contour; S3. calculating a first included angle between the sun and an optical axis of a camera coordinate system according to the pixel plane coordinates of the solar centroid; S4. acquiring an installation included angle between the optical axis of the camera coordinate system and a carrier on which the camera is installed, and obtaining a true azimuth angle of the carrier according to the installation included angle, the first included angle and the solar azimuth angle A s .

Description

Technical Field

[0001] This invention relates to the field of visual positioning and assisted autonomous driving, and in particular to a method and apparatus for calculating the true azimuth of a carrier using the sun. Background Technology

[0002] Visual odometry (VO) is an emerging technology that estimates a vehicle's motion by using a single or multiple cameras as input. Its applications span autonomous driving, robotics, drones, and augmented reality (AR). The concept of visual odometry was coined by Nister in his 2004 Roadmap paper. The term is very similar to wheel odometry, which incrementally estimates a vehicle's motion by integrating the number of wheel rotations. Similarly, visual odometry uses onboard cameras to detect motion changes in images to augment the vehicle's pose. For visual odometry to be more effective, sufficient environmental images are necessary; static images with sufficient texture are required to extract motion features. Additionally, capturing consecutive images to create scene overlap is also necessary.

[0003] Compared to wheel-based distance measurement, visual odometry (VOM) is unaffected by wheel slippage on uneven surfaces or other adverse conditions. Since modern robots, automobiles, and other vehicles are generally equipped with camera sensors, VOM does not significantly increase additional costs. Furthermore, VOM provides more accurate trajectory estimation, with a relative position error range of 0.1% to 2%. This makes VOM a valuable supplement to wheel-based distance measurement, as well as other navigation systems such as GPS, BeiDou, and other Global Navigation Satellite Systems (GNSS), Inertial Measurement Units (IMUs), and radar ranging systems. VOM technology is particularly important in environments where GNSS signals are unavailable, such as in urban high-rise buildings, tunnels, underwater, or in space.

[0004] However, neither visual odometry, inertial measurement, nor vehicle wheel speedometers can determine the true azimuth of a vehicle like GNSS. True azimuth refers to the direction pointing north from any point on the ground, also known as true north. True azimuth information is crucial for navigation; only by knowing the true north direction of the vehicle's movement can navigation, path planning, and other operations be performed quickly and reliably. Summary of the Invention

[0005] The purpose of this invention is to provide a method and apparatus for calculating the true azimuth of a carrier using the sun.

[0006] To achieve the above-mentioned objective, this invention provides a method for calculating the true azimuth of a carrier using the sun, comprising:

[0007] S1. Utilizing the solar altitude angle H s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. s ;

[0008] S2. Acquire a target image containing the sun using a camera, detect and identify the sun's outline from the target image, and calculate the pixel plane coordinates [u,v] of the sun's centroid based on the sun's outline. T ;

[0009] S3. Calculate the first angle between the sun and the optical axis of the camera coordinate system based on the pixel plane coordinates of the sun's centroid;

[0010] S4. Obtain the mounting angle between the optical axis of the camera coordinate system and the carrier on which the camera is mounted, and determine the mounting angle, the first angle, and the solar azimuth angle A based on the mounting angle, the first angle, and the solar azimuth angle A. s The true azimuth angle of the carrier is obtained.

[0011] According to one aspect of the invention, in step S1, the solar altitude angle H is used. s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. s The steps include:

[0012] S11. Calculate the solar declination δ using the number of days N from January 1st of the current year to the calculation date, expressed as:

[0013] δ(deg)=0.006918-0.399912cos(b)+0.070257sin(b)-0.006758cos(2b)+0.000907sin(2b)-0.002697cos(3b)+0.00148sin(3b)

[0014] Where b = 2 × PI × (N-1) / 365, PI represents pi, and deg represents degrees.

[0015] S12. Calculate the true solar time based on the mean solar time and the time difference of the true solar time obtained from the table, and calculate the solar hour angle t based on the true solar time; wherein, the true solar time is expressed as:

[0016] True solar time = Mean solar time + True solar time difference

[0017] The solar hour angle t is expressed as:

[0018] Solar time angle t = (true solar time - 12) × 15°;

[0019] S13. The solar altitude angle H is calculated based on the current geographical latitude φ of the carrier, the solar declination δ, and the solar hour angle t. s It is represented as:

[0020] sinH s =sinφ×sinδ+cosφ×cosδ×cost;

[0021] S14. According to the solar altitude angle H s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. s It is represented as:

[0022] cosA s =(sinφ×sinδ-sinδ)÷(cosH s ×cosφ).

[0023] According to one aspect of the present invention, step S2, which involves acquiring a target image containing the sun using a camera, detecting and identifying the sun's outline from the target image, and calculating the pixel plane coordinates of the sun's centroid based on the sun's outline, includes:

[0024] S21. Construct an image detection model for identifying and detecting the sun's outline in the target image;

[0025] S22. Based on the camera, acquire a target image containing the sun and input it into the image detection model. The image detection model acquires the sun's outline in the target image and calculates the pixel plane coordinates [u,v] of the sun's centroid projected onto the pixel plane coordinate system ouv based on the sun's outline. T .

[0026] According to one aspect of the present invention, step S3, which involves calculating the first angle between the sun and the optical axis of the camera coordinate system based on the pixel plane coordinates of the sun's centroid, includes:

[0027] Based on the pinhole camera model, establish the camera coordinate system Oxyz and the physical imaging coordinate system O′x′y′, and let the spatial coordinates of the spatial point P corresponding to the solar mass center in the camera coordinate system Oxyz be [X,Y,Z]. T The imaging point coordinates of the solar mass center corresponding to the imaging point P′ in the physical imaging coordinate system O′x′y′ are [X′,Y′,Z′]. T ;

[0028] Let the scaling factor between the pixel plane coordinate system ouv and the physical imaging coordinate system O′x′y′ and the origin translation coordinates be [c].x ,c y ] T Based on the coordinate scaling factor and the origin translation coordinates [c x ,c y ] T Construct the imaging point P′ and the pixel plane coordinates [u,v] T The transformation relationship; wherein, the transformation relationship is expressed as:

[0029]

[0030] Alternatively, the transformation relationship can be expressed as:

[0031]

[0032] Wherein, α represents the scaling factor between the u-axis in the pixel plane coordinate system ouv and the x-axis in the physical imaging coordinate system O′x′y′, in pixels per meter; β represents the scaling factor between the v-axis in the pixel plane coordinate system ouv and the y-axis in the physical imaging coordinate system O′x′y′, in pixels per meter.

[0033] The first included angle is obtained based on the transformation relationship and is expressed as follows:

[0034]

[0035] Where θ represents the first included angle, and f represents the focal length in meters.

[0036] According to one aspect of the present invention, in step S4, the mounting angle between the optical axis of the camera coordinate system and the carrier on which the camera is mounted is obtained, and the mounting angle, the first angle, and the solar azimuth angle A are used to determine the appropriate angle. s The step of obtaining the true azimuth angle of the carrier includes:

[0037] Based on the first included angle and the solar azimuth angle A s Obtain the first azimuth angle A of the optical axis of the camera coordinate system. z It is represented as:

[0038] A z =A s +θ

[0039] Based on the first azimuth angle A z The true azimuth angle of the carrier is obtained by combining the installation angle with the azimuth angle, and is expressed as:

[0040] A = A z +A0=A s +θ+A0

[0041] Where A represents the true azimuth angle of the carrier, and A0 represents the installation angle, which is known in advance.

[0042] To achieve the above-mentioned objective, the present invention provides a device for calculating the true azimuth of a carrier using the sun, comprising:

[0043] A camera, mounted on top of the carrier, is used to capture images of the surrounding environment and the sky;

[0044] processor;

[0045] A memory having program instructions executable by the processor;

[0046] When the program instructions are executed by the processor, the apparatus performs the method according to any one of claims 1 to 5.

[0047] According to one aspect of the invention, the camera is a wide-angle camera, a fisheye camera, or a panoramic camera.

[0048] According to one aspect of the present invention, since the motion of celestial bodies such as the sun follows strict laws, the true azimuth of the sun at different locations and times can be calculated quickly and accurately. Furthermore, by employing computer vision technology to identify the sun, the present invention can quickly obtain the true azimuth information of the carrier without relying on GNSS signals and magnetometers, thereby better performing navigation tasks.

[0049] According to one aspect of the present invention, the functionality can be achieved using a vehicle-mounted camera device, eliminating the need for a dedicated solar observation device or strict installation, thus making implementation convenient and cost-effective. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the steps of a method for calculating the true azimuth of a carrier using the sun, according to an embodiment of the present invention.

[0051] Figure 2 This is a pixel plane coordinate system diagram according to an embodiment of the present invention;

[0052] Figure 3 This is a relational diagram of a pinhole camera model according to an embodiment of the present invention;

[0053] Figure 4 This is a diagram showing the similarity of triangles in a pinhole camera model according to an embodiment of the present invention. Detailed Implementation

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0055] In describing embodiments of the present invention, the terms "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer" express orientations or positional relationships based on the orientations or positional relationships shown in the relevant drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the above terms should not be construed as limitations on the present invention.

[0056] like Figure 1 As shown, according to one embodiment of the present invention, a method for calculating the true azimuth of a carrier using the sun includes:

[0057] S1. Utilizing the solar altitude angle H s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. s ;

[0058] S2. Use a camera to acquire a target image containing the sun, detect and identify the sun's outline from the target image, and calculate the pixel plane coordinates [u,v] of the sun's centroid based on the sun's outline. T ;

[0059] S3. Calculate the first angle between the sun and the optical axis of the camera coordinate system based on the pixel plane coordinates of the sun's center of mass;

[0060] S4. Obtain the installation angle between the optical axis of the camera coordinate system and the carrier on which the camera is mounted, and based on the installation angle, the first angle, and the solar azimuth angle A... s The true azimuth angle of the carrier is obtained.

[0061] like Figure 1 As shown, according to one embodiment of the present invention, in step S1, the solar altitude angle H is used... s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. s The steps include:

[0062] S11. Using the number of days N from January 1st of the year of calculation (i.e., N=1 on January 1st, N=2 on January 2nd, and so on), calculate the solar declination δ, which is expressed as:

[0063] δ(deg)=0.006918-0.399912cos(b)+0.070257sin(b)-0.006758cos(2b)+0.000907sin(2b)-0.002697cos(3b)+0.00148sin(3b)

[0064] Where b = 2 × PI × (N-1) / 365, PI represents pi, and deg represents degrees.

[0065] S12. Calculate the true solar time based on the mean solar time and the time difference of the true solar time obtained from the table, and calculate the solar hour angle t based on the true solar time; where true solar time is expressed as:

[0066] True solar time = Mean solar time + True solar time difference

[0067] It's important to note that mean solar time is based on local longitude, not "Beijing time." True solar time, on the other hand, varies daily and doesn't follow a specific pattern. Relevant astronomical research institutions and some organizations have pre-established forecasts for true solar time, which can be obtained by referring to tables. For example, Table 1 shows the true solar time difference for January 2016.

[0068] Table 1

[0069]

[0070]

[0071] Based on this, the solar time angle t, calculated using true solar time, is expressed as:

[0072] Solar time angle t = (true solar time - 12) × 15°;

[0073] In this embodiment, the solar hour angle is zero at noon (i.e., at the exact center of the sky), positive in the morning and negative in the afternoon, -90° at sunrise and +90° at sunset, with an average hourly change of 15°.

[0074] S13. Calculate the solar altitude angle H based on the carrier's current geographical latitude φ, solar declination δ, and solar hour angle t. s It is represented as:

[0075] sinH s =sinφ×sinδ+cosφ×cosδ×cost;

[0076] S14. Based on the solar altitude angle H s The solar azimuth angle A is calculated from the geographical latitude φ and the solar declination δ. sIt is represented as:

[0077] cosA s =(sinφ×sinδ-sinδ)÷(cosH s ×cosφ).

[0078] like Figure 1 As shown, according to one embodiment of the present invention, step S2, which involves acquiring a target image containing the sun using a camera, detecting and identifying the sun's outline from the target image, and calculating the pixel plane coordinates of the sun's centroid based on the sun's outline, includes:

[0079] S21. Construct an image detection model for identifying and detecting the outline of the sun in the target image; in this embodiment, the image detection model can be obtained based on a traditional edge detection algorithm or using a deep learning algorithm (e.g., the YOLO deep learning algorithm).

[0080] S22. Acquire a target image containing the sun using a camera and input it into an image detection model. The image detection model obtains the sun's outline in the target image and calculates the pixel plane coordinates [u,v] of the sun's centroid projected onto the pixel plane coordinate system ouv based on the sun's outline. T In this embodiment, the pixel plane coordinate system ouv is a two-dimensional coordinate system with the image geometric center o as the origin, and the u and v axes parallel to the edge lines of the image frame, as shown below. Figure 2 As shown.

[0081] like Figure 1 As shown, according to one embodiment of the present invention, step S3, which involves calculating the first angle between the sun and the optical axis of the camera coordinate system based on the pixel plane coordinates of the sun's centroid, includes:

[0082] Based on the pinhole camera model, establish the camera coordinate system Oxyz and the physical imaging coordinate system O′x′y′, and let the spatial coordinates of the point P corresponding to the sun's mass center in the camera coordinate system Oxyz be [X,Y,Z]. T The imaging point coordinates of the Sun's mass center in the physical imaging coordinate system O′x′y′ are [X′,Y′,Z′]. T ;like Figure 3 As shown, in the step of establishing the camera coordinate system Oxyz, its z-axis (i.e., optical axis) points forward, the x-axis is to the right, and the y-axis is downward. O is the optical center of the camera, and also the pinhole in the pinhole model. The spatial point P corresponding to the sun's center of mass, after being projected through the pinhole O, falls on the physical imaging coordinate system O′x′y′, and the imaging point is P'. In this embodiment, the distance from the physical imaging coordinate system O′x′y′ to the pinhole O is f (focal length). Then, according to the similarity relationship of triangles (see... Figure 4),have:

[0083]

[0084] The first angle θ between the sun's center of mass and the optical axis (i.e., the z-axis) of the camera coordinate system can be calculated by the following formula:

[0085]

[0086] In this embodiment, the pixel plane coordinate system ouv and the physical imaging coordinate system O′x′y′ (see...) Figure 3 The difference between the pixel plane coordinate system ouv and the physical imaging coordinate system O′x′y′ is a scaling factor and an origin translation factor [c]. x ,c y ] T (where c) x ,c y The unit is pixels), based on the coordinate scaling factor and the origin translation coordinates [c x ,c y ] T Construct the imaging point P′ and the pixel plane coordinates [u,v] T The transformation relationship is expressed as follows:

[0087]

[0088] Alternatively, the transformation relation can be expressed as:

[0089]

[0090] Where α represents the scaling factor between the u-axis in the pixel plane coordinate system ouv and the x'-axis in the physical imaging coordinate system O′x′y′, in pixels per meter; β represents the scaling factor between the v-axis in the pixel plane coordinate system ouv and the y'-axis in the physical imaging coordinate system O′x′y′, in pixels per meter.

[0091] The first included angle is obtained based on the transformation relationship, and it is represented as:

[0092]

[0093] Where θ represents the first included angle and f represents the focal length, in meters.

[0094] like Figure 1 As shown, according to one embodiment of the present invention, in step S4, the mounting angle between the optical axis of the camera coordinate system and the carrier on which the camera is mounted is obtained, and the mounting angle, the first angle, and the solar azimuth angle A are used to determine the optimal angle for the camera. s The steps for obtaining the true azimuth of the carrier include:

[0095] Based on the first included angle and the solar azimuth angle A s Obtain the first azimuth angle A of the optical axis of the camera coordinate system. z It is represented as:

[0096] A z =A s +θ

[0097] Based on the first azimuth angle A z The true azimuth angle of the carrier is obtained by taking the angle between the carrier and the installation angle, and it is expressed as:

[0098] A = A z +A0=A s +θ+A0

[0099] Where A represents the true azimuth of the carrier, and A0 represents the installation angle, which is known in advance.

[0100] According to one embodiment of the present invention, an apparatus for calculating the true azimuth of a carrier using solar energy includes a camera, a processor, and a memory. In this embodiment, the camera is mounted on top of the carrier and is used to acquire images of the surrounding environment and the sky; the memory has program instructions executable by the processor; when the program instructions are executed by the processor, the apparatus performs the aforementioned method.

[0101] According to one embodiment of the present invention, the camera is a wide-angle camera, a fisheye camera, or a panoramic camera.

[0102] The above description is merely an example of a specific solution of the present invention. For any devices and structures not described in detail herein, it should be understood that they are implemented using common devices and methods already available in the art.

[0103] The above description is merely one embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the invention by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for calculating the true azimuth of a carrier using the sun, characterized in that, include: S1. Utilizing the solar altitude angle Geographical latitude Solar declination The solar azimuth angle was calculated. Among them are: S11. Calculate the solar declination using the number of days N from January 1st of the current year to the calculation date. It is represented as: in, , Represents pi (π). Indicates the angle in degrees; S12. Calculate the true solar time based on the mean solar time and the true solar time difference obtained from the table, and calculate the solar hour angle based on the true solar time. True solar time is expressed as: True solar time = Mean solar time + True solar time difference Solar hour angle Represented as: Solar hour angle ; S13. Based on the current geographical latitude of the carrier The solar declination The solar hour angle The solar altitude angle was calculated. It is represented as: ； S14. Based on the solar altitude angle The geographical latitude The solar declination The solar azimuth angle was calculated. It is represented as: ； S2. Acquire a target image containing the sun using a camera, detect and identify the sun's outline from the target image, and calculate the pixel plane coordinates of the sun's centroid based on the sun's outline. ; S3. Based on the pixel plane coordinates of the sun's centroid, calculate the first angle between the sun and the optical axis of the camera coordinate system; wherein, it includes: Based on the pinhole camera model, establish the camera coordinate system Oxyz and the physical imaging coordinate system of the camera. Let the spatial coordinates of the point P corresponding to the solar mass center in the camera coordinate system Oxyz be... And the solar mass center in the physical imaging coordinate system The corresponding imaging point below The coordinates of the imaging point are ; Let the pixel plane coordinate system ouv and the physical imaging coordinate system be... Coordinate scaling factor and origin translation coordinates Based on the coordinate scaling factor and the origin translation coordinates Construct the imaging points With the pixel plane coordinates The transformation relationship; wherein, the transformation relationship is expressed as: Alternatively, the transformation relationship can be expressed as: in, In the pixel plane coordinate system ouv Axis and the physical imaging coordinate system middle Scaling factor between axes, in pixels per meter. In the pixel plane coordinate system ouv Axis and the physical imaging coordinate system middle Scaling factor between axes, in pixels per meter; The first included angle is obtained based on the transformation relationship and is expressed as follows: in, Indicates the first included angle. This indicates focal length, expressed in meters. S4. Obtain the mounting angle between the optical axis of the camera coordinate system and the carrier on which the camera is mounted, and determine the mounting angle, the first angle, and the solar azimuth angle based on the mounting angle. To obtain the true azimuth angle of the carrier; wherein, the following is included: Based on the first included angle and the solar azimuth angle Obtain the first azimuth angle of the optical axis of the camera coordinate system. It is represented as: Based on the first azimuth angle The true azimuth angle of the carrier is obtained by combining the installation angle with the azimuth angle, and is expressed as: in, This represents the true azimuth angle of the carrier. This indicates the installation angle, and it is known in advance.

2. The method according to claim 1, characterized in that, Step S2, which involves acquiring a target image of the sun using a camera, detecting and identifying the sun's outline from the target image, and calculating the pixel plane coordinates of the sun's centroid based on the sun's outline, includes: S21. Construct an image detection model for identifying and detecting the sun's outline in the target image; S22. Based on the camera, acquire a target image containing the sun and input it into the image detection model. The image detection model acquires the sun's outline in the target image and calculates the pixel plane coordinates of the sun's centroid projected onto the pixel plane coordinate system ouv based on the sun's outline. .

3. A device for calculating the true azimuth angle of a carrier using the sun, characterized in that, include: A camera, mounted on top of the carrier, is used to capture images of the surrounding environment and the sky; processor; A memory having program instructions executable by the processor; When the program instructions are executed by the processor, the apparatus performs the method according to any one of claims 1 to 2.

4. The apparatus according to claim 3, characterized in that, The camera used is a wide-angle camera, a fisheye camera, or a panoramic camera.

Images