Solar vector orientation method, device and computer equipment based on heat-sensitive vision

Abstract

The present application relates to the technical field of photoelectric detection and navigation, and discloses a solar vector orientation method, device and computer equipment based on thermal sensitive vision, to solve the problem of signal attenuation and low reliability of existing visible light or polarized light solar detection technology under complex weather. The method comprises: acquiring a thermal infrared image of a sky area by using a near-infrared thermal sensitive vision sensor; performing image preprocessing to enhance the solar target information; detecting and segmenting the solar target area in the image, and accurately positioning the solar center coordinates by using a centroid estimation or ellipse fitting method; converting the image coordinates into the solar position observation vector in the carrier coordinate system of the bearing sensor according to the sensor calibration parameters and installation parameters; and combining the theoretical solar vector calculated by an astronomical algorithm to solve the orientation information of the carrier. The present application is suitable for complex lighting conditions, has the advantages of strong anti-interference and high precision, and can be used for attitude determination in aerospace and marine navigation.

Description

Technical Field

[0001] This invention belongs to the field of photoelectric detection and navigation technology, specifically relating to a solar vector orientation method, device, and computer equipment based on thermal vision. Background Technology

[0002] Precise autonomous orientation capability is a key technology for maximizing the operational efficiency of unmanned platforms, aerospace vehicles, and various transportation vehicles. Currently, commonly used navigation and orientation technologies mainly include Global Navigation Satellite Systems (GNSS), micro inertial orientation systems (INS), and magnetic compass-based orientation methods. However, due to current technological limitations, the orientation accuracy of micro inertial orientation systems is only maintained at the order of degrees per hour; satellite dual-antenna orientation systems are difficult to adopt due to installation size limitations; and miniature magnetic compasses are susceptible to electromagnetic interference, making reliability difficult to guarantee and failing to meet the navigation requirements of unmanned platforms in complex missions. Against this backdrop, developing precise autonomous orientation technology has become crucial for improving the operational efficiency of unmanned platforms.

[0003] Existing technologies mainly include biomimetic polarization orientation technology based on atmospheric polarization patterns and GNSS-based orientation technology. The former captures sky polarization pattern information using a polarization camera and determines the sun's position vector by analyzing the symmetry or E-vector orthogonality of the sky polarization pattern, thereby achieving orientation. This type of method is effective under ideal clear weather conditions and has been applied to some solar sensors and biomimetic polarization compasses, but it has significant limitations: under strong obstruction conditions such as cloudy or hazy weather, the visible light signal of the sun is severely attenuated by strong scattering and absorption by particles such as water droplets, ice crystals, and aerosols in the atmosphere. This causes the polarization image information captured by the polarization camera to become blurred, distorted, or even completely submerged by background noise, making it difficult to achieve effective detection and accurate orientation. At the same time, the sky polarization pattern will undergo significant distortion under non-Rayleigh scattering conditions (such as the influence of clouds), the symmetry will be destroyed, and the neutral point position will shift, resulting in a significant decrease in the accuracy or even failure of the orientation method based on polarization pattern analysis. The latter, GNSS-based orientation technology, uses signals from multiple satellites for positioning and orientation calculations. It is currently the most widely used navigation and orientation method, and is often integrated with inertial navigation (such as micro-inertial / geomagnetic / satellite combined navigation) to improve performance. GNSS can also provide heading information through multi-antenna differential technology; however, GNSS signals are susceptible to electromagnetic interference and malicious spoofing, leading to reduced reliability in complex electromagnetic environments. Furthermore, in signal-blocked areas (such as urban canyons, indoors, and underwater) or specific signal rejection environments, GNSS becomes completely inoperable, causing the carrier to lose its primary navigation and orientation method.

[0004] In addition, other related technologies, such as traditional inertial navigation systems, while highly autonomous, suffer from the problem of errors accumulating over time, requiring external information for correction.

[0005] In summary, existing technologies struggle to guarantee the continuity, stability, and accuracy of solar position vector estimation under complex weather conditions such as heavy cloud cover, resulting in insufficient reliability and multi-weather adaptability for autonomous orientation. Therefore, a new technology capable of effectively achieving precise autonomous orientation under various meteorological conditions is urgently needed. Summary of the Invention

[0006] To address the aforementioned technical problems, this invention proposes a solar vector orientation method, device, and computer equipment based on thermal vision. By introducing a thermal vision sensor operating in the near-infrared band, and utilizing the physical characteristics of the near-infrared band compared to the visible light band—stronger atmospheric penetration and less scattering and absorption by clouds—it achieves accurate detection and vector estimation of the sun's position under complex weather conditions. This aims to solve the core problem of existing technologies struggling to accurately and stably estimate the sun's position vector under various complex meteorological conditions, thus affecting the accuracy and reliability of autonomous orientation of the carrier.

[0007] This invention provides a solar vector orientation method based on thermal vision, comprising the following steps:

[0008] Step 110: Acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band;

[0009] Step 120, preprocessing the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image.

[0010] Step 130: Detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system;

[0011] Step 140: Based on the pre-calibrated internal parameters of the thermal vision sensor and the installation parameters of the carrier, and combined with the center coordinates of the solar target area in the image coordinate system, the solar position observation vector in the carrier coordinate system is estimated; the carrier is used to carry the thermal vision sensor.

[0012] Step 150: By comparing the theoretical solar vector with the observed solar position vector in the coordinate system of the carrier, the orientation information of the carrier is calculated; the theoretical solar vector is obtained by astronomical algorithm based on the current time and current geographical location.

[0013] On the other hand, the present invention also provides a solar vector orientation device based on thermal vision. Utilizing the steps of the aforementioned solar vector orientation method based on thermal vision, the device includes the following modules:

[0014] The first module is used to acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band.

[0015] The second module is used to preprocess the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image.

[0016] The third module is used to detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system.

[0017] The fourth module is used to estimate the solar position observation vector in the carrier coordinate system based on the pre-calibrated internal parameters of the thermal vision sensor and the installation parameters of the carrier, combined with the center coordinates of the solar target area in the image coordinate system; the carrier is used to carry the thermal vision sensor.

[0018] The fifth module is used to calculate the orientation information of the carrier by comparing the theoretical solar vector with the observed solar position vector in the carrier's coordinate system; the theoretical solar vector is calculated using astronomical algorithms based on the current time and current geographical location.

[0019] Furthermore, the present invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned solar vector orientation method based on thermal vision.

[0020] Compared with the prior art, the specific beneficial effects achieved by the solution of the present invention include:

[0021] (1) Utilizing the performance characteristics of the near-infrared (NIR) band, the penetration ability of atmospheric particles such as clouds, fog, haze, and smoke is much stronger than that of visible light. With the sun as a high-temperature blackbody, the radiation intensity in this band is high and stable. This allows the thermal vision sensor to still obtain a solar image with sufficient contrast even under conditions such as thin clouds, overcast skies, fog, haze, and dust that cause visible light sensors to fail or have a sharp drop in accuracy. This fundamentally breaks through the meteorological limitations of traditional optical solar sensors and achieves true all-weather orientation capability.

[0022] (2) Make full use of the characteristic that thermal infrared images mainly reflect the difference in radiation temperature of the target. In particular, the sky background is weak and uniform in the near-infrared band, while the sun is an extremely high-temperature point source with a very high signal-to-noise ratio, which can effectively suppress complex background interference such as bright spots in the sky, strong light at the edge of the cloud, and reflections from the sea / land. In scenarios with severe glare, such as dawn and dusk or low elevation angles, traditional sensors may saturate or misjudge, while the detection based on radiation characteristics in this solution is more robust.

[0023] (3) The method of this invention makes the subsequent image processing (segmentation, center localization) algorithms more reliable and the results more reliable. Since the imaging profile of the sun in the thermal infrared band is stable and is minimally affected by atmospheric dispersion and chromatic aberration, both the centroid method and the fitting method can be performed on a target with clear features, which improves the repeatability and noise resistance of center localization, thereby directly improving the accuracy of the observation vector. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0025] Figure 1 This is a flowchart illustrating a solar vector orientation method based on thermal vision in one embodiment of the present invention.

[0026] Figure 2 This is a schematic diagram of solar flare images captured by a polarization camera and a near-infrared camera under cloudy conditions in the experiment of this invention. Figure 2 (a) and Figure 2 (b) is an image of the sky sunspots obtained by a polarizing camera; Figure 2 (c) and Figure 2 (d) is the image of the sky sunspot obtained by the near-infrared camera;

[0027] Figure 3 This is a schematic diagram of the original image from the polarization camera during synchronous rotation of the polarization camera and the near-infrared camera under cloudy conditions in the experiment of this invention. Figure 3 (a) Figure 3 (b) Figure 3 (c) Figure 3 (d) Figure 3 (e) and Figure 3 (f) represent the original images of the polarization camera at different rotation directions during continuous rotation, with the rotation order being: (a) → (b) → (c) → (d) → (e) → (f);

[0028] Figure 4 This is a schematic diagram of the original near-infrared camera image during the synchronous rotation of the polarization camera and the near-infrared camera under cloudy conditions in the experiment of this invention. Figure 4 (a) Figure 4 (b) Figure 4 (c) Figure 4 (d) Figure 4 (e) and Figure 4 (f) represent the original near-infrared camera images at different rotational positions during continuous rotation, with the rotation order being: (a) → (b) → (c) → (d) → (e) → (f);

[0029] Figure 5 This is a schematic diagram illustrating the visualization of the solar position vector extracted from near-infrared images using morphological centroid estimation when the polarization camera and near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 5 (a) Figure 5 (b) Figure 5 (c) Figure 5 (d) Figure 5 (e) and Figure 5 (f) shows near-infrared images of different rotation directions during continuous rotation and corresponding solar position vector diagrams marked with short straight lines. The rotation order is as follows: (a) → (b) → (c) → (d) → (e) → (f).

[0030] Figure 6 This is a schematic diagram illustrating the visualization of the solar position vector extracted from near-infrared images using an ellipse fitting method when the polarization camera and near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 6 (a) Figure 6 (b) Figure 6 (c) Figure 6 (d) Figure 6 (e) and Figure 6 (f) shows the near-infrared images and corresponding solar position vector diagrams at different rotation directions during continuous rotation, with the extracted solar meridian direction marked by short straight lines. The rotation order is as follows: (a) → (b) → (c) → (d) → (e) → (f).

[0031] Figure 7 This is a schematic diagram illustrating the polarization orientation results based on the symmetry characteristics of atmospheric polarization modes when a polarization camera and a near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 7 (a) Figure 7 (b) Figure 7 (c) Figure 7 (d) Figure 7 (e) and Figure 7(f) represents the polarization orientation results of different rotation directions during continuous rotation. The extracted solar meridian is marked with a straight line. The rotation order is as follows: (a) → (b) → (c) → (d) → (e) → (f). Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] Regarding the key problem that this invention aims to solve:

[0034] Existing biomimetic polarization light orientation technology based on atmospheric polarization patterns suffers from severe polarization signal attenuation and blurred or difficult-to-identify solar spots under weather conditions such as cloudy conditions.

[0035] The distortion of the sky polarization mode under non-Rayleigh scattering conditions leads to a decrease in the accuracy or failure of traditional polarization navigation models.

[0036] External signal sources such as GNSS are susceptible to interference or may not function under certain environments, affecting the reliability of autonomous orientation.

[0037] There is a lack of a multi-weather, robust technique for estimating solar position vectors and orienting carriers.

[0038] By introducing a near-infrared (NIR) thermal vision sensor, and taking advantage of the physical properties of the near-infrared band (especially 0.7-1.1 μm) compared to the visible light band, which has stronger atmospheric penetration and is less affected by cloud scattering and absorption, accurate detection and vector estimation of the sun's position can be achieved under complex weather conditions.

[0039] In one embodiment, such as Figure 1 As shown, this invention provides a solar vector orientation method based on thermal vision, comprising:

[0040] Step 110: Acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band;

[0041] Step 120, preprocessing the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image.

[0042] Step 130: Detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system;

[0043] Step 140: Based on the pre-calibrated internal parameters of the thermal vision sensor and the installation parameters of the carrier, and combined with the center coordinates of the solar target area in the image coordinate system, the solar position observation vector in the carrier coordinate system is estimated; the carrier is used to carry the thermal vision sensor.

[0044] Step 150: By comparing the theoretical solar vector with the observed solar position vector in the coordinate system of the carrier, the orientation information of the carrier is calculated; the theoretical solar vector is obtained by astronomical algorithm based on the current time and current geographical location.

[0045] Specifically, in step 110, a near-infrared thermal vision sensor is introduced to capture the near-infrared portion of solar radiation in order to reduce visible light interference after atmospheric scattering and provide stable image input for subsequent solar position vector estimation.

[0046] Thermal vision sensors consist of a large number of independent sensitive pixels. The analog signal output by each pixel is digitized to form the pixel value of the corresponding position in the thermal infrared image. Due to microscopic differences in manufacturing processes, material inhomogeneity, and characteristic drift of the readout circuit, even when receiving perfectly uniform radiation input, the response output of each pixel varies, manifesting as fixed pattern noise (FPN). This mainly includes offset noise (inconsistent baseline output values ​​of pixels under zero or constant input) and gain noise (inconsistent gain coefficients of pixels in response to the same change in radiation intensity). FPN produces fixed grid-like or speckled noise on the image, severely degrading image quality, obscuring the true radiation distribution of the scene, and interfering with subsequent accurate segmentation of solar targets. Therefore, preprocessing of the thermal infrared images (data) acquired by the thermal vision sensor is necessary.

[0047] In one embodiment, the operating wavelength range of the thermal vision sensor is 0.7 micrometers to 1.1 micrometers.

[0048] Furthermore, in step 120, the thermal infrared image data is preprocessed.

[0049] To obtain high-precision solar vector orientation results, the thermal infrared image data must first be corrected for non-uniformity to remove the fixed-pattern noise of the thermal vision sensor.

[0050] Non-uniformity correction is a routine key technique in the preprocessing of thermal infrared imaging systems. It suppresses or eliminates fixed-pattern noise caused by the inherent differences in the response characteristics of each pixel in the sensor's focal plane array, thereby improving the uniformity and accuracy of thermal infrared image data. This mainly includes:

[0051] The gain correction coefficient and bias correction coefficient of the thermal vision sensor were determined using the calibration parameter method.

[0052] Using the calibrated gain and bias correction coefficients, each frame of the original image acquired in real time by the thermal infrared vision sensor is corrected point by point:

[0053] For each pixel in the original input image, read the gain correction coefficient and bias correction coefficient of its corresponding coordinates, perform linear operations (multiplication and addition), and generate the corrected pixel value;

[0054] For bad pixels that are unresponsive or have abnormal responses identified during the calibration process, the correction output is replaced by interpolation of the effective pixels in the neighborhood to ensure the integrity of the image.

[0055] Output a homogenized image with significantly suppressed fixed-pattern noise for subsequent analysis and applications.

[0056] Then, a method based on background suppression and / or contrast enhancement is designed for near-infrared image preprocessing to enhance solar target information in the image. Specifically, histogram specification is used to achieve this goal.

[0057] (1) Define the target: The target histogram of the present invention is generated by the exponential decay function obtained by fitting the ideal image in advance, that is, the target probability density function is reflected on the discrete gray level.

[0058] In one embodiment, the exponential decay function is given by the following equation:

[0059] ;

[0060] in, For scale parameters, For attenuation parameters, This corresponds to the gray level. The scale parameter and attenuation parameter can be set and adjusted according to specific circumstances.

[0061] In one embodiment, the scale parameter is set as follows: The attenuation parameter is .

[0062] (2) Construct the target cumulative distribution function (CDF): Based on this target histogram, calculate its normalized target cumulative distribution function (CDF).

[0063] (3) Insight into the input image: Calculate the actual histogram of the input image and its normalized actual cumulative distribution function.

[0064] (4) Establish a mapping bridge: For each gray level of the input image, under the guidance of its actual CDF value, find the most suitable corresponding gray level in the target CDF and construct a gray-level mapping lookup table.

[0065] (5) Perform the conversion: Apply this grayscale lookup table to convert each pixel value of the input image into a new grayscale value that conforms to the preset enhancement target.

[0066] In step 130, the solar target region is detected and segmented from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system.

[0067] Specifically, two methods are used to detect and segment the solar target region, including:

[0068] Edge detection operators (such as Sobel operator and Canny operator) are used to perform edge detection on the preprocessed thermal infrared image data to obtain the edge points of the solar target region; the edge points of the solar target region are discretized into a scatter set to be fitted; the scatter set is fitted with an ellipse, and the center coordinates of the solar target region in the image coordinate system are determined according to the center of the fitted ellipse.

[0069] A threshold segmentation algorithm is used to binarize the preprocessed thermal infrared image data; morphological processing, including opening and closing operations, is performed on the binarized image; connected component analysis is performed on the morphologically processed image, and candidate solar regions are selected based on the expected characteristics of the solar target; the center of the candidate solar regions is calculated using a weighted average method, which is used as the center coordinate of the solar target region in the image coordinate system.

[0070] Correspondingly, an algorithm based on ellipse fitting or morphological estimation is designed to locate the image center coordinates of the solar target region:

[0071] (1) Edge ellipse fitting method:

[0072] It is clear that an elliptic curve is a special case of a general quadratic curve. For coordinates on an elliptic curve... The equations of elliptic curves are described using implicit second-order polynomials:

[0073] ;

[0074] use , indicating correspondence Given discrete points on the edge of an ellipse, a matrix can be designed:

[0075] ;

[0076] and the coefficient vector of the elliptic curve equation to be determined ;

[0077] To address the problem of locating the image center coordinates of a solar target region using ellipse fitting methods, we transform it into a least-squares fitting problem:

[0078] ;

[0079] To ensure that the fitting result for the discrete points on the edge of the ellipse is an elliptic curve rather than some other quadratic curve, the following constraints need to be added: ;

[0080] in, This is the constraint matrix.

[0081] In one embodiment, the constraint matrix for:

[0082] .

[0083] Therefore, the ellipse fitting problem can be transformed into a constrained optimization problem:

[0084] ;

[0085] in, ;

[0086] The optimization problem is then transformed into a generalized eigenvalue problem: ,in, For generalized eigenvalues, the coefficient vector This is the corresponding generalized feature vector;

[0087] Finally, the inverse power method can be used to iteratively solve for the generalized eigenvector corresponding to the smallest eigenvalue. The iterative formula is as follows:

[0088] ;

[0089] in, It is an estimate of the eigenvalues, usually taking a value close to the smallest eigenvalue; It is the first Estimation of the second-generalized eigenvectors It is the first Estimation of the second generalized eigenvector.

[0090] Stop when the change in the generalized eigenvectors of adjacent iterations is sufficiently small: ,in, Represents Euclidean distance. This is a preset tolerance used to determine when iteration stops.

[0091] The vector of undetermined coefficients is obtained using the method described above. Then, calculate the coordinates of the ellipse center. Image center coordinates for the solar target region:

[0092] , .

[0093] (2) Morphological centroid estimation method:

[0094] First, the input original grayscale image is binarized using a threshold:

[0095] ;

[0096] in, pixel coordinates binary image, The segmentation threshold is... This represents the original grayscale image (corresponding to the preprocessed thermal infrared image data) in pixel coordinates. The grayscale value at that location.

[0097] Then use structural elements (such as radius) The disk is used to perform an opening operation on the binary image to remove small noise and retain only the connected components of the main spot. Then, a closing operation is performed to fill the small holes in the spot to obtain a more complete main spot area.

[0098] Then, the region labeling analysis function is applied to label the connected components of the purified binary image, resulting in a set of connected components. , ; This represents the total number of connected components. Assuming the main light spot is the connected component with the largest area in the graph, then the set of pixels of the main light spot... You can select the option based on the largest area:

[0099] ;

[0100] in, Indicates the first The total number of discrete points (pixels) containing the main spot in each connected region.

[0101] for In The centroids of the main light spots are then clearly marked in the original image using the region centroid formula, based on the discrete points of the main light spots. To estimate the sun's position, the image center coordinates of the target sun region are obtained.

[0102] ; ;

[0103] in, yes The Middle The main light spot is discrete. .

[0104] Furthermore, step 140, the process of estimating the solar position observation vector in the carrier coordinate system, includes:

[0105] Based on the pre-calibrated internal parameters of the thermal vision sensor, the center coordinates of the solar target area in the image coordinate system are converted into the solar position observation vector in the sensor coordinate system using the pinhole imaging principle.

[0106] By using the attitude transformation relationship determined by the installation parameters of the thermal vision sensor relative to the carrier, the solar position observation vector in the sensor coordinate system is transformed to obtain the solar position observation vector in the carrier coordinate system.

[0107] Finally, in step 150, the process of calculating the theoretical solar vector mainly includes two methods: one based on the local geographic coordinate system (elevation angle and azimuth angle) and the other based on the geocentric inertial coordinate system (right ascension and declination).

[0108] The former includes: calculating the solar altitude angle and azimuth angle using astronomical algorithms based on the current time and geographical location; constructing a theoretical solar vector in the local northeast-northeast coordinate system based on the altitude angle and azimuth angle; the eastward component, northward component and celestial component of the theoretical solar vector are determined by a trigonometric function combination of the altitude angle and azimuth angle.

[0109] The latter includes: calculating the right ascension and declination of the sun using astronomical algorithms based on the current time; constructing a theoretical solar vector in a geocentric equatorial inertial coordinate system based on the right ascension and declination; and determining the three rectangular coordinate components of the theoretical solar vector by a combination of trigonometric functions of the right ascension and declination.

[0110] In summary, the technical principles of this invention and the beneficial effects obtained mainly include:

[0111] (1) The core principle of this invention is based on the following scientific facts:

[0112] The sun still exhibits strong blackbody radiation characteristics in the near-infrared band (especially 0.7μm-1.1μm);

[0113] The atmospheric transmittance in the near-infrared band is significantly higher than that in the visible light band, and it is less affected by meteorological factors such as clouds and haze.

[0114] The atmospheric scattering coefficient is inversely proportional to the fourth power of wavelength (Rayleigh scattering), which means that the scattering effect in the infrared band is much smaller than that in the visible light band.

[0115] Clouds have a relatively low absorption coefficient for near-infrared radiation, allowing solar radiation to penetrate them more effectively.

[0116] (2) Main advantages and beneficial effects:

[0117] Strong all-weather adaptability: It can still work stably in various severe weather conditions such as cloudy and foggy weather;

[0118] High autonomy: It does not rely on external signal sources and has a completely autonomous orientation capability;

[0119] Good real-time performance: The algorithm has moderate complexity and can realize real-time solar tracking and directional solution;

[0120] Wide applicability: It can be applied to various carrier platforms such as drones, unmanned vehicles, and aerospace vehicles.

[0121] Furthermore, the present invention obtains a set of experimental results on solar vector orientation based on thermally sensitive vision through actual scene experiments, which further demonstrates the beneficial effects obtained by the present invention. Figure 2 A schematic diagram of solar flare images captured by a polarization camera and a near-infrared camera under cloudy conditions is given. Figure 2 (a) and Figure 2 (b) is an image of the sky sunspots obtained by a polarizing camera; Figure 2 (c) and Figure 2 (d) is the sky sunspot image obtained by the near-infrared camera.

[0122] Figures 3-7 The results of the visualization extraction of the solar flare image and the corresponding solar position vector obtained by synchronously rotating a polarization camera and a near-infrared camera under cloudy conditions are presented. The order of rotation direction is as follows: (a) → (b) → (c) → (d) → (e) → (f).

[0123] Specifically, Figure 3 This is a schematic diagram of the original image from the polarization camera during synchronous rotation of the polarization camera and the near-infrared camera under cloudy conditions in the experiment of this invention. Figure 3 (a) Figure 3 (b) Figure 3 (c) Figure 3 (d) Figure 3 (e) and Figure 3 (f) These represent the original polarization camera images at different rotational positions during continuous rotation; Figure 4 This is a schematic diagram of the original near-infrared camera image during the synchronous rotation of the polarization camera and the near-infrared camera under cloudy conditions in the experiment of this invention. Figure 4 (a) Figure 4 (b) Figure 4(c) Figure 4 (d) Figure 4 (e) and Figure 4 (f) These represent the original near-infrared camera images at different rotational positions during continuous rotation; Figure 5 This is a schematic diagram illustrating the visualization of the solar position vector extracted from near-infrared images using morphological centroid estimation when the polarization camera and near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 5 (a) Figure 5 (b) Figure 5 (c) Figure 5 (d) Figure 5 (e) and Figure 5 (f) These are near-infrared images of different rotation directions during continuous rotation and schematic diagrams of the corresponding solar position vectors marked with short straight lines. Figure 6 This is a schematic diagram illustrating the visualization of the solar position vector extracted from near-infrared images using an ellipse fitting method when the polarization camera and near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 6 (a) Figure 6 (b) Figure 6 (c) Figure 6 (d) Figure 6 (e) and Figure 6 (f) shows near-infrared images of different rotation directions during continuous rotation and corresponding schematic diagrams of solar position vectors; Figure 7 This is a schematic diagram illustrating the polarization orientation results based on the symmetry characteristics of atmospheric polarization modes when a polarization camera and a near-infrared camera rotate synchronously under cloudy conditions in the experiment of this invention. Figure 7 (a) Figure 7 (b) Figure 7 (c) Figure 7 (d) Figure 7 (e) and Figure 7 (f) represents the polarization orientation results at different rotational positions during continuous rotation. Figure 6 The solar meridian is extracted from short straight lines in near-infrared images; Figure 7 The straight line in the polarization orientation method corresponds to the obtained solar meridian.

[0124] The relative error calculated using the solar vector orientation method based on thermally sensitive vision provided by this invention is shown in Table 1 below. Here, the relative error is defined as the difference between the true value of the rotational angle and the value calculated using a near-infrared camera or polarized light orientation method.

[0125] Table 1. Relative error results of solar vector orientation for locating solar target areas using different algorithms.

[0126]

[0127] In one embodiment, the present invention also provides a solar vector orientation device based on thermal vision, which utilizes the steps of the aforementioned solar vector orientation method based on thermal vision, and the device includes the following modules:

[0128] The first module is used to acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band.

[0129] The second module is used to preprocess the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image.

[0130] The third module is used to detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system.

[0131] The fourth module is used to estimate the solar position observation vector in the carrier coordinate system based on the pre-calibrated internal parameters of the thermal vision sensor and the installation parameters of the carrier, combined with the center coordinates of the solar target area in the image coordinate system; the carrier is used to carry the thermal vision sensor.

[0132] The fifth module is used to calculate the orientation information of the carrier by comparing the theoretical solar vector with the observed solar position vector in the carrier's coordinate system; the theoretical solar vector is calculated using astronomical algorithms based on the current time and current geographical location.

[0133] In one embodiment, the present invention provides a computer device including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the solar vector orientation method based on thermal vision provided in any of the above embodiments. The computer device may be a server. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device stores sample data. The network interface of the computer device is used for communication with an external PC via a network connection.

[0134] On the other hand, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the solar vector orientation method based on thermal vision provided in any of the above embodiments.

[0135] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by hardware related to computer program instructions. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0136] Matters not covered in this invention are common knowledge.

[0137] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

[0139] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A heat-sensitive vision based sun vector orientation method, characterized by, Includes the following steps: Step 110: Acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band; Step 120, preprocessing the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image. Step 130: Detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system; Step 140: Based on the pre-calibrated internal parameters of the thermal vision sensor and the mounting parameters of the carrier, and combined with the center coordinates of the solar target area in the image coordinate system, estimate the solar position observation vector in the carrier coordinate system, including: Based on the pre-calibrated internal parameters of the thermal vision sensor, the center coordinates of the solar target area in the image coordinate system are converted into the solar position observation vector in the sensor coordinate system using the pinhole imaging principle. The attitude transformation relationship determined by the installation parameters of the thermal vision sensor relative to the carrier is used to obtain the solar position observation vector in the carrier coordinate system from the solar position observation vector in the sensor coordinate system through attitude transformation. The carrier is used to hold the thermal vision sensor; Step 150: By comparing the theoretical solar vector with the observed solar position vector in the coordinate system of the carrier, the orientation information of the carrier is calculated; the theoretical solar vector is obtained by astronomical algorithm based on the current time and current geographical location.

2. The heat-sensitive vision based sun vector orientation method according to claim 1, characterized in that, In step 110, the operating wavelength range of the thermal vision sensor is 0.7 micrometers to 1.1 micrometers.

3. The heat-sensitive vision based sun vector orientation method of claim 1, wherein, In step 120, the histogram specification includes: Based on the preset enhancement targets: background suppression and / or contrast enhancement, a target histogram is generated; Based on the target histogram, calculate the normalized target cumulative distribution function; Calculate the actual histogram of the input image and the corresponding normalized actual cumulative distribution function; For each gray level of the input image, based on the value of the actual cumulative distribution function, the most suitable corresponding gray level is found in the target cumulative distribution function, and a gray-level mapping lookup table is constructed. According to the grayscale lookup table, each pixel value of the input image is converted into a grayscale value that conforms to the preset enhancement target.

4. The solar vector orientation method based on thermal vision according to claim 1, characterized in that, In step 130, detecting and segmenting the solar target region from the preprocessed thermal infrared image data includes: An edge detection operator is used to perform edge detection on the preprocessed thermal infrared image data to obtain the edge points of the solar target area; The edge points of the solar target region are discretized into a scatter set to be fitted; Ellipse fitting is performed on the scatter point set, and the center coordinates of the solar target area in the image coordinate system are determined based on the center of the fitted ellipse.

5. The heat-sensitive vision based sun vector orientation method of claim 1, wherein, In step 130, detecting and segmenting the solar target region from the preprocessed thermal infrared image data includes: A threshold segmentation algorithm is used to binarize the preprocessed thermal infrared image data. Morphological processing is performed on the binarized image, including opening and closing operations; Connectivity analysis was performed on the morphologically processed image, and candidate solar regions were selected based on the expected features of the solar target. The center of the candidate solar region is calculated using a weighted average method and used as the center coordinates of the solar target region in the image coordinate system.

6. The heat-sensitive vision based sun vector orientation method of claim 1, wherein, In step 150, the theoretical solar vector is calculated using an astronomical algorithm based on the current time and current geographical location, including: Based on the current time and geographical location, the solar altitude angle and azimuth angle are calculated using astronomical algorithms; Based on the elevation angle and azimuth angle, a theoretical solar vector is constructed in the local northeast-north sky coordinate system; The eastward, northward, and celestial components of the theoretical solar vector are determined by a trigonometric function combination of altitude and azimuth angles.

7. The heat-fusion visual-based sun vector orientation method of claim 1, wherein, In step 150, the theoretical solar vector is calculated using an astronomical algorithm based on the current time and current geographical location, including: Based on the current time, the right ascension and declination of the sun are calculated using astronomical algorithms; Based on the aforementioned right ascension and declination, a theoretical solar vector is constructed in the geocentric equatorial inertial coordinate system; The three rectangular coordinate components of the theoretical solar vector are determined by a combination of trigonometric functions of right ascension and declination.

8. A solar vector orientation device based on thermal vision, characterized in that, The device is used to implement the steps of the solar vector orientation method based on thermal vision as described in any one of claims 1-7, wherein the device comprises the following modules: The first module is used to acquire thermal infrared image data of the sky area using a thermal vision sensor operating in the near-infrared band. The second module is used to preprocess the thermal infrared image data, including: performing non-uniformity correction to remove fixed-pattern noise from the thermal vision sensor; and performing background suppression and / or contrast enhancement through histogram specification to enhance solar target information in the image. The third module is used to detect and segment the solar target region from the preprocessed thermal infrared image data, including using morphological centroid estimation or edge ellipse fitting to accurately locate the center coordinates of the solar target region in the image coordinate system. The fourth module is used to estimate the solar position observation vector in the carrier coordinate system based on the pre-calibrated internal parameters of the thermal vision sensor and the installation parameters of the carrier, combined with the center coordinates of the solar target area in the image coordinate system. This includes: Based on the pre-calibrated internal parameters of the thermal vision sensor, the center coordinates of the solar target area in the image coordinate system are converted into the solar position observation vector in the sensor coordinate system using the pinhole imaging principle. The attitude transformation relationship determined by the installation parameters of the thermal vision sensor relative to the carrier is used to obtain the solar position observation vector in the carrier coordinate system from the solar position observation vector in the sensor coordinate system through attitude transformation. The carrier is used to hold the thermal vision sensor; The fifth module is used to calculate the orientation information of the carrier by comparing the theoretical solar vector with the observed solar position vector in the carrier's coordinate system; the theoretical solar vector is calculated using astronomical algorithms based on the current time and current geographical location. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. When the processor executes the computer program, it implements the steps of the solar vector orientation method based on thermal vision as described in any one of claims 1-7.

Images