A star image recognition method suitable for ultra-large field of view
Through fisheye image correction, partition weighting and neural network recognition, the distortion, interference and calculation complexity problems of star map recognition under super large field of view are solved, and fast and accurate star map recognition is achieved.
Patent Information
- Application Number
- CN202210982821.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-16
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2042-08-16
AI Technical Summary
Existing star map recognition methods are difficult to effectively deal with the problems of image distortion, environmental interference and computational complexity under ultra-large field of view, resulting in slow recognition speed or unsuccessful identification.
The neural network recognition method adopts fisheye image correction, image partition weighting, false star interference removal and mixed mode features. Through fisheye distortion correction, filtering processing, partition weight assignment and neural network recognition, the rapid and large field of view star map is achieved. Accurate identification.
It improves the efficiency and accuracy of star map recognition under super large field of view, reduces recognition time, and is suitable for star maps obtained by fisheye lenses with field of view ranging from 120° to 180°.
Smart Images

Figure CN115507844B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of ultra-large field of view celestial navigation systems, and in particular to a star map recognition method suitable for ultra-large field of view. Background Art
[0002] There is no published patent on the method of star map recognition with an ultra-large field of view.
[0003] According to the search, the current star map recognition method is relatively simple. The star point image captured by the sensor can be quickly recognized by simple filtering. However, this method is mainly limited to astronomical navigation measurement under small field of view conditions. Due to the small field of view, the image distortion is small, and the number of stars that can be observed at the same time is limited. The star sensor designed using this method needs to be used in conjunction with a servo turntable to achieve full-sky star map recognition and star tracking.
[0004] Due to its special optical structure, the fisheye lens has a field of view of 180°, so it can obtain the star image of the entire sky in a single image. However, it is well known that as the field of view increases, the geometric aberration of the optical system will become stronger, resulting in more serious distortion of the captured image. Therefore, it is necessary to calibrate the parameters of the ultra-large field of view optical system and use the correction model to correct the star point position deviation caused by the phase difference imaging. Otherwise, successful recognition or incorrect recognition may occur.
[0005] In addition, for the full-sky star map, some areas in the image may be interfered by stray light, clouds or other targets. The results of star map recognition in these interference areas are often unreliable. Therefore, it is necessary to use algorithms to identify and record such areas and remove them during the star map recognition process to avoid affecting the recognition effect. According to the distortion model of fisheye images, the degree of distortion in different areas of the image is also different. Therefore, it is necessary to adopt a reasonable partitioning algorithm and define the recognition priority of each valid partition. In addition, since there are many non-stellar interference targets in the atmosphere and space, their reflected light from stars such as the sun may enter the star sensor and produce false star interference. Therefore, it is necessary to design a suitable algorithm to remove false star interference as much as possible to improve recognition accuracy.
[0006] Star map recognition technology is to match and identify the stars in the star map taken by the star sensor with the stars in the navigation star library. This technology is the key prerequisite for the star sensor to achieve positioning and attitude determination. The existing star map recognition algorithms are generally divided into two categories: sub-graph isomorphism algorithms use the triangle principle to calculate the angular distance between three or more stars and compare them with the standard star catalog to achieve star recognition; pattern recognition algorithms use other stars within a certain range of the observed star to construct the pattern characteristics of the observed star, and achieve star recognition by searching for this unique pattern feature.
[0007] As for the complexity of the star map recognition algorithm, there are a large number of star points in the whole sky, and the number of star points in a single image can reach dozens of times more than the traditional star sensor star map. Using traditional star map recognition methods to process ultra-large field star map data will result in slow recognition speed or even failure to successfully recognize. Therefore, it is necessary to design a fast and accurate star map recognition algorithm to apply to ultra-large field conditions.
[0008] In summary, current star map recognition algorithms often lack comprehensive consideration of issues such as image distortion, environmental interference, calculation and data complexity under ultra-large field of view conditions. Summary of the invention
[0009] In view of this, the present invention provides a star map recognition method suitable for ultra-large field of view, which performs a series of processing such as filtering, correction, and partitioning on the starry sky pictures taken by the ultra-large field of view camera, and then performs star map recognition in the entire sky area. Under the premise of ensuring recognition accuracy and high recognition rate, the recognition efficiency and recognition continuity are improved as much as possible, and the recognition time is reduced.
[0010] To achieve the above object, the technical solution of the present invention comprises the following steps:
[0011] Step 1: Get the captured star map, perform fisheye distortion correction, and obtain the corrected image.
[0012] Step 2: Filter the corrected image to remove some background noise. Then, compare the information of adjacent frames of the filtered image to suppress the interference of false stars caused by space targets. Partition and weight the image after removing false stars, record the invalid areas, and remove the invalid areas to execute step 3.
[0013] Step 3: Enter the star map recognition stage, sort according to the partition weights, give priority to the partitions with larger weights, and then select appropriate navigation stars in the partition to construct star pattern features and perform neural network star map recognition with mixed pattern features.
[0014] If the recognition fails once, the partition where the recognition failed is removed, and then the weight judgment is performed again. The partition with the highest weight is selected to repeat the star map recognition until the navigation star is successfully recognized or all valid partitions are searched. If all valid partitions are traversed and still not successfully recognized, the recognition of this image fails, and the recognition of the next image is carried out.
[0015] Furthermore, step one is specifically as follows:
[0016] The distance r from the image point to the center of the sensor in the case of pinhole imaging is different from the distance r from the image point to the center of the sensor in the real case. d The calculation formula is
[0017]
[0018] where k i is the coefficient of the Taylor expansion for the incident angle θ, i takes the value of [1,n], and n is the number of Taylor expansion coefficients; the distortion correction of the star map taken with a super large field of view is completed.
[0019] Furthermore, for the coefficient k i , a standard checkerboard image is captured by a star sensor in advance, and then calculated using a calibration algorithm.
[0020] Furthermore, in step 2, the image from which the false stars are removed is partitioned and weighted, the invalid areas are recorded, and the invalid areas are removed, specifically:
[0021] After completing the removal of false stars, the entire star map needs to be divided into several circular partitions. The size of these small circular areas is set to ensure that a sufficient number of stars are included in the range.
[0022] Each area is traversed, and the partitions whose grayscale mean exceeds the set value are recorded as invalid areas and removed; all the remaining partitions are assigned weights from large to small from the inside to the outside and indexed and sorted by weight.
[0023] Furthermore, in step 3, the star map recognition stage specifically includes the following steps:
[0024] Step 301: construct a feature vector for each circular partition. Specifically, the feature vector is constructed by the following steps:
[0025] The circular area is divided into k equally spaced annular areas, and the number of companion stars falling into each annular area is N1, N2, ..., N respectively. k .
[0026] Select the observation star closest to the center of the region as the primary star.
[0027] Find the two companion stars closest to the primary star outside the field of the primary star. T1 is the angular distance from the primary star to the nearest companion star, T2 is the angular distance from the primary star to the next nearest star, and T3 is the angular distance between the two nearest companion stars.
[0028] From N1, N2, ..., N k Together with T1, T2, and T3, it constitutes the feature vector of the observation area.
[0029] Step 302: After the construction of the feature vector is completed, it is input into a self-organizing competitive neural network for star map recognition; the self-organizing competitive neural network refers to the lateral inhibition phenomenon of human neurons, that is, each weight update does not update all weights of the entire network, but only updates the weight of the winning node.
[0030] Step 303: Repeat the above steps 301 and 302 according to the partition weight until all partitions are successfully identified or searched. If no match is found after searching all partitions, restart the process from the next frame of the star map.
[0031] Beneficial effects:
[0032] 1. The present invention aims to solve the many shortcomings of the current star map method in the case of ultra-large field of view, and proposes a star map recognition method suitable for ultra-large field of view. The method can solve the serious distortion problem of star maps taken with ultra-large field of view, and can remove unreliable parts of the sky area and possible interference from false stars in the star map, while overcoming the problem of huge data volume of star maps in the whole sky area, and realizing rapid star map recognition in the whole sky area under complex sky environment. The present invention is suitable for star maps obtained by fisheye lenses with a field of view range of 120° to 180°.
[0033] 2. This star map recognition method realizes star map recognition under ultra-large field of view conditions through fisheye image correction, image partition weighting, false star interference removal, and neural network recognition. In the process of fisheye image processing, the correction time is shortened by directly constructing a correction model to improve the processing speed. In the image preprocessing process, the image is weighted and partitioned to remove invalid areas, thereby improving the recognition efficiency. Inter-frame matching technology is used to remove some false star interference to improve the recognition accuracy. The neural network star map recognition using mixed mode features can solve the problem of the huge number of navigation stars under ultra-large field of view, and greatly reduce the time and space complexity of the search algorithm.
[0034] 3. The establishment of the ultra-large field of view star map recognition method of the present invention fundamentally solves the technical problems of lightweight, small and low-cost astronomical navigation devices. It not only meets the requirements of fast and accurate measurement under static conditions, but also has the ability of fast and accurate positioning under dynamic conditions. It is widely applicable to various types of near-ground astronomical attitude measurement devices. It can completely solve the measurement and calibration of small positioning and navigation equipment under static and dynamic conditions. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 A flow chart of a method for identifying a star map with a very large field of view;
[0036] Figure 2 Schematic diagram of the fisheye imaging model. DETAILED DESCRIPTION
[0037] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0038] The present invention provides a method for identifying a star map with a very large field of view. The implementation process of the method for identifying a star map with a very large field of view is as follows: Figure 1As shown, the following steps are included:
[0039] Step 1: Correct the fisheye distortion of the captured star image. This correction step first uses a pre-calibrated fisheye lens model to directly correct the image, and then uses an optimization algorithm to further reduce the partial distortion that still exists after correction.
[0040] Step 2: After the image correction is completed, the corrected image is filtered to remove some background noise, and then the invalid area is recorded so that it can be eliminated in the partitioning stage. Finally, the information of adjacent frames is compared to suppress the interference of false stars caused by space targets.
[0041] After completing the filtering and processing of the image, it is necessary to divide the area according to the distance between the star point position and the image center, and assign priorities from high to low from near to far. This step can directly affect the recognition speed and the accuracy of the recognition results.
[0042] Step 3: After completing the above image preprocessing, the algorithm enters the star map recognition stage. In this stage, the partition weights are first sorted, and the partitions with larger weights are given priority. Then, suitable navigation stars are selected in the partition to construct star pattern features and perform neural network star map recognition with mixed pattern features. This method only requires one match to complete the matching of observed stars with many navigation stars, so it has higher recognition accuracy and recognition speed for situations with a large number of star points.
[0043] If a recognition fails, it is necessary to remove the partition where the recognition failed, then return to the weight judgment stage, select the partition with the highest weight and repeat the star map recognition until the navigation star is successfully recognized or all valid partitions are searched. If all valid partitions are traversed and still not successfully recognized, the recognition of this image fails, and the recognition of the next image is carried out.
[0044] Specifically, step one adopts the following steps and principles:
[0045] According to the principle of fisheye imaging, the object points in the three-dimensional space are first linearly projected onto a spherical surface, the center of which coincides with the imaging center, and the spherical surface is a unit sphere. This projection process is a linear projection and does not produce distortion. Then the points on the unit sphere are projected onto the image sensor plane. This process is a nonlinear process, and the distortion produced by this process becomes more serious as the field of view increases. The detailed imaging process is as follows: Figure 2 As shown, P represents a point in space, P c is the point where the object point is projected onto the unit sphere, X c ,Y c ,Z c is the camera coordinate system, θ is the incident angle, θ dTo take into account the actual incident angle after distortion, x, y are the physical image coordinate system, u, v are the pixel coordinate system of the image sensor, O is the center of the image sensor, r is the distance from the image point on the image sensor to the center of the sensor in an ideal situation, and r d is the distance from the image point on the image sensor to the center of the sensor in real conditions, and p is the image point of the spatial object point P on the image sensor.
[0046] There are four main types of fisheye lens imaging models currently available:
[0047] (1) Isometric projection model: r d =fθ
[0048] (2) Equal solid angle projection model:
[0049] (3) Stereoscopic projection model:
[0050] (4) Orthogonal projection model: r d =fsinθ
[0051] Taylor expansion of the terms containing θ in the above four models yields:
[0052] θ d =k1θ+k2θ 3 +L+k n θ 2n-1 ,(n=1,2,...) (1)
[0053] Therefore, we can use r d =fθ d Represents the four fisheye projection models mentioned above.
[0054] For an ordinary pinhole camera, its projection model can be expressed as:
[0055] r=ftanθ (2)
[0056] Where r represents the distance from the image point on the image sensor to the center of the sensor in the case of pinhole imaging.
[0057] The analysis shows that:
[0058] θ=arctan(r / f)≈r / f (3)
[0059] Incorporate the focal length f into the coefficient k of the Taylor expansion i You can get:
[0060]
[0061] Combining equations (3) and (4) and merging f, we can get:
[0062]
[0063] From this, we can get the distance r from the image point to the center of the sensor in the pinhole imaging case and the distance r from the image point to the center of the sensor in the real case. d The calculation formula is used to complete the distortion correction of star images taken with a super large field of view.
[0064] For the coefficient k in formula (5) i , it is necessary to use a star sensor to take a standard checkerboard image in advance, and then calculate it through a calibration algorithm.
[0065] In step 2, the image to be removed of false stars is partitioned and weighted, the invalid areas are recorded, and the invalid areas are removed, specifically:
[0066] Star map preprocessing begins with image filtering. First, the grayscale average of the entire image is calculated and used as the threshold for image filtering. That is, the pixel values below the threshold are assigned a value of 0, and the grayscale values of the pixel values above the threshold remain unchanged. After filtering, some stray light interference can be removed.
[0067] The filtered image will be subjected to false star removal, and the current frame will be compared with the previous frame or multiple frames. Since the distance between the interference target and the star sensor is much smaller than the distance between the star and the star sensor, the position change pattern of the interference target in different frames is often different from that of the normal navigation star. Some faster interference targets can even be reflected as a short line on the image. Therefore, the use of inter-frame matching and probability statistics can effectively remove interference targets and retain enough navigation stars for identification.
[0068] After the removal of false stars, the entire star map needs to be divided into several small circular areas. These small circular areas need to be large enough to ensure that there are enough stars in the range, because for the neural network recognition algorithm, more star points represent more features, which can improve the recognition accuracy to a certain extent. The reason for dividing into circular areas is to make the area rotationally invariant, which can reduce the complexity of the algorithm. Since the field of view of the star map is close to or even greater than 180°, large interfering targets such as buildings, trees, and clouds may appear in the field of view. Therefore, each area is simply traversed, and all areas with too large grayscale mean values can be regarded as invalid areas and removed. All remaining areas are assigned weights from large to small from the inside to the outside and indexed and sorted by weight.
[0069] In step 3, the star map recognition phase includes the following steps:
[0070] The recognition algorithm uses the mixed features of adjacent triangles and radial distribution vectors to form a feature vector, and uses a neural network to complete star map recognition.
[0071] For each circular area, the feature vector can be constructed by the following steps:
[0072] (1) The circular region is divided into k equally spaced annular regions, and the number of companion stars falling into each annular region is N1, N2, LN respectively. k .
[0073] (2) Select the observation star closest to the center of the region as the main star.
[0074] (3) Find the two companion stars closest to the primary star outside the field of the primary star. T1 is the angular distance from the primary star to the nearest companion star, T2 is the angular distance from the primary star to the next nearest companion star, and T3 is the angular distance between the two nearest companion stars.
[0075] (4) From N1, N2, LN k Together with T1, T2, and T3, it constitutes the feature vector of the observation area.
[0076] After the feature vector is constructed, it is input into the self-organizing competitive neural network for star map recognition. This neural network refers to the lateral inhibition phenomenon of human neurons, that is, each weight update does not update all weights of the entire network, but only updates the weights of the winning nodes, so it is more like a clustering method. This method does not require complex data sets for training, and the trained network can be directly used for star map recognition.
[0077] The above recognition process is repeated according to the partition weight until all partitions are successfully recognized or searched. If no match is found after searching all partitions, the process starts again from the next frame of the star map.
[0078] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for identifying star images with a very large field of view, characterized in that: The steps include: Step 1: Obtain the captured star map, perform fisheye distortion correction processing, and obtain a corrected image; Step 2: Filter the corrected image to remove some background noise, then compare the information of adjacent frames of the filtered image to suppress the interference of false stars caused by space targets, partition and weight the image after removing false stars, record the invalid area, and remove the invalid area, and then execute step 3; Step 3: Enter the star map recognition stage, sort according to the partition weights, give priority to the partition with larger weights, then select the appropriate navigation star in the partition to construct the star pattern feature and perform neural network star map recognition with mixed pattern features; If the recognition fails once, the partition where the recognition failed is removed, and then the weight judgment is performed again. The partition with the highest weight is selected to repeat the star map recognition until the navigation star is successfully recognized or all valid partitions are searched. If all valid partitions are traversed and still not successfully recognized, the recognition of this image fails, and the recognition of the next image is carried out.
2. The method for recognizing a star map with a super-large field of view according to claim 1, characterized in that: The step 1 is specifically as follows: The distance r from the image point to the center of the sensor in the case of pinhole imaging is different from the distance r from the image point to the center of the sensor in the real case. d The calculation formula is where k i is the coefficient of the Taylor expansion for the incident angle θ, i takes the value of [1,n], and n is the number of Taylor expansion coefficients; the distortion correction of the star map taken with a super large field of view is completed.
3. The method for recognizing a star map with a super-large field of view as claimed in claim 2, characterized in that: For the coefficient k i , a standard checkerboard image is captured by a star sensor in advance, and then calculated using a calibration algorithm.
4. A method for identifying a star map with a very large field of view as claimed in any one of claims 1 to 3, characterized in that: In the step 2, the image from which the false stars are removed is partitioned and weighted, the invalid areas are recorded, and the invalid areas are removed, specifically: After the false stars are removed, the entire star map needs to be divided into several circular partitions. The size of these small circular areas is set to ensure that there are enough stars within the range. Traverse each area, and record the partitions whose grayscale mean exceeds the set value as invalid areas and remove them; Assign weights from large to small to all remaining partitions from the inside to the outside and sort them by index.
5. A method for identifying a star map with a very large field of view as claimed in claim 4, characterized in that: In step 3, the star map recognition stage specifically includes the following steps: Step 301: construct a feature vector for each circular partition. Specifically, the feature vector is constructed by the following steps: The circular area is divided into k equally spaced annular areas, and the number of companion stars falling into each annular area is N1, N2, ..., N respectively. k ; Select the observation star closest to the center of the region as the primary star; Find the two companion stars closest to the primary star outside the field of the primary star. T1 is the angular distance from the primary star to the nearest companion star, T2 is the angular distance from the primary star to the next nearest companion star, and T3 is the angular distance between the two nearest companion stars. From N1, N2, ..., N k Together with T1, T2, and T3, it constitutes the feature vector of the observation area; Step 302: After the construction of the feature vector is completed, it is input into a self-organizing competitive neural network for star map recognition; the self-organizing competitive neural network refers to the lateral inhibition phenomenon of human neurons, that is, each weight update does not update all weights of the entire network, but only updates the weight of the winning node; Step 303: Repeat the above steps 301 and 302 according to the partition weight until all partitions are successfully identified or searched. If no match is found after searching all partitions, restart the process from the next frame of the star map.
Citation Information
Patent Citations
Spider web mode and convolutional neural network based star map identification method
CN107816987A
Star pattern recognition method applicable to shipborne super-wide field-of-view celestial navigation system
CN108469261A
Representation learning-based star identification method
US20200380712A1
