Manufacturing method of on-satellite lightweight vector control library

Abstract

The invention relates to a satellite lightweight vector control library manufacturing method, and belongs to the technical field of satellite positioning. The method comprises the following steps: firstly, acquiring road network data, extracting node and road center line information, constructing a road network topological relation, and forming a ground vector control library; secondly, extracting a local vector construction mode based on random walk in allusion to missing and fracture phenomena which cannot be avoided in road extraction in an on-satellite environment; and finally, providing a hierarchical matching strategy based on a hidden Markov model (HMM) to complete matching, and correcting the RPC by using the offset of a matching transformation model to improve the image positioning precision. The lightweight vector control library to be constructed in the invention refers to extraction of vector network node coordinates with stable structures and obvious features from an image. Compared with a traditional ground control point, field measurement and acquisition are not needed, and the control data acquisition cost is greatly reduced; compared with an image control point, local image blocks do not need to be stored, so that the control data is further lightened.

Description

Technical Field

[0001] The invention relates to a method for manufacturing an on-board lightweight vector control library, belonging to the technical field of satellite positioning. Background Art

[0002] With the rapid development of domestic remote sensing satellites, the types of satellites, imaging methods and the number of satellites in orbit are increasing. Corresponding to the large and diverse satellite population is the limited satellite-to-ground transmission capacity and the cumbersome and slow ground processing technology. Therefore, using the autonomous processing function on the satellite to directly generate products and distribute them to users, and realizing end-to-end remote sensing and mapping application services has become an important development direction in the current remote sensing and mapping field.

[0003] As one of the key technologies of on-board autonomous processing and the basic work of subsequent applications, on-orbit geometric positioning is an indispensable part of remote sensing applications. Most of my country's existing high-resolution remote sensing satellites are non-stereo mapping satellites, which only have high resolution and high mobility, but do not have high positioning capabilities, and it is difficult to meet the high-precision positioning requirements of important targets, such as "Gaofen-2", "Jilin-1" and some high-resolution remote sensing satellites. Therefore, improving the positioning capabilities of such satellites on-orbit is of great significance to the realization of end-to-end remote sensing mapping applications. At present, on-board on-orbit geometric positioning can be mainly divided into satellite-ground joint mode and on-board on-orbit mode. With the continuous improvement of the performance of on-board processors, the satellite-ground joint mode has gradually begun to shift to the on-board on-orbit mode. The premise of realizing on-board on-orbit geometric positioning is to inject control information into the on-board processing system. However, the amount of control information data required for existing ground geometric positioning is huge and cannot be directly loaded into the on-board processing system. At present, only a few satellites in the world have successfully realized on-board on-orbit geometric positioning experiments using on-board control information in local areas. Therefore, how to lighten the global control information with huge data volume has become one of the problems that need to be solved in the current development of on-board autonomous processing technology.

[0004] At present, there are still few studies on the construction of onboard control libraries at home and abroad. Wang Yueli et al. proposed to establish a multi-level and multi-source control point image database system, and to realize the graphical management of control points, multiple query methods, and the extraction and output of GCP. In view of the problem that the geometric accuracy of images is difficult to ensure due to multi-source heterogeneity, Pu Shengliang et al. proposed to automatically match the images of multi-source heterogeneous sensors in the air and space, and to build generalized image control points and control point libraries based on traditional ordinary ground control points. However, these early control libraries were designed for ground processing systems and did not consider the problem of limited onboard processing resources. In response to this, someone proposed a complete method for making onboard lightweight image control points, designed a lightweight representation mode for onboard image control points, and used hash mapping learning to convert feature vectors into hash codes, realizing the deep lightweight of onboard image control points; however, the control library constructed using point features only has low-level single-point information, which is easily affected by environmental noise and has poor globality, thus affecting the subsequent image positioning accuracy. Summary of the invention

[0005] The purpose of the present invention is to provide a method for producing a lightweight on-board vector control library to solve the problems of poor globality and poor image positioning accuracy caused by susceptibility to external environmental interference in the current lightweight on-board vector control library.

[0006] In order to solve the above technical problems, the present invention provides a method for manufacturing a satellite lightweight vector control library, which comprises the following steps:

[0007] 1) Extract the road network from the ground image to obtain the topological structure of the road network, and build a ground vector control library based on the topological structure and node coordinates of the road network;

[0008] 2) Obtaining a random walk vector by random walk on the image to be matched;

[0009] 3) Back-project the constructed ground vector control library to the image coordinate system and match it with the obtained random walk vector to obtain the on-board lightweight vector control library.

[0010] The present invention first extracts the road network from the ground image, and establishes a ground vector control library based on the topological structure and node coordinates of the road network. Compared with traditional control points, the present invention greatly reduces the amount of data required to establish control points, which is easier to process on the satellite; then, in the on-satellite part, the present invention selects random walk vectors for subsequent matching to eliminate the impact of road missing on matching; finally, the constructed ground vector control library is back-projected to the image coordinate system, and matched with the obtained random walk vector, and a lightweight vector control library on the satellite is obtained according to the matching result. The present invention uses the road network to construct the control library, does not require local images or image point descriptors, and greatly reduces the amount of data required, which can meet the lightweight requirements on the satellite, and has good globality and is not easily affected by the external environment, providing accurate positioning information for the subsequent.

[0011] Furthermore, the step 3) adopts a vector matching method of HMM.

[0012] The present invention introduces the hidden Markov model into vector matching, assumes the vector generated by random walk of the image as the observed state and the vector control library as the hidden state, takes the matching problem between the two as the decoding problem of HMM, solves the most likely hidden state sequence, and quickly realizes the matching between the ground vector control library and the random walk vector.

[0013] Furthermore, the transition probability between hidden states corresponding to two consecutive observed states when the HMM vector matching is used in step 3) is:

[0014]

[0015] where d n,n+1 Represents the distance between two consecutive observation state nodes, that is, the distance between two random walk vector nodes; Represents the distance between two consecutive hidden state nodes, that is, the distance between two vector control library nodes.

[0016] In order to avoid the problems in vector matching caused by encrypted random walk vectors, the present invention uses the ratio of the larger value to the smaller value of the distance between two consecutive observed state nodes and the distance between hidden state nodes as the transition probability when calculating the transition probability between vectors, and measures the transition probability by the degree of distance proximity, thereby avoiding errors caused by the increase of the search radius in the matching process.

[0017] Furthermore, the step 3) also includes encrypting the obtained random walk vectors at equal intervals.

[0018] Furthermore, the method also includes adopting a hierarchical matching method when performing vector matching of the HMM, and the hierarchical matching process is:

[0019] The random walk vector is translated in the image coordinate system in both horizontal and vertical directions by a first set step length to determine the vector structure with the highest matching score between the random walk vector at each position and the vector control library;

[0020] The random walk vector is translated between the minimum and the second minimum distance score in the horizontal and vertical directions with a second set step length, and the position of the random walk vector closest to the vector control library after this translation is determined, and HMM matching is performed at this position.

[0021] The present invention adopts a hierarchical matching strategy, that is, moving the random walk vector structure in the image coordinate system, obtaining the random walk vector structure position closest to the vector library matching point, and matching here, thereby obtaining a high-precision matching result, ensuring the matching accuracy and reducing the matching operation amount.

[0022] Furthermore, the process of determining the random walk vector in step 2) is as follows:

[0023] A node is randomly selected as the starting point on the image to be matched, and the nodes connected to the node are searched, and the next node connected to the starting point is randomly walked; then the next node is used as the starting point, and the connected nodes are searched and randomly walked until the number of wandering nodes reaches the set threshold or the wandering reaches the endpoint; during the random walk, each node is passed at most once.

[0024] Furthermore, the construction process of the ground vector control library in step 1) is as follows:

[0025] A. Extract road information from ground images through deep learning algorithms to extract road networks;

[0026] B. Extract the road centerline network, obtain the pixel points next to the nodes or endpoints in the road centerline network, and obtain a node set;

[0027] C. Use the node set to break the road network to obtain a curve set representing the centerline of the road, and use the node set and curve set to determine the connected node set to obtain the topological relationship of the road network.

[0028] Furthermore, step 1) also includes using stereo images or DEM data as an aid to perform regional network adjustment on the images that have completed vector extraction, and determining the geographic coordinates of the vector network nodes through stereo intersection, which together with the topological relationship obtained in step C constitute a ground vector control library. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 It is a flow chart of extracting the vector network in the present invention;

[0030] Figure 2a Schematic diagram of the random walk process in an embodiment of the present invention;

[0031] Figure 2b is a schematic diagram of the termination of the random walk in an embodiment of the present invention;

[0032] Figure 3a is the random walk vector node distance d in the embodiment of the present invention n,n+1 Greater than the candidate vector node distance Schematic diagram of the calculation of the transition probability when ;

[0033] Figure 3b is the random walk vector node distance d in the embodiment of the present invention n,n+1 Less than the candidate vector node distance Schematic diagram of the calculation of the transition probability when ;

[0034] Figure 4a is a schematic diagram of a candidate area when the candidate radius is too large in an embodiment of the present invention;

[0035] Figure 4b is a schematic diagram of a candidate area when the candidate radius is too small in an embodiment of the present invention;

[0036] Figure 4c is a schematic diagram of a candidate region after the first translation in an embodiment of the present invention;

[0037] Figure 4d is a schematic diagram of a candidate region after the second translation in an embodiment of the present invention;

[0038] Figure 5 It is a flow chart of the method for making a lightweight vector control library on board a satellite according to the present invention. DETAILED DESCRIPTION

[0039] The specific implementation of the present invention will be further described below in conjunction with the accompanying drawings.

[0040] The present invention first extracts the road network from the ground image, and establishes a ground vector control library based on the topological structure and node coordinates of the road network. Compared with traditional control points, the present invention greatly reduces the amount of data required to establish control points, which is easier to process on the satellite; then, in the on-satellite part, the present invention selects random walk vectors for subsequent matching to eliminate the impact of road missing on matching; finally, the constructed ground vector control library is back-projected to the image coordinate system and matched with the obtained random walk vector, and a lightweight vector control library on the satellite is obtained according to the matching results. The present invention uses the road network to construct the control library, and does not require local images or image point descriptors. The required amount of data is greatly reduced, which can meet the lightweight requirements on the satellite, and has good globality and is not easily affected by the external environment, providing more accurate positioning information for the subsequent period. Therefore, the lightweight vector control library to be constructed by the present invention refers to the vector network node coordinates extracted from the image with stable structure and obvious characteristics. Compared with traditional ground control points, there is no need to obtain them by field measurement, which greatly reduces the cost of obtaining control data; compared with image control points, there is no need to store local image blocks, which makes the control data further lightweight. The implementation process of this method is as follows Figure 5 As shown, the specific implementation process of this method is described in detail below.

[0041] 1. Extract the road network from the ground image and build a ground vector control library using the road network information.

[0042] Classic vector extraction algorithms mostly use local peak detection in the transform domain or grayscale, gradient and other information in the image domain to extract edge vectors. In the face of high-resolution remote sensing images, the vectors directly extracted by this approach will be small, numerous and disorganized. However, the road network, water system and boundary lines of land objects have relatively fixed structures and obvious features. Using this type of information to extract vector features is simpler and more effective in practical applications. The present invention uses road information as an example to construct a vector control library. The ground image in this embodiment is an image that has been unloaded from the ship to the ground in the early stage and is used to make a database. Its main steps are as follows: Figure 1 shown.

[0043] 1) Extract road network

[0044] With the continuous development of deep learning algorithms, the extraction accuracy of current road extraction algorithms has reached a relatively high level. The D-LinkNet model in this embodiment performs road mask extraction. The model can fully consider the narrowness, connectivity, complexity and long-span spatial performance characteristics of roads in images, and the model network is relatively simple, which can achieve a good balance between algorithm performance, robustness and complexity. In addition, considering the problem of robustness of the algorithm, this embodiment uses the existing road network data to improve and supplement the extraction results when using the D-LinkNet model to extract the road network, so that the road network information in the vector library is as complete as possible.

[0045] 2) Determine the node set based on the road centerline network

[0046] The road network binary image is subjected to skeleton thinning to obtain a single-pixel road network, i.e., a road centerline network. On this basis, the road centerline network is slid along the road centerline network according to the set window size. Since the number of pixels at the node is greater than the number of pixels at the non-node in the road, the present invention calculates the number of road pixels m in each window. When m is greater than the set threshold, the coordinates of the center point of the window (x p ,y p ), thereby forming a preliminary node set p n The preliminary nodes in this embodiment refer to a number of points around the intersection of the road centerline.

[0047] 3) Get the node coordinates of the road centerline network

[0048] According to step 2, since the node set p n The node set p includes multiple pixel points near the nodes or endpoints in the road centerline network, that is, it includes multiple points on the road centerline near the nodes. Therefore, clustering is required to obtain the nodes on each road. The density-based noise application spatial clustering (DBSCAN) method is used to cluster the node set p. n Clustering is performed within the network to obtain the node coordinates of the road centerline network.

[0049] 4) Determine the topological relationship

[0050] According to the node set p n Break the road network, obtain a set of curves C representing the center lines of the roads, and traverse the node set p n , use the curve set to query the connected node set. Specifically, if the node p i With node p j There is a curve C m So that the two ends of the curve are connected to the node p i or p j The distance is less than the threshold T dis , then we consider that node p i With node p j Connected, add the connected nodes to the topology relationship table T topology In addition, it is a deep lightweight vector control library. The curve set is only used to build the topological relationship table and is not used to register satellites.

[0051] 5) Using foreign high-precision stereo images or high-precision DEM data as an aid, perform regional network adjustment on the images that have completed vector extraction; stereo intersection to obtain the geographic coordinates of the vector network nodes, combined with the topological relationship table generated in step 4) to form a lightweight vector control library.

[0052] Through the above steps, lightweight vector control data can be obtained on the ground. The position accuracy of the lightweight vector control data obtained by the present invention is at the meter level, and it is mainly used in remote sensing satellites with non-stereoscopic imaging. In addition, the lightweight vector control library constructed in this paper only contains node coordinates and topological relationship files. Compared with traditional control points, the amount of data is greatly reduced and easier to process on the satellite. Therefore, the lightweight vector control library is injected into the satellite, which can be used for remote sensing satellites with low positioning capabilities such as video satellites and micro-nano satellites.

[0053] 2. Get the on-star vector.

[0054] In the satellite part, in order to match with the lightweight vector control library on the ground, the captured image also needs to extract the vector structure, and the extraction method is roughly the same as the ground processing process. However, in the road extraction stage, the satellite part cannot use the existing road information to supplement and improve the extraction results, and there will inevitably be road missing and broken phenomena, resulting in vector network missing. To address this problem, the present invention abandons the global vector network matching strategy and instead chooses random walk vector structure matching to eliminate the impact of road missing on matching.

[0055] Specifically, Figure 2a As shown in the figure, firstly, n nodes are randomly selected on the image to be matched (the image taken in real time on the satellite), and the node p i For example, the nodes connected to this node are p j 、p k 、p l and p m , starting from this node, randomly walk to the next node connected to this node, such as walking to node p l ; and then p l As the origin, query the connected nodes and walk randomly. Repeat the above process until the number of wandering nodes reaches the threshold T num Or it stops when it reaches the endpoint. In addition, to avoid the wandering route from falling into a local loop, the nodes in the wandering route cannot be repeated, that is, each node can only be passed once. Figure 2b In the case shown, when walking to the node p s , whose connection point p t 、p u 、p v and p w The walk cannot be repeated, so the walk will terminate early. According to the above conditions, the random walk node sequence {p a ,p b ,...,p x}.

[0056] 3. Perform vector matching

[0057] According to the image positioning file, the lightweight vector control library determined in step 1 is back-projected to the image coordinate system and matched with the random walk vector structure in the image.

[0058] In order to make the vector structure participate in the matching as a whole, this embodiment first encrypts the random walk vector node sequence at equal intervals; then introduces the hidden Markov model into the vector matching, and assumes that the vector structure generated by the random walk of the image is the observation state and the vector control library is the hidden state. The matching problem between the two can be approximately regarded as the decoding problem of HMM - given the model parameters and a set of observation values, the most likely hidden state sequence is solved.

[0059] A key issue in the HMM-based vector matching process is to quickly determine the candidate hidden state (lightweight vector control library node) based on the observed state (random walk vector node). The present invention sets the candidate range radius to n meters. When the vector control library contains or intersects with the random walk vector node When is the origin and n is the radius, The projection points on these vectors are Candidate hidden states of . Usually, Corresponding to multiple vectors with candidate points (p a ,p b ,p c ,...,p n ), each candidate point and The corresponding probability follows Gaussian distribution and can be obtained by formula (1):

[0060]

[0061] in Is a node To the candidate point p j The Euclidean distance of , ρ and e are the standard deviation and expected error of the Gaussian distribution respectively.

[0062] In addition, the HMM model parameters also include the transition probability tp between the hidden states corresponding to two consecutive observation states; the distance d between the two observation state nodes is generally used. n,n+1 Distance to hidden state node The ratio of represents the transition probability, as shown in formula (2):

[0063]

[0064] However, after the present invention adopts the encrypted random walk vector structure node, the formula (2) may cause problems in the vector matching of the present invention. Figure 3a In the random walk vector node distance d n,n+1 Greater than the candidate vector node distance When , the transfer probability will be greater than 1; Figure 3b In the example, two random walk vector nodes are matched to the same candidate vector, making is 0, which will cause formula (2) to fail to correctly calculate the transition probability. In view of this, the present invention chooses to use the degree of distance proximity to measure the transition probability, that is, rewrite formula (2) into formula (3):

[0065]

[0066] Formula (3) ensures that Figure 3a In this case, the transition probability is less than 1. Figure 3b In this case, the transition probability is 0, thus avoiding errors caused by the increase of the search radius n during the matching process.

[0067] After the above calculation, the walking vector node sequence can be obtained Multiple sets of candidate vectors The matching score calculation formula for each candidate vector set is shown in formula (4):

[0068]

[0069] The optimal matching vector Ci, that is, the path with the highest score, can be obtained based on the matching score and the Viterbi algorithm.

[0070] From the above description, we can see that the candidate radius n is crucial to the success of the match. When n is large enough, it can be guaranteed that the candidate area contains the correct matching object, but the candidate area will also contain a large number of wrong objects, resulting in an exponential increase in the time cost, such as Figure 4a As shown in ; when n is reduced, the candidate area may not contain the correct matching object, resulting in matching errors, such as Figure 4b In addition, the random walk vector node When determining the vector projection point, if the distance between the random walk vector and the control library vector is too large, the projection point accuracy is poor. Therefore, when only using HMM to match vectors in a large range, the matching results will have huge occasional errors and the RPC offset cannot be effectively corrected.

[0071] In order to ensure the matching accuracy and reduce the amount of matching operations, the present invention proposes a hierarchical matching strategy, that is, moving the random walk vector structure in the image coordinate system, obtaining the position of the random walk vector structure closest to the vector library matching point, and matching here, thereby obtaining a high-precision matching result.

[0072] Specifically, first, the random walk vector is translated in both the horizontal and vertical directions with a step size of S1, such as Figure 4c As shown, the vector structure with the highest matching score between the random walk vector at each position and the vector library is found, and the distance score C between the two is calculated. The calculation formula is shown in formula (5):

[0073]

[0074] Where P represents all nodes and encrypted points of the random walk vector, (x i ,y i ) represents the coordinates of the i-th point in the random walk vector, It means (x i ,y i ) is the coordinate of the projection point on the vector structure with the highest matching score, where t represents the equation (6):

[0075] Score t =max(Score1,Score2,...,Score n ) (6)

[0076] After obtaining the distance score between the two positions through the first translation, a translation with a step length of S2 is performed between the minimum and the second minimum distance in the horizontal and vertical directions, such as Figure 4d As shown, S2 is smaller than S1. Similarly, the position of the random walk vector closest to the lightweight vector library is obtained through equations (5) and (6), and the best matching result can be obtained by matching at this position.

Claims

1. A method for preparing a lightweight vector control library on a satellite, characterized in that: The method comprises the following steps: 1) Extract the road network from the ground image to obtain the topological structure of the road network, and build a ground vector control library based on the topological structure and node coordinates of the road network; 2) Obtaining a random walk vector by random walking on the image to be matched; 3) Back-project the constructed ground vector control library to the image coordinate system and match it with the obtained random walk vector to obtain a lightweight vector control library.

2. The method for producing a satellite lightweight vector control library according to claim 1, characterized in that: The step 3) adopts the vector matching method of HMM.

3. The method for making a satellite lightweight vector control library according to claim 2, characterized in that: The transition probability between the hidden states corresponding to two consecutive observed states when the vector matching of HMM is adopted in step 3) is: where d n,n+1 Represents the distance between two consecutive observation state nodes, that is, the distance between two random walk vector nodes; Represents the distance between two consecutive hidden state nodes, that is, the distance between two vector control library nodes.

4. The method for producing a satellite lightweight vector control library according to claim 2, characterized in that: The step 3) also includes encrypting the obtained random walk vectors at equal intervals.

5. The method for making a satellite lightweight vector control library according to claim 2, characterized in that: The method also includes adopting a hierarchical matching method when performing vector matching of the HMM, and the hierarchical matching process is: The random walk vector is translated in the image coordinate system in both horizontal and vertical directions by a first set step length to determine the vector structure with the highest matching score between the random walk vector at each position and the vector control library; The random walk vector is translated between the minimum and the second minimum distance score in the horizontal and vertical directions with a second set step length, and the position of the random walk vector closest to the vector control library after this translation is determined, and HMM matching is performed at this position.

6. The method for manufacturing a satellite lightweight vector control library according to claim 1, characterized in that: The process of determining the random walk vector in step 2) is as follows: A node is randomly selected as the starting point on the image to be matched, and the nodes connected to the node are searched, and the next node connected to the starting point is randomly walked; then the next node is used as the starting point, and the connected nodes are searched and randomly walked until the number of wandering nodes reaches the set threshold or the wandering reaches the endpoint; during the random walk, each node is passed at most once.

7. The method for making a satellite lightweight vector control library according to claim 1, characterized in that: The construction process of the ground vector control library in step 1) is as follows: A. Extract road information from ground images through deep learning algorithms to extract road networks; B. Extract the road centerline network, obtain the pixel points next to the nodes or endpoints in the road centerline network, and obtain a node set; C. Use the node set to break the road network to obtain a curve set representing the centerline of the road, and use the node set and curve set to determine the connected node set to obtain the topological relationship of the road network.

8. The method for manufacturing a satellite lightweight vector control library according to claim 7, characterized in that: The step 1) also includes using stereo images or DEM data as an aid to perform regional network adjustment on the images that have completed vector extraction, and determining the geographic coordinates of the vector network nodes through stereo intersection, which together with the topological relationship obtained in step C constitute a ground vector control library.

Images