A large-area matching method for high-resolution multi-view remote sensing images of lunar orbiter

Abstract

The application relates to a large-area matching method for high-resolution multi-view remote sensing images of a lunar orbiter, comprising the following steps: acquiring a data set composed of multi-time and multi-view high-resolution lunar orbiter images, screening the images in combination with relevant information and calculating stereo pairs; adopting a multi-node parallel mode to input each stereo pair information, carrying out a matching task in a multi-node parallel mode based on a master-slave structure communication mode; completing a single matching task in each node, extracting key points, constructing key point descriptors and matching the descriptors in a GPU space based on preset geometric constraints, obtaining a preliminary connection point result and returning to a CPU space; eliminating gross error points from the connection points based on RANSAC to generate a connection point file, namely, completing the matching. Compared with the prior art, the application can greatly improve the matching efficiency while guaranteeing the matching precision of the lunar high-resolution remote sensing images, and meets the connection point matching demand of large-range lunar DEM production.

Description

Technical Field

[0001] This invention relates to the field of photogrammetry and satellite data processing technology, and in particular to a large-area matching method for high-resolution multi-view remote sensing images from a lunar orbiter. Background Technology

[0002] A lunar probe is an unmanned spacecraft that explores the Moon, Earth's natural satellite. It is capable of imaging the lunar surface at different resolutions and with varying coverage. Mapping the lunar topography and obtaining two-dimensional and three-dimensional images is a task required in the lunar exploration programs of all countries. For example, the US Lunar Reconnaissance Orbiter (LRO) launched in June 2009 carries a wide-angle (WAC) camera and a narrow-angle (NAC) camera. The LRO NAC can acquire high-resolution lunar orbiter images with a north-south orientation, a swath width of approximately 5000 meters, and a spatial resolution of 0.5–1.5 meters at an altitude of about 50 km above the lunar surface. In October 2010, my country's CE-2 (Chang...) lunar probe... The E-2 satellite was successfully launched from the Xichang Satellite Launch Center. It uses a linear array pushbroom imaging method to acquire two linear array image stripes along the satellite's flight direction, with forward-looking and backward-looking perspectives. The two linear arrays share a single optical system. At a 100km orbit, the spatial resolution is approximately 7m and the imaging swath width is approximately 43km; at a 15km orbit, the spatial resolution is approximately 1.5m and the swath width is approximately 9.2km.

[0003] A lunar surface DEM (Digital Elevation Model) is created by obtaining the image coordinates of corresponding points on a CCD image through image matching, and then calculating the spatial coordinates of the corresponding points on the lunar surface. The key to this method lies in selecting a suitable image matching algorithm. Image matching is actually a core component of digital photogrammetry research; its essence is to establish the correspondence between elements of multiple images. The task of image matching is to find the optimal transformation relationship between images, which involves multiple elements such as feature space, search space, similarity measure, and matching strategy. Therefore, to better conduct high-precision, high-quality, and high-efficiency lunar exploration, proposing a matching method suitable for high-resolution multi-view remote sensing images from a lunar orbiter is an important prerequisite for creating a high-quality DEM.

[0004] In existing research, classic feature matching methods are typically detector-based. Some classic detectors based on manual feature extraction, such as SIFT, SURF, BRIEF, and ORB, were first proposed and widely used in various 3D computer vision tasks. With the advent of deep learning, many detectors based on convolutional neural networks, such as R2D2, SuperPoint, D2-Net, and LF-Net, have been proposed to further improve the robustness of keypoints and descriptors under changes in illumination and viewpoint. However, these methods all originate from Earth imagery or natural images. If applied to matching tasks using high-resolution lunar imagery, they struggle to simultaneously meet the multiple demands for efficiency and accuracy, and are also difficult to extend to high-resolution image matching over large areas. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a large-area matching method for high-resolution multi-view remote sensing images of lunar orbiters. This method can greatly improve the matching efficiency while ensuring the matching accuracy of high-resolution lunar remote sensing images, and meet the tie-point matching requirements for large-scale lunar DEM production.

[0006] The objective of this invention can be achieved through the following technical solution: a large-area matching method for high-resolution multi-view remote sensing images from a lunar orbiter, comprising the following steps:

[0007] S1. Acquire a dataset consisting of high-resolution lunar orbiter images from multiple times and views, combine relevant information to filter images and calculate stereo image pairs;

[0008] S2. Each stereo image pair is transmitted in a multi-node parallel manner, and the matching task is carried out in parallel by multiple nodes based on the master-slave structure communication mode.

[0009] S3. Complete a single matching task within each node. Based on preset geometric constraints, extract key points, construct key point descriptors, and match descriptors in GPU space to obtain preliminary connection point results and send them back to CPU space.

[0010] S4. Based on RANSAC, perform coarse error removal on the connection points to generate the connection point file, thus completing the matching.

[0011] Furthermore, step S1 specifically includes the following steps:

[0012] S11. Select multiple images of the study area to form an input set based on the image thumbnails and the center latitude and longitude at the time of shooting;

[0013] S12. Based on ephemeris information and image attributes, images are screened from multiple aspects such as illumination conditions, image visibility, and image pair conditions. By calculating whether there is an overlap area above a preset percentage threshold between pairs of images, images that can form stereo image pairs are determined. These images are used as the final matching image set, and matching element files are generated at the same time.

[0014] Furthermore, the image attributes in step S12 include the image's emission angle, incident angle, illumination direction, and resolution.

[0015] Furthermore, the matching element file includes matching overlapping areas and an image matching list.

[0016] Furthermore, the preset percentage threshold in step S12 is specifically 30%.

[0017] Furthermore, in step S2, the master-slave communication mode includes a master node and multiple child nodes. The master node traverses the matching element file and sends the remaining tasks to the multiple child nodes in sequence. After receiving the tasks, the child nodes carry out a single matching task within their own nodes, and the operation of multiple child nodes is simultaneous and parallel.

[0018] Furthermore, step S2 specifically includes the following steps:

[0019] S21. Adopting a master-slave communication mode, node 0 is designated as the master node and nodes 1 to n as child nodes. The matching image set and matching feature file are read into the master node respectively.

[0020] S22. The master node iterates through the matching element file in sequence and sends the corresponding matching image pair image number and overlapping area information to each child node. After receiving the information, the child node completes one serial image matching task.

[0021] After the master node has distributed a complete set of matching feature files for all child nodes, if it receives a matching task completion signal from a child node, the master node will send a new task. At this time, the operation of child nodes 1 to n is carried out simultaneously.

[0022] Furthermore, the number of the child node n ≤ 8.

[0023] Furthermore, step S3 specifically includes the following steps:

[0024] S31. Read the stereo image pair and divide the image into blocks in the overlapping area of ​​the stereo image pair;

[0025] S32. Upload the segmented image to the GPU space. Based on the preset geometric constraints, extract key points, construct key point descriptors, and match descriptors in sequence in the GPU space to obtain preliminary connection point results and send them back to the CPU space.

[0026] Furthermore, the preset geometric constraints are specifically as follows:

[0027] Calculate the relationship between the resolution and pixel aspect ratio between the reference image and the matching image between stereo image pairs, and statistically analyze the proportional relationship between 1 pixel in the reference image and 1 pixel in the matching image.

[0028] Based on the pixel correspondence ratio, calculate the image splitting in the overlapping area;

[0029] The latitude and longitude coordinates of the first and last rows within the overlapping area are calculated. The starting row for matching after image splitting is determined by comparison. After splitting, the key point descriptors within the split are matched in GPU space.

[0030] Compared with the prior art, the present invention has the following advantages:

[0031] This invention targets a dataset composed of high-resolution lunar orbiter images from multiple times and views. It combines relevant information to filter images and calculate stereo pairs. Based on a master-slave communication model, it enables parallel matching across multiple nodes. In a single matching task, mechanically repeatable steps such as keypoint extraction, keypoint description, and descriptor matching are uploaded to GPU space for acceleration. After matching, the connection points are returned to CPU space for coarse-grained point removal before being written to a file. This approach balances the imaging characteristics of orbiter images with the caching capabilities of GPUs. High-resolution images are segmented before GPU acceleration, and geometric constraints are applied during connection point matching. This ensures matching accuracy while effectively improving matching efficiency, thus meeting the connection point matching requirements for large-scale lunar DEM creation.

[0032] This invention overcomes the problem of large-area rapid matching of high-resolution multi-view remote sensing images of the moon, and proposes a matching process strategy based on multi-node parallelism combined with GPU acceleration of filtered stereo image pairs. Certain geometric constraints are added to the matching task, which increases the number of matching connection points while ensuring the accuracy of connection points.

[0033] This invention employs a parallel structure, making full use of computational resources and improving the overall matching speed of the algorithm. Experimental results show that this invention has significant advantages over traditional image feature matching algorithms in terms of total matching time, matching accuracy, and root mean square error.

[0034] This invention utilizes image segmentation and combines it with the RANSAC algorithm to remove mismatched points (i.e. gross errors), which can further improve matching accuracy. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0036] Figure 2 Flowchart for multi-node parallel matching;

[0037] Figure 3 This is a schematic diagram of a multi-node parallel master-slave communication mode.

[0038] Figure 4a This is a schematic diagram of image segmentation and matching.

[0039] Figure 4b This is a schematic diagram of matching based on geometric constraints. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0041] Example

[0042] like Figure 1 As shown, a large-area matching method for high-resolution multi-view remote sensing images from a lunar orbiter includes the following steps:

[0043] S1. Acquire a dataset consisting of high-resolution lunar orbiter images from multiple times and views, combine relevant information to filter images and calculate stereo image pairs;

[0044] S2. Each stereo image pair is transmitted in a multi-node parallel manner, and the matching task is carried out in parallel by multiple nodes based on the master-slave structure communication mode.

[0045] S3. Complete a single matching task within each node. Based on preset geometric constraints, extract key points, construct key point descriptors, and match descriptors in GPU space to obtain preliminary connection point results and send them back to CPU space.

[0046] S4. Based on RANSAC, perform coarse error removal on the connection points to generate the connection point file, thus completing the matching.

[0047] This embodiment applies the above solution, and its main contents include:

[0048] Step 1: Obtain high-resolution lunar orbiter image data corresponding to the target study area, then combine relevant information to filter images and calculate stereo image pairs. Specifically:

[0049] Step 1.1: Based on the latitude and longitude of the study area, search for all LRO NAC images within the study area in NASA's Planetary Data System (PDS) (Source: https: / / ode.rsl.wustl.edu / moon / productsearch) and download them.

[0050] Step 1.2, when downloading images, first select multiple scenes of images in the research area to form an input set according to the image thumbnails and the central latitude and longitude at the time of shooting;

[0051] Then, the images are screened from three aspects of lighting condition, image quality and stereo pair condition according to ephemeris information and attributes of the images such as emission angle, incident angle, illumination direction and resolution, and images that have an overlapping area of more than 30% while ensuring image illumination, that is, can form a stereo pair are obtained by calculation and screening. These images are used as the final image set, and matching element files such as matching overlapping area and image matching list are generated.

[0052] Step 2, the screened stereo pairs are input in a multi-node parallel manner, and a master-slave structure is adopted for the communication mode. The master node traverses the task list and sends remaining tasks to multiple slave nodes sequentially, and after receiving the task, the slave node carries out a single matching task within the node, wherein the operation of multiple slave nodes is performed simultaneously;

[0053] See Figure 2 , the implementation process can be divided into the following steps, and the specific method flow of each step is as follows:

[0054] Step 2.1, as shown in Figure 3 , a master-slave communication mode is adopted, that is, node 0 is recorded as the master node, and nodes 1 to n are recorded as operating slave nodes (considering the computer performance in the laboratory, generally n ≤ 8), and the image set screened in step 1 and matching element files such as the matching list and overlapping area are respectively read into the master node;

[0055] Step 2.2, the master node traverses sequentially according to the matching element files, and sends the corresponding matching stereo pair image numbers and overlapping area information to each slave node. After receiving the information, the slave node completes a serial matching task process of one image. After the master node distributes all the matching element files of one complete round, it prepares to receive the task completion signal sent back by the slave nodes, and at this time, the operation of nodes 1 to n is performed simultaneously;

[0056] Step 2.3, since the performance of each slave node is different, after node i (i < n) finishes its task, node i+1 has not completed the task. At this time, the master node will send a new matching task to node i until the master node has distributed all matching tasks.

[0057] Step 3: After the child node receives the information, a serial image matching task is completed. A single matching process consists of keypoint extraction, keypoint description, descriptor matching, and coarse-grained point removal. Mechanically repeatable steps such as keypoint extraction, keypoint description, and descriptor matching are uploaded to the GPU space for acceleration. After matching, preliminary connection points are returned to the CPU space, and coarse-grained points are removed and written to a file. By considering both the imaging characteristics of the orbiter image and the GPU's caching capabilities, the high-resolution image is divided into blocks before being uploaded to the GPU space. Certain geometric constraints are applied during connection point matching.

[0058] Specifically, the process involves: reading stereo image pairs, dividing the images into blocks based on the overlapping areas of the stereo image pairs, uploading the block images to the GPU space, extracting key points, constructing key point descriptors, and matching descriptors in sequence in the GPU space, then sending the preliminary connection points back to the CPU space, and removing coarse errors based on the RANSAC algorithm on the block images to finally generate a connection point file.

[0059] See Figure 4a Keypoint extraction and descriptor construction can be completed by considering only a single patch image. However, descriptor matching needs to consider the correspondence between keypoints; matching keypoints from a single region will inevitably result in errors, and a single patch image will greatly increase this error rate. Therefore, a certain degree of geometric constraint is required, as detailed in the following steps:

[0060] Step 3.1: Calculate the relationship between the resolution and pixel aspect ratio between the reference image and the matching image between stereo image pairs, and statistically analyze the corresponding ratio of 1 pixel between the reference image and the matching image.

[0061] Step 3.2: Considering the small field of view of LRO NAC images, the imaging parameters are 50,000 rows * 5,000 columns. See [link / reference] Figure 4b Based on the pixel ratio, image splitting is performed in the overlapping area, which can improve matching efficiency and greatly avoid incorrect matching of connection points.

[0062] Step 3.3: Due to the differences in the flight direction of the orbiters, the row numbers of different images of the same area vary greatly. Therefore, it is necessary to calculate the latitude and longitude coordinates of the first and last rows in the overlapping area, and determine the starting row for matching after the image is split by comparison. After splitting, the key point descriptors in the split row are matched in the GPU space.

[0063] To verify the effectiveness of this solution, a comparative experiment was conducted in this embodiment as follows:

[0064] I. Experimental Data

[0065] Using LRO NAC images of the Apollo 17 and Chang'e 4 landing areas as experimental data, 32 images from the Apollo 17 landing area were used to form 20 stereo image pairs, and 24 images from the Chang'e 4 landing area were used to form 15 stereo image pairs. The matching method of this invention was then used to test the experimental data.

[0066] II. Experimental Results

[0067] Table 1 summarizes the results of using the matching strategy of this invention and directly using the SIFT matching algorithm. Experiments show that the matching strategy of this invention can effectively increase the number of connection points and improve the image matching accuracy between stereo pairs while ensuring efficiency. Table 2 compares the matching efficiency of the matching strategy of this invention by incorporating parallel methods. Reasonable use of computing resources can quickly and effectively complete the matching task of a high-resolution lunar orbiter over a large area.

[0068] Table 1 Results of the matching strategy and SIFT matching algorithm of this invention

[0069]

[0070] Note: In the table, the number of hops is the average of all hops in the study area to the number of hops in each valid matched pair (number of matched hops > 100), and RMSE is the mean root mean square error in the study area for each valid matched pair.

[0071] Table 2 Comparison of matching efficiency between parallel and serial methods

[0072]

[0073] In summary, compared with existing mature computer vision libraries, this invention effectively improves matching efficiency while ensuring accuracy, and can well meet the task of rapid matching of large areas of high-resolution multi-view remote sensing images from lunar orbiters.

Claims

1. A method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter, characterized in that, Includes the following steps: S1. Acquire a dataset consisting of high-resolution lunar orbiter images from multiple times and views, combine relevant information to filter images and calculate stereo image pairs; S2. Each stereo image pair is transmitted in a multi-node parallel manner, and the matching task is carried out in parallel by multiple nodes based on the master-slave structure communication mode. In step S2, the master-slave communication mode includes a master node and multiple child nodes. The master node traverses the matching element file and sends the remaining tasks to the multiple child nodes in turn. After receiving the task, the child node performs a single matching task within its own node, and the operation of multiple child nodes is simultaneous and parallel. Step S2 specifically includes the following steps: S21. Adopting a master-slave communication mode, node 0 is designated as the master node and nodes 1 to n as child nodes. The matching image set and matching feature file are read into the master node respectively. S22. The master node iterates through the matching element file in sequence and sends the corresponding matching image pair image number and overlapping area information to each child node. After receiving the information, the child node completes one serial image matching task. After the master node has distributed a complete set of matching feature files for all child nodes, if it receives a matching task completion signal from a child node, the master node will send a new task. At this time, the operation of child nodes 1 to n is carried out simultaneously. S3. Complete a single matching task within each node. Based on preset geometric constraints, extract key points, construct key point descriptors, and match descriptors in GPU space to obtain preliminary connection point results and send them back to CPU space. The preset geometric constraints are specifically as follows: Calculate the relationship between the resolution and pixel aspect ratio between the reference image and the matching image between stereo image pairs, and statistically analyze the proportional relationship between 1 pixel in the reference image and 1 pixel in the matching image. Based on the pixel correspondence ratio, calculate the image splitting in the overlapping area; Calculate the latitude and longitude coordinates of the first and last rows within the overlapping area, determine the starting row for matching after image splitting by comparison, and then match the key point descriptors within the split rows in GPU space. S4. Based on RANSAC, perform coarse error removal on the connection points to generate the connection point file, thus completing the matching.

2. The method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 1, characterized in that, Step S1 specifically includes the following steps: S11. Select multiple images of the study area to form an input set based on the image thumbnails and the center latitude and longitude at the time of shooting; S12. Based on ephemeris information and image attributes, images are screened from multiple aspects such as illumination conditions, image visibility, and image pair conditions. By calculating whether there is an overlap area above a preset percentage threshold between pairs of images, images that can form stereo image pairs are determined. These images are used as the final matching image set, and matching element files are generated at the same time.

3. The method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 2, characterized in that, The image attributes in step S12 include the image's emission angle, incident angle, illumination direction, and resolution.

4. The method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 2, characterized in that, The matching element file includes matching overlapping areas and an image matching list.

5. A method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 2, characterized in that, The preset percentage threshold in step S12 is specifically 30%.

6. The method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 1, characterized in that, The child node number n≤8.

7. A method for large-area matching of high-resolution multi-view remote sensing images from a lunar orbiter according to claim 4, characterized in that, Step S3 specifically includes the following steps: S31. Read the stereo image pair and divide the image into blocks in the overlapping area of ​​the stereo image pair; S32. Upload the segmented image to the GPU space. Based on the preset geometric constraints, extract key points, construct key point descriptors, and match descriptors in sequence in the GPU space to obtain preliminary connection point results and send them back to the CPU space.

Images