Remote sensing image registration method based on key target identification and neighborhood graph evaluation
By using a method based on key target identification and neighborhood graph evaluation, the problem of feature point matching in remote sensing image registration is solved, achieving high-precision and automated image registration, applicable to remote sensing images from different sensors.
Patent Information
- Application Number
- CN202411341601.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-25
- Publication Date
- 2025-01-10
AI Technical Summary
Traditional remote sensing image registration methods are difficult to meet the requirements of high precision, especially due to the difficulty in matching key feature points caused by the differences in shooting conditions of remote sensing images at different times.
A method based on key target identification and neighborhood graph evaluation is adopted. Key targets are screened by the YOLOv5 multi-class target identification model, and feature points are matched by SIFT feature extraction and fast nearest neighbor search algorithm. Finally, the coordinate transformation parameters are calculated by the least squares method for image registration.
It achieves high-precision registration of two phases of remote sensing images, with a registration accuracy of 1 pixel. It adapts to changes in ground features, improves the accuracy and automation of feature point matching, and is applicable to images acquired by different sensors.
Smart Images
Figure CN119295518A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of remote sensing image processing, and in particular to a two-phase remote sensing image high-precision registration method based on key target and neighborhood graph evaluation. Background Art
[0002] With the continuous development of remote sensing technology, remote sensing images have been widely used in many fields such as land and resources monitoring, environmental change assessment, and urban planning. However, due to the differences in shooting conditions of remote sensing images in different periods, such as different lighting, atmospheric conditions, sensor angles, etc., the changes in objects in the images are large, which makes the feature points of the same target different, resulting in difficulty in matching key feature points. Traditional remote sensing image registration methods are difficult to meet high-precision requirements. Therefore, it is necessary to study a high-precision registration method for two-period remote sensing images based on key targets. Summary of the invention
[0003] In view of the above problems, the present invention discloses a remote sensing image registration method based on key target recognition and neighborhood graph evaluation, which relates to the technical field of remote sensing image processing, effectively solves the problem of difficulty in matching key feature points, and improves the accuracy of remote sensing image registration.
[0004] The method comprises the following steps:
[0005] S1. Data collection and preprocessing: collect two periods of remote sensing image data, namely the standard period image data and the image data to be registered period, and convert them into a unified format. Perform radiation correction and geometric correction on the two periods of remote sensing image data to obtain the image to be identified;
[0006] S2. Key target identification: According to the content of the image to be identified and the application requirements, the type of key target is determined, the key target is identified and the identified key targets are screened to obtain the final key target;
[0007] S3, neighborhood graph construction: A neighborhood graph is established for each final key target in the two phases of remote sensing images;
[0008] S4, feature point matching: extract features of the final key target to obtain key feature points, and match and screen the key feature points to obtain key feature point pairs;
[0009] S5, coordinate transformation: transform the coordinates of key feature points and obtain the coordinate transformation parameters between two phases of remote sensing images by calculation;
[0010] S6, Image registration: Use the coordinate transformation parameters to register the two remote sensing images, update the neighborhood map of the image to be registered, and calculate the neighborhood map G of the standard image. x and the updated neighborhood graph G of the image to be registered yThe edit distance between them is x and G y When the edit distance is 0, it can be determined that the two-phase image registration is completed, otherwise it returns to step S4 to continue execution.
[0011] Furthermore, the radiation correction method is histogram equalization, and the geometric correction method is polynomial fitting.
[0012] Furthermore, when performing key target recognition, the trained YoloV5 multi-category target recognition model is used.
[0013] Furthermore, when the identified targets are screened, the screening is performed by setting the area of the targets.
[0014] Furthermore, the method for extracting the final key target is a SIFT feature extraction algorithm, and the method for matching the key feature points is a fast nearest neighbor search algorithm.
[0015] Furthermore, the method for transforming the coordinates of the key feature point pairs is a projection transformation method, which transforms the coordinates of the key feature point pairs into coordinates under the WGS84 coordinate system.
[0016] Furthermore, the least squares method is used to calculate the coordinate transformation parameters between the two phases of remote sensing images.
[0017] Furthermore, when the coordinate transformation parameters are used to register the two remote sensing images, the coordinates of the image to be registered are transformed by bilinear interpolation so that it is aligned with the image of the standard period in spatial position, thus completing the registration.
[0018] The beneficial effects of the present invention are:
[0019] (1) High-precision registration: By obtaining key targets through target recognition and using a series of optimization algorithms, high-precision registration of two phases of remote sensing images is achieved. The registration accuracy reaches 1 pixel, meeting the needs of high-precision analysis and application of remote sensing images.
[0020] (2) Adapting to changes in ground objects: In order to address the problem of large changes in ground objects in remote sensing images at different times, the target recognition method can accurately extract key targets, thereby improving the accuracy and reliability of feature point matching.
[0021] (3) High degree of automation: The registration process of the present invention can be automated, reducing manual intervention and improving work efficiency.
[0022] (4) Strong applicability: The present invention is applicable to remote sensing images acquired by different sensors and has strong versatility and applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1This is a flow chart of the method described in an embodiment of the present invention;
[0024] Figure 2 This is a schematic diagram of key target identification in an embodiment of the present invention;
[0025] Figure 3 This is a schematic diagram of feature point matching in an embodiment of the present invention;
[0026] Figure 4 This is a schematic diagram of image registration in an embodiment of the present invention;
[0027] Figure 5 Schematic diagram of image registration effect in an embodiment of the present invention. DETAILED DESCRIPTION
[0028] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0029] This embodiment provides a remote sensing image registration method based on key target recognition and neighborhood graph evaluation. The flowchart of the method is as follows: Figure 1 , the method comprises the following steps:
[0030] Step 1: Data collection and preprocessing:
[0031] S1. Collect two periods of remote sensing image data, namely the standard period image data and the image data to be registered period, and convert them into a unified TIFF format.
[0032] S2. Use the histogram equalization method and polynomial fitting method to perform radiation correction and geometric correction on the standard period image data and the image data of the registration period in succession; the role of radiation correction is to eliminate factors such as sensor response differences and atmospheric influences, and to ensure that the radiation of the image is as consistent as possible; the role of geometric correction is to eliminate geometric deformation caused by factors such as sensor posture and terrain undulations.
[0033] Step 2: Identification of key targets:
[0034] S1: Determine the type of key target based on the image content and application requirements. For example, buildings, roads, platforms, water bodies, bridges and other features with obvious characteristics can be selected as key targets.
[0035] S2: Use the trained YoloV5 multi-category target recognition model to identify key targets from two phases of remote sensing images. The model’s multiple categories include buildings, roads, platforms, water bodies, and bridges.
[0036] S3: By setting the area of the target (the area is greater than 50 square meters and less than 200 square meters), the identified key targets are screened and optimized, and noise targets and unstable feature points are removed to obtain the final key targets.
[0037] The relevant operations of key target identification are introduced with specific examples:
[0038] Environment preparation: Install PyTorch and YoloV5 libraries, and configure CUDA to accelerate model inference.
[0039] Model loading: Use the trained YoloV5 model weights, in which the classification types of key targets include "building", "road", "platform", "water body" and "bridge".
[0040] Data processing: Directly acquire remote sensing images and resize them to the input size required by the model (such as 640x640) through libraries such as OpenCV, without additional enhancement processing.
[0041] Target recognition: The processed image is input into the YoloV5 model for reasoning, and the model automatically outputs the category, location, and confidence of each target.
[0042] Result processing: Filter the results according to the preset confidence threshold of 0.5 and draw a bounding box on the original image, such as Figure 2 The box shown is the bounding box obtained after key target recognition processing.
[0043] Step 3: Neighborhood graph construction:
[0044] S1. Establish a neighborhood graph G for each final key target in the two phases of images. x and G y , the construction method is as follows: according to the position and size in the two phases of remote sensing images, a neighborhood graph G = (V, E) is dynamically constructed, where V represents the final key target, V = {v|v∈N, S}, N represents the classification name, S represents the pixel position of the center point of the final key target, and v represents the element in the V set; E represents the spatial relationship between the final key targets, E = {e|e∈dir, dis}, dir is v x -v y direction, dis is v x -v y The distance, “v x -v y " indicates that from v x to v y , v x and v y Represents the two final key targets in a phase of imaging, v x ∈V,vy ∈V.
[0045] Step 4: Feature point matching:
[0046] S1. Use SIFT feature extraction algorithm to extract features of the final key target and extract key feature points that can describe the target features.
[0047] S2. Use the fast nearest neighbor search algorithm to match the key feature points, and use the random sampling consensus (RANSAC) algorithm to screen the matching results, remove wrong matches, and obtain key feature point pairs.
[0048] Step 5: Coordinate transformation:
[0049] S1. Use the projection transformation method to transform the coordinates of the key feature points into coordinates in the WGS84 coordinate system.
[0050] S2. Use the least squares method to calculate the coordinate transformation parameters between the two remote sensing images. Suppose the coordinates of the feature points corresponding to the standard period image data are (x i ,y i ), the coordinates of the feature points corresponding to the image data to be registered are (x′ i ,y′ i ), the parameters of the affine transformation model are (a, b, c, d, e, f), then:
[0051]
[0052] Step 6: Image registration:
[0053] S1. Use coordinate transformation parameters to register the two remote sensing images. Use bilinear interpolation to transform the coordinates of the image to be registered so that it is aligned with the standard image in spatial position. After registration, interpolation is performed on the transformed image to improve the image quality.
[0054] S2. Update the neighborhood graph of the image to be registered and calculate the neighborhood graph G of the image in the standard period. x and the updated neighborhood graph G of the image to be registered y The edit distance between them is x and G y When the edit distance is 0, it can be determined that the registration accuracy of the two-phase image registration is better than 1 pixel, that is, the registration is completed, otherwise it returns to step S4 to continue execution.
[0055] like Figure 4 FIG. 1 is a schematic diagram of image registration using the registration method of the present invention. In order to verify the effect of the registration method of the present invention, two images are superimposed with a transparency of 50% to form a new image. The new image is as shown in FIG. Figure 5As shown, it is used to visually check the registration effect of the two images. Figure 5 The specific operations are as follows:
[0056] Read two images: Use Pillow's Image.open() method to read two images separately. Make sure that the two images have the same size, or at least the target areas can correspond, so that the registration accuracy can be accurately evaluated. Adjust transparency: Set the transparency (or alpha channel) of one of the images to 50%. In Pillow, this can be achieved by converting the image to RGBA mode (if the original image is not) and then modifying the alpha channel.
[0057] Overlay image: Overlay the image with adjusted transparency with another image. This can be achieved by blending the two images together, and Pillow's Image.blend() function can be used directly for this purpose. Save new image: Save the overlay image as a new image file.
Claims
1. A remote sensing image registration method based on key target recognition and neighborhood graph evaluation, characterized in that: The method comprises the following steps: S1. Data collection and preprocessing: collect two periods of remote sensing image data, namely the standard period image data and the image data to be registered period, and convert them into a unified format. Perform radiation correction and geometric correction on the two periods of remote sensing image data to obtain the image to be identified; S2. Key target identification: According to the content of the image to be identified and the application requirements, the type of key target is determined, the key target is identified and the identified key targets are screened to obtain the final key target; S3, neighborhood graph construction: A neighborhood graph is established for each final key target in the two phases of remote sensing images; S4, feature point matching: extract features of the final key target to obtain key feature points, and match and screen the key feature points to obtain key feature point pairs; S5, coordinate transformation: transform the coordinates of key feature points and obtain the coordinate transformation parameters between two phases of remote sensing images by calculation; S6, Image registration: Use the coordinate transformation parameters to register the two remote sensing images, update the neighborhood map of the image to be registered, and calculate the neighborhood map G of the standard image. x and the updated neighborhood graph G of the image to be registered y The edit distance between them is x and G y When the edit distance is 0, it can be determined that the two-phase image registration is completed, otherwise it returns to step S4 to continue execution.
2. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: The method used for the radiation correction is histogram equalization, and the method used for the geometric correction is polynomial fitting.
3. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: When performing key target recognition, the trained YoloV5 multi-category target recognition model is used.
4. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: When filtering the identified targets, the filtering is performed by setting the area of the target.
5. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: In step S3, the neighborhood graph is constructed as follows: according to the position and size in the two phases of remote sensing images, a neighborhood graph G = (V, E) is dynamically constructed, wherein V represents the final key target, V = {v|v∈N, S}, N represents the classification name, S represents the pixel position of the center point of the final key target, and v represents the element in the V set; E represents the spatial relationship between the final key targets, E = {e|e∈dir, dis}, dir is v x -v y direction, dis is v x -v y The distance, "v x -v y " indicates that from v x to v y , v x and v y Represents the two final key targets in a phase of imaging, v x ∈V,v y ∈V.
6. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: The method for extracting the final key target is the SIFT feature extraction algorithm, and the method for matching the key feature points is the fast nearest neighbor search algorithm.
7. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: The method for transforming the coordinates of the key feature point pairs is a projection transformation method, which transforms the coordinates of the key feature point pairs into coordinates under the WGS84 coordinate system.
8. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: The least squares method is used to calculate the coordinate transformation parameters between two phases of remote sensing images.
9. The remote sensing image registration method based on key target recognition and neighborhood graph evaluation according to claim 1, characterized in that: When two remote sensing images are registered using coordinate transformation parameters, bilinear interpolation is used to transform the coordinates of the image to be registered so that it is aligned with the standard image in spatial position, thus completing the registration.