A high-precision spatial positioning method and system based on multi-source image automatic matching

Abstract

The application discloses a high-precision space positioning method and system based on multi-source image automatic matching, and the method comprises the following steps: acquiring a tower video image and a satellite remote sensing image from a satellite remote sensing image acquisition module and a tower camera module; selecting homonymic feature points of the tower video image and the satellite remote sensing image, and constructing a projection transformation relationship of the satellite remote sensing image and the tower video image; obtaining image 1 and image 2 according to the projection transformation relationship and selected points to be solved; inputting the image 1 and the image 2 into an improved SURF algorithm feature matching module to obtain a transformation matrix; and performing mutual conversion through the transformation matrix to output a satellite remote sensing image coordinate or a tower video image coordinate. The method greatly improves the image matching success rate, greatly reduces the influence of environmental factors such as weather and season on image matching, removes the matching link of a video image to be positioned and a reference library video image, and does not need to perform a large amount of image matching work.

Description

Technical Field

[0001] This invention relates to the field of video monitoring technology for iron towers, and in particular to a high-precision spatial positioning method and system based on automatic matching of multi-source images. Background Technology

[0002] Currently, the main approach to achieving spatial positioning of targets in video images is to establish a matching relationship between satellite remote sensing images and video images by using the known projection transformation relationship between feature points in video images and corresponding feature points of spatial targets, and then extending this relationship to the point to be located for calculation. However, directly applying this method to the spatial positioning of targets in tower video, which involves complex monitoring targets and a wide monitoring range, has certain limitations, mainly manifested in: "accurate positioning in good weather, inaccurate positioning in bad weather," and "accurate positioning when close to the control point, inaccurate positioning when far from the control point." Therefore, conducting research on high-precision spatial positioning methods for tower video images is of great significance for the in-depth application of tower video in related fields.

[0003] Currently, there are three main technical approaches to achieve the mutual conversion between video image coordinates and satellite remote sensing image coordinates: First, constructing a virtual 3D geographic scene model of the camera's surrounding environment and mapping the image captured by the camera or the imaging line between the camera's optical center and the target to be located onto the constructed virtual 3D geographic scene for solution; second, constructing a camera imaging model based on intrinsic and extrinsic parameters such as camera focal length, distortion coefficient, and rotation matrix, and using it to solve for the intersection coordinates of the imaging line between the camera's optical center and the target to be located with auxiliary data such as the Digital Elevation Model (DEM) and Digital Model of Height (DSM); third, using dense matching algorithms to establish a matching relationship between satellite remote sensing images and video images. However, tower videos are captured by PTZ cameras mounted at high altitudes that can rotate 360 ​​degrees, covering a large monitoring range. Constructing a virtual 3D geographic scene around the camera is labor-intensive, and the camera imaging model requires many parameters. Therefore, using dense matching algorithms to establish a matching relationship between satellite remote sensing images and tower video images is currently the main approach to achieving target localization. The specific steps are as follows:

[0004] First, a spatial positioning benchmark database is constructed, containing video images of iron towers, satellite remote sensing images, and the projection transformation relationship between the two. Second, an image matching algorithm is used to obtain the iron tower video image with the highest overlap with the iron tower video image to be located from the benchmark database, and the distortion of the iron tower video image to be located is corrected to make the distortion ratio consistent with the iron tower video image obtained from the benchmark database. Third, the projection transformation relationship between the iron tower video image obtained from the benchmark database and the satellite remote sensing image is established and applied to the iron tower video image to be located for calculation, so that the spatial position of the iron tower video image to be located can be obtained.

[0005] However, the above method has the following problems:

[0006] (1) Establishing a benchmark database involves a large amount of work: a large amount of video image data of iron towers needs to be collected, and the same feature points need to be selected from satellite remote sensing images (at least 8 points need to be selected for each image, and hundreds of points need to be selected for a single camera); at the same time, due to the influence of factors such as changes in the land surface, the benchmark database needs to be continuously updated and maintained, which consumes a huge amount of manpower and material resources.

[0007] (2) Poor stability of feature matching: The influence of factors such as weather and lighting will cause problems such as reduced clarity and image distortion in the video images of the iron tower, which will lead to the failure of matching and positioning using the same feature points; at the same time, since the texture, color and other features of the video targets in the iron tower video images are very different in different seasons, it will also lead to the failure of matching and positioning.

[0008] (3) Low matching efficiency: In order to determine the video image with the greatest overlap, the video image to be located needs to be matched one by one with the iron tower video images in the benchmark database. Furthermore, the integral image is not fully utilized to accelerate feature extraction, which takes a long time and is inefficient, and cannot meet the real-time monitoring requirements.

[0009] (4) Insufficient positioning accuracy: The closer the video target is to the corresponding feature point in the image, the higher the positioning accuracy; the farther the video target is from the corresponding feature point, the worse the positioning accuracy. Summary of the Invention

[0010] This invention combines photogrammetry theory and image matching algorithms to propose a high-precision spatial positioning method and system based on automatic multi-source image matching. It achieves the conversion of tower video image coordinates to geospatial coordinates and vice versa. The method includes:

[0011] The tower video images and satellite remote sensing images are acquired from the satellite remote sensing image acquisition module and the tower camera module.

[0012] Select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved.

[0013] The initial horizontal deflection angle and initial pitch angle of the point to be solved are calculated according to the projection transformation relationship. The tower camera is controlled to capture images under the initial horizontal deflection angle and initial pitch angle to obtain image 1. The initial coordinates of the point to be solved are calculated according to the projection transformation relationship. The initial coordinates of the point to be solved are used as the center point, and twice the distance from the center point to the tower is used as the side length. A square image is cropped from the satellite remote sensing image to obtain image 2.

[0014] Image 1 and Image 2 are input into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. The accurate conversion between the coordinates of the satellite remote sensing image and the coordinates of the tower video image is achieved based on the transformation matrix.

[0015] The improved SURF algorithm feature matching module first uses the SURF algorithm to extract feature points from Image 1 and Image 2. Centered on the feature points, the main direction of the feature points is determined based on the Haar template and the rotational cumulative response of the fan-shaped window. By fusing HSV color features, a descriptor for the feature points is generated. Then, the feature points of Image 1 and Image 2 are matched using a K-nearest neighbor feature matching algorithm with a large threshold. Initial matching point pairs are selected, and RANSAC filtering is used to remove mismatches, resulting in the final matching point pairs. Finally, a coefficient matrix is ​​constructed based on the final matching point pairs, and the transformation matrix between the satellite remote sensing image and the tower video image is obtained by singular value decomposition of the coefficient matrix.

[0016] Preferably, calculating the initial horizontal deflection angle and initial pitch angle of the point to be solved based on the projection transformation relationship includes:

[0017] When the point to be solved is located in the video image of the tower, the horizontal rotation angle and the initial elevation angle of the point at the time of shooting are taken as the initial horizontal rotation angle and the initial elevation angle. When the point to be solved is located in the satellite remote sensing image, the coordinates of the point to be solved on the satellite remote sensing image are first obtained. Through the projection transformation relationship, the coordinates of the point to be solved are calculated and transformed to obtain the initial horizontal rotation angle and the initial elevation angle of the point to be solved on the video image of the tower.

[0018] Preferably, calculating the initial coordinates of the point to be solved based on the projection transformation relationship includes:

[0019] When the point to be solved is located in the video image of the tower, the horizontal rotation angle and pitch angle of the point to be solved are first obtained. The coordinates of the point to be solved are calculated and transformed through the projection transformation relationship to obtain the initial coordinates of the point to be solved on the satellite remote sensing image. When the point to be solved is located in the satellite remote sensing image, the coordinates of the point to be solved on the satellite remote sensing image are taken as the initial coordinates.

[0020] Preferably, the SURF algorithm is used to extract feature points from the satellite remote sensing image and the tower video image to calculate descriptors, including:

[0021] First, feature point extraction is performed, and Image 1 and Image 2 are converted into integral images. Then, the fast Hessian matrix determinant approximation value of each pixel is calculated. Finally, multi-scale feature detection is used to detect maxima and optimize sub-pixel interpolation.

[0022] Then, the descriptor is calculated. A circular region with a radius of 6s is taken, centered on the feature point, where 's' is the feature point scale. The gradient is calculated using a 4s-scale Haar template. The rotational cumulative response of the fan-shaped window with the angular angle is used to calculate the principal direction of the feature point, and then the direction centered on the feature point is calculated. The area is divided into Sub-blocks, within the statistical area ,in, The horizontal and vertical gradients are used to form feature vectors, which are then fused with HSV color features to generate descriptors for the feature points.

[0023] Preferably, the step of using the SAD similarity metric for matching and filtering out initial matching point pairs specifically involves filtering... is the initial matching point pair.

[0024] Preferably, the precise conversion between satellite remote sensing image coordinates and tower video image coordinates is achieved based on the transformation matrix, specifically as follows:

[0025] When it is necessary to convert satellite remote sensing image coordinates to tower video image coordinates, the pixels of the target point in the satellite remote sensing image are extracted. The geographic coordinates of the target point are mapped to video pixels through the transformation matrix H. The center of the moving camera to Read the actual horizontal yaw angle P value and pitch angle T value;

[0026] When it is necessary to convert the coordinates of a tower video image to satellite remote sensing image coordinates, and extract the pixels of the target point in the tower video image... Through homography matrix Calculate the geographic coordinates of the target point .

[0027] Furthermore, the satellite remote sensing image acquisition module supports the access of 0.5-meter resolution multispectral images and digital elevation models; the tower camera module includes a high-definition camera with known three-dimensional spatial coordinates, adjustable horizontal deflection angle and pitch angle, and capable of focal length transformation, and a tower site with known three-dimensional spatial coordinates.

[0028] This invention also provides a high-precision spatial positioning system based on automatic matching of multi-source images. The various modules of the system are connected via a data bus to achieve data interaction, specifically including:

[0029] The satellite remote sensing image acquisition module supports the integration of 0.5-meter resolution multispectral images and digital elevation models for acquiring satellite remote sensing images.

[0030] The tower camera module includes a high-definition camera with adjustable horizontal deflection and pitch angles and focal length transformation based on known three-dimensional spatial coordinates, and a tower site with known three-dimensional spatial coordinates, used to acquire video images of the tower.

[0031] The benchmark database construction module is used to select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved; calculate the initial horizontal deflection angle and the initial pitch angle of the point to be solved according to the projection transformation relationship, control the tower camera to capture images at the initial horizontal deflection angle and the initial pitch angle to obtain image 1, calculate the initial coordinates of the point to be solved according to the projection transformation relationship, take the initial coordinates of the point to be solved as the center point, and take twice the distance from the center point to the tower as the side length to crop a square image on the satellite remote sensing image to obtain image 2.

[0032] An improved SURF algorithm feature matching module includes feature point extraction, HSV-fused feature description, similarity measurement and mismatch removal, and transformation matrix solving. This module is used to input the satellite remote sensing image and the tower video image into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. The improved SURF algorithm feature matching module first uses the SURF algorithm to extract feature points from image 1 and image 2. Centered on the feature points, the main direction of the feature points is determined based on the Haar template and the rotational accumulation response of the fan-shaped window. By fusing HSV color features, descriptors for the feature points are generated. Then, a high-threshold K-nearest neighbor feature matching algorithm is used to match the feature points of image 1 and image 2, filtering out initial matching point pairs. RANSAC filtering is then used to remove mismatches, resulting in final matching point pairs. Finally, a coefficient matrix is ​​constructed based on the final matching point pairs, and the coefficient matrix is ​​decomposed using singular value decomposition to obtain the transformation matrix between the satellite remote sensing image and the tower video image.

[0033] The tower video image coordinate and geospatial coordinate conversion module is used to convert the coordinates of the tower video image and the coordinates of the satellite remote sensing image to each other through the transformation matrix.

[0034] The results output module supports the output of satellite remote sensing image coordinates and the horizontal deflection angle and elevation angle of the tower video image.

[0035] This invention proposes a high-precision spatial positioning method and system based on automatic multi-source image matching. Combining photogrammetry theory and image matching algorithms, it constructs a projection transformation relationship between tower video images and satellite remote sensing images, and calculates and calibrates the spatial position of video targets in the tower video images. Compared with traditional methods, it has the following advantages:

[0036] (1) Only a very small number of identical feature points are needed to construct the projection transformation relationship between the video image of the tower and the satellite remote sensing image (only 1-10 points are selected for each camera).

[0037] (2) The SURF (Speeded-Up Robust Features) algorithm is improved by integrating HSV color features, K nearest neighbor feature matching algorithm with large threshold and RANSAC mismatch elimination algorithm, which greatly improves the image matching success rate and significantly reduces the impact of environmental factors such as weather and season on image matching.

[0038] (3) The matching process between the video image to be located and the benchmark video image is eliminated, eliminating the need for a large amount of image matching work. At the same time, the integral image is used to accelerate feature extraction, which can greatly improve the efficiency of the algorithm and meet the needs of real-time monitoring. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of the basic process of the high-precision spatial positioning method based on automatic matching of multi-source images according to an embodiment of the present invention.

[0040] Figure 2 This is a flowchart of the improved SURF algorithm of the method in the embodiment of the present invention.

[0041] Figure 3 A schematic diagram illustrating the conversion of satellite remote sensing imagery into video image coordinates of the tower.

[0042] Figure 4 A schematic diagram illustrating the conversion of video images of iron towers into satellite remote sensing image coordinates.

[0043] Figure 5 This is a block diagram of a high-precision spatial positioning system based on automatic multi-source image matching, according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] As one implementation scheme, Figure 1 This is a schematic flowchart illustrating a high-precision spatial positioning method based on automatic multi-source image matching, as described in an embodiment of this application. Figure 1 As shown, this embodiment describes a high-precision spatial positioning method based on automatic multi-source image matching, which specifically includes:

[0046] The first step is to acquire video images of the iron tower and satellite remote sensing images from the satellite remote sensing image acquisition module and the iron tower camera module.

[0047] The satellite remote sensing image acquisition module is used to acquire high-resolution multispectral satellite remote sensing images and DEM data, and supports the access of 0.5-meter resolution multispectral images and digital elevation models.

[0048] The tower camera module includes a high-definition camera with adjustable horizontal deflection and pitch angles and focal length transformation, based on known three-dimensional spatial coordinates, and a tower site with known three-dimensional spatial coordinates. It is used to control the rotation of the PTZ high-definition camera and take pictures, while comparing it with satellite remote sensing imagery. The corresponding control points in the tower video image and satellite remote sensing image are manually selected, and the P-value, T-value, and Z-value of the corresponding control points in the tower video image and their latitude and longitude (x, y) in the satellite remote sensing image are recorded.

[0049] The second step is to select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved.

[0050] The third step is to calculate the initial horizontal deflection angle and initial pitch angle of the point to be solved based on the projection transformation relationship, and control the tower camera to capture images at the initial horizontal deflection angle and initial pitch angle to obtain image 1. The initial coordinates of the point to be solved are calculated based on the projection transformation relationship. The initial coordinates of the point to be solved are used as the center point, and twice the distance from the center point to the tower is used as the side length. A square image is cropped from the satellite remote sensing image to obtain image 2.

[0051] The fourth step involves inputting Image 1 and Image 2 into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. Based on the transformation matrix, the accurate conversion between the coordinates of the satellite remote sensing image and the coordinates of the tower video image is achieved.

[0052] The process of the improved SURF algorithm feature matching module is as follows: Figure 2 As shown, the specific process is as follows:

[0053] (1) Feature point extraction

[0054] (a) Create integral images of Image 1 and Image 2 to reduce the computation time for feature point extraction.

[0055] For any pixel (i,j) in Image 1 and Image 2, based on the formula , The OpenCV interface `integral()` is called to obtain the integral image of Image 1 and Image 2. Here, `i` and `j` are the coordinates of pixels in the image, and `S(i,j)` represents the column integral from the beginning of the row to the j-th column at position `i`, which is the sum of the grayscale values ​​from left to right to the j-th column in a row. Grayscale value This represents the value of the pixel at coordinates (i,j) in the image.

[0056] (b) Construct a Hessian matrix and use the Hessian matrix to determine whether the pixel is an extreme point.

[0057] First, scale to the current scale using the function resizeHaarPattern(), then use the function calcLayerDetAndTrace() to calculate the approximate value of the Hessian determinant.

[0058] (c) Multi-scale feature detection for precise feature point localization.

[0059] First, a scale space is created. During image matching, differences in shooting angle, shooting distance, and other factors can cause the same target to appear at different sizes in different images. Therefore, to address the matching failure problem caused by this, a scale space is created to detect features of the integral image at multiple scales, ensuring the scale invariance of the features. In this embodiment, a scale space with 4 groups (n0ctaves=4) and 3 layers (n0ctaveLayers=3) is constructed.

[0060] Secondly, nonmaximum suppression is used to initially determine feature points in the scale space, that is, the function fastHessianDetector() is used to detect the initially determined feature points in the constructed scale space.

[0061] Finally, the feature points are precisely located. After initially determining the feature points at different scales, the function interpolateKeypoint() is used to perform sub-pixel interpolation optimization (positioning accuracy ≤ 0.1 pixels). The remaining extreme points are the extracted feature points.

[0062] (2) Feature description (fusion HSV)

[0063] Main direction determined: Centered on the feature point, take a radius of 6s ( s is the feature point scale, L is the layer index of the feature point detected in the scale space (values ​​0-11), and the gradient is calculated using a 4s-scale Haar template for a circular region. The fan-shaped window of the angle is rotated to accumulate the response, and the main direction is the direction of the maximum response vector (the feature point is copied when the secondary peak is ≥ 80% of the main peak).

[0064] Generate descriptors: Centered on feature points, The area is divided into Sub-blocks, statistics ,in, The horizontal and vertical gradients are used to form a 64-dimensional feature vector, which is then fused with the HSV color mean, i.e., H / S / V normalization, to generate a 67-dimensional descriptor.

[0065] (3) Similarity measurement and mismatch elimination

[0066] SAD similarity measure: Filter initial matching pairs with SAD≤50;

[0067] KNN+RANSAC filtering: K=2 (nearest neighbor / second nearest neighbor ratio <0.8), RANSAC in-point threshold = 2.0 pixels, the false match rate is reduced to below 5%.

[0068] (4) Solving for the transformation matrix (SVD optimization)

[0069] Build based on the final matching point pairs The coefficient matrix A, where n is the final number of matched point pairs, is obtained by singular value decomposition of matrix A. ,Pick The last column serves as the transformation matrix, where, Σ is a left singular vector matrix, and Σ is a singular value diagonal matrix. It is a right singular vector matrix. yes The transpose of .

[0070] The above embodiments combine photogrammetry theory and image matching algorithms to construct the projection transformation relationship between tower video images and satellite remote sensing images, and to calculate and calibrate the spatial position of video targets in the tower video images. Compared with traditional methods, it has the following advantages:

[0071] (1) Only a very small number of homonymous feature points are needed to construct the projection transformation relationship between the tower video image and the satellite remote sensing image. Only 1-10 homonymous points need to be selected for a camera.

[0072] (2) The SURF (Speeded-Up Robust Features) algorithm is improved by integrating HSV color features, K nearest neighbor feature matching algorithm with large threshold and RANSAC mismatch elimination algorithm, which greatly improves the image matching success rate and significantly reduces the impact of environmental factors such as weather and season on image matching.

[0073] (3) The matching process between the video image to be located and the benchmark video image is eliminated, eliminating the need for a large amount of image matching work. At the same time, the integral image is used to accelerate feature extraction, which can greatly improve the efficiency of the algorithm and meet the needs of real-time monitoring.

[0074] In another embodiment, it is necessary to convert geospatial coordinates into tower video image coordinates, that is, to convert satellite remote sensing images into tower video image coordinates, the process of which is as follows: Figure 3 As shown, the specific process is as follows:

[0075] 1. Manually select a feature point with the same name, denoted as point A. The geospatial coordinates of point A are then given as follows: The video image coordinates are as follows .

[0076] 2. Select a point in the satellite remote sensing image as the spatial coordinate point to be solved, denoted as point B. The geographic spatial coordinates of point B are then given as follows: .

[0077] 3. The geographic spatial coordinates of the iron tower are marked as points. The elevation of camera point C is denoted as The parameters required to convert geospatial coordinates to tower video image coordinates are detailed in Table 1.

[0078] Table 1 details the parameters required to convert geospatial coordinates to tower video image coordinates:

[0079]

[0080] 4. Solve for the horizontal deflection angle P of point B. b Value, and rotate the camera to a horizontal deflection angle P. b Place.

[0081] (1) Construct a Cartesian plane rectangular coordinate system with due east as the positive x-axis, due north as the positive y-axis, and the tower point O as the origin. At this time, the vectors in geographic space can be known. The angle between the x-axis and the positive x-axis is the horizontal deflection angle P of the video coordinate point B to be solved. b value.

[0082] (2) Draw a perpendicular line from the x-axis to the video coordinate point B to be solved, and a right triangle can be constructed. Then, the horizontal deflection angle P of the unknown point B can be solved using the slope formula. b Value, see the following formula:

[0083] ,

[0084] In the formula, y b Let x be the latitude coordinate of the video coordinate point B to be solved. bHere are the longitude coordinates of the video coordinate point B to be solved; y t Let x be the latitude coordinates of point O on the tower. t Here are the longitude coordinates of point O on the iron tower; To determine the latitude difference between video coordinate point B and tower point O, The longitude difference between the video coordinate point B and the tower point O is to be solved.

[0085] (3) Control the camera to rotate to the horizontal deflection angle P b .

[0086] 5. Obtain the matching image.

[0087] Find the pitch angle T of point B. b Value: Construct a Cartesian coordinate system with east as the positive x-axis, north as the positive y-axis, and tower point O as the origin. Connect the video coordinate point B to be solved with the camera point C to construct a right triangle in the coordinate system. The pitch angle T of the video coordinate point B to be solved is... b The value is the angle between the line connecting BC and the x-axis, and the calculation formula is shown below:

[0088] ,

[0089] In the formula, h is the elevation difference between the camera and the solution video coordinate point B, and R is the distance between the tower O and the solution video coordinate point B. The calculation formula is as follows:

[0090] ,

[0091] ,

[0092] In the formula, For the camera's elevation, To determine the elevation of video coordinate point B, y b Let x be the latitude coordinate of the video coordinate point B to be solved. b Here are the longitude coordinates of the video coordinate point B to be solved; y t Let x be the latitude coordinates of point O on the tower. t Here are the longitude coordinates of point O on the iron tower;

[0093] After obtaining the PT value of the video coordinate point B to be solved, control the camera to capture an image at that PT value as the video image of the tower.

[0094] Centered on the video coordinate point B to be solved, and with twice the distance R from the tower to the video coordinate point B to be solved as the side length, a square image is extracted as the satellite remote sensing image.

[0095] 6. Input the tower video image and satellite remote sensing image into the improved SURF matching algorithm module to perform image matching and transformation matrix solution, and output the transformation matrix H.

[0096] 7. Calculate video pixel location: Map the geographic coordinates of point B to the video pixel using the homography matrix H. , , These are the x and y coordinates of the pixels, respectively, moved from the center of the camera to... Read the actual horizontal yaw angle P value and pitch angle T value.

[0097] In another embodiment, it is necessary to convert the tower video image coordinates into geospatial coordinates, that is, to convert satellite remote sensing images into tower video image coordinates. The process is as follows: Figure 4 As shown, the specific process is as follows:

[0098] 1. Manually select a feature point with the same name, denoted as point A. The geospatial coordinates of point A are (xa, ya, za), and the video image coordinates are (P...). a , T a Z a );

[0099] 2. Select a point in the video image of the iron tower as the coordinate point to be solved, denoted as point B. The video image coordinates of point B are then (P... b , T b Z b );

[0100] 3. The geographic spatial coordinates of the iron tower are denoted as point O(x). t , y t , z t The elevation of camera point C is denoted as z0. The parameters required to convert the geographic spatial coordinates to the coordinates of the tower video image are shown in Table 2.

[0101] Table 2 details the parameters required to convert geospatial coordinates to tower video image coordinates:

[0102]

[0103] Calculate the horizontal and vertical components of the geospatial coordinate point B in the Cartesian plane coordinate system, and extract video images and satellite remote sensing images of the tower.

[0104] (1) Construct a Cartesian plane rectangular coordinate system with due east as the positive x-axis, due north as the positive y-axis, and the iron tower point O as the origin.

[0105] (2) The horizontal component of the geographic spatial coordinate point B in the Cartesian plane coordinate system can be obtained by using trigonometric function formulas. Vertical component The calculation formula is as follows:

[0106] ,

[0107] ,

[0108] In the formula, h is the elevation difference between the camera and point O on the tower.

[0109] (3) Control the camera to capture an image at the PT value of the geographic spatial coordinate point B to be solved, and use it as the video image of the tower;

[0110] (4) Determine the horizontal component of the geographic spatial coordinate point B to be solved. Vertical component Then, taking the video coordinate point B to be solved as the center, and using twice the distance R from the tower to the video coordinate point B to be solved as the side length, a square image is extracted as the satellite remote sensing image.

[0111] The improved SURF matching algorithm module inputs video images of the iron tower and satellite remote sensing images to perform image matching and transformation matrix solving, and outputs the homography matrix H.

[0112] Reverse geographic coordinates: Extracting pixels of target point B from the video image of the iron tower. The geographic coordinates of point B are calculated using the homography matrix H−1. .

[0113] Another embodiment also provides a high-precision spatial positioning system based on automatic multi-source image matching, such as Figure 5 As shown, the various modules of the system are connected via a data bus to achieve data interaction, specifically including:

[0114] The satellite remote sensing image acquisition module supports the integration of 0.5-meter resolution multispectral images and digital elevation models for acquiring satellite remote sensing images.

[0115] The tower camera module includes a high-definition camera with adjustable horizontal deflection and pitch angles and focal length transformation based on known three-dimensional spatial coordinates, and a tower site with known three-dimensional spatial coordinates, used to acquire video images of the tower.

[0116] The benchmark database construction module is used to select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved; calculate the initial horizontal deflection angle and the initial pitch angle of the point to be solved according to the projection transformation relationship, control the tower camera to capture images at the initial horizontal deflection angle and the initial pitch angle to obtain image 1, calculate the initial coordinates of the point to be solved according to the projection transformation relationship, take the initial coordinates of the point to be solved as the center point, and take twice the distance from the center point to the tower as the side length to crop a square image on the satellite remote sensing image to obtain image 2.

[0117] An improved SURF algorithm feature matching module includes feature point extraction, HSV-fused feature description, similarity measurement and mismatch removal, and transformation matrix solving. This module is used to input the satellite remote sensing image and the tower video image into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. The improved SURF algorithm feature matching module first uses the SURF algorithm to extract feature points from image 1 and image 2. Centered on the feature points, the main direction of the feature points is determined based on the Haar template and the rotational accumulation response of the fan-shaped window. By fusing HSV color features, descriptors for the feature points are generated. Then, a high-threshold K-nearest neighbor feature matching algorithm is used to match the feature points of image 1 and image 2, filtering out initial matching point pairs. RANSAC filtering is then used to remove mismatches, resulting in final matching point pairs. Finally, a coefficient matrix is ​​constructed based on the final matching point pairs, and the coefficient matrix is ​​decomposed using singular value decomposition to obtain the transformation matrix between the satellite remote sensing image and the tower video image.

[0118] The tower video image coordinate and geospatial coordinate conversion module is used to convert the coordinates of the tower video image and the coordinates of the satellite remote sensing image to each other through the transformation matrix.

[0119] The results output module supports the output of satellite remote sensing image coordinates and the horizontal deflection angle and elevation angle of the tower video image.

Claims

1. A high-precision spatial positioning method based on automatic matching of multi-source images, characterized in that, The method includes: The tower video images and satellite remote sensing images are acquired from the satellite remote sensing image acquisition module and the tower camera module. Select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved. The initial horizontal deflection angle and initial pitch angle of the point to be solved are calculated according to the projection transformation relationship. The tower camera is controlled to capture images under the initial horizontal deflection angle and initial pitch angle to obtain image 1. At the same time, the initial coordinates of the point to be solved are calculated according to the projection transformation relationship. The initial coordinates of the point to be solved are used as the center point, and a square image is cropped on the satellite remote sensing image with twice the distance from the center point to the tower as the side length to obtain image 2. Image 1 and Image 2 are input into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. The accurate conversion between the coordinates of the satellite remote sensing image and the coordinates of the tower video image is achieved based on the transformation matrix. The improved SURF algorithm feature matching module first extracts feature points from Image 1 and Image 2. Centered on the feature points, the main direction of the feature points is determined based on the Haar template and the rotational cumulative response of the fan-shaped window. By fusing HSV color features, a descriptor for the feature points is generated. Then, the feature points of Image 1 and Image 2 are matched using a K-nearest neighbor feature matching algorithm with a large threshold. Initial matching point pairs are selected, and RANSAC filtering is used to remove mismatches, resulting in the final matching point pairs. Finally, a coefficient matrix is ​​constructed based on the final matching point pairs, and the transformation matrix between the satellite remote sensing image and the tower video image is obtained by singular value decomposition of the coefficient matrix.

2. The high-precision spatial positioning method according to claim 1, characterized in that, The feature points extracted from Image 1 and Image 2 are used as the center. The main direction of the feature points is determined based on the Haar template and the rotational accumulation response of the fan-shaped window. By fusing HSV color features, a descriptor for each feature point is generated, including: First, feature points are extracted, and images 1 and 2 are converted into integral images. Then, the fast Hessian matrix determinant approximation value of each pixel is calculated. Finally, multi-scale feature detection is used to detect maxima and optimize sub-pixel interpolation. Then, the descriptor is calculated. A circular region with a radius of 6s is taken, centered on the feature point, where 's' is the feature point scale. The gradient is calculated using a 4s-scale Haar template. The rotational cumulative response of the fan-shaped window with the angular angle is used to calculate the principal direction of the feature point, and then the direction centered on the feature point is calculated. The area is divided into Sub-blocks, within the statistical area ,in, The horizontal and vertical gradients are used to form feature vectors, which are then fused with HSV color features to generate descriptors for the feature points.

3. The high-precision spatial positioning method according to claim 1, characterized in that, The initial horizontal deflection angle and initial pitch angle of the point to be solved are calculated based on the projection transformation relationship, including: When the point to be solved is located in the video image of the iron tower, the horizontal rotation angle and the pitch angle at the time of the capture of the point to be solved are taken as the initial horizontal rotation angle and the initial pitch angle. When the point to be solved is located in a satellite remote sensing image, the coordinates of the point to be solved on the satellite remote sensing image are first obtained. Then, through the projection transformation relationship, the coordinates of the point to be solved are calculated and transformed to obtain the initial horizontal rotation angle and initial pitch angle of the point to be solved on the tower video image.

4. The high-precision spatial positioning method according to claim 1, characterized in that, The initial coordinates of the point to be solved are calculated based on the projection transformation relationship, including: When the point to be solved is located in the video image of the tower, the horizontal rotation angle and pitch angle of the point to be solved are first obtained. Then, the coordinates of the point to be solved are calculated and transformed through the projection transformation relationship to obtain the initial coordinates of the point to be solved on the satellite remote sensing image. When the point to be solved is located in a satellite remote sensing image, the coordinates of the point to be solved on the satellite remote sensing image are taken as the initial coordinates.

5. The high-precision spatial positioning method according to claim 1, characterized in that, The precise conversion between satellite remote sensing image coordinates and tower video image coordinates is achieved based on the transformation matrix, specifically as follows: When it is necessary to convert satellite remote sensing image coordinates to tower video image coordinates, the pixels of the target point in the satellite remote sensing image are extracted. The geographic coordinates of the target point are mapped to video pixels through the transformation matrix H. The center of the moving camera to Read the actual horizontal yaw angle P value and pitch angle T value; When it is necessary to convert the coordinates of a tower video image to satellite remote sensing image coordinates, and extract the pixels of the target point in the tower video image... Through homography matrix Calculate the geographic coordinates of the target point .

6. The high-precision spatial positioning method according to any one of claims 1-5, characterized in that, The satellite remote sensing image acquisition module supports the access of 0.5-meter resolution multispectral images and digital elevation models; the tower camera module includes a high-definition camera with known three-dimensional spatial coordinates, adjustable horizontal deflection angle and pitch angle, and capable of focal length transformation, and a tower site with known three-dimensional spatial coordinates.

7. A high-precision spatial positioning system based on automatic multi-source image matching, used to implement the method described in any one of claims 1-6, wherein the modules of the system are connected via a data bus to achieve data interaction, characterized in that... The system includes: The satellite remote sensing image acquisition module supports the integration of 0.5-meter resolution multispectral images and digital elevation models for acquiring satellite remote sensing images. The tower camera module includes a high-definition camera with adjustable horizontal deflection and pitch angles and focal length transformation based on known three-dimensional spatial coordinates, and a tower site with known three-dimensional spatial coordinates, used to acquire video images of the tower. The benchmark database construction module is used to select corresponding feature points from the tower video image and the satellite remote sensing image, construct the projection transformation relationship between the satellite remote sensing image and the tower video image based on the corresponding feature points, and arbitrarily select a point on the tower video image or the satellite remote sensing image as the point to be solved; calculate the initial horizontal deflection angle and the initial pitch angle of the point to be solved according to the projection transformation relationship, control the tower camera to capture images at the initial horizontal deflection angle and the initial pitch angle to obtain image 1, calculate the initial coordinates of the point to be solved according to the projection transformation relationship, take the initial coordinates of the point to be solved as the center point, and take twice the distance from the center point to the tower as the side length to crop a square image on the satellite remote sensing image to obtain image 2; An improved SURF matching module, including feature point extraction, HSV-fused feature description, similarity measurement and mismatch removal, and transformation matrix solving, is used to input the satellite remote sensing image and the tower video image into the improved SURF algorithm feature matching module to obtain the transformation matrix between the satellite remote sensing image and the tower video image. The improved SURF algorithm feature matching module first extracts feature points from image 1 and image 2. Centered on the feature points, the main direction of the feature points is determined according to the Haar template and the fan-shaped window rotation accumulation response. By fusing HSV color features, a descriptor of the feature points is generated. Then, the k-nearest neighbor feature matching algorithm with a large threshold is used to match the feature points of image 1 and image 2, and the initial matching point pairs are selected. RANSAC filtering is used to remove mismatches to obtain the final matching point pairs. Finally, a coefficient matrix is ​​constructed based on the final matching point pairs, and the transformation matrix between the satellite remote sensing image and the tower video image is obtained by singular value decomposition of the coefficient matrix. A module for converting tower video image coordinates to geospatial coordinates is used to convert the coordinates of the tower video image to the coordinates of the satellite remote sensing image through the transformation matrix. The results output module supports the output of satellite remote sensing image coordinates and the horizontal deflection angle and elevation angle of the tower video image.

Images