Image control point automatic pricking method, device and equipment and storage medium

Abstract

The invention discloses an image control point automatic pricking method, device and equipment and a storage medium, and relates to the technical field of aerial photography image processing, and the method comprises the steps: inputting an image control point coordinate, a plurality of alternative pricking point images and image information; selecting a point position index image based on the image information; selecting a target thorn point image based on the image control point coordinates and the POS data of the alternative thorn point images, and cutting the target thorn point image to obtain a target image; performing feature extraction on the point location index image and the target image and calculating a homography matrix; calling the homography matrix to carry out azimuth correction on the target image to obtain a second target image; cutting out a template image from the point position index image, and carrying out template matching on the second target image by adopting the template image to obtain an integer pixel registration coordinate; performing quadric surface fitting on the integer pixel registration coordinate to obtain a target thorn point coordinate; and performing point pricking on the target point pricking image based on the target point pricking coordinates. According to the invention, sub-pixel-level automatic point pricking of aerial photography images can be realized, and the point pricking efficiency and precision are greatly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of aerial image processing, and in particular, to a method, device, equipment and storage medium for automatically piercing image control points. Background Art

[0002] The remote sensing images obtained through aerial photography tasks are the main data sources for surveying and mapping geographic information products such as digital elevation models (digital surface models), digital orthophotos, and real-scene three-dimensional models. At present, the dynamic network RTK (Real-Time Kinematic) measurement method based on the CORS (Continuously Operating Reference Stations) network can achieve control-free measurement in small-scale aerial photography tasks, but large-scale aerial photography tasks still require the use of image control points for geometric accuracy control.

[0003] The process of obtaining image control points mainly includes interior point selection, field measurement, and manual piercing. However, at present when aerial photography tasks are becoming increasingly heavy, the efficiency of manual piercing is low and cannot meet the actual application requirements. In response to the above problems, some studies have proposed automatic piercing methods at present, but they can only meet the geometric control requirements of aerial images with a resolution of meters and cannot meet the geometric control accuracy requirements of high-resolution aerial images with a centimeter-level resolution in the context of urban real-scene three-dimensional construction. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a method, device, equipment and storage medium for automatically piercing image control points, which can achieve sub-pixel-level automatic piercing of aerial images, greatly improving the piercing efficiency and accuracy.

[0005] To achieve the above purpose, the embodiments of the present invention provide a method for automatically piercing image control points, including:

[0006] Inputting the coordinates of the image control point in the geographic coordinate system, a number of alternative piercing point images, and image information; wherein, the image information includes the aerial photography range and resolution;

[0007] Based on the image information, extracting a point index image from the image control point database;

[0008] Based on the coordinates of the image control point in the geographic coordinate system and the POS data of the alternative piercing point images, selecting a target piercing point image from the number of alternative piercing point images, and cropping the target piercing point image to obtain a target image;

[0009] Performing feature extraction on the point index image and the target image to obtain a first initial feature point and a second initial feature point respectively;

[0010] Based on the first initial feature point and the second initial feature point, establish a homography matrix between the point index image and the target image;

[0011] Call the homography matrix to perform azimuth correction on the target image to obtain a second target image;

[0012] Crop a template image containing image control points from the point index image, and use the template image to perform template matching on the second target image to obtain integer pixel registration coordinates;

[0013] Perform quadratic surface fitting on the integer pixel registration coordinates to obtain target piercing point coordinates;

[0014] Pierce the target piercing point image based on the target piercing point coordinates and output a piercing point file.

[0015] As an improvement to the above solution, the target piercing point image and the target image are obtained through the following methods:

[0016] Input the POS data of the alternative piercing point image; the POS data includes the coordinates of the camera center in the geodetic coordinate system, the rotation matrix, and the sensor focal length;

[0017] Perform a difference operation on the coordinates of the image control point in the geodetic coordinate system and the coordinates of the camera center in the geodetic coordinate system to obtain the coordinates of the image control point in the auxiliary image space coordinate system;

[0018] Based on the coordinates of the image control point in the auxiliary image space coordinate system, the rotation matrix, and the sensor focal length, calculate the image coordinates of the image control point;

[0019] When the image coordinates are within a predetermined range on the alternative piercing point image, use the alternative piercing point image as the target piercing point image;

[0020] Centered on the image coordinates, crop the target image from the target piercing point image based on a preset first cropping size.

[0021] As an improvement to the above solution, both the first initial feature point and the second initial feature point are calculated through the following methods:

[0022] Downsample the image to be processed to obtain a second image to be processed; where the image to be processed is the point index image or the target image;

[0023] Calculate the Hessian matrix and the Hessian matrix discriminant of each pixel point on the second image to be processed;

[0024] Convert the second image to be processed into a transformed image based on the Hessian matrix discriminant;

[0025] Construct an image pyramid based on the transformed image;

[0026] Based on the image pyramid, obtain the neighboring pixel points within the three-dimensional neighborhood of each pixel point. When the gray value of the pixel point is greater than the gray values of all the neighboring pixel points or less than the gray values of all the neighboring pixel points, take the pixel point as an initial feature point.

[0027] As an improvement to the above solution, it further includes calculating the feature vector of the initial feature point in the following manner:

[0028] Taking the initial feature point as the center, calculate the first wavelet features in different directions, and take the direction with the largest first wavelet feature as the main direction of the initial feature point;

[0029] Taking the initial feature point as the center, take a region block containing several sub-regions along the main direction, and calculate the second wavelet features of each sub-region, where the second wavelet features include the sum in the horizontal direction, the sum in the vertical direction, the sum of the absolute values in the horizontal direction, and the sum of the absolute values in the vertical direction;

[0030] Summarize the second wavelet features of each sub-region to obtain the feature vectors of each initial feature point.

[0031] As an improvement to the above solution, the establishing of the homography matrix between the point index image and the target image based on the first initial feature point and the second initial feature point includes:

[0032] Calculate the Euclidean distance between the first initial feature point and the second initial feature point, and take the point pairs with the Euclidean distance less than the preset distance threshold as initial feature point pairs;

[0033] Randomly select a first preset number of the initial feature point pairs to obtain second initial feature point pairs;

[0034] Based on the second initial feature point pairs, use the least squares estimation algorithm to calculate the intermediate homography matrix;

[0035] Take the initial feature point pairs with the deviation from the intermediate homography matrix less than the preset deviation threshold as inlier points;

[0036] When the number of inlier points is greater than the preset inlier point number threshold, take the inlier points as the new second initial feature point pairs, and return to the step of calculating the intermediate homography matrix; when the number of inlier points is not greater than the inlier point number threshold, return to the step of randomly selecting the second initial feature point pairs;

[0037] When the iteration end condition is met, take the intermediate homography matrix with the largest number of inlier points as the homography matrix.

[0038] As an improvement of the above solution, the second target image and the template image are calculated in the following manner:

[0039] Taking the image control point as the center, an initial template image is cropped from the point index image based on a preset second cropping size;

[0040] Based on the homography matrix, the initial template image is expanded to obtain the template image;

[0041] The template image is mapped to the target image by using the homography matrix to obtain a coordinate matrix;

[0042] Bilinear interpolation is performed on the target image by using the coordinate matrix to obtain the second target image; wherein, the size of the template image is smaller than the size of the second target image.

[0043] As an improvement of the above solution, using the template image to perform template matching on the second target image includes:

[0044] Sliding the template image on the second target image to calculate the normalized mean square difference between different positions of the second target image and the template image;

[0045] Taking the coordinates of the pixel point with the smallest normalized mean square difference as the integer pixel registration coordinates;

[0046] Wherein, the normalized mean square difference is calculated in the following manner:

[0047] Based on the size and sliding position of the template image, a matching image is extracted from the second target image;

[0048] Performing gray normalization on the pixel points on the matching image and the template image to obtain a target matching image and a target template image;

[0049] Calculating the gray mean square difference between the target matching image and the target template image to obtain the normalized mean square difference.

[0050] To achieve the above object, an embodiment of the present invention further provides an automatic image control point piercing device, including:

[0051] An alternative piercing image input module, configured to input the coordinates of the image control point in the geographic coordinate system, a plurality of alternative piercing images, and image information; wherein, the image information includes the aerial photography range and resolution;

[0052] A point index image extraction module, configured to extract a point index image from the image control point database based on the image information;

[0053] A target puncture point image screening module, configured to select a target puncture point image from the plurality of alternative puncture point images based on the coordinates of the image control point in the geographical coordinate system and the POS data of the alternative puncture point images, crop the target puncture point image to obtain a target image;

[0054] A feature extraction module, configured to extract features from the point index image and the target image respectively to obtain a first initial feature point and a second initial feature point;

[0055] A homography matrix calculation module, configured to establish a homography matrix between the point index image and the target image based on the first initial feature point and the second initial feature point;

[0056] An orientation correction module, configured to call the homography matrix to perform orientation correction on the target image to obtain a second target image;

[0057] A fine matching module, configured to crop a template image containing the image control point from the point index image, and perform template matching on the second target image using the template image to obtain integer pixel registration coordinates;

[0058] A sub-pixel level matching module, configured to perform quadratic surface fitting on the integer pixel registration coordinates to obtain target puncture point coordinates;

[0059] A puncture point module, configured to perform puncture on the target puncture point image based on the target puncture point coordinates and output a puncture point file.

[0060] To achieve the above object, an embodiment of the present invention further provides an automatic image control point puncture device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the automatic image control point puncture method described in any of the above embodiments is implemented.

[0061] To achieve the above object, an embodiment of the present invention further provides a computer-readable storage medium, the computer-readable storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the automatic image control point puncture method described in any of the above embodiments.

[0062] Compared with the prior art, the method, apparatus, device and storage medium for automatic image control point piercing provided by the embodiments of the present invention input the geographic coordinate system coordinates of the image control points, a number of alternative piercing images and image information; wherein, the image information includes the aerial photography range and resolution; based on the image information, the point index image is extracted from the image control point database; based on the geographic coordinate system coordinates of the image control points and the POS data of the alternative piercing images, the target piercing image is selected from the number of alternative piercing images, and the target piercing image is cropped to obtain the target image; feature extraction is performed on the point index image and the target image to obtain the first initial feature points and the second initial feature points respectively; based on the first initial feature points and the second initial feature points, a homography matrix is established between the point index image and the target image; the homography matrix is called to perform azimuth correction on the target image to obtain the second target image; the template image containing the image control points is cropped from the point index image, and the template image is used to perform template matching on the second target image to obtain the whole pixel registration coordinates; the whole pixel registration coordinates are subjected to quadratic surface fitting to obtain the target piercing coordinates; piercing is performed on the target piercing image based on the target piercing coordinates, and the piercing file is output. Compared with the prior art, the embodiments of the present invention can realize sub-pixel level automatic piercing of aerial images by adopting a three-level matching strategy of "coarse matching + fine matching + sub-pixel level matching", greatly improving the piercing efficiency and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 is a flowchart of a method for automatic image control point piercing provided by an embodiment of the present invention;

[0064] Figure 2 is a flowchart of a method for automatic image control point piercing provided by another embodiment of the present invention;

[0065] Figure 3 is a schematic diagram of a template image provided by an embodiment of the present invention;

[0066] Figure 4 is a schematic diagram of a second target image provided by an embodiment of the present invention;

[0067] Figure 5 is a schematic structural diagram of an apparatus for automatic image control point piercing provided by an embodiment of the present invention;

[0068] Figure 6 is a schematic structural diagram of a device for automatic image control point piercing provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0070] In the description of the present application, it should be understood that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation to the present application.

[0071] The terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0072] In the description of the present application, it should be noted that unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances.

[0073] It is worth noting that with the rapid development of information-based surveying and mapping, in view of the disadvantages of long acquisition cycle, low application efficiency, and low reusability of conventional phase control points, surveying and mapping research institutions and production units also pay more attention to the information-based management of image control points, and have proposed and implemented the project of constructing an image database for phase control points.

[0074] The First Institute of Photogrammetry and Remote Sensing of the National Administration of Surveying, Mapping and Geoinformation has achieved a multi-level and multi-source image database system for photo control points. It has proposed a top-down unified numbering strategy and attribute information coding for different data sources for the photo control point database, realizing the graphical management of photo control points. The system provides various query methods for photo control points, as well as the extraction and output of photo control points, and has data interfaces with remote sensing image processing software such as ERDAS and PCI. It has also designed a process for quickly extracting photo control points that meet the geometric correction of images based on the photo control point image database and the images to be corrected, and has studied the determination of database retrieval conditions such as image range, resolution, and accuracy. Relevant research shows that retrieving and extracting the photo control point information of the photo control point image database using the determined image range, accuracy, and resolution conditions can greatly narrow the search range and improve the effectiveness of the extracted photo control points. However, this research has not truly realized the automatic extraction of photo control points for multi-source remote sensing images.

[0075] The Chinese Academy of Surveying and Mapping has put forward the idea and technical design of establishing a national multi-level and multi-resolution graphic image photo control point library for different types of remote sensing mapping data (remote sensing satellite images, aerial images, and digital line graphs) and different resolutions (1-meter level, 10-meter level, and 100-meter level), including the data source selection criteria for the national multi-level and multi-resolution graphic image photo control point database of data sources, the technical indicators for photo control point collection, the database establishment and management mode, and the database sharing mechanism, etc.

[0076] The Third Institute of Oceanography of the State Oceanic Administration has also developed a sub-system of "Fujian Island and Coastal Zone Photo Control Point Image Database" during the development of the "Fujian Island and Coastal Zone Remote Sensing Image Library Management System". A large amount of photo control point image data has been obtained through two methods: image collection and field investigation and measurement. On this basis, the effective management and reasonable and efficient utilization of photo control point data have been realized through the establishment of a photo control point image database. They have carried out research on the geometric precise correction of the original images based on the photo control point image database. And using the resources of the photo control point image library, they have focused on studying the matching technology between the photo control point images and the original images, and verified 2,875 photo control points in the photo control point library. The experiments show that the matching has a relatively fast speed, high accuracy, and strong stability.

[0077] The construction of the above photo control point databases can meet the geometric control requirements of satellite and aerial images with a resolution of meters, but it cannot meet the geometric control accuracy requirements of high-resolution aerial images with a centimeter-level resolution under the background of urban real-scene three-dimensional construction. In such scenarios, manual point layout and puncturing are still generally used for geometric control, which has a large workload, high cost, and it is difficult to reuse the photo control points. Moreover, manual puncturing cannot be directly connected to the computer automatic aerial triangulation of remote sensing images and cannot adapt to the current development trend.

[0078] To solve the above problems, embodiments of the present invention provide a method, apparatus, device, and storage medium for automatically piercing image control points. Refer to Figure 1 which is a flowchart of the method for automatically piercing image control points provided by an embodiment of the present invention, including steps S1 to S9:

[0079] S1. Input the coordinates of the image control point in the geographic coordinate system, several alternative piercing images, and image information; wherein, the image information includes the aerial photography range and resolution;

[0080] S2. Based on the image information, extract the point index image from the image control point database;

[0081] S3. Based on the coordinates of the image control point in the geographic coordinate system and the POS data of the alternative piercing images, select the target piercing image from the several alternative piercing images, and crop the target piercing image to obtain the target image;

[0082] S4. Extract features from the point index image and the target image respectively to obtain the first initial feature points and the second initial feature points;

[0083] S5. Based on the first initial feature points and the second initial feature points, establish a homography matrix between the point index image and the target image;

[0084] S6. Call the homography matrix to perform azimuth correction on the target image to obtain the second target image;

[0085] S7. Crop the template image containing the image control point from the point index image, and use the template image to perform template matching on the second target image to obtain the integer-pixel registration coordinates;

[0086] S8. Perform quadratic surface fitting on the integer-pixel registration coordinates to obtain the target piercing coordinates;

[0087] S9. Pierce the target piercing image based on the target piercing coordinates and output the piercing file.

[0088] In some embodiments of the present invention, a point index image of image control points is pre-produced using low-altitude remote sensing images with centimeter-level resolution, and an image control point database is established by constructing attribute fields using information such as the coordinates of image control points and the resolution of the point index image. Further, in step S2, the image control points and the corresponding point index images in the image control point database are filtered out according to the aerial photography range and resolution. In step S3, alternative piercing point images that meet the requirements of aerotriangulation piercing points are filtered out using the geographic coordinate system coordinates of the image control points and the POS data of the alternative piercing point images, and used as target piercing point images. An image block of a certain size is cropped with the initial position of the image control point in the target piercing point image as the center to obtain the target image. Further, the embodiment of the present invention adopts a three-level matching strategy of "coarse matching + fine matching + sub-pixel level matching" to match the target piercing point coordinates. Specifically, in step S4, a feature extraction algorithm is used for fast feature extraction, and in step S5, a homography matrix is established between the target image and the point index image. Then, in step S6, the target image is transformed through the homography matrix to be consistent with the pose of the point index image. Usually, there is still an error within 10 pixels for the image control points after transformation. Therefore, in step S7, the error is reduced to 1 pixel through template matching. Further, in step S8, the matching error is reduced to within 1 pixel using the surface fitting method to obtain the image coordinates of the image control points in the target piercing point image. Finally, in step S9, the target piercing point image is pierced and an image control point piercing file supported by mainstream modeling software (such as ContextCapture and Dazhi Intelligence, etc.) is output, so that it can be directly imported into the software for aerotriangulation calculation in the subsequent process.

[0089] Exemplarily, in some embodiments, the root mean square error between the matching results can also be calculated to evaluate the accuracy of the matching (piercing) results.

[0090] See Figure 2 , which is a flowchart of the automatic piercing method for image control points provided by another embodiment of the present invention. It can be seen from Figure 2 that first, it is necessary to retrieve the image control point database based on the alternative piercing point images. If no image control points are retrieved, that is, there are no image control points within the aerial photography range of the alternative piercing point images in the image control point database, it is necessary to select the original aerial photography images for manual piercing, use the obtained images as point index images, and synchronously add them to the image control point database. If image control points can be retrieved, the existing point index images are directly called from the image control point database; then, the target piercing point images are filtered out based on the POS data of the alternative piercing point images and the target images are cropped; then, the target piercing point coordinates are identified through SURF matching, template matching, and surface fitting matching for automatic piercing; finally, accuracy evaluation is performed and the final piercing file is output.

[0091] Compared with the prior art, in the embodiment of the present invention, by adopting a three-level matching strategy of "coarse matching + fine matching + sub-pixel level matching", sub-pixel level automatic piercing points of aerial images can be realized, greatly improving the efficiency and accuracy of piercing points.

[0092] Exemplarily, before step S1, the image control point database is constructed in the following manner: image control points are arranged within the survey area, original aerial images with less distortion are selected (usually the distortion of image control points is the smallest at the center of the image), the positions of the image control points are selected (piercing points) and the original aerial images are cropped. The cropping size is generally 1024*1024 pixels, and the cropping size can be adjusted according to actual needs. After cropping, an image block centered on the image control point will be obtained, which is called a point index image; the attributes of the image control points and the point index images are edited and stored in the database in batches to establish an image control point database.

[0093] Further, for areas where the distribution of image control points is insufficient or the image control points are damaged, it is necessary to supplement the measurement of image control points. Specifically, the geographical range of the flight line is used to query in the image control point database to obtain the image control points within the aerial photography range, and information such as their coordinates and the paths of point index images is obtained. The coverage analysis of the queried image control points is carried out. If the coverage density does not meet the modeling requirements, a high-quality original aerial image is selected for each supplementary image control point for manual piercing to obtain a point index image, and the supplementary image control points and their point index images are imported into the image control point database.

[0094] Further, when supplementing the measurement of image control points, ground feature points such as the corner points of traffic indication lines that can be preserved for a long time and the ground corner points in open and flat areas such as park squares can be selected as the image control points to be measured. The original aerial images of the image control points to be measured are obtained by performing a flight mission and the coordinates of the selected image control points are measured; when piercing points, the original aerial images with a downward-looking lens and the image control points at the center of the image can be preferentially selected for manual piercing.

[0095] As an optional implementation manner, in step S3, the target piercing point image and the target image are obtained in the following manner:

[0096] Input the POS data of the alternative piercing point image; the POS data includes the coordinates of the camera center in the geodetic coordinate system, the rotation matrix, and the sensor focal length;

[0097] Perform a difference operation on the geodetic coordinate system coordinates of the image control point and the geodetic coordinate system coordinates of the camera center to obtain the coordinates of the image control point in the image space auxiliary coordinate system;

[0098] Based on the coordinates of the image control point in the image space auxiliary coordinate system, the rotation matrix, and the sensor focal length, calculate the image coordinates of the image control point;

[0099] When the image coordinates are within a predetermined range on the alternative fiducial point image, the alternative fiducial point image is taken as the target fiducial point image;

[0100] Centered on the image coordinates, the target image is cropped from the target fiducial point image based on a preset first cropping size.

[0101] It can be understood that in the embodiments of the present invention, it is necessary to inversely calculate the image coordinates of each image control point on each alternative fiducial point image according to the POS data of the alternative fiducial point image; wherein, the coordinates of the camera center in the geodetic coordinate system refer to the coordinates of the camera center in the geodetic coordinate system of the alternative fiducial point image.

[0102] Specifically, the relationship between the image plane coordinates of the image control point and the auxiliary image space coordinates is:

[0103]

[0104] wherein, (x, y) represents the image plane coordinates of the image control point; f represents the focal length of the sensor, which can be found in the original POS (Position and Orientation System) data; (x, y, -f) represents the coordinates in the image space coordinate system, and (X, Y, Z) represents the coordinates of the image control point in the auxiliary image space coordinate system; R represents the transformation matrix between the image space coordinate system and the auxiliary image space coordinate system; T represents the transpose operation; a i represents the element in the first row and the i-th column of the transformation matrix; b i represents the element in the second row and the i-th column of the transformation matrix; c i represents the element in the third row and the i-th column of the transformation matrix.

[0105] From Equation (1), it can be known that if the coordinates of the image control point in the auxiliary image space coordinate system, the rotation matrix, and the focal length of the sensor are known, the image coordinates of the image control point can be obtained, as shown in Equation (2):

[0106]

[0107] It can be understood that the coordinates of the auxiliary image space coordinate system are a space rectangular coordinate system with the camera center as the origin. The coordinates of the image control point in the auxiliary image space coordinate system can be calculated by taking the difference between the coordinates of the image control point in the geodetic coordinate system and the coordinates of the camera center in the geodetic coordinate system at the imaging instant. And usually, in the original POS data of aerial photography, the geodetic coordinate system of the camera center is longitude, latitude, and geodetic height. Therefore, the geodetic coordinate system of the image control point also needs to adopt this coordinate, and by calculating the X and Y direction distances between two points through the Haversine formula, the X, Y, and Z in Equation (2) can be obtained.

[0108] Furthermore, the transformation matrix is the rotation matrix between the three coordinate axes of the two coordinate systems, which is determined by the three exterior orientation elements of the principal optical axis of the camera at the imaging instant. The POS data contains the above three angle information, so the transformation matrix can be calculated by solving the three exterior orientation elements.

[0109] Taking the exterior orientation elements ω, κ system as an example, the image space coordinate system is the position after the image space auxiliary coordinate system (equivalent to the starting position of the photographic beam) rotates around the corresponding coordinate axes successively by ω and κ three angles. At this time, the rotation matrix R can be expressed as:

[0110]

[0111] Among them, R represents the rotation matrix; represents the heading angle rotation matrix; R ω represents the pitch angle rotation matrix; R κ represents the roll angle rotation matrix; represents the heading angle; ω represents the pitch angle; κ represents the roll angle; a i represents the element in the i-th column of the first row of the transformation matrix; b i represents the element in the i-th column of the second row of the transformation matrix; c i represents the element in the i-th column of the third row of the transformation matrix.

[0112] Furthermore, according to the image coordinates, the alternative images for image control point piercing are screened. Specifically, according to whether the calculated image coordinates are within the pixel numbers of the length and width of the alternative piercing points, the alternative images that may form images for each image control point are screened. Furthermore, since there are distortions in the alternative piercing point images, the alternative piercing point images suitable for piercing can be screened by restricting the range of the image coordinates, that is, when the image coordinates are within the predetermined range on the alternative piercing point images, the alternative piercing point images are used as the target piercing point images. Exemplarily, the predetermined range can be the middle 2 / 3 area of the length and width of the alternative piercing point images.

[0113] Furthermore, it is necessary to crop the target piercing point images. To improve the matching efficiency, with the image coordinates of the image control points in the target piercing point images as the center, an image block of a certain size is cropped as the target image. Among them, the size of the image block is related to the original POS accuracy of the sensor. For higher POS accuracy, it can be cropped to be the same size as the image control point position index image. For poorer POS accuracy, the cropping size should be increased according to the specific situation.

[0114] As one of the optional implementation manners, in step S4, both the first initial feature point and the second initial feature point are calculated by the following method:

[0115] S40. Downsample the image to be processed to obtain a second image to be processed; wherein, the image to be processed is the point index image or the target image;

[0116] S41. Calculate the Hessian matrix and the Hessian matrix discriminant for each pixel point on the second image to be processed;

[0117] S42. Convert the second image to be processed into a transformed image based on the Hessian matrix discriminant;

[0118] S43. Construct an image pyramid based on the transformed image;

[0119] S44. Based on the image pyramid, obtain the neighboring pixel points within the three-dimensional neighborhood of each pixel point. When the gray value of the pixel point is greater than or less than the gray values of all the neighboring pixel points, use the pixel point as an initial feature point.

[0120] It can be understood that in the embodiment of the present invention, the SURF (Speeded Up Robust Features) algorithm is used for feature extraction, that is, rough matching, with the aim of narrowing the matching range and reducing the point registration accuracy to within two digits, saving computational cost for subsequent fine matching. Therefore, the most important thing in rough matching is fast matching. Thus, the SURF algorithm is used in the embodiment of the present invention, so the matching speed is faster than the SIFT algorithm. Further, before using the SURF algorithm in the embodiment of the present invention, the point index image and the target image are first downsampled, which not only greatly improves the matching efficiency but also avoids the problem of reduced global matching accuracy caused by uneven texture distribution. Exemplarily, it is appropriate to downsample the image to a size of 200 pixels for the adjusted length and width of the image with a lower resolution.

[0121] Further, in step S41, assuming the pixel point position is I(x, y) and the scale of the Hessian matrix is σ, the Hessian matrix of the pixel point at I(x, y) can be expressed as:

[0122]

[0123] where H represents the Hessian matrix; σ is the scale; L xx (x, σ), L xy (x, σ) and L yy (x, σ) represent the second-order partial derivatives of the pixel point at I(x, y) after convolution with the Gaussian filter function.

[0124] Further, in step S41, the Hessian matrix discriminant is as shown in Equation (5):

[0125] det(H) = Lxx L yy -L xy 2 (5)

[0126] Among them, det(H) represents the Hessian matrix discriminant.

[0127] Furthermore, in step S43, without changing the size of the image, directly use a filter to blur the image, and an image pyramid can be obtained. The number of layers of this image pyramid is n, and each layer contains a certain number of blurred images.

[0128] Furthermore, in step S44, obtain 27 pixel points within the three-dimensional neighborhood of each pixel point. When the gray value of this pixel point is the maximum or minimum value among these 27 pixel points, the pixel point is used as an initial feature point. Among them, in the embodiment of the present invention, the initial feature point of the point index image is denoted as the first initial feature point, and the initial feature point of the target image is denoted as the second initial feature point.

[0129] As an optional implementation manner, in step S4, it further includes calculating the feature vector of the initial feature point in the following manner:

[0130] S45. Taking the initial feature point as the center, calculate the first wavelet features in different directions, and use the direction with the maximum first wavelet feature as the main direction of the initial feature point;

[0131] S46. Taking the initial feature point as the center, take a region block including several sub-regions along the main direction, and calculate the second wavelet features of each sub-region. Among them, the second wavelet features include the sum in the horizontal direction, the sum in the vertical direction, the sum of absolute values in the horizontal direction, and the sum of absolute values in the vertical direction;

[0132] S47. Summarize the second wavelet features of each sub-region to obtain the feature vectors of each initial feature point.

[0133] It can be understood that, in order to ensure the rotation invariance of the image, in step S45, it is necessary to record different wavelet features in the vicinity of the initial feature point to determine the main direction.

[0134] Furthermore, in step S46, taking the initial feature point as the origin, rotate the coordinate axes along the main direction, select a region block with a certain size in the main direction, then divide it into 16 sub-regions of the same size, and statistically analyze the wavelet features in the vertical and horizontal directions of the pixel points in each sub-region to obtain 4 eigenvalue of each sub-region, and thus a feature vector containing 64 eigenvalues can be obtained.

[0135] As one of the alternative embodiments, in step S5, establishing the homography matrix between the point index image and the target image based on the first initial feature points and the second initial feature points includes:

[0136] S50. Calculate the Euclidean distance between the first initial feature points and the second initial feature points, and use the point pairs with the Euclidean distance less than the preset distance threshold as the initial feature point pairs;

[0137] S51. Randomly select the first preset number of the initial feature point pairs to obtain the second initial feature point pairs;

[0138] S52. Based on the second initial feature point pairs, use the least variance estimation algorithm to calculate the intermediate homography matrix;

[0139] S53. Use the initial feature point pairs with the deviation from the intermediate homography matrix less than the preset deviation threshold as the inlier points;

[0140] S54. When the number of the inlier points is greater than the preset inlier point number threshold, use the inlier points as the new second initial feature point pairs and return to the step of calculating the intermediate homography matrix; when the number of the inlier points is not greater than the inlier point number threshold, return to the step of randomly selecting the second initial feature point pairs;

[0141] S55. When the iteration end condition is satisfied, use the intermediate homography matrix with the largest number of inlier points as the homography matrix.

[0142] It can be understood that when the Euclidean distance between a certain first initial feature point and a certain second initial feature point is less than the preset distance threshold, the point pair composed of these two pixel points is recorded as the initial feature point pair.

[0143] Further, in order to improve the reliability of the matching points, the embodiment of the present invention also uses the RANSAC (Random Sample Consensus) algorithm to eliminate the error point pairs and calculate the homography matrix. Specifically, a sample subset (i.e., a set composed of the second initial feature point pairs) is randomly selected from the initial feature point pairs. Exemplarily, the sample subset may include 8 point pairs; then, the least squares estimation algorithm is used to calculate the intermediate homography matrix for the sample subset, and then the deviation between all the initial feature point pairs and the intermediate homography matrix is calculated. When the deviation is less than the deviation threshold, the corresponding initial feature point pair is used as the inlier points of the homography matrix, otherwise it is an outlier point; when the number of inlier points is greater than the preset inlier point threshold, the intermediate homography matrix is recalculated based on the inlier points, that is, the inlier points are used as the new second initial feature point pairs and a new round of iteration is performed. When the number of inlier points is not greater than the preset inlier point threshold, the step of randomly extracting the second feature point pairs is returned to perform a new round of iteration; when the iteration end condition is reached, the intermediate homography matrix with the largest number of inlier points is used as the homography matrix. Further, if the homography matrix cannot be calculated after the sampling times reach the preset sampling threshold, it is considered that there is no correct matching point pair between the point index image and the target image, that is, the point piercing cannot be performed.

[0144] As an optional implementation manner, in steps S6 and S7, the second target image and the template image are calculated in the following manner:

[0145] Centered on the image control point, an initial template image is cropped from the point index image based on a preset second cropping size;

[0146] The initial template image is expanded based on the homography matrix to obtain the template image;

[0147] The template image is mapped to the target image by using the homography matrix to obtain a coordinate matrix;

[0148] Bilinear interpolation is performed on the target image by using the coordinate matrix to obtain the second target image; wherein, the size of the template image is smaller than the size of the second target image.

[0149] It should be noted that since there is usually a certain degree of rotation between the point index image and the target image block, and template matching essentially uses a convolution kernel of a certain size to perform convolution on the image, which requires the orientation between the matching images to be consistent. Therefore, before template matching, the embodiment of the present invention also corrects the target image to be consistent with the orientation of the point index image through transformation and resampling to obtain the second target image.

[0150] See Figure 3, which is a schematic diagram of the template image provided by an embodiment of the present invention. Specifically, centering on the image control point, an image block of a certain size is cropped from the point index image as the initial template image. Among them, the size of the initial template image can be set to 30×30 pixels. Further, the initial template image is appropriately expanded by a certain size according to the homography matrix to obtain the template image, where the expansion size can be 100 pixels.

[0151] See Figure 4 , which is a schematic diagram of the second target image provided by an embodiment of the present invention. Specifically, the template image is mapped to the target image by the homography matrix to obtain the coordinate matrix corresponding to the target image, and bilinear interpolation (resampling) is performed on the target image through the coordinate matrix to obtain the second target image.

[0152] As one of the optional implementation manners, in step S7, using the template image to perform template matching on the second target image includes:

[0153] Sliding the template image on the second target image to calculate the normalized mean square difference between different positions of the second target image and the template image;

[0154] Taking the coordinates of the pixel point with the smallest normalized mean square difference as the integer pixel registration coordinates;

[0155] Among them, the normalized mean square difference is calculated by the following method:

[0156] Based on the size and sliding position of the template image, a matching image is extracted from the second target image;

[0157] Normalizing the pixel points on the matching image and the template image to obtain the target matching image and the target template image;

[0158] Calculating the mean square difference of the gray levels of the target matching image and the target template image to obtain the normalized mean square difference.

[0159] It can be understood that template matching needs to compare the gray level values of each pixel of two images, and the size of the template image is smaller than that of the second target image. Therefore, a sliding window with the same size as the template image can be set and moved on the target image, and then the normalized mean square difference between the second target image and the target template at different positions is calculated, and the integer pixel registration coordinates are obtained. The embodiment of the present invention can improve the matching accuracy to 1 pixel by using template matching.

[0160] In some embodiments of the present invention, the normalized mean square difference is used to evaluate the similarity, that is, the pixel point with the smallest normalized mean square difference has the highest similarity.

[0161] Specifically, in order to weaken the difference in gray values between the template image and the second target image, the embodiment of the present invention normalizes the template image and the second target image using Equation (6):

[0162]

[0163] where g represents the normalized gray value; g0 represents the gray value before normalization; g min represents the minimum gray value in the two images before normalization; g max represents the maximum gray value in the two images before normalization.

[0164] Further, the embodiment of the present invention calculates the squared difference of gray values between the template image and the second target image using Equation (7):

[0165] C = ∑[g l - g r 2 (7)

[0166] where C represents the squared difference of gray values (normalized squared difference); g l represents the gray value of the corresponding pixel point of the target template image; g r represents the gray value of the corresponding pixel point of the target matching image.

[0167] Exemplarily, in some embodiments, the correlation coefficient or the squared difference may also be used to replace the normalized squared difference to evaluate the similarity, which is not limited herein.

[0168] Further, since the accuracy of template matching can usually only reach 1 pixel, therefore, in order to improve the matching accuracy to the sub-pixel level, the embodiment of the present invention performs quadratic surface fitting on the integer-pixel registration coordinates in step S8 to achieve sub-pixel level matching.

[0169] Specifically, when performing quadratic surface fitting, the correlation coefficients of the integer-pixel registration coordinates and their neighboring pixel points are used for quadratic surface fitting, and then the sub-pixel offset value is calculated using the characteristic that the partial derivative at the extreme point of the surface is zero. Then, the sub-pixel level registration coordinates, that is, the target fiducial point coordinates, are calculated by combining the sub-pixel offset value and the integer-pixel registration coordinates. By adopting quadratic fitting, the embodiment of the present invention can effectively balance the calculation accuracy and the calculation efficiency.

[0170] Specifically, the normalized squared differences of the nine pixel points in the 3×3 neighborhood of the integer-pixel registration point coordinates (x p , y p ) can be taken for quadratic surface fitting, and the fitting function is as shown in Equation (8):

[0171] C(x, y) = α0 + α1x + α2y + α3x 2 + α4xy + α5y​2 (8)

[0172] Among them, (x, y) represents the coordinates of a pixel point; C(x, y) represents the normalized squared difference of the pixel point with coordinates (x, y); α0 to α5 represent the coefficients of the quadratic fitting surface.

[0173] Furthermore, it is assumed that there is 1 best-fitting point within a certain range around the target puncture point, and the similarity between the two matching images at this point is the highest, that is, the normalized squared difference is the smallest.

[0174] Specifically, substituting the coordinates of nine pixel points and their normalized squared differences into Equation (8) can obtain a system of equations consisting of nine equations. Solving the system of equations, for example, the least squares method can be used to solve the system of equations to obtain α0 to α5 in Equation (8).

[0175] Furthermore, the quadratic surface obtained by fitting satisfies Equation (9) at the extreme point:

[0176]

[0177] Among them, (x, y) represents the coordinates of a pixel point; C(x, y) represents the normalized squared difference of the pixel point with coordinates (x, y); α1 to α5 represent the coefficients of the quadratic fitting surface.

[0178] Furthermore, solving Equation (9) gives the sub-pixel-level target puncture point coordinates as shown in Equation (10):

[0179]

[0180] Among them, (x m , y m ) represents the target puncture point coordinates; α1 to α5 represent the coefficients of the quadratic fitting surface.

[0181] Furthermore, in some embodiments, the matching accuracy is evaluated after calculating the target puncture point coordinates.

[0182] Specifically, with the image control point as the center, Equation (11) is used to calculate the image correlation coefficient within the neighborhood of the target puncture point.

[0183]

[0184] Among them, r represents the correlation coefficient; n represents the number of pixel points in the neighborhood of the image control point used to calculate the correlation coefficient; g d and g m respectively represent the gray values at the corresponding positions of the point index image and the target image.

[0185] It can be understood that the value of the correlation coefficient ranges from -1 to 1. Then, a correlation coefficient threshold (such as 0.8) can be set. When the absolute value of the correlation coefficient is lower than the correlation coefficient threshold, the matching result is determined as a false match; otherwise, it is a correct match. Further, the frequency of false matches for each image control point can be counted and updated as attribute data to the image control point database. When the frequency of false matches for a certain image control point exceeds the preset fault tolerance threshold, or when all image control points within a certain flight line fail to match, the administrator of the image control point database is reminded to check the currency of the corresponding image control point.

[0186] Compared with the prior art, the image control point automatic piercing method provided by the embodiment of the present invention inputs the geographical coordinate system coordinates of the image control point, a number of alternative piercing images and image information; wherein, the image information includes the aerial photography range and resolution; based on the image information, a point index image is extracted from the image control point database; based on the geographical coordinate system coordinates of the image control point and the POS data of the alternative piercing images, a target piercing image is selected from the number of alternative piercing images, and the target piercing image is cropped to obtain a target image; feature extraction is performed on the point index image and the target image respectively to obtain a first initial feature point and a second initial feature point; based on the first initial feature point and the second initial feature point, a homography matrix between the point index image and the target image is established; the homography matrix is called to perform azimuth correction on the target image to obtain a second target image; a template image containing the image control point is cropped from the point index image, and the second target image is subjected to template matching using the template image to obtain an integer pixel registration coordinate; quadratic surface fitting is performed on the integer pixel registration coordinate to obtain a target piercing coordinate; piercing is performed on the target piercing image based on the target piercing coordinate, and a piercing file is output. Compared with the prior art, the embodiment of the present invention can achieve sub-pixel level automatic piercing of aerial photography images by adopting a three-level matching strategy of "coarse matching + fine matching + sub-pixel level matching", greatly improving the piercing efficiency and accuracy.

[0187] See Figure 5 , the embodiment of the present invention also provides an image control point automatic piercing device 10, including:

[0188] An alternative piercing image input module 11, configured to input the geographical coordinate system coordinates of the image control point, a number of alternative piercing images and image information; wherein, the image information includes the aerial photography range and resolution;

[0189] A point index image extraction module 12, configured to extract a point index image from the image control point database based on the image information;

[0190] The target puncture point image screening module 13 is configured to select a target puncture point image from the plurality of alternative puncture point images based on the coordinates of the image control point in the geographic coordinate system and the POS data of the alternative puncture point images, and crop the target puncture point image to obtain a target image;

[0191] The feature extraction module 14 is configured to perform feature extraction on the point index image and the target image to respectively obtain a first initial feature point and a second initial feature point;

[0192] The homography matrix calculation module 15 is configured to establish a homography matrix between the point index image and the target image based on the first initial feature point and the second initial feature point;

[0193] The orientation correction module 16 is configured to call the homography matrix to perform orientation correction on the target image to obtain a second target image;

[0194] The fine matching module 17 is configured to crop a template image including the image control point from the point index image, and perform template matching on the second target image using the template image to obtain integer pixel registration coordinates;

[0195] The sub-pixel level matching module 18 is configured to perform quadratic surface fitting on the integer pixel registration coordinates to obtain target puncture point coordinates;

[0196] The puncture point module 19 is configured to perform puncture on the target puncture point image based on the target puncture point coordinates and output a puncture point file.

[0197] The image control point automatic puncture device provided by the embodiment of the present invention can implement all the process steps of the image control point automatic puncture method described in the above embodiment. The functions and the achieved technical effects of each module and unit in the device respectively correspond to the functions and the achieved technical effects of the image control point automatic puncture method described in the above embodiment, and the specific implementation manners are not elaborated herein.

[0198] See Figure 6 , the embodiment of the present invention further provides an image control point automatic puncture device 20, including a processor 21, a memory 22, and a computer program stored in the memory 22 and configured to be executed by the processor 21. When the processor 21 executes the computer program, the steps in the image control point automatic puncture method embodiment described above are implemented, such as Figure 1 the steps S1 to S9 described in; or, when the processor 21 executes the computer program, the functions of each module in the above device embodiments are implemented.

[0199] The image control point automatic spotting device may be a computing device such as a desktop computer, notebook, handheld computer, and cloud server. The image control point automatic spotting device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the schematic diagram is only an example of the image control point automatic spotting device, and does not constitute a limitation on the image control point automatic spotting device. It may include more or fewer components than shown, or combine certain components, or different components. For example, the image control point automatic spotting device may also include input / output devices, network access devices, buses, etc.

[0200] The processor may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the image control point automatic spotting device, and connects various parts of the entire image control point automatic spotting device through various interfaces and lines.

[0201] The memory can be used to store the computer programs and / or modules. The processor realizes various functions of the image control point automatic spotting device by running or executing the computer programs and / or modules stored in the memory, and by calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system, application programs required for at least one function, etc.; the data storage area may store data created according to the use of the controller, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0202] Among them, if the modules integrated in the image control point automatic piercing device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0203] Compared with the prior art, the image control point automatic piercing device, equipment and storage medium provided by the embodiments of the present invention input the coordinates of the image control point in the geographic coordinate system, several alternative piercing images and image information; among them, the image information includes the aerial photography range and resolution; based on the image information, a point index image is extracted from the image control point database; based on the coordinates of the image control point in the geographic coordinate system and the POS data of the alternative piercing images, a target piercing image is selected from the several alternative piercing images, the target piercing image is cropped to obtain a target image; feature extraction is performed on the point index image and the target image to obtain a first initial feature point and a second initial feature point respectively; based on the first initial feature point and the second initial feature point, a homography matrix between the point index image and the target image is established; the homography matrix is called to perform azimuth correction on the target image to obtain a second target image; a template image containing the image control point is cropped from the point index image, and the second target image is subjected to template matching using the template image to obtain an integer-pixel registration coordinate; quadratic surface fitting is performed on the integer-pixel registration coordinate to obtain a target piercing coordinate; piercing is performed on the target piercing image based on the target piercing coordinate, and a piercing file is output. Compared with the prior art, the embodiments of the present invention can achieve sub-pixel-level automatic piercing of aerial photography images by adopting a three-level matching strategy of "coarse matching + fine matching + sub-pixel-level matching", greatly improving the piercing efficiency and accuracy.

[0204] The above are the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. An automatic tie point piercing method, characterized in that, Including: Inputting the geodetic coordinate of image control points, several alternative fiducial images and image information; wherein, the image information includes the aerial photography range and resolution; Based on the image information, extracting the point index image from the image control point database; Based on the geodetic coordinate of the image control points and the POS data of the alternative fiducial images, selecting the target fiducial image from the several alternative fiducial images, and cropping the target fiducial image to obtain the target image; Performing feature extraction on the point index image and the target image respectively to obtain the first initial feature points and the second initial feature points; Based on the first initial feature points and the second initial feature points, establishing a homography matrix between the point index image and the target image; Invoking the homography matrix to perform azimuth correction on the target image to obtain the second target image; Cropping a template image containing image control points from the point index image, and performing template matching on the second target image with the template image to obtain the integer pixel registration coordinates; Performing quadratic surface fitting on the integer pixel registration coordinates to obtain the target fiducial coordinates; Performing fiducial marking on the target fiducial image based on the target fiducial coordinates and outputting a fiducial marking file.

2. The automatic tie point piercing method according to claim 1, wherein The target fiducial image and the target image are obtained through the following methods: Inputting the POS data of the alternative fiducial images; the POS data includes the geodetic coordinate of the photography center, the rotation matrix and the sensor focal length; Performing a difference operation on the geodetic coordinate of the image control points and the geodetic coordinate of the photography center to obtain the image space auxiliary coordinate of the image control points; Based on the image space auxiliary coordinate of the image control points, the rotation matrix and the sensor focal length, calculating the image coordinates of the image control points; When the image coordinates are within a predetermined range on the alternative fiducial image, taking the alternative fiducial image as the target fiducial image; Centering on the image coordinates, cropping the target image from the target fiducial image based on a preset first cropping size.

3. The automatic image control point piercing method according to claim 1, characterized in that Both the first initial feature points and the second initial feature points are calculated through the following methods: Performing downsampling on the image to be processed to obtain a second image to be processed; wherein, the image to be processed is the point index image or the target image; Calculating the Hessian matrix and the Hessian matrix discriminant of each pixel point on the second image to be processed; Based on the Hessian matrix discriminant, converting the second image to be processed into a transformed image; Constructing an image pyramid based on the transformed image; Based on the image pyramid, obtaining the neighboring pixel points within the three-dimensional neighborhood of each pixel point, and when the gray value of the pixel point is greater than the gray values of all the neighboring pixel points or less than the gray values of all the neighboring pixel points, taking the pixel point as an initial feature point.

4. The method for automatically puncturing image control points according to claim 3, characterized in that, It also includes calculating the feature vector of the initial feature points through the following methods: Centering on the initial feature point, calculating the first wavelet features in different directions, and taking the direction with the largest first wavelet feature as the main direction of the initial feature point; Taking the initial feature point as the center, a region block containing a number of sub-regions is taken along the main direction, and the second wavelet feature of each sub-region is calculated, where the second wavelet feature includes the sum in the horizontal direction, the sum in the vertical direction, the sum of the absolute values in the horizontal direction, and the sum of the absolute values in the vertical direction; Summarize the second wavelet features of each sub-region to obtain the feature vectors of each initial feature point.

5. The automatic image control point piercing method according to claim 1, characterized in that Establishing the homography matrix between the point index image and the target image based on the first initial feature point and the second initial feature point includes: Calculating the Euclidean distance between the first initial feature point and the second initial feature point, and taking the point pairs with the Euclidean distance less than the preset distance threshold as the initial feature point pairs; Randomly extracting the first preset number of the initial feature point pairs to obtain the second initial feature point pairs; Based on the second initial feature point pairs, using the least variance estimation algorithm to calculate the intermediate homography matrix; Taking the initial feature point pairs with the deviation from the intermediate homography matrix less than the preset deviation threshold as the inlier points; When the number of inlier points is greater than the preset inlier point number threshold, taking the inlier points as the new second initial feature point pairs and returning to the step of calculating the intermediate homography matrix; when the number of inlier points is not greater than the inlier point number threshold, returning to the step of randomly extracting the second initial feature point pairs; When the iteration end condition is satisfied, taking the intermediate homography matrix with the largest number of inlier points as the homography matrix.

6. The automatic image control point puncturing method according to claim 1, characterized in that The second target image and the template image are calculated in the following way: Taking the image control point as the center, an initial template image is cropped from the point index image based on the preset second cropping size; Based on the homography matrix, the initial template image is expanded to obtain the template image; Using the homography matrix to map the template image to the target image to obtain the coordinate matrix; Performing bilinear interpolation on the target image using the coordinate matrix to obtain the second target image; where the size of the template image is smaller than the size of the second target image.

7. The automatic image control point piercing method according to claim 1, wherein The template matching of the second target image using the template image includes: Sliding the template image on the second target image to calculate the normalized squared difference between different positions of the second target image and the template image; Taking the coordinates of the pixel point with the smallest normalized squared difference as the integer pixel registration coordinates; Where the normalized squared difference is calculated in the following way: Based on the size and sliding position of the template image, a matching image is extracted from the second target image; Normalizing the pixel points on the matching image and the template image to obtain the target matching image and the target template image; Calculating the squared difference of the grayscales of the target matching image and the target template image to obtain the normalized squared difference.

8. An automatic image control point piercing device, characterized in that, Including: An alternative piercing point image input module for inputting the coordinates of the image control point in the geographical coordinate system, a number of alternative piercing point images, and image information; where the image information includes the aerial photography range and the resolution; A point index image extraction module, which is used to extract a point index image from an image control point database based on the image information; A target piercing point image screening module, which is used to select a target piercing point image from the several alternative piercing point images based on the geodetic coordinate system coordinates of the image control point and the POS data of the alternative piercing point images, and crop the target piercing point image to obtain a target image; A feature extraction module, which is used to extract features from the point index image and the target image respectively to obtain a first initial feature point and a second initial feature point; A homography matrix calculation module, which is used to establish a homography matrix between the point index image and the target image based on the first initial feature point and the second initial feature point; An orientation correction module, which is used to call the homography matrix to perform orientation correction on the target image to obtain a second target image; A fine matching module, which is used to crop a template image containing an image control point from the point index image, and perform template matching on the second target image with the template image to obtain integer-pixel registration coordinates; A sub-pixel level matching module, which is used to perform quadratic surface fitting on the integer-pixel registration coordinates to obtain target piercing point coordinates; A piercing point module, which is used to perform piercing on the target piercing point image based on the target piercing point coordinates and output a piercing point file.

9. An automatic control point piercing device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the image control point automatic piercing method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program. Wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the image control point automatic piercing method according to any one of claims 1 to 7.

Images