A new system splicing type surveying and mapping camera target positioning method

Abstract

This invention discloses a novel target localization method for a stitched mapping camera, comprising: obtaining the interior orientation elements of each sub-image of the camera based on the calibration parameters of the stitched mapping camera; obtaining the exterior orientation elements of each sub-image based on the POS data of each exposure point in the aerial photography operation and the transformation relationship between each sub-image; establishing an overlap relationship table between each sub-image based on the image overlap rate and the transformation relationship between each sub-image; finding each pair of sub-images with an overlap relationship based on the overlap relationship table, extracting corresponding feature points within the overlapping area of ​​the sub-images, and performing feature matching to obtain the image plane coordinates of the corresponding feature points in each sub-image; treating each sub-image with obtained image plane coordinates as an independent camera, performing aerial triangulation, and using bundle adjustment to jointly solve for the ground positioning result. This invention avoids the accuracy loss caused by the near single-center projection processing of the stitched mapping camera, thus improving the ground positioning accuracy.

Description

Technical Field

[0001] This invention belongs to the field of surveying and computer vision technology, and in particular relates to a new type of target localization method for stitched surveying cameras. Background Technology

[0002] Based on current practice, constructing a complete large-format image using a multi-lens, multi-detector stitching method not only meets the inevitable trend of the development of ultra-large image size, wide field of view, and high resolution for aerial mapping cameras, but also has significant advantages over the method of using a single ultra-large format detector to cover the entire field of view of the camera, with superior overall performance and significantly reduced difficulty and cost in implementation.

[0003] However, current data processing methods for mosaic cameras still involve constructing a centrally projected image through image stitching, and then achieving ground positioning according to the traditional photogrammetric processing workflow. This workflow is relatively mature, but mosaic cameras are not strictly single-center projections in nature, and the stitching process only constructs an approximate equivalent central projection, which inevitably results in a loss of image accuracy. Summary of the Invention

[0004] The technical problem solved by this invention is to overcome the shortcomings of the prior art and provide a new type of target positioning method for stitched mapping cameras. By directly using the original images for ground positioning, the accuracy loss caused by the approximate single-center projection process is avoided, thereby improving the accuracy of ground positioning.

[0005] The objective of this invention is achieved through the following technical solution: a novel target localization method for a stitched mapping camera, comprising: obtaining the interior orientation elements of each sub-image of the camera based on the calibration parameters of the stitched mapping camera; obtaining the transformation relationship between each sub-image based on the calibration parameters of the stitched mapping camera and the interior orientation elements of each sub-image; obtaining the exterior orientation elements of each sub-image based on the POS data of each exposure point image in a preset aerial photography operation and the transformation relationship between each sub-image; obtaining the image overlap rate based on the POS data of each exposure point image in a preset aerial photography operation, and obtaining an overlap relationship table between each sub-image based on the image overlap rate and the transformation relationship between each sub-image; finding each pair of sub-images with an overlap relationship based on the overlap relationship table between each sub-image, extracting corresponding feature points in the overlapping area of ​​the sub-images, and performing feature matching to obtain the image plane coordinates of the corresponding feature points in each sub-image; treating each sub-image with obtained image plane coordinates as an independent camera, performing aerial triangulation, and using bundle adjustment to solve for the ground positioning result.

[0006] In the above-mentioned new system of stitched mapping camera target positioning method, the calibration parameters of the stitched mapping camera include lens calibration parameters and detector installation position parameters.

[0007] In the above-mentioned new system stitching type mapping camera target positioning method, the lens calibration parameters include lens focal length and distortion model.

[0008] In the above-mentioned new system stitching type mapping camera target positioning method, the detector installation position parameters include the camera image principal point coordinates.

[0009] In the above-mentioned new system of stitched mapping camera target localization method, the distortion model is obtained through the following formula:

[0010]

[0011] Where Δx is the distortion of the image point's x-coordinate, Δy is the distortion of the image point's y-coordinate, x is the image point's x-coordinate, y is the image point's y-coordinate, x0 is the principal image point's x-coordinate, y0 is the principal image point's y-coordinate, and k1, k2, k3, and k n All are radial distortion parameters, p1 and p2 are tangential distortion coefficients, r is the radiation distance of the actual imaging point, and n is the number of terms in the selected radial distortion polynomial.

[0012] In the above-mentioned new system of stitched mapping camera target localization method, the transformation relationship between each sub-image is obtained by the following formula (taking the transformation relationship between image a and image b as an example):

[0013]

[0014] Where, λ a , λ b f represents the scaling factors of sub-images a and b relative to the reference image (the scaling factor of the reference image is 1), respectively. a f b Let x be the focal length of the camera corresponding to sub-image a and sub-image b, respectively. a0 ,y a0 (x) are the image plane coordinates of the principal point of sub-image a, and (x) are the coordinates of the principal point of sub-image a. b0 ,y b0 (x) are the image plane coordinates of the principal point of sub-image b, and (x) are the coordinates of the principal point of sub-image b. a ,y a (x) represents the image plane coordinates of the corresponding image point in sub-image a, where (x) b ,y b (X) represents the image plane coordinates of the corresponding image point in sub-image b. a ,Y a Z a (X) represents the three-dimensional coordinates of the optical center of camera a in the camera system coordinate system. b ,Y b Z b () represents the three-dimensional coordinates of the optical center of camera b in the camera system coordinate system. These are the rotation matrices between sub-images a and b and the reference image, respectively.

[0015] A novel stitched mapping camera target positioning system includes: a first module for obtaining the interior orientation elements of each sub-image of the camera based on the calibration parameters of the stitched mapping camera; obtaining the transformation relationship between each sub-image based on the calibration parameters of the stitched mapping camera and the interior orientation elements of each sub-image; and obtaining the exterior orientation elements of each sub-image based on the POS data of each exposure point image in a preset aerial survey and the transformation relationship between each sub-image; a second module for obtaining the image overlap rate based on the POS data of each exposure point image in a preset aerial survey, and establishing an overlap relationship table between each sub-image based on the image overlap rate and the transformation relationship between each sub-image; a third module for finding each pair of sub-images with an overlap relationship based on the overlap relationship table between each sub-image, extracting corresponding feature points in the overlapping area of ​​the sub-images, and performing feature matching to obtain the image plane coordinates of the corresponding feature points in each sub-image; and a fourth module for treating each sub-image with obtained image plane coordinates as an independent camera, performing aerial triangulation, and using bundle adjustment to solve for the ground positioning result.

[0016] In the aforementioned new system of stitched mapping camera target positioning system, the calibration parameters of the stitched mapping camera include lens calibration parameters and detector installation position parameters.

[0017] In the aforementioned new system of stitched mapping camera target positioning system, the lens calibration parameters include lens focal length and distortion model.

[0018] In the aforementioned new type of stitched mapping camera target positioning system, the detector installation position parameters include the coordinates of the principal point of the camera image.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] This invention improves the accuracy of geolocation by directly using original images for geolocation, avoiding the accuracy loss caused by approximate single-center projection. Attached Figure Description

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0022] Figure 1 This is a flowchart of the new system stitching type mapping camera target localization method provided in the embodiments of the present invention;

[0023] Figure 2This is a schematic diagram of image overlay provided in an embodiment of the present invention;

[0024] Figure 3 This is a schematic diagram illustrating the overlapping relationship between various sub-images provided in an embodiment of the present invention;

[0025] Figure 4 This is a schematic diagram of feature matching of sub-images provided in an embodiment of the present invention;

[0026] Figure 5 This is a schematic diagram of aerial triangulation performed by treating each sub-image as an independent camera, as provided in an embodiment of the present invention. Detailed Implementation

[0027] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0028] Figure 1 This is a flowchart of a new type of stitched mapping camera target localization method provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps:

[0029] The interior orientation elements of each sub-image of the camera are obtained based on the calibration parameters of the stitched mapping camera; the transformation relationship between each sub-image is obtained based on the calibration parameters of the stitched mapping camera and the interior orientation elements of each sub-image; the exterior orientation elements of each sub-image are obtained based on the POS data of each exposure point image of the aerial photography operation and the transformation relationship between each sub-image.

[0030] The image overlap rate is obtained based on the POS data of each exposure point image in the aerial photography operation. An overlap relationship table between each sub-image is established based on the image overlap rate and the conversion relationship between each sub-image.

[0031] Based on the overlap relationship table between sub-images, find each pair of sub-images with overlapping relationship, extract corresponding feature points in the overlapping area of ​​the sub-images, and perform feature matching to obtain the image plane coordinates of the corresponding feature points in each sub-image.

[0032] Each sub-image with obtained image plane coordinates is treated as an independent camera, and aerial triangulation is performed. The bundle adjustment method is used to solve the problem together to obtain the ground positioning result.

[0033] The calibration parameters for a stitched mapping camera include lens calibration parameters and detector mounting position parameters. The lens calibration parameters include the lens focal length and distortion model. The detector mounting position parameters include the coordinates of the camera image principal point.

[0034] The distortion model is obtained through the following formula:

[0035]

[0036] Where Δx is the distortion of the image point's x-coordinate, Δy is the distortion of the image point's y-coordinate, x is the image point's x-coordinate, y is the image point's y-coordinate, x0 is the principal image point's x-coordinate, and y0 is the principal image point's y-coordinate. k1, k2, k3, and k n All are radial distortion parameters, p1 and p2 are tangential distortion coefficients, r is the radiation distance of the actual imaging point, and n is the number of terms in the selected radial distortion polynomial.

[0037] The transformation relationship between each sub-image is obtained by the following formula (taking the transformation relationship between image a and image b as an example):

[0038]

[0039] Where, λ a , λ b f represents the scaling factors of sub-images a and b relative to the reference image (the scaling factor of the reference image is 1), respectively. a f b Let x be the focal length of the camera corresponding to sub-image a and sub-image b, respectively. a0 ,y a0 (x) are the image plane coordinates of the principal point of sub-image a, and (x) are the coordinates of the principal point of sub-image a. b0 ,y b0 (x) are the image plane coordinates of the principal point of sub-image b, and (x) are the coordinates of the principal point of sub-image b. a ,y a (x) represents the image plane coordinates of the corresponding image point in sub-image a, where (x) b ,y b (X) represents the image plane coordinates of the corresponding image point in sub-image b. a ,Y a Z a (X) represents the three-dimensional coordinates of the optical center of camera a in the camera system coordinate system. b ,Y b Z b () represents the three-dimensional coordinates of the optical center of camera b in the camera system coordinate system. These are the rotation matrices between sub-images a and b and the reference image, respectively. In this formula, parameters such as camera focal length, scaling factor, principal point image plane coordinates, and rotation matrix between the image and the reference image are all given by the calibration parameters of the stitched mapping camera.

[0040] Specifically, the method includes the following steps:

[0041] Step (1): Calculate the interior and exterior orientation elements of each sub-image from the camera.

[0042] (1) Calculate the interior orientation elements of each sub-image of the camera based on the calibration parameters of the stitched surveying camera; obtain the transformation relationship (i.e., relative position and rotation relationship) between each sub-image based on the calibration parameters of the stitched surveying camera and the interior orientation elements of each sub-image of the camera.

[0043] The calibration parameters for a stitched mapping camera include lens calibration parameters and detector mounting position parameters. Lens calibration parameters include the lens focal length f and distortion model, such as:

[0044] Where k1, k2, and k3 are radial distortion parameters; p1 and p2 are tangential distortion coefficients; and r is the radiation distance of the actual imaging point, i.e. The detector is installed at the principal point of the camera image, with coordinates x0, y0. The exterior orientation elements of each camera represent their relative position and attitude relationships. The geometric relationships at the imaging moment satisfy the collinearity equation:

[0045]

[0046] In the formula, x, y are the image plane coordinates of the image point; x0, y0, f are the interior orientation elements of the image; X S ,Y S Z S The object space coordinates of the camera station; X, Y, Z are the object space coordinates of the object point; a i ,b i ,c i (i = 1, 2, 3) are the nine direction cosines formed by the three exterior azimuth elements of the image.

[0047] The transformation relationship between each sub-image is obtained by the following formula (taking the transformation relationship between image a and image b as an example):

[0048]

[0049] Where, λ a , λ b f represents the scaling factors of images a and b relative to the reference image (the scaling factor of the reference image is 1), respectively. a f b Let x be the focal length of the camera corresponding to image a and image b, respectively. a0 ,y a0 (x) are the image plane coordinates of the principal point of image a, and (x) are the coordinates of the principal point of image a. b0 ,y b0(x) represents the image plane coordinates of the principal point of image b, and (x) represents the image plane coordinates of the principal point a ,y a (x) is the coordinate of a certain image point of the same name on the image plane of image a, (x) b ,y b (X) is the coordinate of the corresponding image point on the image plane of image b. a ,Y a Z a (X) represents the three-dimensional coordinates of the optical center of camera a in the camera system coordinate system. b ,Y b Z b () represents the three-dimensional coordinates of the optical center of camera b in the camera system coordinate system. These are the rotation matrices between image a and image b and the reference image, respectively. In this formula, parameters such as camera focal length, scaling factor, principal point image plane coordinates, and rotation matrices between the image and the reference image are all given by the calibration parameters of the stitched mapping camera.

[0050] Based on this formula, the relative orientation elements of each camera with respect to the reference can be calculated.

[0051] (2) Based on the POS data of the aerial photography operation image, and combined with the transformation relationship between each sub-image (i.e., relative position and rotation relationship), calculate the exterior orientation elements of each sub-image.

[0052] Based on the line-of-sight (POS) calibration results, the absolute imaging positions of the reference camera and each camera are calculated using the eccentricity component of the POS installation. Using the line-of-sight calibration results and the attitude angles measured by the POS, the imaging attitudes of the reference camera and each camera are calculated. This yields the interior and exterior orientation elements of each camera.

[0053] Step (2): Estimate the overlap relationship of each sub-image

[0054] (1) Calculate the image overlap rate based on the POS data of the aerial photography images, such as... Figure 2 As shown; based on the position and height of the shooting point obtained from the POS and the camera parameters, the ground coverage area corresponding to each overall image is calculated, thereby calculating the overlap rate between the overall images.

[0055] (2) Establish an overlap relationship table between each sub-image based on the image overlap rate and the transformation relationship between each sub-image. For example... Figure 3 As shown, the same area coverage relationship of different image sub-images is constructed to form the overlapping relationship table of sub-images of the entire working area.

[0056] Step (3): Image Connectivity Matching and Conversion

[0057] Based on the predicted image overlap relationship, each pair of overlapping sub-images is identified. Corresponding feature points are extracted within the overlapping region of each sub-image, and feature matching is performed to obtain the image plane coordinates of these feature points in each sub-image. After matching, mismatched points are removed based on the image relative geometric relationship model. For example... Figure 4 As shown.

[0058] Step (4): Treat each sub-image with obtained image plane coordinates as an independent camera, perform aerial triangulation, and use bundle adjustment to solve for the ground positioning result. The ground positioning result includes the precise exterior orientation elements of each sub-image and the ground coordinates corresponding to the corresponding feature points.

[0059] like Figure 5 As shown, each sub-image is treated as an independent camera, and aerial triangulation of the detection area is carried out. The bundle adjustment method is used to jointly solve for the accurate exterior orientation elements of each image and the ground coordinates of the connection points, thereby achieving the Earth positioning solution.

[0060] The method also includes the following steps:

[0061] Step (5): Accuracy Verification

[0062] The positioning accuracy is evaluated based on the measured ground control points to confirm the ground positioning results.

[0063] The formula for estimating positioning accuracy is as follows:

[0064]

[0065] Where, μ X μ is the estimated error value of the X coordinate of the ground control point. Y Let μ be the estimated value of the Y-coordinate error of the control point. Z X is the estimated value of the Z-coordinate error of the control point. 真实 X is the X coordinate of the ground measurement of the image control point. 平差 For the X and Y coordinates of the image control point adjustment solution, 真实 Y is the Y coordinate of the ground measurement of the image control point. 平差 For the Y-coordinate of the image control point adjustment solution, Z... 真实 Z is the Z coordinate of the ground measurement of the image control point. 平差 Here is the Z-coordinate of the control point adjustment solution, and n is the number of control points used for accuracy verification.

[0066] The concepts and methods of image feature point extraction, aerial triangulation, ground phase control points, and positioning accuracy in this embodiment are common knowledge in the surveying and mapping discipline and will not be elaborated further.

[0067] This embodiment also provides a novel system for positioning targets using a stitched mapping camera. The system includes: a first module for obtaining the interior orientation elements of each sub-image of the camera based on the camera's calibration parameters; obtaining the transformation relationship between sub-images based on the camera's calibration parameters and the interior orientation elements of each sub-image; and obtaining the exterior orientation elements of each sub-image based on the POS data of each exposure point image during aerial photography and the transformation relationship between sub-images; a second module for obtaining the image overlap rate based on the POS data of each exposure point image during aerial photography, and establishing an overlap relationship table between sub-images based on the image overlap rate and the transformation relationship between sub-images; a third module for finding each pair of overlapping sub-images based on the overlap relationship table, extracting corresponding feature points within the overlapping area of ​​the sub-images, performing feature matching, and obtaining the image plane coordinates of the corresponding feature points in each sub-image; and a fourth module for treating each sub-image with obtained image plane coordinates as an independent camera, performing aerial triangulation, and using bundle adjustment to jointly solve for the ground positioning result.

[0068] This embodiment uses the original imagery directly for ground positioning, avoiding the accuracy loss caused by approximate single-center projection and improving ground positioning accuracy.

[0069] Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications to the technical solutions of the present invention by utilizing the methods and techniques disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solutions of the present invention shall fall within the protection scope of the technical solutions of the present invention.

Claims

1. A new type of stitched mapping camera target localization method, characterized in that... include: The interior orientation elements of each sub-image of the camera are obtained based on the calibration parameters of the stitched mapping camera; The conversion relationship between each sub-image is obtained based on the calibration parameters of the stitched mapping camera and the interior orientation elements of each sub-image of the camera; the exterior orientation elements of each sub-image are obtained based on the POS data of each exposure point image of the preset aerial photography operation and the conversion relationship between each sub-image. The image overlap rate is obtained based on the POS data of each exposure point image in the preset aerial photography operation. The overlap relationship table between each sub-image is obtained based on the image overlap rate and the conversion relationship between each sub-image. Based on the overlap relationship table between sub-images, find each pair of sub-images with overlapping relationships, extract corresponding feature points in the overlapping area of ​​the sub-images, and perform feature matching to obtain the image plane coordinates of the corresponding feature points in each sub-image. Each sub-image with obtained image plane coordinates is treated as an independent camera, and aerial triangulation is performed. The bundle adjustment method is used to solve the problem together to obtain the ground positioning result.

2. The target localization method for the novel stitching type mapping camera according to claim 1, characterized in that: The calibration parameters for a stitched mapping camera include lens calibration parameters and detector installation position parameters.

3. The target positioning method for the new type of stitched mapping camera according to claim 2, characterized in that: The lens calibration parameters include lens focal length and distortion model.

4. The target localization method for the new type of stitched mapping camera according to claim 2, characterized in that: The detector installation location parameters include the coordinates of the principal point of the camera image.

5. The target positioning method for the novel stitching type mapping camera according to claim 3, characterized in that: The distortion model is obtained through the following formula: Where Δx is the distortion of the image point's x-coordinate, Δy is the distortion of the image point's y-coordinate, x is the image point's x-coordinate, y is the image point's y-coordinate, x0 is the principal image point's x-coordinate, y0 is the principal image point's y-coordinate, and k1, k2, k3, and k n All are radial distortion parameters, p1 and p2 are tangential distortion coefficients, r is the radiation distance of the actual imaging point, and n is the number of terms in the selected radial distortion polynomial.

6. The target localization method for a novel stitching-type mapping camera according to claim 1, characterized in that: The transformation relationship between the sub-images is obtained by the following formula: Where, λ a , λ b f represents the scaling factors of sub-images a and b relative to the reference image, respectively. a f b Let x be the focal length of the camera corresponding to sub-image a and sub-image b, respectively. a0 ,y a0 (x) are the image plane coordinates of the principal point of sub-image a, and (x) are the coordinates of the principal point of sub-image a. b0 ,y b0 (x) are the image plane coordinates of the principal point of sub-image b, and (x) are the coordinates of the principal point of sub-image b. a ,y a (x) represents the image plane coordinates of the corresponding image point in sub-image a, where (x) b ,y b (X) represents the image plane coordinates of the corresponding image point in sub-image b. a ,Y a Z a (X) represents the three-dimensional coordinates of the optical center of camera a in the camera system coordinate system. b ,Y b Z b () represents the three-dimensional coordinates of the optical center of camera b in the camera system coordinate system. These are the rotation matrices between sub-images a and b and the reference image, respectively.

7. A novel stitching-type mapping camera target positioning system, characterized in that... include: The first module is used to obtain the interior orientation elements of each sub-image of the camera based on the calibration parameters of the stitched mapping camera. The conversion relationship between each sub-image is obtained based on the calibration parameters of the stitched mapping camera and the interior orientation elements of each sub-image of the camera; the exterior orientation elements of each sub-image are obtained based on the POS data of each exposure point image of the preset aerial photography operation and the conversion relationship between each sub-image. The second module is used to obtain the image overlap rate based on the POS data of the images at each exposure point of the preset aerial photography operation, and to establish an overlap relationship table between each sub-image based on the image overlap rate and the conversion relationship between each sub-image. The third module is used to find each pair of overlapping sub-images according to the overlap relationship table between each sub-image, extract the same feature points in the overlapping area of ​​the sub-images, and perform feature matching to obtain the image plane coordinates of the same feature points in each sub-image. The fourth module is used to treat each sub-image with obtained image plane coordinates as an independent camera, perform aerial triangulation, and use bundle adjustment to solve for the ground positioning result.

8. The novel stitching-type mapping camera target positioning system according to claim 7, characterized in that: The calibration parameters for a stitched mapping camera include lens calibration parameters and detector installation position parameters.

9. The novel stitching-type mapping camera target positioning system according to claim 8, characterized in that: The lens calibration parameters include lens focal length and distortion model.

10. The novel stitching-type mapping camera target positioning system according to claim 8, characterized in that: The detector installation location parameters include the coordinates of the principal point of the camera image.

Images