Direct geographic positioning method and system based on airborne LiDAR point cloud assistance

Through the direct geolocation method of the same-machine image based on the onboard LiDAR point cloud, the SURF operator and virtual control point technology are used to solve the difficulty of direct geolocation of LiDAR point cloud and aerial images, and efficient visual axis error calibration and imaging are achieved Geopositioning reduces costs and time and is suitable for fast response to needs.

CN114255457BActive Publication Date: 2024-11-15WUHAN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111445327.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-30
Publication Date
2024-11-15
Estimated Expiration
2041-11-30

AI Technical Summary

Technical Problem

In the prior art, there are difficulties in direct geolocation of airborne LiDAR point clouds and aerial images, especially because the image sensing frame and IMU frame cannot maintain parallelism, resulting in a complicated and costly calibration process of the visual axis error, and traditional system calibration. It is difficult to respond quickly due to environmental changes.

Method used

Direct geolocation method for the same-machine image based on airborne LiDAR point cloud assisted is adopted. The same-name image point is extracted through the SURF operator, and virtual control points are obtained by using LiDAR point cloud interpolation. Combined with planarity, slope and reliability tests, collinearity is established. The equation performs compensation for the perspective axis offset to achieve direct geolocation of the image.

Benefits of technology

It reduces the need for manual selection of control points, reduces the cost of setting up the test field, improves calibration accuracy and efficiency, and is suitable for rapid response situations where new calibration fields cannot be established, improving productivity and user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114255457B_ABST
    Figure CN114255457B_ABST
Patent Text Reader

Abstract

The present invention discloses a method and system for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance, including selecting a sub-block image set, intercepting airborne LiDAR point cloud data according to the area where the image is located, and using SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images; obtaining the ground target point of the same-name image point through forward intersection in photogrammetric space, replacing the elevation value of the target point with the elevation value interpolated from the LiDAR point cloud to obtain a candidate virtual control point; brushing a suitable virtual control point set based on an automatic VCP selection process, the automatic VCP selection process includes a flatness test, a slope test, and a reliability test; establishing an adjustment equation based on the VCP point set and a collinearity equation to perform line of sight offset compensation, and supporting the acquisition of the same-machine image direct geolocation results based on airborne LiDAR point cloud assistance. The present invention uses airborne LiDAR point cloud data as the control information source for direct geolocation of images, thereby effectively reducing labor costs and time costs, and ensuring calibration accuracy.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to an image direct geographic positioning technology, in particular to an on-board image direct geographic positioning solution based on airborne LiDAR point cloud assistance, and belongs to the field of photogrammetry and remote sensing technology. Background Art

[0002] Now science and technology are developing rapidly, and the new generation of sensors in the field of remote sensing is updated very quickly, but this has also caused the level of corresponding data processing software to fail to keep up with the development speed of hardware. Taking the airborne LiDAR system as an example, the initial system only had the function of laser scanning. After several years of development, a complete airborne LiDAR system can not only obtain laser point clouds but also corresponding optical images. However, in the corresponding software market, there is no set of software with a high degree of automation. At present, most of the fusion processing methods for point cloud and image data are completed through software preprocessing, assisted by frequent manual or semi-automatic work of operators, and then the relevant surveying and mapping products can be produced. Such a process is often costly and has a long production cycle. Not only that, the rapid development of the remote sensing surveying and mapping industry has brought a series of problems, including that the surveying and mapping products obtained by processing single sensor data are increasingly difficult to meet social needs. For example, the airborne LiDAR system can obtain the geographic coordinates of the surface of the objects in the survey area, but the geometric features and texture information of the target cannot be directly obtained. It is still difficult to extract point cloud data with 3D coordinates from overlapping images through traditional photogrammetry matching methods. In addition, the point cloud data extracted through matching is dense and requires operations such as point cloud thinning. At the same time, the accuracy is not ideal. Fusion of airborne LiDAR point cloud and aerial imagery, two different data sources, can give full play to their respective advantages and make up for each other's shortcomings.

[0003] LiDAR systems all come with a POS system, so in theory the high-resolution aerial images acquired can be directly positioned to achieve the purpose of alignment with the LiDAR point cloud. However, since strict parallelism cannot be maintained between the image sensing frame and the IMU frame, the attitude and position relationship between the camera and the IMU (called the boresight error) must be determined before direct geo-registration of the image can be performed. Typically, this step is determined indirectly by using a reference block with known ground control points (located in the project area or a special test area), a process called system calibration.

[0004] Although traditional system calibration is widely used, it has the following disadvantages: 1) Environmental conditions (e.g., temperature, humidity, etc.) may vary dramatically between the test area and the mapping area. Therefore, the geometry of the camera during operation may also change relative to the test situation due to changes in environmental conditions. 2) It is expensive and sometimes impractical to establish a new test site for each mapping project and collect a large number of ground control points. In recent years, the continuous improvement of the performance and accuracy of LiDAR systems has made LiDAR point cloud data a source of control information for photogrammetric applications, used to perform aerial triangulation or determine the axis alignment error for direct georeferencing. Direct positioning of optical images on the same machine through airborne LiDAR systems has great potential, which is conducive to improving the production efficiency of related industries, promoting the development of the field of heterogeneous remote sensing data fusion processing, and diversifying mapping products. Summary of the invention

[0005] In view of the challenges and difficulties faced by the registration of LiDAR point clouds and aerial images, the present invention proposes a direct geolocation scheme for same-aircraft images based on the assistance of airborne LiDAR point clouds, so as to achieve the purpose of registration with aerial images and LiDAR point clouds.

[0006] The technical solution of the present invention provides a method for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance, including the following processing:

[0007] Select a sub-block image set, intercept the airborne LiDAR point cloud data according to the area where the image is located, and use the SURF operator to extract the image points with the same name in the overlapping area of ​​the sub-block images;

[0008] The ground target points with the same image points are obtained by forward intersection in photogrammetry space, and the elevation values ​​of the target points are replaced with the elevation values ​​interpolated from the LiDAR point cloud to obtain candidate virtual control points;

[0009] Selecting a suitable set of virtual control points based on an automatic VCP selection process, wherein the automatic VCP selection process includes a flatness test, a slope test, and a reliability test;

[0010] The adjustment equation is established based on the VCP point set and the collinearity equation to perform the line of sight offset compensation, which supports the acquisition of direct geolocation results based on the same-aircraft images assisted by the airborne LiDAR point cloud.

[0011] Moreover, the SURF operator is used to extract the same-name image points in the overlapping area of ​​the sub-block images. The implementation process is as follows:

[0012] In the image scale space, the image is convolved using window filtering to obtain local feature points;

[0013] The main direction of the feature point is determined by counting the pixel values ​​around the feature point in a 60° fan-shaped area;

[0014] Perform Haar wavelet filtering on the data around the feature point along the main direction of the feature point and construct a feature vector as the descriptor of the point;

[0015] The correlation coefficients between different image feature points are calculated based on direction and grayscale to determine the image homonymous points.

[0016] Moreover, the implementation process of obtaining candidate virtual control points is as follows,

[0017] Project the point cloud data onto a two-dimensional plane, and construct a 2D irregular triangulated network based on the projection points. The values ​​of the vertices of the triangulated network are the elevation values ​​of the corresponding point cloud.

[0018] Assign the plane coordinates of the target point to the virtual control point VCP, and determine which triangle the VCP falls on;

[0019] Calculate the candidate virtual control point VCP coordinates. If the target point is located at a vertex of the triangle, assign the vertex value to the VCP point. Otherwise, estimate it through the inverse distance weighted interpolation of the triangle vertices.

[0020] Moreover, the implementation process of selecting a suitable virtual control point set based on the automatic VCP selection process is as follows:

[0021] Flatness test, including searching for point cloud data within a preset scale window centered on the VCP, fitting the plane equation to calculate the flatness of the area, and comparing it with the corresponding set threshold;

[0022] Slope test, including calculating the slope of the area where the VCP is located and comparing it with the corresponding set threshold;

[0023] Reliability testing includes constraining the distribution of the selected VCP point set, agreeing that it is evenly distributed in the measurement area and that some VCPs are located in the overlapping area of ​​multiple images.

[0024] Furthermore, the adjustment equation is established based on the VCP point set and the collinearity equation to perform the line of sight offset compensation implementation process as follows:

[0025] Calculate the corresponding image point coordinates for each VCP point, and use the exterior orientation elements with errors to calculate the VCP image point coordinates according to the collinear conditional formula;

[0026] Calculate the coefficients and constant terms of the error equation point by point to form the error equation;

[0027] Step 1: Calculate the coefficient matrix and constant term of the normal equation to form the normal equation;

[0028] Solving the exterior orientation elements includes solving the exterior orientation element correction number according to the normal equation and summing it with the exterior orientation element to obtain a new exterior orientation element.

[0029] On the other hand, the present invention also provides a direct geolocation system for same-machine images based on airborne LiDAR point cloud assistance, which is used to implement the above-mentioned direct geolocation method for same-machine images based on airborne LiDAR point cloud assistance.

[0030] Furthermore, the following modules are included,

[0031] The first module is used to select a sub-block image set, intercept the airborne LiDAR point cloud data according to the area where the image is located, and use the SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images;

[0032] The second module is used to obtain ground target points of the same-name image points through forward intersection in photogrammetric space, and replace the elevation values ​​of the target points with the elevation values ​​interpolated from the LiDAR point cloud to obtain candidate virtual control points;

[0033] A third module is used to select a suitable set of virtual control points based on an automatic VCP selection process, wherein the automatic VCP selection process includes a flatness test, a slope test, and a reliability test;

[0034] The fourth module is used to establish adjustment equations based on the VCP point set and the collinearity equation to perform line of sight offset compensation, and supports obtaining direct geolocation results of the same-machine images based on the assistance of the airborne LiDAR point cloud.

[0035] Alternatively, it includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned method of direct geolocation of same-machine images based on airborne LiDAR point cloud assistance.

[0036] Alternatively, it includes a readable storage medium having a computer program stored thereon, and when the computer program is executed, the method for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance as described above is implemented.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] 1. The present invention uses airborne lidar point cloud data as the control information source for direct image geolocation. There is no need to establish a new test field for field collection of a large number of ground control points, thereby effectively reducing the manpower and time costs of the project. It is particularly suitable for calibrating the line of sight error in areas where a new calibration field cannot be established or where a rapid response is required in situations such as earthquake rescue.

[0039] 2. The present invention aims to automatically and directly geolocate images, designs a set of virtual control point selection processes to avoid manual selection of control point information from the point cloud, and uses an iterative method to gradually calibrate the image line of sight misalignment error. It has good robustness and a high guarantee for calibration accuracy.

[0040] The solution of the present invention is simple and convenient to implement, has strong practicality, solves the problems of low practicality and inconvenience in actual application existing in related technologies, can improve user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 The present invention is a flowchart of a method for direct geolocation of same-aircraft images based on airborne LiDAR point cloud assistance according to an embodiment of the present invention.

[0042] Figure 2 Schematic diagram of image same-name points and VCP inversely calculated image points according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0044] like Figure 1 As shown, an embodiment of the present invention provides a method for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance, comprising the following steps:

[0045] Step 1: Select a sub-block image set collected on a flat area from the image set, and intercept the airborne LiDAR point cloud data according to the area where the image is located.

[0046] The image set refers to all the images acquired during the flight, and the sub-block image set refers to the images that meet the conditions selected from all the images.

[0047] Step 2: Use the SURF operator to extract the image points with the same name in the overlapping area of ​​the sub-block images.

[0048] In step 2 as described above, the method of using the SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images is:

[0049] Step 2.1, feature detection: In the image scale space, use window filtering to perform a convolution process on the image to obtain local feature points.

[0050] In the embodiment, the feature detection of the image is based on the scale space, and the scale space of the image is the representation of the image at different resolutions. For a certain image, a multi-scale window is used to convolve the image to construct a multi-level pyramid image, and the transformed Hessian matrix Δ(H) is obtained, which is expressed as follows:

[0051] Δ(H)=DXX D YY -(0.9D XY ) 2 (1)

[0052] Where D XX , D YY , D XY They are the results of convolving the box filter with the image.

[0053] According to formula (1), the image points are calculated and the value of the determinant Δ(H) is determined. If it is a positive value and the corresponding point feature value is also a positive value, then this point is the local feature point being searched.

[0054] Step 2.2, main direction determination: the main direction of the feature point is determined by counting the pixel values ​​around the feature point in a 60° fan-shaped area.

[0055] In the embodiment, with the feature point as the center, a circular neighborhood with a preset radius (preferably set to 6s, s is the scale value of the feature point) is calculated, and the sum of the horizontal and vertical Harr wavelet features of all points in the fan-shaped area (usually the angle of the fan-shaped area is preferably set to 60°) is counted. Then the fan is rotated at a certain angle (usually preferably set to 0.2 radians) and the Harr wavelet eigenvalues ​​in the area are counted again. Finally, the direction of the fan with the largest value is taken as the main direction of the feature point.

[0056] Step 2.3, feature point description vector: perform Haar wavelet filtering on the data around the feature point along the main direction of the feature point, and construct a 64-bit feature vector as the descriptor of the point.

[0057] In the embodiment, a 4*4 rectangular area block is selected along the main direction of the feature point, and the Haar wavelet features of 25 pixels in each sub-area in the horizontal and vertical directions are counted, where the horizontal and vertical directions are relative to the main direction. The Haar wavelet feature is a four-dimensional vector V (∑dx,∑dy,∑|dx|,∑|dy|), which is the sum of the horizontal direction, the sum of the vertical direction, the sum of the absolute value of the horizontal direction, and the sum of the absolute value of the vertical direction. Feature extraction is performed on each small block in the rectangular area to obtain a 64-bit feature vector, i.e., the feature point description vector.

[0058] Step 2.4, feature point matching: Calculate the correlation coefficient between feature points of different images based on direction and grayscale to determine the same-name points in the images.

[0059] In the embodiment, a feature point p is selected from an image P. i , calculate the feature point description vector V one by one i and feature point q in another image Q jThe feature point description vector V j The Euclidean distance dis is calculated as follows:

[0060] dis=||V i -V j || 2 j=1,2,3,…n (2)

[0061] Where V i Represents the i-th feature point p in the first image P i The feature point description vector, V j Represents the jth feature point q in another image Q j The feature point description vector,|| || 2 represents the L2 norm.

[0062] Select the two feature points q with the smallest Euclidean distance a With q b , which is related to p i The Euclidean distances are dis a and dis b (dis a <dis b ), if dis a / dis b is less than a given threshold T (the reasonable value range of T is 0.4-0.6), and the feature point p i The feature point description vector V i With feature point q a The feature point description vector V a The directions are consistent, then it is considered that point p i With q a are feature points with the same name.

[0063] Step 3: Use the initial image exterior orientation elements obtained by the POS system to calculate the target point coordinates (X, Y, Z) corresponding to the same-name image points based on the spatial forward intersection method.

[0064] The spatial forward intersection method is an existing technology. For the sake of reference, the method of calculating the target point coordinates (X, Y, Z) corresponding to the same-name image points based on the spatial forward intersection method using the initial image exterior orientation elements obtained by the POS system in step 3 as described above is provided as follows:

[0065] The target point coordinates (X, Y, Z) are calculated based on the collinearity condition equation, and the calculation formula is as follows:

[0066]

[0067] Where (x, y) is the image coordinate of the image point, -f is the focal length of the camera, (X S, Y S , Z S ) is the geographic 3D coordinate of the sensor location, λ is the scale factor, is the exterior azimuth element, a i , b i , c i is the rotation matrix elements, i=1,2,3.

[0068] When the selected image connection point is in the overlapping area of ​​n images, 2n error equations can be listed according to the formula, and the coordinates of the target point can be obtained by solving them using the least squares method.

[0069] Step 4: Replace the elevation value Z of the target point with the elevation value Z′ interpolated from the LiDAR point cloud. (X, Y, Z′) is called a candidate virtual control point. The specific method is as follows: project the point cloud onto the xy plane, and then build a 2D irregular triangulated network (TIN) based on the projected points. For a given target point (X, Y, Z), determine the triangle in which it is located. If it is located at a vertex of the triangle, the Z value will be replaced by the elevation value of the vertex. Otherwise, it is estimated by inverse distance weighted interpolation.

[0070] In step 4 above, the method for calculating the coordinates of the candidate virtual control points is:

[0071] In step 4.1, the point cloud data of the test area is projected onto the xy plane, and then a 2D triangulated irregular network (TIN) is constructed based on the projection points. The values ​​of the vertices in the TIN network are the elevation values ​​of the corresponding point clouds.

[0072] Step 4.2: Optionally select a target point calculated in step 3, and determine which triangle in the TIN its projection point falls on based on the X and Y coordinate values ​​of the target point.

[0073] Step 4.3, calculate the coordinates of the candidate virtual control point VCP. If the target point is exactly at a vertex of the triangle, assign the vertex value to the VCP point. Otherwise, estimate it through the inverse distance weighted interpolation of the vertices of the triangle. That is, if the target projection point is exactly at a vertex of the triangle, the Z value will be replaced by the vertex value point; otherwise, estimate it through inverse distance weighted interpolation to obtain the elevation value Z′ interpolated from the LiDAR point cloud, thereby obtaining the candidate virtual control point (X, Y, Z′), that is, the candidate VCP.

[0074] Step 5: Select a suitable set of virtual control points based on the automatic VCP selection process, including flatness test, slope test, and reliability test, and select a VCP set that is located in a flat area and has a suitable distribution. VCP in Chinese is virtual control point.

[0075] In step 5 as described above, the method for selecting a suitable set of virtual control points based on the automatic VCP selection process is:

[0076] Step 5.1, flatness test: Search the point cloud data within the k-scale window centered on the VCP, fit the plane equation to calculate the flatness of the area, and compare it with the corresponding set threshold.

[0077] In the embodiment, the search is performed with k centered on the VCP (preferably, it is recommended to set σ is the density of the point cloud, and max() is the maximum value function. The point cloud data in the scale window is fitted with the plane equation, and the distance from all point clouds in the area to the fitting plane is counted. The maximum distance is recorded as d max , if d max If it is greater than the corresponding given threshold, the point is marked as a non-virtual control point, otherwise, the next test is performed.

[0078] Step 5.2, slope test: Calculate the slope of the VCP area and compare it with the corresponding set threshold.

[0079] In the embodiment, the slope T of the area where the VCP is located is calculated slope , compare it with the corresponding set threshold (preferably the recommended threshold setting range is 5 to 10°), if d is greater than the set threshold, mark the point as a non-virtual control point, otherwise, proceed to the next test.

[0080] Step 5.3, reliability test: constrain the distribution of the selected VCP point set, and agree that it is evenly distributed in the measurement area, and some VCPs are located in the overlapping area of ​​multiple images.

[0081] In the embodiment, the preferred implementation method is to set the number of virtual control points to be selected, such as 12, and then divide the test area into 12 sub-areas. The virtual control point selection rules are as follows: select a virtual control point in each sub-area, and the virtual control point is located at the center of the sub-area block as much as possible; select 3-5 sub-area blocks, and constrain the virtual control points selected in the area, requiring them to be located in the overlapping area of ​​n (n≥3) images.

[0082] Step 6: Establish an adjustment equation based on the VCP point set and the collinearity equation to perform line of sight offset compensation.

[0083] The present invention further proposes that step 6 comprises the following steps:

[0084] Step (a) Calculate the coordinates of the corresponding image points of each VCP point. Calculate the coordinates of the VCP image points according to the collinear condition using the exterior orientation elements with errors.

[0085] Step (b) calculates the coefficients and constant terms of the error equation point by point to form the error equation.

[0086] Step (c) calculates the coefficient matrix and constant term of the normal equation to form the normal equation.

[0087] Step (d) Calculate the exterior orientation element. Calculate the exterior orientation element correction number according to the normal equation and sum it with the exterior orientation element to obtain a new exterior orientation element.

[0088] In the embodiment, the method of establishing the adjustment equation based on the VCP point set and the collinearity equation to perform the visual axis offset compensation is:

[0089] Set the rotation matrix of the exterior orientation element And the rotation matrix of the line of sight error in represents the visual axis error angle, or is called the external azimuth element compensation value. Then the collinearity equation can be expressed as:

[0090]

[0091] in,

[0092]

[0093]

[0094] In the above formula, (x, y) is the image coordinate of the image point; f is the focal length of the camera; λ is the scale factor; (X S , Y S , Z S ) is the geographic three-dimensional coordinate of the sensor location; (X, Y, Z′) is the geographic coordinate of the selected virtual control point VCP; a i , b i , c i is the rotation matrix elements, i = 1, 2, 3; a i ′,b i ′,c i ′ is the compensation matrix elements, i=1,2,3.

[0095] because The values ​​of ω′ and κ′ are very small. According to the knowledge of photogrammetry, the matrix can be simplified as follows:

[0096]

[0097] Formula (5) can be adjusted to:

[0098]

[0099]

[0100] Among them, the matrix is the difference between the two matrices.

[0101] Take the x,y coordinates of the image tie points as observations and misalign the visual axis ω′, κ′ are unknowns. Using the expansion of Taylor's formula, we get two error equations:

[0102]

[0103] Among them, the variable F X , F Y , V X , V Y , e x , e y , e z , Δe x , Δe y , Δe z It is a variable representation of the result of derivation of the function, which will not be described in detail in the present invention.

[0104]

[0105] Among them, A i , B i , C i It is a variable set for the convenience of statistics.

[0106] The error equation can also be expressed as:

[0107] V=AE-L (9)

[0108] Wherein, each expression V, A, E, L can be expressed as V = [V X V Y ] T , E=[e x , e y , e z ] T , L=[l x , l y ] T ,

[0109] The normal equation is expressed as:

[0110] A T WAE=A T WL (10)

[0111] In the formula, W is the unit matrix, which represents the weight matrix of the observation value. The transformation can be obtained as follows:

[0112] E=(A T A) -1 AT L (11)

[0113] The obtained E represents the compensation value of the exterior azimuth element.

[0114] Step 7, update the exterior azimuth elements, repeat steps 3, 4, and 6, and reduce the error through iteration until the total distance between all image points with the same name and the corresponding virtual control point back-calculated image points remains stable during the iteration, or reaches the maximum number of iterations, then the iteration can be stopped to obtain the direct geolocation results of the same-machine image based on the airborne LiDAR point cloud assistance. The image points and the corresponding virtual control point back-calculated image points can be back-calculated by referring to step 3, and the schematic diagram is attached Figure 2 . S 1 and S 2 are the projection centers of image 1 and image 2 respectively; S 1 -XYZ is the image space coordinate system; 1 and O 2 They are the image plane coordinate system O where image 1 is located. 1 -x 1 y 1 and the image plane coordinate system O where image 2 is located 2 -x 2 y 2 The origin of 1 and a 2 is a pair of image points with the same name, where a 1 The image plane coordinates are (x, y); P image is a 1 and a 2 The spatial position of the feature point is calculated using the initial exterior orientation elements and the forward intersection method; P laser It is a virtual control point obtained by interpolation of LiDAR point cloud, which is consistent with P image have the same plane coordinates; a′ 1 It is by P laser The image point position obtained by inverse calculation based on the collinear equation has an image plane coordinate of (x′, y′).

[0115] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0116] In some possible embodiments, a direct geolocation system based on airborne LiDAR point cloud assistance and same-aircraft imagery is provided, comprising the following modules:

[0117] The first module is used to select a sub-block image set, intercept the airborne LiDAR point cloud data according to the area where the image is located, and use the SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images;

[0118] The second module is used to obtain ground target points of the same-name image points through forward intersection in photogrammetric space, and replace the elevation values ​​of the target points with the elevation values ​​interpolated from the LiDAR point cloud to obtain candidate virtual control points;

[0119] A third module is used to select a suitable set of virtual control points based on an automatic VCP selection process, wherein the automatic VCP selection process includes a flatness test, a slope test, and a reliability test;

[0120] The fourth module is used to establish adjustment equations based on the VCP point set and the collinearity equation to perform line of sight offset compensation, and supports obtaining direct geolocation results of the same-machine images based on the assistance of the airborne LiDAR point cloud.

[0121] In some possible embodiments, a system for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance is provided, including a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a method for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance as described above.

[0122] In some possible embodiments, a system for direct geolocation of same-aircraft images based on airborne LiDAR point cloud assistance is provided, comprising a readable storage medium having a computer program stored thereon,

[0123] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for direct geolocation of same-aircraft images based on airborne LiDAR point cloud assistance, characterized by: The following processing is included: selecting a sub-block image set, intercepting the airborne LiDAR point cloud data according to the area where the image is located, and using the SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images; The ground target points with the same image points are obtained by forward intersection in photogrammetry space, and the elevation values ​​of the target points are replaced with the elevation values ​​interpolated from the LiDAR point cloud to obtain candidate virtual control points; Selecting a suitable set of virtual control points based on an automatic VCP selection process, wherein the automatic VCP selection process includes a flatness test, a slope test, and a reliability test; According to the VCP point set and collinear equation, the adjustment equation is established to perform the line of sight offset compensation, which supports the acquisition of direct geolocation results of the same-aircraft image based on the assistance of the airborne LiDAR point cloud; The SURF operator is used to extract the same-name image points in the overlapping area of ​​the sub-block images. The implementation process is as follows: In the image scale space, the image is convolved using window filtering to obtain local feature points; The main direction of the feature point is determined by counting the pixel values ​​around the feature point in a 60° fan-shaped area; Perform Haar wavelet filtering on the data around the feature point along the main direction of the feature point and construct a feature vector as the descriptor of the point; Calculate the correlation coefficient between different image feature points based on direction and grayscale to determine the image homonymous points; The implementation process of obtaining candidate virtual control points is as follows: Project the point cloud data onto a two-dimensional plane, and construct a 2D irregular triangulated network based on the projection points. The values ​​of the vertices of the triangulated network are the elevation values ​​of the corresponding point cloud. Assign the plane coordinates of the target point to the virtual control point VCP, and determine which triangle the VCP falls on; Calculate the coordinates of the candidate virtual control point VCP. If the target point is located at a vertex of the triangle, assign the vertex value to the VCP point. Otherwise, estimate it through the inverse distance weighted interpolation of the triangle vertices. The process of selecting a suitable set of virtual control points based on the automatic VCP selection process is as follows: Flatness test, including searching for point cloud data within a preset scale window centered on the VCP, fitting the plane equation to calculate the flatness of the area, and comparing it with the corresponding set threshold; Slope test, including calculating the slope of the area where the VCP is located and comparing it with the corresponding set threshold; Reliability test, including constraining the distribution of the selected VCP point set, stipulating that it is evenly distributed in the measurement area, and some VCPs are located in the overlapping area of ​​multiple images; The adjustment equation is established based on the VCP point set and the collinearity equation to perform the line of sight offset compensation. The implementation process is as follows: Calculate the corresponding image point coordinates for each VCP point, and use the exterior orientation elements with errors to calculate the VCP image point coordinates according to the collinear condition. in, In the above formula, (x, y) is the image coordinate of the image point; f is the focal length of the camera; λ is the scale factor; (X S , Y S , Z S ) is the geographic three-dimensional coordinate of the sensor location; (X, Y, Z′) is the geographic coordinate of the selected virtual control point VCP; a i , b i , c i is the rotation matrix elements, i = 1, 2, 3; a i ′,b i ′,c i ′ is the compensation matrix elements, i = 1, 2, 3; Take the x,y coordinates of the image tie points as observations and misalign the visual axis ω′, κ′ are taken as unknowns, and two error equations are obtained by expanding Taylor's formula: Among them, the variable F X , F Y , V X , V Y , e x , e y , e z , Δe x , Δe y , Δe z It is the variable representation of the result of taking the derivative of a function; Among them, A i , B i , C i It is to facilitate the statistical setting of variables; The coefficients and constant terms of the error equation are calculated point by point to form the error equation, which is expressed as follows: V=AE-L Wherein, each expression V, A, E, L is expressed as V = [V X V Y ] T , E=[e x , e y , e z ] T , L=[l x , l y ] T , Calculate the coefficient matrix and constant term of the normal equation to form the normal equation, which is expressed as follows: And=(A T TO) -1 TO T THE The obtained E represents the compensation value of the exterior azimuth element; Solving the exterior orientation elements includes solving the exterior orientation element correction number according to the normal equation and summing it with the exterior orientation element to obtain a new exterior orientation element.

2. A direct geolocation system based on airborne LiDAR point cloud assistance, characterized by: Used to implement a method for direct geolocation of same-machine images based on airborne LiDAR point cloud assistance as described in claim 1.

3. The direct geographic positioning system based on airborne LiDAR point cloud assistance according to claim 2, characterized in that: Includes the following modules, The first module is used to select a sub-block image set, intercept the airborne LiDAR point cloud data according to the area where the image is located, and use the SURF operator to extract the same-name image points in the overlapping area of ​​the sub-block images; The second module is used to obtain ground target points of the same-name image points through forward intersection in photogrammetric space, and replace the elevation values ​​of the target points with the elevation values ​​interpolated from the LiDAR point cloud to obtain candidate virtual control points; A third module is used to select a suitable set of virtual control points based on an automatic VCP selection process, wherein the automatic VCP selection process includes a flatness test, a slope test, and a reliability test; The fourth module is used to establish adjustment equations based on the VCP point set and the collinearity equation to perform line of sight offset compensation, and supports obtaining direct geolocation results of the same-machine images based on the assistance of the airborne LiDAR point cloud.

4. The direct geographic positioning system based on airborne LiDAR point cloud assistance according to claim 2, characterized in that: It includes a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the direct geographic positioning method of the same-machine image based on the assistance of airborne LiDAR point cloud as described in claim 1.

5. The direct geographic positioning system based on airborne LiDAR point cloud assistance according to claim 2, characterized in that: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, the method for direct geographic positioning of same-machine images based on airborne LiDAR point cloud assistance as described in claim 1 is implemented.