A method and system for back-projection of aerial linear array images guided by a three-dimensional grid

Abstract

The present invention provides a method and system for back-projection of aerial linear array images guided by a three-dimensional grid, including constructing a virtual three-dimensional grid on the object side; calculating the corresponding image points of the three-dimensional grid points on the original linear array image; calculating the corresponding image points of any object point on the original linear array image under the guidance of the three-dimensional grid, including for any object point, first finding its position in the three-dimensional grid, then calculating its approximate position in the linear array image, and then constructing search windows before and after the scanning line where the approximate position is located to calculate the best corresponding image point, thus completing the back-projection from the object point to the image point. The method proposed by the present invention adopts the idea of "exchanging space for time", only increasing the calculation time and storage space for the corresponding image points of the three-dimensional grid points, but greatly shortening the subsequent back-projection time, which has important significance for the processing of a large number of image points.

Description

Technical Field

[0001] The invention belongs to the technical field of photogrammetry, and in particular relates to a back-projection scheme of aerial linear array images under the guidance of a three-dimensional grid. Background Art

[0002] Aerial line array camera is a device for quickly acquiring ground image information. Compared with traditional frame imaging systems and multi-view oblique imaging systems, aerial line array cameras have the characteristics of high data acquisition efficiency, low overall cost, and strong practicality. The most well-known aerial line array camera in the world is the ADS series camera produced by Leica. After years of hard work, domestic research institutes in related technical fields have successfully developed the domestically produced aerial three-line array camera AMS-3000. This camera not only has higher resolution, but also has faster data acquisition efficiency. It can complete the acquisition of 180 square kilometers of ground data within 10 minutes, and has broad application prospects. In terms of ground positioning processing, there is a big difference between line array cameras and traditional imaging systems. Its biggest feature is continuous push-scan imaging, continuous within the image line, and independent between lines. In order to restore the relationship between image lines, the line array camera is equipped with a high-precision POS system composed of GPS and IMU to record the position and attitude parameters of each image line at the imaging moment. According to the principle of photogrammetry, the collinear equation is the basis of geometric processing, which describes the relationship between the coordinates of the object point and the coordinates of the image point. For frame images, each image has only one set of external orientation elements (i.e., position and attitude parameters), and collinear equations can be used to conveniently realize the forward and reverse calculations between the object point coordinates and the image point coordinates. However, for linear array images, each scan line has its own external orientation elements, that is, the external orientation elements are determined by the image coordinates. When back-projecting, that is, when using the object point to calculate the corresponding image point coordinates, the image point coordinates are unknown, and the external orientation elements of the collinear equations are of course unknown, which leads to the fact that the linear array image cannot be directly back-projected using collinear equations. The commonly used method is the iterative calculation method, which first assumes a set of external orientation elements, then uses the collinear equations to calculate the image point coordinates, obtains the external orientation elements based on the image point coordinates, and then brings them into the collinear equations, and repeats this process until the calculated image point coordinate change is less than a certain threshold value, and then the iteration ends. Obviously, the iterative calculation method is inefficient and has a great impact on the processing of linear array images. In order to improve this problem, researchers have also proposed some improved methods, such as the image-side bisection iteration method, the basic idea of which is to use the bisection principle to quickly narrow the search range. However, these improved methods are still iterative methods and are not perfect for the processing of linear array images. In particular, when the linear array images are displayed and calculated in real time, it can be clearly felt that the position of the mouse displayed in the image cannot keep up with the actual movement of the mouse, which greatly affects the application of the linear array images. In view of the shortcomings of the prior art, the present invention proposes a back-projection scheme for aerial linear array images under the guidance of a three-dimensional grid. Summary of the invention

[0003] In view of the problems in the prior art, the present invention proposes a back-projection scheme for airborne linear array images guided by a three-dimensional grid. This scheme adopts the idea of "exchanging space for time". First, a virtual three-dimensional grid is constructed in the object space, and then the image point coordinates corresponding to the three-dimensional grid points are calculated according to the image-side binary iteration method and saved. Next, for any object point, its position in the three-dimensional grid can be found first, and then the bilinear interpolation method is used to calculate its approximate position in the linear array image. Then, search windows are constructed before and after the scanning line where the approximate position is located, and the best corresponding image point is calculated, thus completing the back-projection from the object point to the image point.

[0004] To achieve the above object, the technical solution proposed by the present invention is a method for back-projecting airborne linear array images guided by a three-dimensional grid, including the following steps:

[0005] Step 1, construct a virtual three-dimensional grid in the object space;

[0006] Step 2, calculate the image points corresponding to the three-dimensional grid points on the original linear array image;

[0007] Step 3, calculate the image points corresponding to any object point on the original linear array image under the guidance of the three-dimensional grid, including for any object point, first finding its position in the three-dimensional grid, then calculating its approximate position in the linear array image, and then constructing search windows before and after the scanning line where the approximate position is located, calculating the best corresponding image point, and completing the back-projection from the object point to the image point.

[0008] Moreover, the implementation method of the above Step 1 includes the following sub-steps:

[0009] Step 1.1, calculate the ground range of the virtual three-dimensional grid corresponding to the original linear array image;

[0010] Step 1.2, determine the spacings dX, dY, dZ of the virtual three-dimensional grid;

[0011] Step 1.3, establish a virtual three-dimensional grid and record the coordinate information of the grid points;

[0012] The implementation method of establishing the virtual three-dimensional grid is to start from the lower left corner point (Xmin, Ymin, Zmin) of the area with dX, dY, dZ as the spacings.

[0013] Moreover, when calculating the image points corresponding to the three-dimensional grid points on the original linear array image in Step 2, the image-side binary iteration method is adopted.

[0014] Moreover, Step 3 includes the following sub-steps:

[0015] Step 3.1, calculate the grid where the object point P(X, Y, Z) is located. The implementation method is as follows:

[0016] First, calculate the position (I, J, K) of the object point P in the virtual three-dimensional grid.

[0017]

[0018] Wherein, [] represents the rounding operation, Round() represents the rounding operation, and (Xmin, Ymin, Zmin) is the lower left corner coordinate of the virtual three-dimensional grid.

[0019] Denote the grid points of the plane grid where P is located as P1(X1, Y1, Z1), P2(X2, Y2, Z2), P3(X3, Y3, Z3), P4(X4, Y4, Z4), and the corresponding image point coordinates are respectively denoted as p1(x1, y1), p2(x2, y2), p3(x3, y3), p4(x4, y4).

[0020] Step 3.2, perform bilinear interpolation on the object plane coordinates (X, Y) and the image coordinates (x, y) to find the approximate position (x, y) of the corresponding image point of the object point P on the linear array image. The calculation formula is:

[0021]

[0022] Among them,

[0023]

[0024]

[0025] w x and w y are the weights of the grid point P3(X3, Y3, Z3) in the X direction and the Y direction respectively.

[0026] Step 3.3, construct a search window around the approximate position (x, y) and search row by row for the best image point corresponding to the object point.

[0027] Moreover, the size of the search window is [Lx - 3, Lx + 3], and Lx is the scan line number corresponding to the approximate position (x, y).

[0028] On the other hand, the present invention also provides an aerial linear array image back-projection system guided by a three-dimensional grid for implementing a three-dimensional grid-guided aerial linear array image back-projection method as described above.

[0029] Moreover, it includes the following modules.

[0030] The first module is used to construct the object virtual three-dimensional grid.

[0031] The second module is used to calculate the corresponding image points of the three-dimensional grid points on the original linear array image.

[0032] The third module is used to calculate the corresponding image point of any object point on the original linear array image under the guidance of a three-dimensional grid. For any object point, first find its position in the three-dimensional grid, then calculate its approximate position in the linear array image, and then construct a search window before and after the scanning line where the approximate position is located to calculate the optimal corresponding image point, thus completing the back-projection from the object point to the image point.

[0033] Alternatively, it includes a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a method for back-projecting an aerial linear array image under the guidance of a three-dimensional grid as described above.

[0034] Alternatively, it includes a readable storage medium with a computer program stored thereon. When the computer program is executed, it implements a method for back-projecting an aerial linear array image under the guidance of a three-dimensional grid as described above.

[0035] Compared with the prior art, the present invention has the following advantages:

[0036] A virtual three-dimensional grid is constructed in the object space to calculate in advance the corresponding image points of the grid points on the original linear array image. When looking for the corresponding positions of other object points on the original image later, it can be searched within a small range of the original image based on the known information, greatly shortening the search time. Only the calculation and storage steps of the corresponding image points of the three-dimensional grid points are added, but the subsequent back-projection time is greatly shortened, which is of great significance for the processing of a large number of image points.

[0037] The solution of the present invention is simple and convenient to implement, with strong practicability. It solves the problems of low practicability and inconvenience in actual application existing in the related art, can improve the user experience, and has important market value. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a technical flow chart of an embodiment of the present invention.

[0039] Figure 2 It is a schematic diagram of linear array image projection in an embodiment of the present invention.

[0040] Figure 3 It is a schematic diagram of the relationship between the ground range of a three-dimensional grid and the image coverage range in an embodiment of the present invention.

[0041] Figure 4 It is a schematic diagram of a virtual three-dimensional grid in an embodiment of the present invention.

[0042] Figure 5 It is a schematic diagram of bilinear interpolation in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The technical solution of the present invention will be specifically described below with reference to the drawings and embodiments.

[0044] As shown in Figure 1 the figure, the process of a back-projection method for aerial linear array images guided by a three-dimensional grid in an embodiment of the present invention includes the following steps:

[0045] Step 1, construct a virtual three-dimensional grid in the object space;

[0046] The embodiment of the present invention further provides a preferred implementation manner as follows:

[0047] Step 1.1, calculate the ground range of the virtual three-dimensional grid corresponding to the original linear array image;

[0048] According to the approximate elevation data of the survey area, it is recommended to use the publicly available global SRTM data to calculate the average elevation Z0 of the survey area. As shown in Figure 2 the figure, project the end pixels of the first and last image rows of the linear array image onto the average elevation plane according to the collinearity condition (the specific formula is as shown in Equation (1)), and obtain four object space points P_1, P_2, P_3, and P_4. As shown in Figure 3 the figure, the quadrilateral formed by connecting these four object space points is the ground range corresponding to the original linear array image. By comparing the coordinate sizes of these four object space points, Xmin, Xmax, Ymin, and Ymax are obtained, and thus the ground range of the three-dimensional grid is obtained (the lower left corner coordinates of this area are (Xmin, Ymin), and the upper right corner coordinates are (Xmax, Ymax), which is represented as the inner rectangle in Figure 3 ), and this area is the circumscribed rectangle of the ground range corresponding to the original linear array image.

[0049]

[0050] In the formula, is the exterior orientation element of the i-th row, a i , b i , c i (i = 1, 2, 3) are the elements of the rotation matrix composed of , (x, y) are the image point coordinates, (X, Y) are the plane coordinates of the object space point, f is the principal distance of the image, and Z0 is the average elevation of the survey area.

[0051] Step 1.2, determine the spacings dX, dY, and dZ of the virtual three-dimensional grid on the X, Y, and Z axes;

[0052] Take an image block from the original linear array image, and it is recommended to use 128×128. Project the four vertices of the image block onto the average elevation plane according to the collinearity condition, and the calculation formula is formula (1). After calculating the four vertices, four object points can be obtained, and then calculate the difference between the two X coordinates and take the average as dX. Similarly, calculate dY. Query the approximate elevation data according to the ground range covered by the image, and obtain the maximum elevation Zmax and the minimum elevation Zmin of the ground area corresponding to the image. Divide the elevation of the entire area into n parts (the recommended value is 100), and then dZ can be calculated by formula (2).

[0053]

[0054] Step 1.3, establish a virtual three-dimensional grid and record the grid point coordinate information;

[0055] Starting from the lower left corner point (Xmin, Ymin, Zmin) of the area, with dX, dY, and dZ as the intervals, establish a virtual three-dimensional grid in the entire area, as Figure 4 shown. The number of grid points X_num in the X direction can be obtained by subtracting Xmax from Xmin and dividing by the interval dX. In the same way, the number of grid points Y_num and Z_num in the Y and Z directions can be obtained. The calculation formulas for X_num, Y_num, and Z_num are as follows:

[0056]

[0057] The position of each grid point in the virtual three-dimensional grid can be described by integers I, J, and K. I represents the I-th grid point counted from the lower left corner point of the area in the positive X-axis direction. J represents the J-th grid point counted from the lower left corner point of the area in the positive Y-axis direction. K represents the K-th grid point counted from the lower left corner point of the area in the positive Z-axis direction. Then the true object coordinates (X, Y, Z) of a grid point can be calculated by the following formula:

[0058]

[0059] Since the object coordinates of each grid point can be calculated, there is no need to store the grid point coordinates. Only store the lower left corner coordinates (Xmin, Ymin, Zmin) of the grid, the grid intervals (dX, dY, dZ), and the number of grid points in the XYZ directions (X_num, Y_num, Z_num).

[0060] Step 2, calculate the corresponding image points of the three-dimensional grid points on the original linear array image;

[0061] Since the ground coverage of the virtual three-dimensional grid is the circumscribed rectangle of the corresponding ground range of the original image, there will be some redundant grid points in the virtual three-dimensional grid that cannot find corresponding image points on the original linear array image. It is possible to determine whether a grid point is a redundant point by calculating whether the planar coordinates of the grid point are within the ground range covered by the image. For the grid points within the ground range covered by the image, the image bisection method in the image space can be used to calculate the corresponding image points of the three-dimensional grid points on the original linear array image and record their corresponding relationships.

[0062] The embodiments of the present invention further provide the following preferred implementation manners:

[0063] Step 2.1 Determine the initial values of the search window and the optimal scan line for iteration;

[0064] Assume that the linear array image has N scan lines. The left scan line number of the search window is denoted as L left , and the right scan line number of the search window is denoted as L right . The optimal scan line is the central scan line of the search window, denoted as L best . The initial search window is taken as [1, N], then L left = 1, L right = N, L best = N / 2.

[0065] Step 2.2 Calculate the image point coordinates corresponding to the object point;

[0066] In the image plane coordinate system, the axis parallel to the scan line is set as the y-axis, and the axis perpendicular to the scan line is set as the x-axis. The image point coordinates are expressed as (x, y). Calculate the image point coordinates x left , x best , x right corresponding to the object point on these three scan lines L left , x right , x best respectively according to the collinearity condition. The specific calculation method is to substitute the exterior orientation elements of these three scan lines L left , L best , L right into Equation (5).

[0067]

[0068] Among them, is the exterior orientation element of the i-th row, and a i , b i , c i (i = 1, 2, 3) are the elements of the rotation matrix composed of , (X, Y, Z) are the object point coordinates, and f is the principal distance of the image.

[0069] Step 2.3 Calculate x left×x best , and determine its sign;

[0070] If x left ×x best = 0, then L best corresponds to the best scan line;

[0071] If x left ×x best < 0, then bisect the window between L left and L best , that is, L left remains unchanged, and L right is changed to L best , L best is changed to L left and the average value of the new L right .

[0072] If x left ×x best > 0, then bisect the window between L best and L right , that is, L left is changed to L best ,, L right remains unchanged, and L best is changed to the average value of the new L left and L right .

[0073] Step 2.4 Judge the size of the search window |L right - L left |, where || represents taking the absolute value;

[0074] If the size of the search window |L right - L left | ≤ δ (δ is the set minimum window threshold), then proceed to Step 2.5; otherwise, transfer to Step 2.2.

[0075] Step 2.5 Determine the best scan line within the search window [L left , L right ;

[0076] Within the search window, sequentially take the exterior orientation elements corresponding to each scan line to calculate the image point coordinates corresponding to the object point, and the image point closest to the CCD linear array on the focal plane corresponds to the best scan line.

[0077] Calculate the image point coordinates corresponding to each grid point on the original linear array image through image - side bisection method, and then record the image point coordinates corresponding to the grid points in sequence.

[0078] Step 3, under the guidance of the 3D grid, calculate the corresponding image point of any object point on the original linear array image, and complete the back-projection from the object point to the image point.

[0079] Taking an object point P(X, Y, Z) as an example, calculate its corresponding image point on the original linear array image.

[0080] The embodiments of the present invention further provide a preferred implementation manner as follows:

[0081] Step 3.1, calculate the grid where the object point P is located.

[0082] First, according to formula (6), calculate the position (I, J, K) of the object point P in the virtual 3D grid.

[0083]

[0084] In the formula, [] represents the rounding operation, Round() represents the rounding operation, and (Xmin, Ymin, Zmin) is the coordinates of the lower left corner of the virtual 3D grid.

[0085] According to K, the layer where the object point P is located in the 3D grid can be determined. According to I and J, the plane grid to which the object point belongs in this layer of the grid can be determined. As Figure 5 shown, denote the coordinates of the four grid points of this plane grid as P1(X1, Y1, Z1), P2(X2, Y2, Z2), P3(X3, Y3, Z3), P4(X4, Y4, Z4) (since P1, P2, P3, and P4 are in the same plane grid, Z1 = Z2 = Z3 = Z4). The corresponding image point coordinates of these four grid points have been calculated in Step 2, and are denoted as p1(x1, y1), p2(x2, y2), p3(x3, y3), p4(x4, y4) respectively.

[0086] Step 3.2, perform bilinear interpolation on the object plane coordinates (X, Y) and the image coordinates (x, y) to find the approximate position (x, y) of the object point P on the linear array image corresponding image point;

[0087] Bilinear interpolation is often used in image gray interpolation. Using the pixel values of the adjacent 4 image points, different weights are assigned according to their distances from the interpolation point for linear interpolation. Here, the image point coordinates x and y are used as function values respectively, and bilinear interpolation is borrowed for interpolation. The calculation formula is as follows:

[0088]

[0089] Among them,

[0090]

[0091]

[0092] w x and w y are the weights of the grid point P3(X3, Y3, Z3) in the X and Y directions respectively.

[0093] Step 3.3: Construct a small search window before and after the scan line where the approximate position (x, y) is located, and calculate the best image point corresponding to the object point.

[0094] In Step 3.2, the approximate position of the image point corresponding to the object point is found through bilinear interpolation. This approximate position is very close to the best position of the corresponding image point. Therefore, only a small search window needs to be constructed before and after the scan line where the approximate position is located (the recommended window size is [Lx - 3, Lx + 3], where Lx is the scan line number corresponding to the approximate position (x, y)). Take the exterior orientation elements corresponding to each scan line in the search window in turn to calculate the image point coordinates corresponding to the object point. The image point closest to the CCD linear array on the focal plane is the best image point corresponding to the object point.

[0095] In specific implementation, the method proposed by the technical solution of the present invention can be automatically run by those skilled in the art using computer software technology. The system device for implementing the method, such as a computer-readable storage medium storing the corresponding computer program of the technical solution of the present invention and a computer device including running the corresponding computer program, should also be within the protection scope of the present invention.

[0096] In some possible embodiments, there is provided an aerial linear array image back-projection system guided by a three-dimensional grid, including the following modules.

[0097] The first module is used to construct a virtual three-dimensional grid of the object space.

[0098] The second module is used to calculate the image points corresponding to the three-dimensional grid points on the original linear array image.

[0099] The third module is used to calculate the image points corresponding to any object point on the original linear array image under the guidance of the three-dimensional grid. For any object point, first find its position in the three-dimensional grid, then calculate its approximate position in the linear array image, and then construct a search window before and after the scan line where the approximate position is located to calculate the best corresponding image point, thus completing the back-projection from the object point to the image point.

[0100] In some possible embodiments, there is provided an aerial linear array image back-projection system guided by a three-dimensional grid, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the above-mentioned method for aerial linear array image back-projection guided by a three-dimensional grid.

[0101] In some possible embodiments, a back-projection system for aerial linear array images guided by a three-dimensional grid is provided, including a readable storage medium, on which a computer program is stored. When the computer program is executed, a method for back-projecting aerial linear array images guided by a three-dimensional grid as described above is implemented.

[0102] The specific embodiments described herein are merely illustrative of the spirit of the present invention. Those skilled in the art of the present invention can make various modifications or supplements to the described specific embodiments or use similar methods for substitution, but will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for back-projection of aerial linear array images guided by a three-dimensional grid, characterized in that, It includes the following steps: Step 1, constructing an object-space virtual three-dimensional grid; Step 2, calculating the corresponding image points of the three-dimensional grid points on the original linear array image; Step 3, calculating the corresponding image points of any object-space point on the original linear array image under the guidance of the three-dimensional grid, including for any object-space point, first finding its position in the three-dimensional grid, then calculating its approximate position in the linear array image, and then constructing search windows before and after the scan line where the approximate position is located to calculate the best corresponding image point, completing the back-projection from the object-space point to the image point; The following sub-steps are included in Step 3: Step 3.1, calculating the grid where the object-space point P(X, Y, Z) is located. The implementation method is as follows: First, calculate the position (I, J, K) of the object-space point P in the virtual three-dimensional grid, where [], [] represents the rounding operation, Round() represents the rounding operation, and (Xmin, Ymin, Zmin) is the coordinates of the lower left corner of the virtual three-dimensional grid; Denote the grid points of the plane grid where P is located as P1(X1, Y1, Z1), P2(X2, Y2, Z2), P3(X3, Y3, Z3), P4(X4, Y4, Z4), and the corresponding image point coordinates are respectively denoted as p1(x1, y1), p2(x2, y2), p3(x3, y3), p4(x4, y4); Step 3.2, performing bilinear interpolation on the object-space plane coordinates (X, Y) and the image coordinates (x, y) to find the approximate position (x, y) of the corresponding image point of the object-space point P on the linear array image. The calculation formula is: where, w x and w y are the weights of the grid point P3(X3, Y3, Z3) in the X and Y directions, respectively; Step 3.3, constructing a search window around the approximate position (x, y) and searching row by row for the best image point corresponding to the object-space point.

2. The three-dimensional grid-guided back-projection method for aerial linear array images according to claim 1, wherein: The implementation method of the above Step 1 includes the following sub-steps: Step 1.1, calculating the ground range of the virtual three-dimensional grid corresponding to the original linear array image; Step 1.2, determining the grid spacings dX, dY, dZ of the virtual three-dimensional grid; Step 1.3, establishing the virtual three-dimensional grid and recording the grid point coordinate information; The implementation method of establishing the virtual three-dimensional grid is to take the lower left corner point (Xmin, Ymin, Zmin) of this area as the starting point and dX, dY, dZ as the spacings.

3. A method for back projection of aerial linear array images guided by a three-dimensional grid, as claimed in claim 1, wherein: When calculating the corresponding image points of the three-dimensional grid points on the original linear array image in Step 2, the image-space bisection method is adopted.

4. A method for back-projection of aerial linear array images guided by a three-dimensional grid, as claimed in claim 1, wherein: The size of the search window is [Lx - 3, Lx + 3], where Lx is the scan line number corresponding to the approximate position (x, y).

5. A three-dimensional grid-guided aerial linear array image back-projection system, characterized in that: It is used to implement a method for back-projection of an aerial linear array image guided by a three-dimensional grid as described in any one of claims 1 - 4.

6. The aerial linear array image back-projection system guided by a three-dimensional grid according to claim 5, wherein: It includes the following modules: The first module is used to construct an object-space virtual three-dimensional grid; The second module is used to calculate the corresponding image points of the three-dimensional grid points on the original linear array image; The third module is used to calculate the corresponding image points of any object-space point on the original linear array image under the guidance of the three-dimensional grid, including for any object-space point, first finding its position in the three-dimensional grid, then calculating its approximate position in the linear array image, and then constructing search windows before and after the scan line where the approximate position is located to calculate the best corresponding image point, completing the back-projection from the object-space point to the image point.

7. The aerial linear array image back-projection system guided by a three-dimensional grid according to claim 5, wherein: It includes a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a three-dimensional grid-guided aerial linear array image back-projection method according to any one of claims 1-4.

8. The aerial linear array image back-projection system guided by the three-dimensional grid according to claim 5, characterized in that: It includes a readable storage medium, on which a computer program is stored. When the computer program is executed, it implements a three-dimensional grid-guided aerial linear array image back-projection method according to any one of claims 1-4.

Images