An aerial three-line array camera image orthorectification method and system

Abstract

The application provides a kind of aviation three linear array camera image orthorectification method and system, according to imaging equation, original image is directly projected to target image, the gray of original pixel is split according to weight on target image, and target image is recombined by splitting integral pixel into sub-pixel, finally realizing linear array image orthorectification method correction.The application extracts the weight of four pixels of each target position on target image;According to the weight, the gray value of original pixel is decomposed into pixel, and the allocated gray and weight are accumulated into the corresponding variables of target pixel respectively;According to the gray value and weight obtained by target pixel, normalization is carried out to form target image;Detect target image, if the weight is less than the preset threshold, the average value of adjacent pixels is used to compensate.This method can effectively solve the precision problem caused by the discontinuity of adjacent rows of targets in the inverse solution method of aerial linear array image, significantly improve the rectification accuracy and quality of linear array image, and is especially suitable for the image rectification of aviation three linear array camera.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image correction technology, specifically designing a forward resolution correction method for aerial three-line array camera images, wherein target pixel segmentation is the key technology of the forward resolution correction method. Background Technology

[0002] A three-line array camera uses three CCD detectors mounted on the focal plane. During flight, the three CCD arrays image simultaneously: the forward-looking array (F) tilts forward to image, the downward-looking array (N) images vertically towards the ground, and the rear-looking array (B) tilts backward to image. The working principle is as follows: Figure 1 As shown, as the payload platform pushes forward, the camera continuously scans and images the ground at a certain frequency, simultaneously acquiring three linear images of the ground at each exposure time.

[0003] The raw images from a three-line array camera are recorded separately along each CCD line, with each CCD line recording one image file. Based on the principle of continuous pushbroom imaging, each line of the image file represents the data of a specific line on the ground acquired at a particular moment using a center-based imaging method. Each line of the image is imaged once using a center-based imaging method, therefore the CCD pixels are continuously imaged, meaning the direction of the CCD lines aligns with the actual position on the ground. However, the image lines are imaged independently, and there is no adjacency between lines on the ground. Especially when aircraft are affected by atmospheric flow, engine vibrations, etc., and their attitude is not constant, the acquired images will have some distortion, such as… Figure 2 As shown.

[0004] Because the original images from a three-line array camera are highly distorted, making direct observation and measurement impossible, image correction is necessary to reconstruct the images to match the actual positions. Figure 3 As shown. In a three-line array processing system, the original distorted image is called an L0-level image, while the image after distortion correction is called an L1-level image.

[0005] Traditional image correction uses inverse image correction. The basic principle is to solve the position of the target image in the original image according to the integer pixel position (i.e. row and column number) of the target image and then perform sub-pixel interpolation on the original image to obtain its gray level and place it on the target image position.

[0006] In inverse resolving image correction, when performing sub-pixel interpolation on the original image, it is essential that the image being interpolated is a continuous image, meaning that adjacent pixels are continuous on the original target. However, for aerial linear array images, the flight platform is not as stable as a satellite linear array camera due to the influence of the natural environment and the flight vehicle during the imaging process. This results in adjacent scan lines in aerial linear array images not necessarily being continuous targets. Therefore, conventional inverse resolving image correction has theoretical flaws for aerial linear array images, leading to significant loss of accuracy in the correction process.

[0007] To address this, the present invention proposes a forward image correction method for an airborne three-line array camera. The original image is directly projected onto the target image according to the imaging equation. The grayscale of the original pixels on the target image is then weighted and sub-pixel-divided, and the target image is recombined to finally achieve forward image correction of the line array image. Summary of the Invention

[0008] This invention primarily addresses the problem of significant image accuracy loss when using conventional inverse kinematics methods to correct images acquired by an airborne three-line array camera.

[0009] The present invention provides a forward correction method for images from an aerial three-line array camera. The original image is directly projected onto the target image according to the imaging equation. The grayscale of the original pixels on the target image is weighted and split. The whole pixel is split into sub-pixels and then recombined to form the target image, thus achieving forward correction of the line array image.

[0010] Moreover, the implementation process includes the following steps:

[0011] Step 1: First, divide the large linear array image to be corrected into logical image blocks, and then allocate two-dimensional memory space of floating point type and short integer type according to the logical image blocks;

[0012] Step 2: Using the interior and exterior orientation elements of the linear array image, each pixel of the current logical image block is directly projected into the target space through the imaging collinearity equation, and the corresponding target image coordinates are calculated from the image point coordinates of the original image pixels.

[0013] Step 3: Based on the coordinates projected onto the target image obtained in Step 2, the integer pixel is split into sub-pixels and then the target image is recombined. This includes extracting the weights of the four pixels at each target location on the target image; decomposing the grayscale value of the original pixel into four pixels according to the weights, and accumulating the assigned grayscale and weights into the corresponding variables of the target pixel; after projecting all pixels, normalization is performed based on the grayscale value and weights obtained from the target pixel to form the target image; the target image is detected, and if the weight is found to be less than a preset threshold, the average value of the neighboring pixels is used to compensate for the weight of the current pixel.

[0014] Step 4: Repeat steps 2 and 3 for the other logical image blocks of the large linear array image to be corrected until all logical image blocks are corrected and the final corrected image is obtained.

[0015] Furthermore, after allocating two-dimensional memory spaces of floating-point and short integer types in step 1, the floating-point memory space is used to store the cumulative weight of each pixel in subsequent processing, while the short integer memory space is used to store the grayscale information of logical image blocks.

[0016] Furthermore, in step 2, when the target space is directly projected using interior and exterior orientation elements, the forward method of direct projection is adopted to avoid the errors and time consumption introduced by the iterative calculation of the inverse method of linear array images.

[0017] Furthermore, in step 3, the weights of the four adjacent pixels of each target location in the target image are extracted using the following formula.

[0018] c = (int)x

[0019] r = (int)y

[0020] dx=xc

[0021] dy = yr

[0022] q00=(1-dx)×(1-dy)

[0023] q01=(1-dx)×dy

[0024] q10 = dx × (1 - dy)

[0025] q11=dx×dy

[0026] In the formula, (int) represents the rounding operation of the floating-point number; c, r are the number of rows and columns of the target pixel; x, y are the accurate coordinates of the target pixel; dx, dy are the differences between the accurate coordinates and the number of rows and columns, representing the sub-pixel part; q00, q01, q10, q11 are the 4-neighborhood weights of the target pixel.

[0027] Moreover, the preset threshold is 0.15.

[0028] On the other hand, the present invention also provides a forward resolution correction system for aerial three-line array camera images, used to implement the forward resolution correction method for aerial three-line array camera images as described above.

[0029] Moreover, it includes the following modules,

[0030] The first module is used to first divide the large linear array image to be corrected into logical image blocks, and then allocate two-dimensional memory space of floating point and short integer according to the logical image blocks.

[0031] The second module is used to use the interior and exterior orientation elements of the linear array image to directly project each pixel of the current logical image block into the target space through the imaging collinearity equation, and calculate the corresponding target image coordinates from the image point coordinates of the original image pixels.

[0032] The third module is used to split integer pixels into sub-pixels and recombine the target image based on the coordinates projected onto the target image obtained from the second module. This includes extracting the weights of the four pixels at each target location on the target image; decomposing the grayscale value of the original pixel into four pixels according to the weights; accumulating the assigned grayscale and weights into the corresponding variables of the target pixel; after projecting all pixels, normalizing the grayscale value and weights obtained from the target pixel to form the target image; detecting the target image, and if the weight is found to be less than a preset threshold, using the average value of neighboring pixels to compensate for it.

[0033] The fourth module is used to repeat the operations of the second and third modules on other logical image blocks of the large linear array image to be corrected, until all logical image blocks are corrected and the final corrected image is obtained.

[0034] Alternatively, it may include a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute a forward correction method for aerial three-line array camera images as described above.

[0035] Alternatively, it may include a readable storage medium storing a computer program that, when executed, implements a forward resolution correction method for aerial three-line array camera images as described above.

[0036] This method can effectively solve the accuracy loss problem caused by the discontinuity of targets in adjacent rows in the inverse resolution method of airborne linear array images, and significantly improve the accuracy and quality of three-line array image correction. It is particularly suitable for image correction processing of airborne three-line array cameras. Attached Figure Description

[0037] Figure 1 This is a schematic diagram of the working principle of an airborne three-line array camera in the prior art. The left figure shows the three-line array imaging principle, and the right figure shows the three-view intersection geometric positioning principle.

[0038] Figure 2 This is a schematic diagram of the discontinuity phenomenon in image rows acquired by an airborne three-line array in the prior art. The left image shows the discontinuity effect of the ground projection of the image rows, and the right image shows the discontinuity phenomenon manifested as distortion in the image.

[0039] Figure 3 This is a schematic diagram of the correction effect of an airborne three-line array image in the prior art, where the left image is the original uncorrected image and the right image is the corrected image.

[0040] Figure 4This is a flowchart illustrating the processing of an embodiment of the present invention.

[0041] Figure 5 This is a schematic diagram illustrating the target pixel splitting principle of an embodiment of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0043] This invention proposes a forward resolution correction method for images from an airborne three-line array camera. The original image is directly projected onto the target space according to the imaging collinearity equation. The grayscale of the original pixels is weighted and recombined on the target image to finally achieve forward resolution correction of the line array image.

[0044] See Figure 4 This invention provides a method for correcting the resolution of images from an aerial linear array camera. The entire processing flow is as follows: Figure 4 As shown, the target pixel segmentation algorithm is a key technology in the forward correction method during the processing, and its core processing includes the following steps:

[0045] Step 1: First, divide the large linear array image to be corrected into logical image blocks (e.g., 4096×4096, 5120×5120) according to an appropriate number of rows and columns. Then, allocate two-dimensional memory space (i.e., row × column space) of floating-point type (occupying 4 bytes, denoted as float type) and short integer type (occupying 2 bytes, denoted as short) according to the size of the logical image block, denoted by the letters S and T respectively. The floating-point type memory space is used to store the cumulative weight of each pixel in subsequent processing, and the short integer type memory space is used to store the grayscale information of the logical image block.

[0046] In step 1 of this invention, by dividing the image into logical blocks and using float and short types as intermediate data, both accuracy and memory space are guaranteed, effectively improving the practicality and robustness of the processing method and enabling it to process images of any size.

[0047] Step 2: Using the interior and exterior orientation elements of the linear array image, each pixel of the logical image block is directly projected into the target space through the imaging collinearity equation, based on the image point coordinates (p') of the original image pixels. x ,p' y Calculate the corresponding target image coordinates (x, y).

[0048] In step 2 of this invention, the image is directly projected onto the target space using interior and exterior orientation elements. This adopts the forward method of direct projection, which avoids the errors and time consumption introduced by the iterative calculation of the inverse method of linear array images.

[0049] The target image space is defined as an object space with constant elevation. Its coordinate system is usually chosen as a local tangent plane coordinate system, with the origin defined as the geometric center of the flight area, the east direction as the X-axis, the north direction as the Y-axis, and the Z-axis determined according to the right-hand coordinate system principle.

[0050] The collinearity equation for the projection of the original pixel onto the target pixel is:

[0051]

[0052]

[0053] in:

[0054] P x ,P y These are the X and Y coordinates projected onto the target space.

[0055] P z It is the Z-coordinate value of the spatial elevation plane of the target image;

[0056] (X s ,Y s Z s () represents the coordinates of the projection center at the exposure time;

[0057] f is the camera focal length;

[0058] p' x ,p' y It refers to the location of the original image points;

[0059] a1, a2, a3, b1, b2, b3, c1, c2, c3 are the rotation matrices at the imaging time. Row and column elements;

[0060] Calculate the coordinates of the starting point of the corrected image in the target space coordinate system using the formula above. Project a certain number of edge pixels from both ends of the linear array image onto the ground, and find the minimum values ​​(P) of its projected coordinates in the x and y directions. x0 ,P y0 The starting point for the corrected image is determined by the focal length f, pixel size m, and relative flight altitude. The ground sampling interval gsd is then calculated as the sampling interval for the corrected image.

[0061] gsd=m×(P z -Z s ) / f

[0062] Based on this, the target location P is calculated. x ,P y Coordinates on the target image:

[0063] x=(P x-P x0 ) / gsd

[0064] y = (P y -P y0 ) / gsd

[0065] Step 3: Based on the coordinates projected onto the target image obtained in Step 2, the whole pixel is split into sub-pixels, and then the target image is reassembled.

[0066] The coordinates x and y projected onto the target image will not all be integers, therefore it is necessary to split integer pixels into sub-pixels. The process and formula proposed in this embodiment of the invention are as follows:

[0067] (1) Extract the weights of the four adjacent pixels of each target location in the target image. The weight calculation formula is as follows:

[0068] c = (int)x

[0069] r = (int)y

[0070] dx=xc

[0071] dy = yr

[0072] q00=(1-dx)×(1-dy)

[0073] q01=(1-dx)×dy

[0074] q10 = dx × (1 - dy)

[0075] q11=dx×dy

[0076] In the formula, (int) represents the integer part of the floating-point number; c and r are the number of rows and columns of the target pixel (integers); x and y are the exact coordinates of the target pixel (floating-point numbers); dx and dy are the differences between the exact coordinates and the number of rows and columns, i.e., the sub-pixel part; q00, q01, q10, and q11 are the 4-neighborhood weights of the target pixel. The meanings of each parameter are as follows: Figure 5 As shown;

[0077] (2) Accumulate q00, q01, q10, and q11 into the corresponding positions of the target pixel:

[0078] S[r,c]+=q00

[0079] S[r,c+1]+=q01

[0080] S[r+1,c]+=q10

[0081] S[r+1,c+1]+=q11

[0082] Where S[i,j] represents the j-th element in the i-th row of S;

[0083] And according to the corresponding weights, the gray value I of the original pixel p is... p The decomposition is accumulated into 4 pixels:

[0084] T[r,c]+=q00×I p

[0085] T[r,c+1]+=q01×I p

[0086] T[r+1,c]+=q10×I p

[0087] T[r+1,c+1]+=q11×I p

[0088] Where T[i,j] represents the j-th element in the i-th row of T, I p This represents the original pixel grayscale value;

[0089] (3) After projecting all pixels, normalization is performed based on the grayscale values ​​and weights of the target pixels to form the target image. Specifically, for each target pixel, the following operations are performed:

[0090] T[r,c]=T[r,c] / S[r,c]

[0091] (4) For each pixel in the target image, if the weight is found to be less than the threshold, the average value of the neighboring pixels is used to fill the pixel. More specifically, starting from the pixel that needs to be filled, the search is expanded outward to each neighboring pixel, and the average gray value of the four or eight neighboring pixels is taken.

[0092] In this example, the threshold is recommended to be 0.15 based on experience, but it can be adjusted according to the actual situation during implementation.

[0093] In step 3 of this invention, a detailed splitting process and corresponding calculation formula are proposed. The calculation formula and theoretical derivation here come from the inverse operation of bilinear interpolation, and also contain a lot of experimental experience. This is the core and originality of this method.

[0094] Step 4: Repeat steps 2 and 3 for all logical image blocks of the linear array image to be corrected until all logical image block data is corrected and the final corrected image is obtained.

[0095] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0096] In some possible embodiments, a forward correction system for aerial three-line array camera images is provided, including the following modules: a first module is used to first divide the large linear array image to be corrected into logical image blocks, and then allocate two-dimensional memory space of floating point type and short integer type according to the logical image blocks;

[0097] The second module is used to use the interior and exterior orientation elements of the linear array image to directly project each pixel of the current logical image block into the target space through the imaging collinearity equation, and calculate the corresponding target image coordinates from the image point coordinates of the original image pixels.

[0098] The third module is used to split integer pixels into sub-pixels and recombine the target image based on the coordinates projected onto the target image obtained from the second module. This includes extracting the weights of the four pixels at each target location on the target image; decomposing the grayscale value of the original pixel into four pixels according to the weights; accumulating the assigned grayscale and weights into the corresponding variables of the target pixel; after projecting all pixels, normalizing the grayscale value and weights obtained from the target pixel to form the target image; detecting the target image, and if the weight is found to be less than a preset threshold, using the average value of neighboring pixels to compensate for it.

[0099] The fourth module is used to repeat the operations of the second and third modules on other logical image blocks of the large linear array image to be corrected, until all logical image blocks are corrected and the final corrected image is obtained.

[0100] In some possible embodiments, a forward resolution correction system for aerial three-line array camera images is provided, including a processor and a memory. The memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute a forward resolution correction method for aerial three-line array camera images as described above.

[0101] In some possible embodiments, a forward resolution correction system for aerial three-line array camera images is provided, including a readable storage medium storing a computer program, which, when executed, implements a forward resolution correction method for aerial three-line array camera images as described above.

[0102] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for correcting the forward resolution of images from an aerial three-line array camera, characterized in that: According to the imaging equation, the original image is directly projected onto the target image. The grayscale of the original pixels on the target image is weighted and split. The whole pixel is split into sub-pixels and then recombined to form the target image, thus realizing the forward correction of the linear array image. Implementation process Includes the following steps, Step 1: First, divide the large linear array image to be corrected into logical image blocks, and then allocate two-dimensional memory space of floating point type and short integer type according to the logical image blocks; Step 2: Using the interior and exterior orientation elements of the linear array image, each pixel of the current logical image block is directly projected into the target space through the imaging collinearity equation, and the corresponding target image coordinates are calculated from the image point coordinates of the original image pixels. Step 3: Based on the coordinates projected onto the target image obtained in Step 2, the integer pixel is split into sub-pixels and then the target image is recombined. This includes extracting the weights of the four pixels at each target location on the target image; decomposing the grayscale value of the original pixel into four pixels according to the weights, and accumulating the assigned grayscale and weights into the corresponding variables of the target pixel; after projecting all pixels, normalization is performed based on the grayscale value and weights obtained from the target pixel to form the target image; the target image is detected, and if the weight is found to be less than a preset threshold, the average value of the neighboring pixels is used to compensate for the weight of the current pixel. The weights of the four neighboring pixels of each target location in the target image are extracted using the following formula. c=(int)x r=(int)y dx = x - c dy = y - r q00 = (1-dx)×(1-dy) q01 = (1-dx)×dy q10 = dx × (1 - dy) q11 = dx × dy In the formula, (int) represents the rounding operation of floating-point numbers; c, r are the number of rows and columns of the target pixel; x, y are the accurate coordinates of the target pixel; dx, dy are the differences between the accurate coordinates and the number of rows and columns, representing the sub-pixel part; q00, q01, q10, q11 are the 4-neighborhood weights of the target pixel. Step 4: Repeat steps 2 and 3 for the other logical image blocks of the large linear array image to be corrected until all logical image blocks are corrected and the final corrected image is obtained.

2. The forward resolution correction method for aerial three-line array camera images as described in claim 1, characterized in that: After allocating two-dimensional memory spaces for floating-point and short integer types in step 1, the floating-point memory space is used to store the cumulative weight of each pixel in subsequent processing, while the short integer memory space is used to store the grayscale information of the logical image block.

3. The forward resolution correction method for aerial three-line array camera images as described in claim 1, characterized in that: In step 2, when the target space is directly projected using interior and exterior orientation elements, the forward method of direct projection is adopted to avoid the errors and time consumption introduced by the iterative calculation of the inverse method of linear array images.

4. The forward resolution correction method for aerial three-line array camera images as described in claim 1, 2, or 3, characterized in that: The preset threshold is 0.

15.

5. A forward resolution correction system for images from an airborne three-line array camera, characterized in that: This method is used to implement the forward resolution correction method for aerial three-line array camera images as described in any one of claims 1-4.

6. The forward resolution correction system for aerial three-line array camera images according to claim 5, characterized in that: Includes the following modules, The first module is used to first divide the large linear array image to be corrected into logical image blocks, and then allocate two-dimensional memory space of floating point and short integer according to the logical image blocks. The second module is used to use the interior and exterior orientation elements of the linear array image to directly project each pixel of the current logical image block into the target space through the imaging collinearity equation, and calculate the corresponding target image coordinates from the image point coordinates of the original image pixels. The third module is used to split integer pixels into sub-pixels and recombine the target image based on the coordinates projected onto the target image obtained from the second module. This includes extracting the weights of the four pixels at each target location on the target image; decomposing the grayscale value of the original pixel into four pixels according to the weights; accumulating the assigned grayscale and weights into the corresponding variables of the target pixel; after projecting all pixels, normalizing the grayscale value and weights obtained from the target pixel to form the target image; detecting the target image, and if the weight is found to be less than a preset threshold, using the average value of neighboring pixels to compensate for it. The fourth module is used to repeat the operations of the second and third modules on other logical image blocks of the large linear array image to be corrected until all logical image blocks have been corrected and the final corrected image is obtained.

7. An electronic device, characterized in that: It includes a processor and a memory, the memory being used to store program instructions, and the processor being used to call the stored instructions in the memory to execute the positive resolution correction method for aerial three-line array camera images as described in any one of claims 1-4.

8. A readable storage medium, characterized in that: The readable storage medium stores a computer program, which, when executed, implements a forward resolution correction method for aerial three-line array camera images as described in any one of claims 1-4.

Images