Internal view field splicing method and system for high-orbit multi-area-array camera with consistent object space

Through the consistent rational function model of objects and squares, the misalignment problem in the imaging data splicing of high-orbit multi-faceted array cameras is solved, and high-precision seamless splicing is achieved, simplified computing complexity, and meet the standardized production requirements of high-orbit satellite image products. .

CN118898541APending Publication Date: 2024-11-05HUBEI UNIV OF TECH
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410943739.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-15
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

Due to the misalignment between adjacent sub-surface array images, it is difficult to directly form a seamless complete image. In addition, the traditional multi-line array field splicing method is complex in calculations, making it difficult to meet the requirements of high-precision splicing.

Method used

A rational function model based on object-to-parameter consistency is used instead of the rigorous geometric imaging model. By establishing the object-to-parameter positioning consistency between the styling image and the original image, image point coordinate mapping is realized, and styling images are generated through grayscale resampling.

Benefits of technology

The calculation complexity of field of view splicing processing in high-orbit multi-faceted arrays is simplified, and high-precision seamless splicing of multi-faceted array images is realized, reducing the dependence on attitude, track data processing and coordinate system conversion.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118898541A_ABST
    Figure CN118898541A_ABST
Patent Text Reader

Abstract

The invention provides an object space consistent high-orbit multi-area-array camera internal view field splicing method and system. According to the method, a rational function model of an original sub-area array image acquired by a multi-area array camera and a complete spliced image covering a full view field is utilized, and a coordinate mapping relation from a spliced image point to an original image point is established based on an object space positioning consistency principle between the spliced image and the original image; and finally, carrying out original image point gray scale resampling to obtain a spliced image point gray scale value so as to realize the generation of a multi-area-array spliced image. According to the method, the geometric imaging characteristics of the high-orbit multi-area-array camera are combined, a strict geometric relationship exists between the original image and the spliced image, a rational function model is used for replacing a traditional strict geometric model, attitude and orbit data processing and conversion of various coordinate systems required by strict geometric model construction are avoided in the coordinate mapping process, and the method is suitable for the high-orbit multi-area-array camera. And the calculation complexity of field-of-view splicing processing in the high-orbit multi-area array can be simplified.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing satellite image processing, and in particular relates to a method and system for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space. Background Art

[0002] High-orbit satellites operate in geosynchronous orbits 36,000 kilometers from the earth. They are relatively fixed in their observation positions for the earth. They have the characteristics of high temporal resolution and wide observation range, and are an important part of aerospace earth observation imaging technology. In order to take into account both high spatial resolution and large imaging field of view, high-resolution high-orbit optical remote sensing satellite cameras are equipped with multiple array detectors, which make up for the shortcomings of the observation range of a single array through multi-array stitching. Since the original sub-array images obtained by the multi-array are recorded separately for each array, the sub-array images are independent and not convenient for subsequent applications; and affected by factors such as the focal position of the sensor and the deformation of the detector, the overlapping areas between adjacent sub-array images show a certain degree of misalignment, and it is difficult for the original sub-array images to directly form a seamless image covering the entire field of view of the camera. In order to meet the standardized production requirements of subsequent high-orbit satellite imaging products, it is necessary to perform high-precision stitching processing on the imaging data of high-orbit multi-array cameras to obtain seamless stitching of complete images.

[0003] In the ground processing system of remote sensing satellite data, the internal field of view stitching of multiple original sub-images is usually achieved based on the strict geometric relationship between the original sub-images and the stitched images. The high-orbit multi-array camera adopts a frame-type imaging method, which is essentially different from the line-by-line push-scan imaging method of the traditional multi-line array camera. The difference in imaging characteristics leads to application defects in the multi-line array internal field of view stitching model and method. In order to eliminate the impact of platform vibration on the geometric accuracy of the stitched image, the internal field of view stitching of the multi-line array camera will establish a geometric imaging model under steady-state conditions to replace the original non-steady-state push-scan imaging conditions. The mapping of the original image and the stitched image points depends on a strict geometric model. The strict geometric model requires auxiliary data such as attitude, orbit, and flight time measured on the satellite imaging system and the conversion of multiple coordinate systems, which has high computational complexity. However, the multi-array image of the frame-type imaging will not be affected by the platform vibration. If the internal field of view stitching method of the traditional multi-line array camera is directly used, the complexity of the internal field of view stitching processing will increase. Therefore, according to the characteristics of high-orbit multi-faceted array camera imaging, the present invention proposes an internal field of view stitching method based on object space consistency, replaces the strict geometric imaging model with a universal rational function model (RFM), and establishes a coordinate mapping relationship between the image points of the stitched image and the original image based on the consistency of the object space positioning of the same-name image points. The grayscale value of the stitched image point is obtained by resampling the grayscale of the original image point, thereby realizing the generation of multi-faceted array stitching images. Summary of the invention

[0004] In order to solve the problem of stitching the internal field of view of a high-orbit multi-array camera, the present invention proposes a method and system for stitching the internal field of view of a high-orbit multi-array camera with consistent object space.

[0005] The technical solution of the method of the present invention is a method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space, comprising the following steps:

[0006] Step 1: Generate a rational function model of the multi-array image based on the rigorous geometric model of the constructed multi-array original image.

[0007] Step 2: Based on the strict geometric model parameters of the multi-faceted original image, solve the strict geometric model of the stitched image and generate a rational function model of the stitched image.

[0008] Step 3: Use the geometric imaging model of the stitched image to orthographically project its image points onto the object plane to obtain the corresponding object coordinates.

[0009] Step 4: Using the geometric imaging model of the original sub-image, back-project the object coordinates to the image points of the original sub-image to obtain the image point coordinates of the stitched image on the original sub-image, thereby establishing a mapping relationship between the image point coordinates of the stitched image and the original image.

[0010] Step 5: Interpolate the grayscale of the original image to obtain the grayscale value of the image points in the stitched image, and traverse pixel by pixel to generate the stitched image.

[0011] Furthermore, the specific implementation of step 1 is as follows:

[0012] Based on the internal and external orientation model parameters calibrated by on-orbit geometric calibration and the attitude and orbit data at the imaging time, a rigorous imaging geometric model of the multi-faceted array is established, and then the imaging correspondence between each image point on the array and the corresponding ground object point is restored.

[0013]

[0014] Where (s,l) is the image plane coordinate of the sth row and lth column of the image, tan i (ψ x (s,l)) is the optical axis direction of the detector in the sth row and lth column of the ith array in the camera coordinate system in the x direction, tan i (ψ y (s,l)) is the optical axis direction of the detector in the s-th row and l-th column of the ith array in the camera coordinate system in the x-direction, representing the internal orientation model of the array; ax0, ax1, ax2, ..., ax9 represent the 0th, 1st, 2nd, ..., 9th x-direction pointing angle model parameters respectively; ay0, ay1, ay2, ..., ay9 represent the 0th, 1st, 2nd, ..., 9th y-direction pointing angle model parameters respectively; (X i ,Y i ,Zi ) is the coordinate vector of the object point corresponding to the image point in the sth row and lth column of the i-th array in the WGS84 rectangular coordinate system, X i , Y i , Z i Respectively represent the object coordinates in the X, Y, and Z directions of the WGS84 rectangular coordinate system; (X body ,Y body ,Z body ) is the coordinate vector of the satellite position in the WGS84 rectangular coordinate system, X body , Y body , Z body They represent the satellite position coordinates in the X, Y and Z directions of the WGS84 rectangular coordinate system. Since multiple arrays are on the same satellite, the satellite position parameters of each array are the same; The rotation matrix from WGS84 coordinate system to J2000 coordinate system; is the rotation matrix from the J2000 coordinate system to the satellite body coordinate system; It represents the rotation matrix from the satellite body coordinate system to the camera coordinate system, which is the external orientation model parameter; λ is the scale factor;

[0015] The RFM model of each sub-array is obtained by using the terrain-independent rational function model parameter solution method:

[0016]

[0017] Among them, (l n ,s n ) represents the normalized coordinates of the image point in the sth row and the lth column; (U, V, W) is the object coordinate vector of the image point in the sth row and the lth column, and U, V, and W represent the normalized longitude, latitude, and altitude values, respectively; Num L 、Den L 、Num S and Den S is a cubic polynomial, Den L and Den S The constant term is 1, Num L and Den L The ratio of the object coordinates (U, V, W) to the image point column coordinates l n The conversion relationship of Num S and Den S The ratio of the object coordinates (U, V, W) to the image point row coordinates s n conversion relationship.

[0018] Furthermore, the specific implementation of step 2 is as follows:

[0019] According to the internal orientation model of each sub-array, the optical axis orientation of the four corner image points of the original sub-image of the multi-array is calculated, and the coverage range of the multi-array in the camera coordinate system is determined as the range of the stitched image. The minimum and maximum optical axis orientation values ​​in the x direction are set as minψ x and maxψ x , the minimum and maximum optical axis pointing values ​​in the y direction are minψ y and maxψ y ;

[0020] According to the pixel size pixelSize of the multi-faceted array, the optical axis pointing range is divided into equally spaced grids, and the image points of the stitched image are matched one by one with the grid points. The number of grids in the x and y directions corresponds to the number of rows and columns of the stitched image, respectively. Based on this, the relationship between the coordinates of the image points of the stitched image and the corresponding optical axis direction can be obtained:

[0021]

[0022] Among them, (s',l') is the coordinate of the image point in the s'th row and l'th column on the stitched image, (ψ' x ,ψ' y ) is the direction of the optical axis of the image point in the s'th row and l'th column on the stitched image in the x and y directions.

[0023] The same mathematical model as the original sub-array internal orientation model is used to construct the internal orientation model of the stitched image:

[0024]

[0025] Among them, ax0', ax1', ax2', ..., ax9' represent the 0th, 1st, 2nd, ..., 9th x-direction internal orientation model parameters of the stitched image; ay0', ay1', ay2', ..., ay9' represent the 0th, 1st, 2nd, ..., 9th y-direction internal orientation model parameters of the stitched image.

[0026] The point pairs formed by the coordinates of the image points in the stitched images and the corresponding optical axis directions are used as the observation values ​​for solving the parameters of the orientation model in the stitched images, and the model parameters are solved based on the least squares adjustment method.

[0027] Finally, in the manner described in step 1, the rational function model of the stitched image is generated by combining the attitude, trajectory, exterior orientation model parameters and the interior orientation model parameters of the stitched image.

[0028] Furthermore, the specific implementation of step 3 is as follows:

[0029] The RFM model of the image establishes the relationship between the two-dimensional image point coordinates and the three-dimensional object coordinates. Solving the object coordinates from the image coordinates is called coordinate forward projection, and solving the image coordinates from the object coordinates is called coordinate back projection. When a single coordinate is forward projected, it is impossible to obtain the three-dimensional object coordinates from the two-dimensional image point coordinates. Usually, a digital elevation model is introduced as the object projection surface, and the object elevation value is interpolated from the DEM. The accurate object coordinates are obtained through step-by-step iteration.

[0030] Since the orbital altitude of high-orbit satellites reaches 36,000 kilometers, the influence of elevation error on image stitching is minimal. The traditional DEM-based coordinate projection method relies on additional DEM data, and coordinate solution is an iterative and time-consuming calculation process. Therefore, the present invention sets the object space projection surface as a plane with a constant elevation value, regards the object space elevation as a known value, and directly calculates the object space longitude and latitude corresponding to the image point coordinates according to the RFM model.

[0031] According to the geometric imaging model RFM' of the stitched image, the image point (s', l') in the s'th row and l'th column of the stitched image is projected point by point to the object plane to obtain the object coordinates (B', L', H') corresponding to the image point (s', l') of the stitched image. s'l' , B', L', H' represent longitude, latitude and elevation coordinates respectively:

[0032]

[0033] in, It represents the coordinate calculation process of the spliced ​​image geometric imaging model RFM'.

[0034] Furthermore, the specific implementation of step 4 is as follows:

[0035] According to the geometric imaging model RFM of the original sub-image of the i-th array i , calculate the object coordinates (B', L', H') obtained in step 3 s'l' The corresponding image point coordinates (s, l) on the i-th original sub-image of the array i :

[0036]

[0037] in, RFM represents the geometric imaging model of the i-th array image i The coordinate inversion process.

[0038] Thus, we can get the stitched image point (s', l') to the original sub-image point (s, l) i The mapping relationship is:

[0039] (s',l')→(s,l) i

[0040] Among them, → represents the image point (s', l') of the spliced ​​image and the image point (s, l) of the original sub-image of the i-th array i The corresponding relationship.

[0041] Furthermore, the specific implementation of step 5 is as follows:

[0042] The grayscale value of the integer pixel coordinates near the original sub-image pixel corresponding to the stitched image pixel is interpolated by bilinear interpolation or bicubic interpolation to obtain the grayscale value of the pixel. The stitched image is generated by traversing each pixel of the stitched image pixel by pixel, and the corresponding pixel grayscale value is obtained from the interpolation of the original image grayscale.

[0043] The present invention also provides a high-orbit multi-surface array camera internal field of view stitching system with consistent object space, comprising:

[0044] The multi-surface array image rational function model generation module is used to generate the rational function model of the multi-surface array image based on the strict geometric model of the multi-surface array original image.

[0045] The rational function model generation module of the stitched image is used to solve the rigorous geometric model of the stitched image based on the rigorous geometric model parameters of the multi-faceted original image and generate the rational function model of the stitched image.

[0046] The mosaic image point orthographic projection module is used to use the geometric imaging model of the mosaic image to orthographically project its image points onto the object plane to obtain the corresponding object coordinates.

[0047] The module for establishing the coordinate mapping between the stitched image and the original image pixel is used to use the geometric imaging model of the original sub-image to back-project the object coordinates to the original sub-image pixel, obtain the pixel coordinates of the stitched image pixel on the original sub-image, and establish the mapping relationship between the stitched image and the original image pixel coordinates.

[0048] The stitching image generation module is used to interpolate the grayscale of the original image, obtain the grayscale value of the stitching image point, and traverse pixel by pixel to generate the stitching image.

[0049] The present invention provides a method and system for stitching the field of view of a high-orbit multi-faceted array camera in a consistent object space. The method and system combine the geometric imaging principle of the high-orbit multi-faceted array camera to establish a strict geometric relationship between the original image and the stitched image, and realize seamless stitching of the original multi-faceted array image based on a strict theoretical basis. In addition, the present invention uses a rational function model to replace the traditional strict geometric model. In the process of mapping the coordinates of the original image and the stitched image, the method avoids the posture, orbit data processing and conversion of multiple coordinate systems required for the construction of a strict geometric model, which is conducive to simplifying the computational complexity of the stitching process of the field of view of the high-orbit multi-faceted array. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 : Flowchart of an embodiment of the present invention.

[0051] Figure 2 : Schematic diagram of the principle of focal plane position and object consistency of multi-array images and stitched images according to an embodiment of the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0054] The method of the present invention utilizes the original sub-array image data and geometric imaging model acquired by a high-orbit multi-array camera, designs a complete stitched image covering the entire field of view of the multi-array imaging, establishes a strict geometric coordinate mapping relationship between the stitched image points and the original image points based on the object space positioning consistency between the stitched image and the original image, obtains the grayscale value of the stitched image points by grayscale resampling of the original image points, and realizes the generation of the multi-array stitched image.

[0055] See also Figure 1 The embodiment provides a method for stitching the internal field of view of a high-orbit multi-faceted camera with consistent object space. The implementation process can be divided into 5 steps, which are specifically implemented as follows:

[0056] Step 1: Generate a rational function model of the multi-array image based on the rigorous geometric model of the constructed multi-array original image.

[0057] The specific implementation of step 1 is as follows:

[0058] Based on the internal and external orientation model parameters calibrated by on-orbit geometric calibration and the attitude and orbit data at the imaging time, a rigorous imaging geometric model of the multi-faceted array is established, and then the imaging correspondence between each image point on the array and the corresponding ground object point is restored.

[0059]

[0060] Where (s, l) is the image plane coordinate of the sth row and lth column of the image; i represents the array number, i=1...N, N=9 is the number of arrays; tan i (ψ x (s,l)) is the optical axis direction of the detector in the sth row and lth column of the ith array in the camera coordinate system in the x direction, tan i (ψ y (s,l)) is the optical axis direction of the detector in the s-th row and l-th column of the ith array in the camera coordinate system in the x-direction, representing the internal orientation model of the array; ax0, ax1, ax2, ..., ax9 represent the 0th, 1st, 2nd, ..., 9th x-direction pointing angle model parameters respectively; ay0, ay1, ay2, ..., ay9 represent the 0th, 1st, 2nd, ..., 9th y-direction pointing angle model parameters respectively; (X i ,Y i ,Z i ) is the coordinate vector of the object point corresponding to the image point in the sth row and lth column of the i-th array in the WGS84 rectangular coordinate system, X i , Y i , Z i Respectively represent the object coordinates in the X, Y, and Z directions of the WGS84 rectangular coordinate system; (X body ,Y body ,Z body ) is the coordinate vector of the satellite position in the WGS84 rectangular coordinate system, X body , Y body , Z body They represent the satellite position coordinates in the X, Y and Z directions of the WGS84 rectangular coordinate system. Since multiple arrays are on the same satellite, the satellite position parameters of each array are the same; The rotation matrix from WGS84 coordinate system to J2000 coordinate system; is the rotation matrix from the J2000 coordinate system to the satellite body coordinate system; It represents the rotation matrix from the satellite body coordinate system to the camera coordinate system, which is the external orientation model parameter; λ is the scale factor;

[0061] The RFM model of each sub-array is obtained by using the terrain-independent rational function model parameter solution method:

[0062]

[0063] Among them, (l n ,s n ) represents the normalized coordinates of the image point in the sth row and the lth column; (U, V, W) is the object coordinate vector of the image point in the sth row and the lth column, and U, V, and W represent the normalized longitude, latitude, and altitude values, respectively; Num L 、Den L 、NumS and Den S is a cubic polynomial, Den L and Den S The constant term is 1, Num L and Den L The ratio of the object coordinates (U, V, W) to the image point column coordinates l n The conversion relationship of Num S and Den S The ratio of the object coordinates (U, V, W) to the image point row coordinates s n conversion relationship.

[0064] Step 2: Based on the strict geometric model parameters of the multi-faceted original image, solve the strict geometric model of the stitched image and generate a rational function model of the stitched image.

[0065] The specific implementation of step 2 is as follows:

[0066] According to the internal orientation model of each sub-array, the optical axis orientation of the four corner image points of the original sub-image of the multi-array is calculated, and the coverage range of the multi-array in the camera coordinate system is determined as the range of the stitched image. The minimum and maximum optical axis orientation values ​​in the x direction are set as minψ x and maxψ x , the minimum and maximum optical axis pointing values ​​in the y direction are minψ y and maxψ y ;

[0067] According to the pixel size pixelSize of the multi-faceted array, the optical axis pointing range is divided into equally spaced grids, and the image points of the stitched image are matched one by one with the grid points. The number of grids in the x and y directions corresponds to the number of rows and columns of the stitched image, respectively. Based on this, the relationship between the coordinates of the image points of the stitched image and the corresponding optical axis direction can be obtained:

[0068]

[0069] Among them, (s',l') is the coordinate of the image point in the s'th row and l'th column on the stitched image, (ψ' x ,ψ' y ) is the direction of the optical axis of the image point in the s'th row and l'th column on the stitched image in the x and y directions.

[0070] The same mathematical model as the original sub-array internal orientation model is used to construct the internal orientation model of the stitched image:

[0071]

[0072] Among them, ax0', ax1', ax2', ..., ax9' represent the 0th, 1st, 2nd, ..., 9th x-direction internal orientation model parameters of the stitched image; ay0', ay1', ay2', ..., ay9' represent the 0th, 1st, 2nd, ..., 9th y-direction internal orientation model parameters of the stitched image.

[0073] The point pairs formed by the coordinates of the image points in the stitched images and the corresponding optical axis directions are used as the observation values ​​for solving the parameters of the orientation model in the stitched images, and the model parameters are solved based on the least squares adjustment method.

[0074] Finally, in the manner described in step 1, the rational function model of the stitched image is generated by combining the attitude, trajectory, exterior orientation model parameters and the interior orientation model parameters of the stitched image.

[0075] Step 3: Use the geometric imaging model of the stitched image to orthographically project its image points onto the object plane to obtain the corresponding object coordinates.

[0076] The specific implementation of step 3 is as follows:

[0077] The RFM model of the image establishes the relationship between the two-dimensional image point coordinates and the three-dimensional object coordinates. Solving the object coordinates from the image coordinates is called coordinate forward projection, and solving the image coordinates from the object coordinates is called coordinate back projection. When a single coordinate is forward projected, it is impossible to obtain the three-dimensional object coordinates from the two-dimensional image point coordinates. Usually, a digital elevation model is introduced as the object projection surface, and the object elevation value is interpolated from the DEM. The accurate object coordinates are obtained through step-by-step iteration.

[0078] Since the orbital altitude of high-orbit satellites reaches 36,000 kilometers, the influence of elevation error on image stitching is minimal. The traditional DEM-based coordinate projection method relies on additional DEM data, and coordinate solution is an iterative and time-consuming calculation process. Therefore, the present invention sets the object space projection surface as a plane with a constant elevation value, regards the object space elevation as a known value, and directly calculates the object space longitude and latitude corresponding to the image point coordinates according to the RFM model.

[0079] According to the above method, according to the geometric imaging model RFM' of the stitched image, the image point (s', l') in the s'th row and l'th column of the stitched image is projected point by point to the object plane to obtain the object coordinates (B', L', H') corresponding to the image point (s', l') of the stitched image. s'l' , B', L', H' represent longitude, latitude and elevation coordinates respectively:

[0080]

[0081] in, It represents the coordinate calculation process of the spliced ​​image geometric imaging model RFM'.

[0082] Step 4: Using the geometric imaging model of the original sub-image, back-project the object coordinates to the image points of the original sub-image to obtain the image point coordinates of the stitched image on the original sub-image, thereby establishing a mapping relationship between the image point coordinates of the stitched image and the original image.

[0083] The specific implementation of step 4 is as follows:

[0084] According to the geometric imaging model RFM of the original sub-image of the i-th array i , calculate the object coordinates (B', L', H') obtained in step 3 s'l' The corresponding image point coordinates (s, l) on the i-th original sub-image of the array i :

[0085]

[0086] in, RFM represents the geometric imaging model of the i-th array image i The coordinate inversion process.

[0087] Thus, we can get the stitched image point (s', l') to the original sub-image point (s, l) i The mapping relationship is:

[0088] (s',l')→(s,l) i

[0089] Among them, → represents the image point (s', l') of the spliced ​​image and the image point (s, l) of the original sub-image of the i-th array i The corresponding relationship.

[0090] Step 5: Interpolate the grayscale of the original image to obtain the grayscale value of the image points in the stitched image, and traverse pixel by pixel to generate the stitched image.

[0091] The specific implementation of step 5 is as follows:

[0092] The grayscale value of the integer pixel coordinates near the original sub-image pixel corresponding to the stitched image pixel is interpolated by bilinear interpolation or bicubic interpolation to obtain the grayscale value of the pixel. The stitched image is generated by traversing each pixel of the stitched image pixel by pixel, and the corresponding pixel grayscale value is obtained from the interpolation of the original image grayscale.

[0093] An embodiment of the system of the present invention is a high-orbit multi-surface array camera internal field of view stitching system based on object space consistency, comprising:

[0094] The multi-surface array image rational function model generation module is used to generate the rational function model of the multi-surface array image based on the strict geometric model of the multi-surface array original image.

[0095] The rational function model generation module of the stitched image is used to solve the rigorous geometric model of the stitched image based on the rigorous geometric model parameters of the multi-faceted original image and generate the rational function model of the stitched image.

[0096] The mosaic image point orthographic projection module is used to use the geometric imaging model of the mosaic image to orthographically project its image points onto the object plane to obtain the corresponding object coordinates.

[0097] The module for establishing the coordinate mapping between the stitched image and the original image pixel is used to use the geometric imaging model of the original sub-image to back-project the object coordinates to the original sub-image pixel, obtain the pixel coordinates of the stitched image pixel on the original sub-image, and establish the mapping relationship between the stitched image and the original image pixel coordinates.

[0098] The stitching image generation module is used to interpolate the grayscale of the original image, obtain the grayscale value of the stitching image point, and traverse pixel by pixel to generate the stitching image.

[0099] Using the imaging data of my country's high-orbit high-resolution optical satellite Gaofen-4, 2×2 surface array sub-images (denoted as s1, s2, s3 and s4) and the internal and external orientation model parameters of each image are simulated. According to the specific steps of the above embodiment, the rational function model of each original sub-image of the surface array and the rational function model of the spliced ​​image are generated, the coordinate mapping of the image points of the two is established, and the complete spliced ​​image is generated by grayscale interpolation. The technical effect of the present invention is verified by comparing the internal relative accuracy of each surface array image and the spliced ​​image. Table 1 lists the internal relative accuracy of each surface array sub-image and the spliced ​​image in the row and column directions. It can be seen that the internal relative accuracy of the spliced ​​image in both the row and column directions is better than 1 pixel, realizing high-precision internal field of view splicing of multiple surface array images, and the internal relative accuracy of the spliced ​​image is basically consistent with that of the surface array sub-image, indicating that the use of rational function models can effectively replace the strict geometric model and realize the establishment of a strict geometric correspondence between the surface array sub-image and the spliced ​​image.

[0100] Table 1 Internal relative accuracy of high-orbit multi-array images and stitched images

[0101]

[0102] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0103] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space, characterized in that: The following steps are involved: Step 1: Generate a rational function model of the multi-array image based on the rigorous geometric model of the constructed multi-array original image; Step 2: Based on the strict geometric model parameters of the original multi-faceted array image, solve the strict geometric model of the stitched image and generate a rational function model of the stitched image; Step 3: Use the geometric imaging model of the stitched image to orthographically project its image points onto the object plane to obtain the corresponding object coordinates; Step 4: Using the geometric imaging model of the original sub-image, the object coordinates are back-projected to the image points of the original sub-image to obtain the image point coordinates of the stitched image on the original sub-image, thereby establishing a mapping relationship between the image point coordinates of the stitched image and the original image; Step 5: Interpolate the grayscale of the original image to obtain the grayscale value of the image points in the stitched image, and traverse pixel by pixel to generate the stitched image.

2. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 1, characterized in that: The specific implementation of step 1 is as follows: Based on the internal and external orientation model parameters of the on-orbit geometric calibration and the attitude and orbit data at the imaging time, a rigorous imaging geometric model of the multi-faceted array is established, and then the imaging correspondence between each image point on the array and the corresponding ground object point is restored; Where (s, l) is the image plane coordinate of the sth row and lth column of the image, tan i (ψ x (s,l)) is the optical axis direction of the detector in the sth row and lth column of the ith array in the camera coordinate system in the x direction, tan i (ψ y (s,l)) is the optical axis direction of the detector in the s-th row and l-th column of the ith array in the camera coordinate system in the x-direction, representing the internal orientation model of the array; ax0, ax1, ax2, ..., ax9 represent the 0th, 1st, 2nd, ..., 9th x-direction pointing angle model parameters respectively; ay0, ay1, ay2, ..., ay9 represent the 0th, 1st, 2nd, ..., 9th y-direction pointing angle model parameters respectively; (X i ,Y i ,Z i ) is the coordinate vector of the object point corresponding to the image point in the sth row and lth column of the i-th array in the WGS84 rectangular coordinate system, X i , Y i , Z i Respectively represent the object coordinates in the X, Y, and Z directions of the WGS84 rectangular coordinate system; (X body ,Y body ,Z body ) is the coordinate vector of the satellite position in the WGS84 rectangular coordinate system, X body , Y body , Z body They represent the satellite position coordinates in the X, Y and Z directions of the WGS84 rectangular coordinate system. Since multiple arrays are on the same satellite, the satellite position parameters of each array are the same; The rotation matrix from WGS84 coordinate system to J2000 coordinate system; is the rotation matrix from the J2000 coordinate system to the satellite body coordinate system; Represents the rotation matrix from the satellite body coordinate system to the camera coordinate system, which is the exterior orientation model parameter; λ is the scale factor.

3. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 2, characterized in that: The RFM model of each sub-array is obtained by using the terrain-independent rational function model parameter solution method: Among them, (l n ,s n ) represents the normalized coordinates of the image point in the sth row and the lth column; (U, V, W) is the object coordinate vector of the image point in the sth row and the lth column, and U, V, and W represent the normalized longitude, latitude, and elevation values, respectively; Num L 、Den L 、Num S and Den S is a cubic polynomial, Den L and Den S The constant term is 1, Num L and Den L The ratio of the object coordinates (U, V, W) to the image point column coordinates l n The conversion relationship of Num S and Den S The ratio of the object coordinates (U, V, W) to the image point row coordinates s n conversion relationship.

4. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 3, characterized in that: The specific implementation of step 2 is as follows: According to the internal orientation model of each sub-array, the optical axis orientation of the four corner image points of the original sub-image of the multi-array is calculated, and the coverage range of the multi-array in the camera coordinate system is determined as the range of the stitched image. The minimum and maximum optical axis orientation values ​​in the x direction are set as minψ x and maxψ x , the minimum and maximum optical axis pointing values ​​in the y direction are minψ y and maxψ y ; According to the pixel size pixelSize of the multi-faceted array, the optical axis pointing range is divided into equally spaced grids, and the image points of the stitched image are matched one by one with the grid points. The number of grids in the x and y directions corresponds to the number of rows and columns of the stitched image respectively. Based on this, the relationship between the coordinates of the image points of the stitched image and the corresponding optical axis pointing direction can be obtained: Among them, (s',l') is the coordinate of the image point in the s'th row and l'th column on the stitched image, (ψ' x ,ψ' y ) is the direction of the optical axis of the image point in the s'th row and l'th column on the stitched image in the x and y directions.

5. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 4, characterized in that: The same mathematical model as the original sub-array internal orientation model is used to construct the internal orientation model of the stitched image: Among them, ax0', ax1', ax2', ..., ax9' represent the 0th, 1st, 2nd, ..., 9th x-direction internal orientation model parameters of the stitched image; ay0', ay1', ay2', ..., ay9' represent the 0th, 1st, 2nd, ..., 9th y-direction internal orientation model parameters of the stitched image.

6. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 5, characterized in that: The point pairs formed by the coordinates of the image points in the stitched images and the corresponding optical axis directions are used as the observation values ​​for solving the parameters of the orientation model in the stitched images, and the model parameters are solved based on the least squares adjustment method. In the manner described in step 1, the rational function model of the stitched image is generated by combining the attitude, trajectory, exterior orientation model parameters and the interior orientation model parameters of the stitched image.

7. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 6, characterized in that: The specific implementation of step 3 is as follows: The object space projection plane is set as a plane with a constant elevation value, the object space elevation is regarded as a known value, and the object space longitude and latitude corresponding to the image point coordinates are directly calculated according to the RFM model; According to the geometric imaging model RFM' of the stitched image, the image point (s', l') in the s'th row and l'th column of the stitched image is projected point by point to the object plane to obtain the object coordinates (B', L', H') corresponding to the image point (s', l') of the stitched image. s'l' , B', L', H' represent longitude, latitude and elevation coordinates respectively: in, It represents the coordinate calculation process of the mosaic image geometric imaging model RFM'.

8. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 7, characterized in that: The specific implementation of step 4 is as follows: According to the geometric imaging model RFM of the original sub-image of the i-th array i , calculate the object coordinates (B', L', H') obtained in step 3 s'l' The corresponding image point coordinates (s, l) on the i-th original sub-image of the array i : in, RFM represents the geometric imaging model of the i-th array image i The coordinate inverse calculation process; Thus, we can get the stitched image point (s', l') to the original sub-image point (s, l) i The mapping relationship is: (s',l')→(s,l) i Among them, → represents the image point (s', l') of the spliced ​​image and the image point (s, l) of the original sub-image of the i-th array i The corresponding relationship.

9. The method for stitching the internal field of view of a high-orbit multi-surface array camera with consistent object space according to claim 8, characterized in that: The specific implementation of step 5 is as follows: The grayscale value of the integer pixel coordinates near the original sub-image pixel corresponding to the stitched image pixel is interpolated by bilinear interpolation or bicubic interpolation to obtain the grayscale value of the pixel; each pixel of the stitched image is traversed pixel by pixel, and the corresponding pixel grayscale value is obtained from the interpolation of the original image grayscale to generate a stitched image.

10. A high-orbit multi-array camera internal field of view stitching system with consistent object space, characterized in that: include: A multi-surface array image rational function model generation module is used to generate a rational function model of a multi-surface array image based on a rigorous geometric model of the multi-surface array original image; The rational function model generation module of the stitched image is used to solve the rigorous geometric model of the stitched image based on the rigorous geometric model parameters of the original multi-faceted array image and generate the rational function model of the stitched image; The mosaic image point orthographic projection module is used to orthographically project the image points of the mosaic image onto the object plane using the geometric imaging model of the mosaic image to obtain the corresponding object coordinates; A module for establishing the coordinate mapping between the stitched image and the original image is used to use the geometric imaging model of the original sub-image to back-project the object coordinates to the image points of the original sub-image, obtain the image point coordinates of the stitched image on the original sub-image, and establish the mapping relationship between the coordinates of the stitched image and the original image; The stitching image generation module is used to interpolate the grayscale of the original image, obtain the grayscale value of the stitching image point, and traverse pixel by pixel to generate the stitching image.

Citation Information

Cited By

  • Large-area rapid geometric positioning method based on remote sensing image

    CN119784847A