A method for producing and measuring large aerial scenes

Abstract

The present invention discloses a method for producing and measuring large aerial scenes. Existing large-scale stereoscopic scene methods do not systematically propose issues that need to be considered for stereoscopic measurement, such as upper and lower parallax elimination and the convenience of the method for generating stitching lines. The present invention first obtains aerial remote sensing data, performs aerial triangulation, and obtains attitude POS data; generates low-resolution DSM data; performs uniform color and light processing on the aerial image data; uses digital differential correction to generate geographic orthophotos of each image; spatially searches for the best matching pair, and performs stitching line editing on the upper and lower images separately; generates a large scene left image through image differential correction, introduces a parallax function, and generates a large scene right image; performs stereoscopic measurement to obtain a geographic information database. The present invention greatly reduces the time for obtaining geographic three-dimensional information, is more versatile, and has higher accuracy, and the accuracy of stereoscopic measurement is not affected by DSM.

Description

Technical Field

[0001] The invention belongs to the technical field of surveying and mapping, and in particular relates to a method for producing and measuring a large aerial scene. Background Art

[0002] The three-dimensional reconstruction and measurement technology of real scenes has always been one of the hot technologies in the research of basic disciplines such as surveying and mapping, geological exploration, and computer vision. The real-scene perception data collected and reconstructed has been promoting the development of higher-level technology applications in civil and military fields, bringing huge economic and social value.

[0003] The traditional methods of acquiring large three-dimensional scenes mainly include: 1) using the stereo core line image pair switching display mode; 2) using the three-dimensional texture mode; 3) using the DSM+DOM 2.5-dimensional mode. However, the stereo core line mode is limited by the inherent deficiency of small image size. When viewing the stereo scene of the survey area, there are a lot of stereo double-film switching processes, which cannot form a seamless large-scene roaming effect; the scene using the three-dimensional texture mode has a wide range of applications. Thanks to the support of PagedLOD technology, it can achieve the real scene display effect of super-large scenes, but this method is easily affected by factors such as image quality, imaging environment, matching accuracy, weak texture, etc., which makes the three-dimensional real scene have serious problems such as texture distortion and geometric deformation, which affects the interpretation of the real scene and is not suitable for geological surveys and current status surveys. In addition, it requires a lot of reconstruction operations; the 2.5-dimensional method can only display the terrain undulations in the vertical direction of the elevation, and cannot reflect the three-dimensional form of the real scene.

[0004] [Patent CN200310111231.1] proposed a measurable seamless spatial stereo generation method using digital stereo orthophoto mosaics, which introduced parallax using nonlinear projection, but did not consider the correction of up and down parallax. Stereo imaging using their own baseline parameters was prone to spatial dislocation, and the stereo stitching line generation method was poor in operability; [Patent CN201010114571x] replaced the forward intersection mode of [Patent CN200310111230.7] by performing position correction based on the tilted beam, but this method could not completely correct the influence of up and down parallax in real situations, and the accuracy was lost compared to the forward intersection mode of the original film. None of the existing methods systematically proposed the issues that need to be considered in stereo measurement, such as the elimination of up and down parallax, camera distortion correction, and the convenience of the stitching line generation method. Summary of the invention

[0005] In order to make up for the shortcomings of the existing technology, the present invention provides a method for producing and measuring large-scale aviation scenes, unifying the data of large aviation aircraft and unmanned aerial vehicles, and making up for the problems omitted by the existing large-scale scene generation algorithm and the problems of system incompleteness.

[0006] In order to achieve the above object, the technical solution adopted by the present invention is:

[0007] 1. A method for producing and measuring large aerial scenes, comprising the following steps:

[0008] Step 1: Obtain aerial remote sensing data according to needs;

[0009] Step 2: Obtain the POS data of each image through aerial triangulation of aerial images;

[0010] Step 3: Generate low-resolution DSM data;

[0011] Step 4: Perform color and light uniformity processing on the aerial image data;

[0012] Step 5: Generate the geographic orthophoto of each image using digital differential correction based on DSM data and attitude POS data;

[0013] Step 6: Based on the POS data of each image, spatial search is performed to find the best matching pair, and the spatial overlap area of ​​each pair of stereo images is recorded;

[0014] Step 7: Perform stitching line editing on the upper image and the lower image separately, and restrict the editing work to the overlapping area of ​​the respective stereo images;

[0015] Step 8: Count the baseline direction of each image pair. If the difference with the specified reference direction is ≥90°, swap the roles of the left image and the right image.

[0016] Step 9: According to the upper layer image, set the ground resolution and generate the large scene left image through image differential correction;

[0017] Step 10: Introduce the disparity function, correct the two-dimensional disparity of each stereo image pair to the geographic coordinates, and participate in the digital differential correction process of the lower right image, that is, generate the large scene right image;

[0018] Step 11: Perform stereo measurement on the generated large scene left image and large scene right image to obtain a geographic information database.

[0019] Specifically, in step 10, the disparity function is:

[0020]

[0021] Where Q is the parallax, Z is 0 is the average image photography height value, H is the average flight altitude, Z is the ground height, and B is the average baseline length or reference length.

[0022] Beneficial effects of the present invention:

[0023] 1) The present invention realizes an efficient large-scene stereo 3D generation method. Compared with the existing large-scene stereo generation method, the present invention is more versatile, more accurate, and more operable, and the stereo measurement accuracy is not affected by DSM.

[0024] 2) The present invention greatly shortens the time for obtaining geographic three-dimensional information and reduces the cost investment in the application field;

[0025] 3) The present invention is not affected by the traditional up-and-down parallax, and the measurement accuracy is equivalent to the forward intersection accuracy of the original image. Therefore, there is no need to set up a separate stereoscopic large scene data acquisition specification in the data production specification, and the aerial triangulation accuracy is consistent with the ordinary aerial mapping specification;

[0026] 4) The method of the present invention is directly based on a unified and open aerial triangulation data protocol, which is compatible with aerial image data of various products. Then, two-dimensional parallax is introduced, and the stitching line generation process of the stereo image pair is more easy to operate. At the same time, it is not affected by the residual upper and lower parallax. The accuracy of stereo measurement is equivalent to the effect of the forward intersection of the original image, and the measurement process from the same-name point to the three-dimensional object point is completely efficient, automated and completed in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is the operation flow chart of the present invention;

[0028] Figure 2 It is the left and right incompatible schematic diagram;

[0029] Figure 3 It is a three-dimensional stitching line diagram;

[0030] Figure 4 This is the principle diagram of stereo mapping. DETAILED DESCRIPTION

[0031] The present invention is described in detail below in conjunction with specific implementation modes.

[0032] like Figure 1 As shown, the present invention comprises the following steps:

[0033] Step 1: Acquire aerial remote sensing data according to needs. When acquiring data, the relevant mapping specifications must be followed to meet the requirements of 2-degree overlap in heading and 3-degree overlap in lateral direction within the coverage area of ​​the survey area.

[0034] Step 2: Obtain the POS data of each image through aerial triangulation, that is, the exterior orientation element M = [RT] 3×4 and the internal orientation element K 3×3 , as well as the sensor distortion parameter D and platform position correction parameter S = [R`T`] 3×4, forming the mapping relationship between the object space X and the image space x in formula 1, where the column vector is homogenized.

[0035] x=DSKMX=PX (Formula 1)

[0036] Step 3: Generate low-resolution DSM data, which is mainly used to intersect with the real stereo beam survey line within the survey area;

[0037] Step 4: Perform color and light uniformity processing on the aerial image data to make the overall tone and color of the overlapping parts consistent;

[0038] Step 5: Generate the geographic orthophoto of each image using digital differential correction based on DSM data and attitude POS data, which is the indirect process of formula 1;

[0039] Step 6: Based on the POS data of each image, spatial search is performed to find the best matching pair. The initial spatial matching pair can be formed according to the overlapping area and distance parameters. Then, the stereo matching image pair is placed with the left image on the upper layer and the right image on the lower layer, such as Figure 2 As shown. In addition, in the initial state, it is constrained that any image cannot appear in the upper and lower layer images at the same time. And the spatial overlap area of ​​each pair of stereo images is recorded.

[0040] Step 7: Based on the result of step 6, the upper image can be edited separately, and the editing work is restricted to the overlapping area of ​​the respective stereo images, such as Figure 3 As shown, each region corresponds to a stereo pair;

[0041] Similarly, the stitching lines of the lower-layer images can also be edited separately; the stitching lines between stereo image pairs do not have to meet constraints such as obstacle avoidance at the same time, which simplifies the difficulty of real large-scale scene operations;

[0042] Step 8: Count the baseline direction of each image pair. If the difference with the specified reference direction is ≥90°, swap the roles of the left image and the right image.

[0043] Step 9: According to the upper layer image, set the ground resolution and generate the large scene left image through image differential correction;

[0044] When performing digital differential correction at the stitching line, it is necessary to cover the transition area of ​​about 1 pixel more for mosaicking and floating-point processing of different image sources;

[0045] Step 10: Introduce the disparity function, correct the two-dimensional disparity of each stereo image pair to the geographic coordinates, and participate in the digital differential correction process of the lower right image, that is, generate the large scene right image;

[0046] 1) The generation of the right image of the large scene needs to add a disparity function to achieve a three-dimensional effect;

[0047] The disparity function is shown in Formula 2, where Q is the disparity, Z 0 is the average image photography height value, H is the average flight altitude, Z is the ground height, and B is the average baseline length or the reference length. Unifying the baseline length, flight altitude, and photography height can prevent geometric dislocation of adjacent images in the overlapping area.

[0048]

[0049] 2) Correct parallax deviation.

[0050] Since the baseline direction α of each stereo pair deviates from the average baseline β direction, which causes tearing of stereo vision, Formula 3 is used to correct it and convert the original one-dimensional parallax into two-dimensional geographic displacement.

[0051]

[0052] 3) Through step 2), the two-dimensional parallax is corrected to the geographic coordinates, as shown in formula 4, and together with the digital differential correction process of the lower right image, a large scene right image is generated; the two-dimensional geographic parallax needs to be processed homogeneously.

[0053]

[0054] Since large scenes need to synthesize large-format images, the stereoscopic loading process is limited by hardware, so it is necessary to build a pyramid for it to facilitate the real-time tile call of the drawing window. In addition, in the process of mapping, the left and right images of the stereoscopic large scene need to be rotated by an angle of β to make the visual baseline consistent with the average baseline direction.

[0055] Step 11: Perform stereo measurement on the generated large scene left image and large scene right image to obtain a geographic information database.

[0056] In stereo mapping operations, the large scene right image can be displaced, such as Figure 4 As shown, the best stereo vision is formed, and the coordinates of the three-dimensional points are automatically returned by puncturing the same-name points with the cursor. The essence of stereo measurement is the intersection of the same-name points in front, but through clever design, real-time measurement is achieved without adding additional storage. The specific stereo measurement formula is shown in Formula 5. Among them, xL and xR are the same-name points of the left and right images, Z is the elevation component of X, and X has three unknown values ​​to be determined. Four valid equations can be solved. Formula 5 implies characteristics that are not related to DSM. The effect of original artificial parallax is eliminated.

[0057]

[0058] Example:

[0059] The number of images covering a certain survey area is 69, with a resolution of 2 cm. The survey area is distributed in strips, with flat terrain and an average photography height of 100 meters. The method of the present invention is used to generate a large scene stereo model display on the left and right, and to perform measurements. The plane accuracy can reach 2 ground resolutions, and the elevation accuracy can reach 3 ground resolutions. However, in the process of generating large scenes, in addition to the aerial triangulation solution, the most computing-intensive part is the digital differential correction, and the upper limit of the overall time consumption is 2 times the DOM production time. However, compared with the computing power consumed by three-dimensional reconstruction, the time cost required for three-dimensional perception provided by this application is significantly reduced.

[0060] The content of the present invention is not limited to the embodiments listed, and any equivalent changes made to the technical solution of the present invention by ordinary technicians in this field after reading the description of the present invention are covered by the claims of the present invention.

Claims

1. A method for producing and measuring large aerial scenes, characterized by: The steps include: Step 1: Obtain aerial remote sensing data according to needs; Step 2: Obtain the POS data of each image through aerial triangulation of aerial images; Step 3: Generate low-resolution DSM data; Step 4: Perform color and light uniformity processing on the aerial image data; Step 5: Generate the geographic orthophoto of each image using digital differential correction based on DSM data and attitude POS data; Step 6: Based on the POS data of each image, spatial search is performed to find the best matching pair, and the spatial overlap area of ​​each pair of stereo images is recorded; Step 7: Perform stitching line editing on the upper image and the lower image separately, and restrict the editing work to the overlapping area of ​​the respective stereo images; Step 8: Count the baseline direction of each image pair. If the difference with the specified reference direction is ≥90°, swap the roles of the left image and the right image. Step 9: According to the upper layer image, set the ground resolution and generate the large scene left image through image differential correction; Step 10: Introduce the disparity function, correct the two-dimensional disparity of each stereo image pair to the geographic coordinates, and participate in the digital differential correction process of the lower right image, that is, generate the large scene right image; Step 10 is as follows: 1) The generation of the large scene right image needs to add a disparity function to achieve a stereoscopic effect. The disparity function is shown in Formula 1; Q is parallax, Z0 is the average image photography altitude, H is the average flight altitude, Z is the ground altitude, and B is the average baseline length or reference length; 2) Correction of parallax deviation Use formula 3 to correct and convert the original one-dimensional parallax into two-dimensional geographic displacement; α is the baseline direction of the stereo pair, and β is the average baseline of the stereo pair; 3) Through step 2), the two-dimensional parallax is corrected to the geographic coordinates, as shown in formula 4, and together with the digital differential correction process of the lower right image, a large scene right image is generated, in which the two-dimensional geographic parallax needs to be processed homogeneously; Step 11: Perform stereo measurement on the generated large scene left image and large scene right image to obtain a geographic information database; The specific stereo measurement formula is shown in Formula 5, where xL and xR are the same-name points of the left and right images, Z is the elevation component of X, and X is a value with three unknown quantities to be determined. Four valid equations can be solved:

Images