Remote sensing image mosaicking method based on image quality

Abstract

The present invention discloses a remote sensing image mosaicking method based on image quality, which includes the following steps: obtaining evaluation indicators and establishing first, second, and third evaluation criteria respectively according to the evaluation indicators through weighted summation operations; screening images to be mosaicked according to the first and second evaluation standards and determining the mosaicking order to obtain a mosaicking scheme, and performing virtual mosaicking according to the mosaicking scheme to obtain a virtual synthetic image; performing an overall evaluation of multiple virtual synthetic images based on the third evaluation standard to select the optimal mosaicking scheme; and mosaicking the images to be mosaicked using image mosaicking technology according to the optimal mosaicking scheme. This method not only performs local evaluation on the individual images involved in the mosaicking, but also performs an overall evaluation on the virtual mosaicked synthetic image after virtual mosaicking, thereby ensuring that the quality of the images involved in the mosaicking is optimal and that the overall quality of the mosaicked synthetic image is at a high level.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image mosaicking, and in particular to a remote sensing image mosaicking method based on image quality. Background Art

[0002] In recent years, the development and application of remote sensing, unmanned aerial vehicle (UAV), and aerospace technologies have provided an image foundation for large-scale feature classification. Because single-view remote sensing images are typically strip-like, remote sensing image processing requires the technical process of stitching multiple views together according to mosaicking rules to obtain complete image data over a larger area. The quality of mosaicking directly impacts the usability of remote sensing images. Problems such as cracks, ghosting, and blurring can reduce the quality of remote sensing data. However, there is currently no unified standard for quantitative evaluation and analysis of image mosaicking. Because image quality is subjective, subjective evaluation has long dominated. However, human subjectivity varies greatly from person to person, and the evaluation criteria for the same image can fluctuate under different conditions. Furthermore, subjective evaluation consumes a significant amount of human resources, making it inadequate for evaluating the quality of large quantities of remote sensing images. Therefore, an objective evaluation method that aligns with subjective evaluation results is urgently needed.

[0003] Existing objective evaluation standards are based on a panoramic reference image, with which the image to be mosaicked is compared. However, obtaining panoramic reference images is difficult, which necessitates the development of appropriate mosaic image quality evaluation rules that minimize inherent image quality defects and reduce color differences and distortion between different images. Without the need for a panoramic reference image, the quality of the individual images participating in the mosaic can be evaluated to produce a mosaicking solution with a high image quality level.

[0004] Generally speaking, the types of defects that may appear in the image mosaic results are divided into four categories: (1) unclear texture; (2) blur; (3) cracks; (4) unnatural color transitions. Some of the above different types of mosaic defects are global, which will cause obvious transitions between single-view images and the mosaic traces to be too obvious; some are local, which will reduce the quality of the entire mosaicked synthetic image due to the poor quality of the image to be mosaicked. In order to ensure the mosaic quality, it is necessary to comprehensively consider the impact of local defects and global defects on the mosaicked synthetic image to ensure that the final mosaicked synthetic image area is clear, complete, and integrated, and it is not obvious that it is obtained by mosaicking multiple images. In the image mosaicking process, the selection method of the image to be mosaicked is crucial to the final mosaicking effect. The selection principles of the image to be mosaicked are: (1) clear texture information and good performance of key objects; (2) true and accurate restoration of the spectral characteristics of the multispectral image; (3) the mosaicked image can distinguish various objects by visual inspection, and the characteristics of various objects are obvious and the texture is clear. Summary of the Invention

[0005] The present invention proposes a remote sensing image mosaicking method based on image quality, which makes up for the defect that the existing mosaicking method based on image quality must obtain panoramic images as a reference. The present invention can complete the quality evaluation of the images to be mosaicked in the process of pairwise mosaicking, and mosaicking is carried out according to the quality evaluation results. At the same time, the second evaluation standard can dynamically adjust the priority of the images to be mosaicked; at the same time, it overcomes the difficulties existing in the existing image mosaicking evaluation standards and establishes a mosaic image quality evaluation method.

[0006] In order to achieve the above technical objectives, the technical solutions of the present invention are as follows:

[0007] S1 obtains evaluation indicators, and obtains a first evaluation standard, a second evaluation standard, and a third evaluation standard through a weighted summation operation method according to the evaluation indicators;

[0008] S2 evaluates the remote sensing image set based on the first evaluation criteria and the second evaluation criteria to obtain a mosaicking scheme and a virtual synthetic image of the mosaicking scheme, wherein the mosaicking scheme includes a set of images to be mosaicked and a mosaicking order;

[0009] S3 evaluates the virtual synthetic images of the mosaicking schemes based on the third evaluation criterion, and takes the mosaicking scheme corresponding to the virtual synthetic image with the best evaluation result as the optimal mosaicking scheme;

[0010] S4 completes remote sensing image mosaicking through image mosaicking technology according to the optimal mosaicking solution.

[0011] Specifically, step S2 is:

[0012] S201 obtains remote sensing image sets;

[0013] S202 initializes the mosaic scheme;

[0014] S203 scores the remote sensing images in the remote sensing image set one by one according to the first evaluation standard to obtain a first evaluation ranking result;

[0015] S204: screening the remote sensing images in the remote sensing image set according to the first evaluation ranking result to obtain an initial image to be mosaicked, adding the initial image to be mosaicked to the image set to be mosaicked, setting the initial mosaicking order to 1, and using the initial mosaicked image as an initial reference image;

[0016] S205 scores the remote sensing images in the remote sensing image set one by one according to the second evaluation standard based on the current benchmark image to obtain a second evaluation ranking result;

[0017] S206: screening the remote sensing image set according to the second evaluation ranking result to obtain a new image to be mosaicked, adding the new image to be mosaicked to the image set to be mosaicked, and increasing the current mosaicking order by one;

[0018] S207: Virtually mosaicking the current image to be mosaicked with the current reference image to obtain a current composite image as a new reference image;

[0019] S208 iteratively executes steps S205-S207 until the spatial coverage of the current synthetic image reaches a preset coverage, and the iteration ends, completing a mosaic scheme, and using the current synthetic image as a virtual synthetic image of the mosaic scheme;

[0020] S209 iterates steps S202-S208 until the number of output mosaic solutions reaches a preset threshold.

[0021] Specifically, the evaluation indicators include color difference in overlapping areas, information entropy, cloudiness, temporal similarity, and clarity. The specific calculation method of the evaluation indicators is as follows:

[0022] (1) The problem of two images overlapping. The color difference of the overlapping area is calculated by the following formula:

[0023]

[0024] Where ΔE is the total color difference of the overlapping area between the reference image and the remote sensing image, ΔL is the brightness difference of the overlapping area between the reference image and the remote sensing image, Δa is the standard red-green difference of the overlapping area between the reference image and the remote sensing image, and Δb is the standard yellow-blue difference of the overlapping area;

[0025] For the problem of overlapping three or more images, the average color difference of each image is obtained by the following formula:

[0026] The average color difference of image A is:

[0027] ΔE A =(ΔE AB ·S AB +ΔE AC ·S AC ) / (S AB +S AC );

[0028] The average color difference of image B is:

[0029] ΔE B =(ΔE AB ·S AB +ΔE BC ·S BC ) / (S AB +S BC )

[0030] The average color difference of image C is:

[0031] ΔE C =(ΔE AC ·S AC +ΔE BC ·S BC ) / (S AC +S BC )

[0032] The color difference between image A and image B is ΔE AB , the overlapping area is S AB ; The color difference between image B and image C is ΔE BC , the overlapping area is S BC ; The color difference between image A and image C is ΔE AC , the overlapping area is S AC ;

[0033] (2) Information entropy is calculated by the following formula:

[0034]

[0035] where p i is the ratio of the pixel with gray value i of the remote sensing image to the total number of pixels in the image, and L is the number of gray levels;

[0036] (3) The amount of cloud and fog is measured by the ratio of the pixels covered by cloud and fog in the remote sensing image to the total pixels of the image;

[0037] (4) The degree of temporal similarity is measured by the distance between the reference image and the remote sensing image;

[0038] (5) Clarity is measured by the following formula:

[0039] Amb(x)=F -1 (F(xn) / F(z))·zI

[0040] Among them, F -1 is the inverse Fourier transform, F is the Fourier transform, x is the remote sensing image, n is the noise, z is the standard image, is a constant, the greater the clarity of image x, I is the identity operator, is a constant.

[0041] Specifically, the first evaluation criterion is obtained through a weighted summation calculation method based on the evaluation indicators, wherein the weight coefficients of the color difference and the degree of temporal similarity in the overlapping area are 0, and the sum of the weight coefficients of information entropy, cloudiness, and clarity is 1.

[0042] Specifically, the second evaluation criterion and the third evaluation criterion are obtained by a weighted summation calculation method based on the evaluation indicators, wherein the sum of the weight coefficients of the overlapping area color difference, information entropy, cloudiness, temporal similarity, and clarity is 1.

[0043] Specifically, the image mosaic technology includes geometric mosaic, color mosaic, mosaic line generation and mosaic line fusion.

[0044] Specifically, the specific steps of the image mosaic are:

[0045] Step S4 is:

[0046] S41: matching the points at the same position in space of the overlapping positions of adjacent images to be mosaicked in the optimal mosaicking solution one by one;

[0047] S42 performs color adjustment on the mosaicked image;

[0048] S43 generates mosaic lines using an algorithm that calculates the difference in overlapping image areas, and optimizes the mosaic lines between adjacent images to find a stitching method with minimal color difference, thereby removing overlapping areas.

[0049] S44 further uses fusion technology to eliminate color differences and mosaic lines at the mosaic lines. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 A flow chart of the remote sensing image mosaic method based on image quality provided by the present invention;

[0052] Figure 2 FIG. 1 is a schematic diagram illustrating calculation of average color difference when average color differences of multiple images intersect according to an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention are within the scope of protection of the present invention.

[0054] See also Figure 1 The present invention provides a remote sensing image mosaic method based on image quality, the method comprising:

[0055] S1 obtains evaluation indicators, and obtains a first evaluation standard, a second evaluation standard, and a third evaluation standard through a weighted summation operation method based on the evaluation indicators.

[0056] The first, second, and third evaluation criteria are composed of different evaluation indicators, including overlapping area color difference, information entropy, cloud volume, and resolution. The specific calculation methods for each indicator are as follows:

[0057] (1) The color difference of the overlapping area is calculated by the following formula:

[0058]

[0059] Where ΔE is the total color difference of the overlapping area, ΔL is the brightness difference of the overlapping area, Δa is the standard red-green difference of the overlapping area, and Δb is the standard yellow-blue difference of the overlapping area. For the overlapping area, the larger ΔE is, the greater the color difference between the image to be mosaicked and the reference image is;

[0060] The color difference of the overlapping area calculated by the above formula only involves the overlap of two images. When it involves the overlap of three or more images, Figure 2 For example, there are three images A, B, and C, where the color difference between image A and image B is ΔE AB , the overlapping area is S AB ; The color difference between image B and image C is ΔE BC , the overlapping area is S BC ; The color difference between image A and image C is ΔE AC , the overlapping area is S AC The average color difference of each image is obtained by the following formula:

[0061] The average color difference of image A is:

[0062] ΔE A =(ΔE AB ·S AB +ΔE AC ·S AC ) / (S AB +S AC );

[0063] The average color difference of image B is:

[0064] ΔE B =(ΔE AB ·S AB +ΔE BC ·S BC ) / (S AB +S BC )

[0065] The average color difference of image C is:

[0066] ΔE C =(ΔE AC ·S AC +ΔE BC ·S BC ) / (S AC +S BC )

[0067] (2) Information entropy is calculated by the following formula:

[0068]

[0069] where p i is the ratio of the pixel with gray value i to the total number of pixels in the image, L is the number of gray levels, and the greater the information entropy of image x, the richer the information of the image;

[0070] (3) The amount of cloud and fog is measured by the ratio of the pixels obscured by cloud and fog to the total pixels of the image. The higher the amount of cloud and fog, the less effective image is obscured.

[0071] (4) The degree of temporal similarity is measured by the distance between the reference image and the remote sensing image;

[0072] (4) Clarity is obtained by the following formula:

[0073] Amb(x)=F -1 (F(xn) / F(z))·zI

[0074] Wherein, z is a standard image (which can be a reference standard image or inferred), H is an image blur operator, H can be expressed as the convolution of the blur convolution kernel function h and z, and the identity function mapping operator I is a constant.

[0075] The first evaluation criterion is obtained by adding the evaluation indicators with different weight coefficients, wherein the weight coefficients of the color difference and the similarity of the time phase in the overlapping area are 0, and the sum of the weight coefficients of the other evaluation indicators is 1;

[0076] The second evaluation criterion and the third evaluation criterion are both obtained by adding the evaluation indicators with different weight coefficients, and the sum of the weight coefficients of the evaluation indicators is 1;

[0077] The first evaluation standard and the second evaluation standard are local evaluations, and the evaluation objects are the images to be mosaicked; the second evaluation standard is an overall evaluation, and the evaluation object is the virtual synthetic image.

[0078] The first evaluation criterion is obtained by adding the evaluation indicators with different weight coefficients, wherein the weight coefficient of the color difference sum of the overlapping area is 0, and the sum of the weight coefficients of the remaining evaluation indicators is 1;

[0079] The second and third evaluation criteria are both obtained by adding the aforementioned evaluation indicators with different weight coefficients, with the sum of the weight coefficients of the various evaluation indicators being 1. The first and second evaluation criteria are local evaluations, with the evaluation object being the image to be mosaicked; the second evaluation criterion is an overall evaluation, with the evaluation object being the virtual synthetic image.

[0080] S2 evaluates the remote sensing image set based on the first evaluation standard and the second evaluation standard to obtain a mosaic scheme and a virtual synthetic image of the mosaic scheme, wherein the mosaic scheme includes a set of images to be mosaicked and a mosaic order.

[0081] According to the first and second evaluation criteria, the images to be mosaicked are screened and the mosaicking order is determined for virtual mosaicking to obtain the mosaicking scheme and virtual synthetic image. The specific steps are as follows:

[0082] S201 obtains a remote sensing image set, using 50 as a preset threshold;

[0083] S202 initializes the mosaic scheme;

[0084] S203 scores the remote sensing images in the remote sensing image set one by one according to the first evaluation standard to obtain a first evaluation ranking result;

[0085] S204: screening the remote sensing images in the remote sensing image set according to the first evaluation ranking result to obtain an initial image to be mosaicked, adding the initial image to be mosaicked to the image set to be mosaicked, setting the initial mosaicking order to 1, and using the initial mosaicked image as an initial reference image;

[0086] S205 scores the remote sensing images in the remote sensing image set one by one according to the second evaluation standard based on the current benchmark image to obtain a second evaluation ranking result;

[0087] S206: screening the remote sensing image set according to the second evaluation ranking result to obtain a new image to be mosaicked, adding the new image to be mosaicked to the image set to be mosaicked, and increasing the current mosaicking order by one;

[0088] S207: Virtually mosaicking the current image to be mosaicked with the current reference image to obtain a current composite image as a new reference image;

[0089] S208 iteratively executes steps S205-S207 until the spatial coverage of the current synthetic image reaches a preset coverage, and the iteration ends, completing a mosaic scheme, and using the current synthetic image as a virtual synthetic image of the mosaic scheme;

[0090] S209 iterates steps S202 to S208 until the number of output mosaic solutions reaches a preset threshold of 50.

[0091] S3 evaluates the virtual synthetic images of the mosaicking schemes based on the third evaluation criterion, and takes the mosaicking scheme corresponding to the virtual synthetic image having the best evaluation result as the optimal mosaicking scheme.

[0092] Based on the third evaluation criterion, the 50 virtual synthetic images were evaluated as a whole and the optimal mosaic scheme was selected. The weight coefficients of each evaluation indicator in the third evaluation criterion were consistent with those in the second evaluation criterion.

[0093] The third evaluation criterion of overlapping area chromatic aberration involves the problem of overlapping three or more images. Figure 2 For example, there are three images A, B, and C, where the color difference between image A and image B is ΔE AB , the overlapping area is S AB ; The color difference between image B and image C is ΔE BC , the overlapping area is S BC ; The color difference between image A and image C is ΔE AC , the overlapping area is S AC In the third evaluation criterion, the average color difference of each image is obtained by the following formula:

[0094] The average color difference of image A is:

[0095] ΔE A =(ΔE AB ·S AB +ΔE AC ·S AC ) / (S AB +S AC );

[0096] The average color difference of image B is:

[0097] ΔE B =(ΔE AB ·S AB +ΔE BC ·S BC ) / (S AB +S BC )

[0098] The average color difference of image C is:

[0099] ΔE C =(ΔE AC ·S AC +ΔE BC ·S BC ) / (S AC +S BC )

[0100] S4 completes remote sensing image mosaicking through image mosaicking technology according to the optimal mosaicking solution.

[0101] According to the optimal mosaicking scheme, the mosaicked image is mosaicked using image mosaicking technology. The basic mosaicking process includes geometric mosaicking, color mosaicking, and mosaic line elimination:

[0102] (1) Geometric mosaicking: This process is to match the spatial positions of multiple data sources with different parameters under a unified standard, so that all images have the same coordinate system and spatial resolution, and the images are positioned and superimposed.

[0103] (2) Color mosaicking: The object of study in this embodiment is multi-source remote sensing satellite images. There are many differences between images, such as phase, radiation level, spatial resolution, and imaging conditions. In addition to the problem of geometric distortion, there are also problems such as visual grayscale and inconsistent texture discontinuity between adjacent images. Therefore, it is necessary to adjust the color of the mosaicked image. Color mosaicking can be roughly divided into color balance within the image and color balance between images.

[0104] (3) Generation and fusion of mosaic lines:

[0105] a. Mosaic line generation: Calculate the difference of each pixel in the overlapping area to form a two-dimensional difference matrix. Then, use a dynamic programming algorithm to select a path with the minimum difference between the two images on this two-dimensional matrix as the optimal mosaic line.

[0106] b. Mosaic line fusion: Calculate the average value of the pixels of two corresponding lines between adjacent images, and use this value as the pixel value of the mosaic image in the overlapping area to achieve fuzzy processing of the overlapping area of ​​the images. The formula is expressed as:

[0107]

[0108] Where F(x,y) is the pixel average of the corresponding feature points between adjacent images, and f(x,y) and g(x,y) represent the grayscale values ​​of the two feature points in the overlapping area respectively.

[0109] In summary, by utilizing the above-described technical solutions of the present invention, through this method and the quality evaluation method of the present invention, mosaicked image screening is completed, and the mosaicked composite image is evaluated as a whole, thereby ensuring both the optimal quality of the images involved in the mosaicking and the high overall quality of the mosaicked composite image. The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A remote sensing image mosaic method based on image quality, characterized in that: include: S1 obtains evaluation indicators, and obtains a first evaluation standard, a second evaluation standard, and a third evaluation standard through a weighted summation operation method according to the evaluation indicators; S2 evaluates the remote sensing image set based on the first evaluation criteria and the second evaluation criteria to obtain a mosaicking scheme and a virtual synthetic image of the mosaicking scheme, wherein the mosaicking scheme includes a set of images to be mosaicked and a mosaicking order; S3 evaluates the virtual synthetic images of the mosaicking schemes based on the third evaluation criterion, and takes the mosaicking scheme corresponding to the virtual synthetic image with the best evaluation result as the optimal mosaicking scheme; S4 completes remote sensing image mosaicking by image mosaicking technology according to the optimal mosaicking solution; Step S2 is: S201 obtains remote sensing image sets; S202 initializes the mosaic scheme; S203 scores the remote sensing images in the remote sensing image set one by one according to the first evaluation standard to obtain a first evaluation ranking result; S204: screening the remote sensing images in the remote sensing image set according to the first evaluation ranking result to obtain an initial image to be mosaicked, adding the initial image to be mosaicked to the image set to be mosaicked, setting the initial mosaicking order to 1, and using the initial mosaicked image as an initial reference image; S205 scores the remote sensing images in the remote sensing image set one by one according to the second evaluation standard based on the current benchmark image to obtain a second evaluation ranking result; S206: screening the remote sensing image set according to the second evaluation ranking result to obtain a new image to be mosaicked, adding the new image to be mosaicked to the image set to be mosaicked, and increasing the current mosaicking order by one; S207: Virtually mosaicking the current image to be mosaicked with the current reference image to obtain a current composite image as a new reference image; S208 iteratively executes steps S205-S207 until the spatial coverage of the current synthetic image reaches a preset coverage, and the iteration ends, completing a mosaic scheme, and using the current synthetic image as a virtual synthetic image of the mosaic scheme; S209 iteratively executes steps S202-S208 until the number of output mosaic solutions reaches a preset threshold; The evaluation indicators include color difference in overlapping areas, information entropy, cloudiness, temporal similarity, and clarity. The specific calculation method of the evaluation indicators is as follows: Regarding the problem of two images overlapping, the color difference in the overlapping area is calculated using the following formula: Where ΔE is the total color difference of the overlapping area between the reference image and the remote sensing image, ΔL is the brightness difference of the overlapping area between the reference image and the remote sensing image, Δa is the standard red-green difference of the overlapping area between the reference image and the remote sensing image, and Δb is the standard yellow-blue difference of the overlapping area; For the problem of three or more images overlapping, the color difference of the overlapping area is calculated by the following formula: The average color difference of image A is: ΔE A =(ΔE AB ·S AB +ΔE AC ·S AC ) / (S AB +S AC ); The average color difference of image B is: ΔE B =(ΔE AB ·S AB +ΔE BC ·S BC ) / (S AB +S BC ) The average color difference of image C is: ΔE C =(ΔE AC ·S AC +ΔE BC ·S BC ) / (S AC +S BC ) The color difference between image A and image B is ΔE AB , the overlapping area is S AB ; The color difference between image B and image C is ΔE BC , the overlapping area is S BC ; The color difference between image A and image C is ΔE AC , the overlapping area is S AC ; Information entropy is calculated by the following formula: where p i is the ratio of the pixel with gray value i of the remote sensing image to the total number of pixels in the image, and L is the number of gray levels; The amount of cloud and fog is measured by the ratio of the pixels covered by cloud and fog in the remote sensing image to the total pixels of the image; The degree of temporal similarity is measured by the distance between the reference image and the remote sensing image; Clarity is measured by: Amb(x)=F -1 (F(x-n) / F(z))·z-I Among them, F -1 is the inverse Fourier transform, F is the Fourier transform, x is the remote sensing image, n is the noise, z is the standard image, is a constant value, the greater the clarity of image x, I is the identity operator, is a constant value; The first evaluation criterion is obtained by adding the evaluation indicators with different weight coefficients, wherein the weight coefficients of the color difference and the degree of temporal similarity in the overlapping area are 0, and the sum of the weight coefficients of information entropy, cloudiness, and clarity is 1; The second evaluation standard and the third evaluation standard are both obtained by adding the evaluation indicators with different weight coefficients, where the sum of the weight coefficients of overlapping area color difference, information entropy, cloudiness, temporal similarity, and clarity is 1, and the weight coefficients of each evaluation indicator in the third evaluation standard are consistent with the weight coefficients of each evaluation indicator in the second evaluation standard.

2. The method according to claim 1, characterized in that Step S4 is: S41: matching the points at the same position in space of the overlapping positions of adjacent images to be mosaicked in the optimal mosaicking solution one by one; S42 performs color adjustment on the mosaicked image; S43 generates mosaic lines using an algorithm that calculates the difference in overlapping image areas, and optimizes the mosaic lines between adjacent images to find a stitching method with minimal color difference, thereby removing overlapping areas. S44 further uses fusion technology to eliminate color differences and mosaic lines at the mosaic lines.

Images