Multi-scale remote sensing image color homogenizing method based on geographic position matching

Abstract

The invention relates to the technical field of remote sensing image color uniformizing methods, in particular to a multi-scale remote sensing image color uniformizing method based on geographic position matching, which comprises the following steps: acquiring a satellite image covering an image area to be subjected to color uniformizing, and processing the satellite image to form a reference image; carrying out normalization processing on the image to be subjected to color homogenization; the geographic position of the image to be homogenized is matched with the geographic position of the reference image to obtain an image of the image to be homogenized in the corresponding coordinate range on the reference image, and the image is marked as a reference POI image; converting the reference POI image and the to-be-homogenized image from a time domain to a frequency domain; performing three-level multi-scale decomposition on the reference POI image and the image to be subjected to color homogenization; calculating a color feature vector for the reference POI image, constructing a color mapping relation, and performing color mapping on the image to be subjected to color homogenization; a multi-scale restoration strategy is adopted to restore a color-homogenized image; according to the method, effective color adjustment can be realized on the remote sensing image, and uniform color correction processing with rich details can be realized.

Description

Technical Field

[0001] This invention relates to the field of remote sensing image color balancing methods, and in particular to a multi-scale remote sensing image color balancing method based on geographic location matching. Background Technology

[0002] Remote sensing images acquired by satellite sensors often cannot be directly applied to real-world scenarios due to the inherent characteristics of the sensors, human operational errors, and external factors such as weather conditions. These images require systematic correction processing at the ground level, including key steps such as radiometric correction, geometric correction, mosaicking, and color balancing. Color balancing, as a core challenge in image post-processing, faces multiple difficulties: color deviations caused by differences in remote sensors, brightness variations due to sudden weather changes, fluctuations in ground features caused by seasonal changes, changes in ground features caused by human factors, and tonal differences between images from different time periods. Furthermore, the uncertainty introduced by subjective factors further exacerbates the processing difficulty. The interplay of these objective and subjective factors makes color balancing a bottleneck restricting the large-scale application of remote sensing images.

[0003] Color balancing (also known as color correction) is a crucial step in remote sensing image processing, primarily used to address issues of color distortion and uneven brightness in images. Through specialized algorithms, the hue, brightness, and contrast of an image are optimized and adjusted, resulting in a visually harmonious overall tone, uniform brightness distribution, and moderate contrast. This significantly enhances the image's visual appeal, creating a unified, seamless "single map" that provides high-precision, high-quality image data support for applications across large survey areas.

[0004] Traditional color balancing methods include interactive processing, template processing, light equalization, and histogram methods. Interactive processing can achieve good color balancing results after multiple color adjustments, but it suffers from slow processing speed and low efficiency. Template processing uses a single, well-colored image as a color reference for color balancing, which works well for areas with similar terrain, but is less effective for images with significant terrain differences, exhibiting a mismatch between features and ground objects, thus limiting its application to large-scale color balancing. Furthermore, the color balancing results vary greatly depending on the reference image used, lacking a unified evaluation standard. Light equalization uses the mask method, which can solve color consistency within a single image, but color differences remain at the stitching points of multiple scenes. Histogram methods statistically analyze the mean, variance, and histogram of the reference image and the image to be balancing, achieving color balancing through mean-variance mapping and histogram mapping. It achieves good color balancing results under most conditions, but also suffers from detail blurring and distortion. Therefore, how to achieve both effective color adjustment and rich detail in color uniformity processing is a problem that urgently needs to be solved. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-scale remote sensing image color balancing method based on geographic location matching, which can achieve both effective color adjustment and rich detail in the color balancing correction of remote sensing images.

[0006] To achieve the above objectives, the present invention provides a multi-scale remote sensing image color balancing method based on geographic location matching, comprising:

[0007] Acquire satellite images covering the area to be color-matched, perform uniform lighting processing on each satellite image and orthophoto mosaic stitching, smooth the color difference at the edge of the image to form a reference image with consistent tone and rich color.

[0008] Normalization processing, including grayscale normalization and spectral normalization, is performed on the color-matched image.

[0009] Construct a coordinate transformation relationship to achieve geographic location matching between the image to be color-matched and the reference image, and obtain the image of the image to be color-matched within the corresponding coordinate range on the reference image, which is marked as the reference POI image;

[0010] Transform the reference POI image and the image to be color-matched from the time domain to the frequency domain;

[0011] In the frequency domain, a three-level multi-scale decomposition is performed on the reference POI image and the image to be color-balanced. The first level of decomposition is based on the original image, and the second and third levels are performed on the low-frequency components of the previous level decomposition.

[0012] Calculate the color feature vector for the reference POI image;

[0013] Based on the color feature vector of the reference POI image, a color mapping relationship is constructed between the image to be color-matched and the reference base image, and color mapping is performed on the image to be color-matched.

[0014] Based on the color mapping and signal decomposition results, a multi-scale restoration strategy is adopted to reconstruct the image step by step from bottom to top, restoring an image with consistent color tone and rich details after color uniformity.

[0015] The specific steps involved in acquiring satellite images covering the area to be color-matched, performing uniform illumination processing on each satellite image and orthorectifying and mosaicking the images, and smoothing color differences at adjacent scenes to form a reference image with consistent tone and rich color:

[0016] Acquire multiple first-level remote sensing images that have undergone orthorectification and cover the area to be uniformized, and perform format conversion, bit conversion, coordinate reference unification, spatial resolution adjustment, and dimension unification.

[0017] Each image is processed to achieve uniform lighting, ensuring consistent color tone and moderate brightness within each image.

[0018] The images after uniform lighting processing are mosaicked according to geographical location;

[0019] Set the pixel width of the transition area between adjacent scenes, and use feathering to eliminate the color difference at the junction of scenes, so as to form a reference base image with consistent tone, moderate brightness and geographic reference information.

[0020] The step of performing normalization processing on the image to be color-matched, including grayscale normalization and spectral normalization, specifically includes the following steps for grayscale normalization:

[0021] The gray levels of the reference image are selected as the reference gray levels. The gray levels of the image to be color-normalized are then transformed to the range of the reference image gray levels. The formula for gray level normalization is:

[0022]

[0023] In the formula, , , , These are the maximum and minimum gray values ​​of the image to be normalized and the maximum and minimum gray values ​​of the reference image, respectively. SRC is the DN value before normalization, and DST is the DN value after normalization.

[0024] Among them, the normalization process for the image to be color-matched, which includes grayscale normalization and spectral normalization, specifically includes the following steps for spectral normalization:

[0025] By calculating the spectral matching factor, the radiative differences between images of the same radiance formed on different sensors due to differences in the spectral responses of different remote sensors are eliminated. The formula for calculating the spectral matching factor is as follows:

[0026]

[0027] In the formula: , Let be the normalized spectral response function of the image to be color-matched and the reference image. , , , They are respectively , The upper and lower boundaries of the effective spectral range;

[0028] The formula for image correction using the spectral matching factor is as follows:

[0029]

[0030] Where DN is the DN value of the image before correction. This is the corrected image DN value.

[0031] The specific steps involved in constructing a coordinate transformation relationship to achieve geographic location matching between the image to be color-matched and the reference image, obtaining the image of the image to be color-matched within the corresponding coordinate range on the reference image, and marking it as the reference POI image include:

[0032] The georeference information of the reference base image and the image to be color-matched is analyzed to obtain the ellipsoidal reference and affine six parameters of the reference base image and the image to be color-matched, respectively.

[0033] Select the four corner points on the image to be color-matched as image control points;

[0034] Based on the image-space coordinates of the four corner points of the image to be color-matched and their affine six parameters, calculate the object-space coordinates of the four corner points under the ellipsoid reference of the image to be color-matched.

[0035] Based on the Bursa seven parameters, the object coordinates of the four corner points under the ellipsoid reference of the image to be uniformly colored are transformed to the object coordinates under the ellipsoid reference corresponding to the reference image.

[0036] Based on the affine six parameters of the reference image, the object coordinates under the ellipsoidal reference corresponding to the transformed reference image are transformed to the image coordinates of the reference image. The image coordinates are the position of the image to be color-matched on the reference image. After obtaining the row and column information through coordinate matching, the POI image is obtained by position cropping.

[0037] In the step of calculating the object-space coordinates of the four corner points under the ellipsoidal reference of the image to be uniformized, based on the image-space coordinates of the four corner points and their affine six parameters, the calculation formula is as follows:

[0038]

[0039] In the formula, GT[0], GT[1], GT[2], GT[3], GT[4], and GT[5] are the six parameters of the affine transformation, col and row are the image coordinates on the image, and Xgeo and Ygeo are the object coordinates corresponding to the image coordinates.

[0040] In the step of transforming the object coordinates of the four corner points under the ellipsoidal reference of the image to be uniformly colored to the object coordinates under the ellipsoidal reference of the reference image based on the Bursa seven parameters, the coordinate transformation formula is as follows:

[0041]

[0042] In the formula, [X1 Y1 Z1] and [X2 Y2 Z2] are the coordinates of the same point in the two coordinate systems, respectively, and [X0 Y0 Z0] is the translation parameter between the two coordinate systems. is the rotation parameter between the two coordinate systems, and m is the scale parameter.

[0043] The specific steps for converting the reference POI image and the image to be color-matched from the time domain to the frequency domain include:

[0044] Using a two-dimensional Fourier transform, the reference POI image and the image to be color-matched are transformed from the time domain to the frequency domain, and then spectral normalization and dynamic range compression are performed.

[0045] Specifically, a three-level multi-scale decomposition is performed on the reference POI image and the image to be color-balanced in the frequency domain. The first-level decomposition is based on the original image, and the second and third-level decompositions are performed on the low-frequency components of the previous level decomposition.

[0046] Downsampling was used for each level of decomposition of the uniform color image, meaning that the size of the decomposition result of the next level is only 1 / 2 of that of the previous level, and the size of the three-level decomposition is only 1 / 8 of the original image.

[0047] The specific steps involved in reconstructing an image with consistent color tone and rich detail, based on color mapping and signal decomposition results and employing a multi-scale restoration strategy, involve bottom-up, step-by-step reconstruction. These steps include:

[0048] The reference image is subjected to three-level signal decomposition. The low-frequency component Ls3 obtained from the third-level decomposition is subjected to SVD decomposition to obtain the feature vector UV and the feature value E, and then the color transfer matrix T is calculated.

[0049] For the third-level low-frequency component L3 of the color-balanced image, color mapping is performed based on the color transfer matrix T to obtain the third-level low-frequency correction result. ;

[0050] The high-frequency component H3 obtained from the third-level decomposition of the color-neutralized image and the low-frequency correction result obtained after color mapping After merging, we get Then, it is compared with the low-frequency component L2 of the second layer of the image to be uniformly colored to obtain the detail restoration factor of the third layer.

[0051] Based on the results of color mapping, the detail restoration factor of the third layer, and the high-frequency components obtained from the third layer decomposition, the restoration result of the third layer is obtained. ;

[0052] Upsampling is performed on the restored image I3 of the third layer to obtain the restored result of the low-frequency components of the second layer. ;

[0053] The high-frequency component H2 obtained from the second-level decomposition is combined with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then, it is compared with the low-frequency component L1 of the first layer to obtain the detail restoration factor of the second layer;

[0054] Based on the results of low-frequency component restoration The detail reduction factor of the second layer and the high-frequency components obtained from the second layer decomposition yield the restoration result of the second layer. ;

[0055] The result of the second layer restoration Upsampling is performed to obtain the restored result of the first layer of low-frequency components. ;

[0056] The high-frequency component H1 obtained from the first-level decomposition is combined with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then compare it with the image I to be uniformly colored to obtain the detail restoration factor of the first layer;

[0057] Based on the low-frequency component restoration results and the high-frequency components obtained from the first-level decomposition, the restoration result of the first level is obtained. ;

[0058] The result of the first layer restoration Upsampling is performed to obtain the low-frequency component results at the original image size. ;

[0059] Based on the low-frequency components of the first layer of restoration at the original image size, calculate the high-frequency components H0 of the image to be color-matched at the original image size.

[0060] High-frequency components based on the original image size and the restored low-frequency components The result after merging is a uniformly colored product.

[0061] This invention presents a multi-scale remote sensing image color balancing method based on geographic location matching. It proposes constructing a global (national) color balancing reference image, unifying the color scale. The normalization processing proposed in this invention eliminates response differences between different sensors, unifies radiometric dimensions, and enables large-scale, cross-sensor color balancing. The geographic location matching proposed in this invention accurately achieves the correspondence of ground features between the image to be balancing and the reference image, eliminating ground feature differences. The multi-scale signal decomposition technique and multi-scale inverse signal reconstruction technique proposed in this invention in the frequency domain can both transfer the colors of the reference image to the image to be balancing, achieving color consistency, and rely on the recovered detail information at different scales to achieve the dual purpose of color mapping and rich detail. This invention can achieve both effective color adjustment and detail-enriching color balancing correction processing of remote sensing images. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0063] Figure 1 This is a flowchart of a multi-scale remote sensing image color balancing method based on geographic location matching according to the present invention.

[0064] Figure 2 It is a nationwide uniform color reference image that has been constructed.

[0065] Figure 3 This is a diagram of the coordinate transformation relationship.

[0066] Figure 4 This is a diagram of coordinate matching.

[0067] Figure 5 This is a diagram illustrating POI cutting.

[0068] Figure 6 This is a schematic diagram of spatial domain transformation to frequency domain.

[0069] Figure 7 This is a schematic diagram of the signal decomposition of the image to be color-matched and the POI image.

[0070] Figure 8 It is a signal reconstruction flow graph. Detailed Implementation

[0071] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0072] Please see Figures 1-8 This invention provides a multi-scale remote sensing image color balancing method based on geographic location matching, comprising:

[0073] S1 acquires satellite images covering the area to be color-matched, performs uniform lighting processing on each satellite image and orthophoto mosaic stitching, and smooths the color difference at the adjacent scenes to form a reference reference image with consistent tone and rich color.

[0074] The specific steps include:

[0075] S11 acquires multiple first-level remote sensing images that have undergone orthorectification and cover the area to be uniformized, and performs format conversion, bit conversion, coordinate reference unification, spatial resolution adjustment, and dimension unification.

[0076] S12 performs uniform lighting processing on each scene to ensure consistent color tone and moderate brightness within a single image.

[0077] S13 mosaics the images after uniform lighting processing according to geographical location;

[0078] S14 sets the pixel width of the transition area between adjacent scenes and uses feathering to eliminate the color difference at the junction of scenes, forming a reference base image with consistent tone, moderate brightness, and geographic reference information.

[0079] In this embodiment, satellite images covering the area to be color-matched or the global (national) area are collected. Commercial software (such as PS, ArcMAP) is used to perform color-matching processing on each image. Then, the color-matched images are orthorectified and mosaicked. Feathering and other tools are used to smooth the color difference at the junction of the images to form a color-matching reference image (Note: This work only needs to be done once, and subsequent color-matching processing can use this image as the color-matching reference).

[0080] S2 performs normalization processing on the color-matching image, including grayscale normalization and spectral normalization;

[0081] In this step, the specific steps of grayscale normalization include:

[0082] The gray levels of the reference image are selected as the reference gray levels. The gray levels of the image to be color-normalized are then transformed to the range of the reference image gray levels. The formula for gray level normalization is:

[0083]

[0084] In the formula, , , , These are the maximum and minimum gray values ​​of the image to be normalized and the maximum and minimum gray values ​​of the reference image, respectively. SRC is the DN value before normalization, and DST is the DN value after normalization.

[0085] The specific steps of the spectral normalization include:

[0086] By calculating the spectral matching factor, the radiative differences between images of the same radiance formed on different sensors due to differences in the spectral responses of different remote sensors are eliminated. The formula for calculating the spectral matching factor is as follows:

[0087]

[0088] In the formula: , Let be the normalized spectral response function of the image to be color-matched and the reference image. , , , They are respectively , The upper and lower boundaries of the effective spectral range;

[0089] The formula for image correction using the spectral matching factor is as follows:

[0090]

[0091] Where DN is the DN value of the image before correction. This is the corrected image DN value.

[0092] S3 establishes a coordinate transformation relationship to achieve geographic location matching between the image to be color-matched and the reference image, obtaining the image of the image to be color-matched within the corresponding coordinate range on the reference image, which is then marked as the reference POI image;

[0093] The specific steps include:

[0094] S31 analyzes the georeference information of the reference image and the image to be color-matched, and obtains the ellipsoidal reference and affine six parameters of the reference image and the image to be color-matched respectively.

[0095] S32 selects four corner points as image control points on the image to be color-uniformed.

[0096] S33 calculates the object coordinates of the four corner points of the image to be uniformized under the ellipsoid reference of the image to be uniformized based on the image-square coordinates and their affine six parameters.

[0097] In this step, the calculation formula is:

[0098]

[0099] In the formula, GT[0], GT[1], GT[2], GT[3], GT[4], and GT[5] are the six parameters of the affine transformation, col and row are the image coordinates on the image, and Xgeo and Ygeo are the object coordinates corresponding to the image coordinates.

[0100] S34 transforms the object coordinates of the four corner points under the ellipsoidal reference of the image to be uniformly colored to the object coordinates under the ellipsoidal reference corresponding to the reference image based on the Bursa seven parameters.

[0101] In this step, the formula for coordinate transformation is as follows:

[0102]

[0103] In the formula, [X1 Y1 Z1] and [X2 Y2 Z2] are the coordinates of the same point in the two coordinate systems, respectively, and [X0 Y0 Z0] is the translation parameter between the two coordinate systems. is the rotation parameter between the two coordinate systems, and m is the scale parameter.

[0104] S35 uses the affine six parameters of the reference image to transform the object coordinates of the ellipsoidal reference corresponding to the transformed reference image to the image coordinates of the reference image. The image coordinates are the position of the image to be color-matched on the reference image. After obtaining the row and column information through coordinate matching, the POI image is obtained through position cropping.

[0105] S4 converts the reference POI image and the image to be color-matched from the time domain to the frequency domain.

[0106] The specific steps include: using two-dimensional Fourier transform to convert the reference POI image and the image to be color-matched from the time domain to the frequency domain, and then performing spectral normalization and dynamic range compression.

[0107] S5 performs a three-level multi-scale decomposition of the reference POI image and the image to be color-balanced in the frequency domain. The first level of decomposition is based on the original image, and the second and third levels are performed on the low-frequency components of the previous level decomposition.

[0108] In this step, downsampling technology is used for each level of decomposition of the color-uniform image, that is, the size of the decomposition result of the next level is only 1 / 2 of the size of the previous level, that is, the size after three levels of decomposition is only 1 / 8 of the original image.

[0109] S6 calculates the color feature vector for the reference POI image;

[0110] S7 constructs a color mapping relationship between the image to be color-matched and the reference image based on the color feature vector of the reference POI image, and performs color mapping on the image to be color-matched.

[0111] Based on color mapping and signal decomposition results, S8 employs a multi-scale restoration strategy to reconstruct images from the bottom up, restoring images with consistent color tone and rich detail after color homogenization.

[0112] The specific steps include:

[0113] S81 performs a three-level signal decomposition on the reference image, and performs SVD decomposition on the low-frequency component Ls3 obtained from the third-level decomposition to obtain the feature vector UV and the feature value E, and then calculates the color transfer matrix T.

[0114] S82 performs color mapping on the third-level low-frequency component L3 of the color-balanced image based on the color transfer matrix T to obtain the third-level low-frequency correction result. ;

[0115] S83 obtains the high-frequency component H3 from the third-level decomposition of the color-neutralizing image and the low-frequency correction result after color mapping. After merging, we get Then, it is compared with the low-frequency component L2 of the second layer of the image to be uniformly colored to obtain the detail restoration factor of the third layer.

[0116] Based on the results of color mapping, the detail restoration factor of the third layer, and the high-frequency components obtained from the third-layer decomposition, S84 obtains the restoration result of the third layer. ;

[0117] S85 upsamples the restored image I3 of the third layer to obtain the restored result of the low-frequency components of the second layer. ;

[0118] S86 combines the high-frequency component H2 obtained from the second-level decomposition with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then, it is compared with the low-frequency component L1 of the first layer to obtain the detail restoration factor of the second layer;

[0119] S87 Based on Low-Frequency Component Restoration Results The detail reduction factor of the second layer and the high-frequency components obtained from the second layer decomposition yield the restoration result of the second layer. ;

[0120] S88's result on the second layer restoration Upsampling is performed to obtain the restored result of the first layer of low-frequency components. ;

[0121] S89 combines the high-frequency component H1 obtained from the first-level decomposition with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then compare it with the image I to be uniformly colored to obtain the detail restoration factor of the first layer;

[0122] Based on the low-frequency component reconstruction results and the high-frequency components obtained from the first-level decomposition, S810 obtains the reconstruction result of the first level. ;

[0123] S811's result on the first layer restoration Upsampling is performed to obtain the low-frequency component results at the original image size. ;

[0124] S812 calculates the high-frequency component H0 of the image to be color-matched at the original image size based on the low-frequency component of the first layer restoration.

[0125] S813 based on high-frequency components at original image size and the restored low-frequency components The result after merging is a uniformly colored product.

[0126] To better understand the present invention, an embodiment is provided below to illustrate the multi-scale remote sensing image color balancing method based on geographic location matching. The specific process is as follows:

[0127] S1 Constructing a Color-Morphing Reference Baseline Image: This involves collecting multiple orthorectified Level 1 remote sensing images covering the globe or the entire country (ideally images from the same sensor or sensors with similar spectral responses). These images undergo format conversion, bit conversion, coordinate benchmark unification, spatial resolution adjustment, and dimension unification. Each image is then homogenized to ensure consistent tone and moderate brightness within each frame. The homogenized images are then mosaicked according to their location. The transition pixel width between adjacent images is set (referred to as the feathering parameter), and feathering is used to eliminate color differences at the edges, resulting in a large-area map with consistent tone, moderate brightness, and geographic reference information. This map serves as the baseline reference for the color-morphing process. The completed nationwide color-morphing reference baseline image is shown below. Figure 2 As shown.

[0128] S2 Normalization Processing: Normalization processing includes grayscale normalization and spectral normalization, aiming to eliminate differences in images acquired by different remote sensors and under different shooting conditions. Here, it refers to unifying the radiometric dimensions between the color-matching reference image and the image to be color-matched. This includes:

[0129] Gray-level normalization: Different remote sensors have different gray levels (dynamic ranges). For example, the gray level of a TM sensor is 256, while that of a GF2 sensor is 1024. These different gray levels result in different tones and brightness in the image. Gray-level normalization addresses the brightness differences of the same target image at different gray levels. The method for gray-level normalization is as follows: Select the gray level of a color-matching reference image as the reference gray level, and transform the gray level of the image to be color-matched to the range of the reference image. The formula for gray-level normalization is:

[0130]

[0131] In the formula: , , , These are the maximum and minimum gray values ​​of the image to be normalized and the maximum and minimum gray values ​​of the reference image, respectively. SRC is the DN value before normalization, and DST is the DN value after normalization.

[0132] Spectral normalization: Different remote sensors respond differently to the same entrance pupil radiance. The purpose of spectral normalization is to eliminate the radiative differences between images formed on different sensors for the same amount of radiation due to differences in the spectral responses of different remote sensors. This difference can be eliminated by calculating the spectral matching factor. The formula for calculating the spectral matching factor is as follows:

[0133]

[0134] In the formula: , Let be the normalized spectral response function of the image to be color-matched and the reference image. , , , They are respectively , The upper and lower bounds of the effective spectral range. The formula for image correction using the spectral matching factor is as follows:

[0135]

[0136] Where DN is the DN value of the image before correction. This is the corrected image DN value.

[0137] S3 Geographic Location Matching: Geographic location matching is based on the geographic location information of the image to be color-matched and the geographic location information of the reference image. A projection transformation model is constructed between the two, coordinate transformation parameters are calculated, and the location region of the image to be color-matched is matched on the reference image. The process of constructing the projection transformation model is as follows:

[0138] The georeference information of the reference base image and the image to be color-matched is analyzed to obtain the ellipsoidal reference and affine six parameters of the reference base image and the image to be color-matched, respectively.

[0139] Select the four corner points on the image to be color-matched as image control points;

[0140] The object coordinates of the four corner points under the ellipsoid reference of the image to be uniformly colored are calculated based on the affine six-parameter calculation.

[0141] Based on the Bursa seven parameters, the object coordinates of the four corner points under the ellipsoid reference of the image to be uniformly colored are transformed to the object coordinates under the ellipsoid reference of the reference image (Note: If the ellipsoid references of the two are the same, the object coordinates between the two are the same, and this step can be omitted).

[0142] Based on the affine six parameters of the reference reference image, the object coordinates under the ellipsoidal reference corresponding to the transformed reference reference image are transformed to the image coordinates of the reference reference image. The image coordinates are the position of the image to be uniformly colored on the reference reference image.

[0143] The detailed steps for geolocation matching between the reference image and the image to be color-matched are as follows:

[0144] (1) Calculate the object coordinates: Based on the image-space coordinates of the four corner points of the image to be uniformly colored and their affine six parameters, calculate the object coordinates of the four corner points under the ellipsoidal reference of the image to be uniformly colored. The calculation formula is:

[0145]

[0146] In the formula: GT[0], GT[1], GT[2], GT[3], GT[4], GT[5] are the six parameters of the affine transformation, col and row are the image coordinates on the image, and Xgeo and Ygeo are the object coordinates corresponding to the image coordinates.

[0147] (2) Constructing coordinate transformation relationships: Based on the ellipsoidal information between the images, construct the coordinate transformation relationship between the object-space coordinates of the image to be color-matched and the reference image. A schematic diagram of constructing the coordinate transformation relationship is shown below. Figure 3 As shown in the figure, the origins of the two coordinate systems are not the same, the corresponding coordinate axes are not parallel, and there may also be inconsistencies in scale between the two coordinate systems. This results in the same point in space having different coordinates in different coordinate systems. The purpose of coordinate transformation is to convert the coordinates of a point in one coordinate system to another and then solve for its coordinates. The coordinate transformation relationship is constructed using the Bursa-Taylor seven-parameter formula, which is as follows:

[0148]

[0149] In the formula: [X1 Y1 Z1] and [X2 Y2 Z2] are the coordinates of the same point in the two coordinate systems, respectively, and [X0 Y0 Z0] is the translation parameter between the two coordinate systems. is the rotation parameter between the two coordinate systems, and m is the scale parameter.

[0150] By establishing coordinate transformation relationships, the object coordinates corresponding to the four corner points of the image to be uniformly colored can be transformed to the object coordinates under the reference ellipsoid of the reference image.

[0151] (3) Coordinate matching: By constructing coordinate transformation relationships, the image to be uniformly colored can be projected onto the reference image, and the projection coordinates of the image to be uniformly colored on the reference image can be obtained, thus achieving coordinate matching. A schematic diagram of coordinate matching is shown below. Figure 4 As shown.

[0152] Based on the affine six parameters of the reference image, the object-space coordinates under the ellipsoidal reference of the reference image can be transformed to the image-space coordinates of the reference image through inverse affine transformation. The transformation formula is as follows:

[0153]

[0154] In the formula: GT[0], GT[1], GT[2], GT[3], GT[4], GT[5] are the six parameters of the affine transformation, Xgeo and Ygeo are the object coordinates, and col and row are the image coordinates on the image.

[0155] After coordinate matching, col and row are obtained, which are the row and column positions of the image to be color-matched on the reference image.

[0156] (4) POI cropping: After obtaining row and column information through coordinate matching, the POI image can be obtained by cropping the position. See the diagram below. Figure 5 As shown.

[0157] S4 Frequency Domain Feature Extraction: The POI image and the image to be color-matched are transformed into the frequency domain, where signal decomposition and feature extraction are performed. Specifically, the POI image and the image to be color-matched are transformed from the time domain to the frequency domain. A two-dimensional Fourier transform is used to convert the image from the time domain to the frequency domain, followed by spectral normalization and dynamic range compression. A schematic diagram after spatial transformation is shown below. Figure 6 As shown.

[0158] S5 Multi-Scale Signal Decomposition: A three-level signal decomposition is performed on the POI image and the image to be color-balanced in the frequency domain, obtaining the high-frequency and low-frequency components at each level. The next level of decomposition is performed on the low-frequency components of the previous level. (Note: To improve algorithm speed, downsampling is used for each level of decomposition of the image to be color-balanced: the size of the decomposition result at the next level is only half that of the previous level, meaning the size after three levels of decomposition is only 1 / 8 of the original image). A schematic diagram of the signal decomposition of the image to be color-balanced and the POI image is shown below. Figure 7 As shown (I represents the image to be color-matched, and S represents the reference image).

[0159] S6 Feature Extraction: Feature vector extraction is based on singular value decomposition (SVD). Here, feature extraction is performed only on the results of the final (third) level decomposition of the POI image. The principle of singular value decomposition is as follows:

[0160] First, we need to construct the equations. Where A is an n*n square matrix. It is an n-dimensional vector. It is an eigenvalue of matrix A. These are the eigenvalues ​​of matrix A. The corresponding eigenvector.

[0161] After obtaining the eigenvalues ​​and eigenvectors, the matrix can be eigenvalued by solving the equation, thus obtaining the eigenvectors corresponding to multiple different sets of eigenvalues. The characteristic equation is expressed as follows:

[0162]

[0163]

[0164] Where W is the n*n dimensional matrix spanned by these n eigenvectors, and E is the n*n dimensional matrix with these n eigenvalues ​​as the main diagonal. The n eigenvectors of W are standardized, i.e., they must satisfy... ,or At this point, the n eigenvectors of W are orthogonal. This satisfies... ,Right now In other words, W is a unitary matrix. Therefore, its characteristic expression can be decomposed as: At this point, the matrix can be decomposed.

[0165] Suppose matrix A is not a square matrix, but an m*n dimensional matrix, that is... Where U is an m*m dimensional matrix, E is an m*n dimensional matrix with all elements except those on the diagonal being 0, and the elements on the main diagonal being non-zero, and V is an n*n dimensional matrix, and both U and V are unitary matrices, i.e., satisfying .

[0166] The method to find the eigenvectors of a non-square matrix is ​​to left-multiply A by its transpose to obtain an n*n square matrix. The specific calculation is as follows:

[0167]

[0168] This allows us to obtain the matrix. Given n eigenvalues ​​and their corresponding n eigenvectors v, then... All the eigenvectors span an n*n matrix V, which is the matrix V mentioned above.

[0169] The method for finding U is similar: multiplying A by its transpose matrix on the right results in an m*m square matrix.

[0170]

[0171] This allows us to obtain the matrix. Given m eigenvalues ​​and their corresponding m eigenvectors u, we can... All the eigenvectors span an m*m matrix U, which is the matrix U mentioned above.

[0172] S7 Color Mapping: Based on the Singular Value Decomposition (SVD) results, the colors of the POI image are mapped to the image to be color-matched, achieving the purpose of color adjustment. The specific implementation is as follows:

[0173] Constructing the color transfer matrix: Based on the feature matrices U and V obtained from feature extraction, calculate the color transfer matrix. The calculation formula is as follows:

[0174]

[0175] Color mapping: Using the color transfer matrix, color mapping is performed on the low-frequency components of the final (third) level of decomposition. The formula is:

[0176]

[0177] S8 Signal Reconstruction (Restore): Signal reconstruction is based on color mapping and signal decomposition results. It proceeds from the bottom up, starting with the last layer, until the first layer reconstructs the entire image. The final restored result is the color-balanced final product. The signal reconstruction flowchart is as follows: Figure 8 As shown. The signal reconstruction is explained in detail below:

[0178] Third layer restoration:

[0179] Constructing the color transfer matrix: Perform three-level signal decomposition on the reference image S, and perform SVD decomposition on the low-frequency component Ls3 obtained from the third-level decomposition to obtain the eigenvectors UV and eigenvalues ​​E, and then calculate the color transfer matrix T.

[0180] Color mapping: The third low-frequency component L3 of the image to be color-matched is color-mapped based on the color transfer matrix T to obtain the third low-frequency correction result. ( * indicates matrix multiplication, corresponding to the operation in the flow graph. Symbols; if the dimensions of the matrices do not match when multiplying, the transition matrix needs to be scaled, corresponding to the flow graph. symbol);

[0181] Detail Restoration: The high-frequency component H3 obtained from the third layer decomposition of the color-matched image and the low-frequency correction result obtained after color mapping. After merging, we get Then, it is compared with the low-frequency component L2 of the second layer of the image to be color-matched to obtain the detail restoration factor of the third layer. The formula is as follows:

[0182]

[0183] merge(): Represents signal merging, corresponding to the [signal] in the flow graph. Symbol; downSample(): indicates downsampling, corresponding to the symbol in the flow graph. The symbol ; / indicates ratio operation, corresponding to the step in the process. symbol.

[0184] Image restoration: Based on the results of color mapping, detail restoration factor, and high-frequency components obtained from decomposition, the third layer of restoration is obtained. The formula is:

[0185]

[0186] This is the result of the third layer of restoration.

[0187] Second layer restoration

[0188] Color mapping: Upsampling is performed on the restored image I3 of the third layer to obtain the restored low-frequency components of the second layer. The formula is:

[0189]

[0190] upSample(): Indicates upsampling, corresponding to the flow graph in the diagram. symbol.

[0191] Detail restoration: Merging the high-frequency component H2 obtained from the second-level decomposition with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then, it is compared with the low-frequency component L1 of the first layer to obtain the detail restoration factor of the second layer. The formula is as follows:

[0192]

[0193] merge(), downSample(), and / have the same meaning as the third layer.

[0194] Image restoration: Based on the restoration results of low-frequency components, the detail restoration factor, and the high-frequency components obtained from decomposition, the second layer of restoration results is obtained. The formula is:

[0195]

[0196] This is the result of the second layer of restoration.

[0197] First layer restoration

[0198] Color mapping: Upsample the result of the second layer restoration to obtain the restoration result of the low-frequency components of the first layer. The formula is:

[0199]

[0200] The meaning of upSample() is the same as above.

[0201] Detail restoration: Merging the high-frequency component H1 obtained from the first layer decomposition with the low-frequency correction result obtained from the second layer restoration. After merging, we get Then, it is compared with the image I to be uniformly colored to obtain the detail restoration factor of the first layer. The formula is as follows:

[0202]

[0203] Among them, merge(), downSample(), and / have the same meaning as the first level.

[0204] Image restoration: Based on the restoration results of low-frequency components and the high-frequency components obtained from the first-level decomposition, the restoration result of the first level is obtained. The formula is:

[0205]

[0206] This is the result of the first layer of restoration.

[0207] Original image restoration

[0208] Low-frequency calculation: Upsample the result of the first layer restoration to obtain the low-frequency component results at the original image size. The calculation formula is:

[0209]

[0210] High-frequency decomposition: Based on the low-frequency components of the first layer of restoration at the original image size, calculate the high-frequency components H0 of the image to be color-balanced at the original image size. The formula is:

[0211]

[0212] Where - indicates signal splitting, corresponding to the flow graph The symbol I represents the image to be uniformly colored.

[0213] Color homogenization restoration: Based on the high-frequency components at the original image size and the restored low-frequency components, the result after color homogenization is obtained by merging them. The formula is as follows:

[0214]

[0215] The final result This is the result of even coloring.

[0216] This invention discloses a multi-scale remote sensing image color balancing method based on geographic location matching. First, it establishes a global (or national) color balancing reference image to unify the color standard. A normalization method is used to unify the dimensions between different sensors, enabling multi-sensor, large-area color balancing. A geographic template matching method is employed to solve the problem of mismatch between ground features in the image to be balancing and the reference image. Finally, the image is converted from the time domain to the frequency domain, and based on multi-scale signal decomposition technology and color transfer techniques, the color of the image to be balancing is mapped onto the reference image to achieve color adjustment. Simultaneously, detailed information of the image to be balancing is restored at different scales, achieving a gradual restoration from coarse to fine detail, resulting in a color-consistent and detail-clear image after balancing. This invention proposes the construction of a global (national) color-matching reference image, unifying the color scale; its proposed normalization processing eliminates response differences between different sensors, unifies radiometric dimensions, and enables large-scale, cross-sensor color matching; its proposed geographic location matching accurately achieves the correspondence of ground features between the image to be color-matched and the reference image, eliminating ground feature differences; and its proposed multi-scale signal decomposition technology in the frequency domain for feature extraction and multi-scale inverse signal reconstruction not only transfers the colors of the reference image to the image to be color-matched, achieving color consistency, but also leverages the recovered detail information at different scales to achieve the dual goals of color mapping and rich detail. This invention enables effective color adjustment and detail-enriching color-matching correction of remote sensing images.

[0217] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. A multi-scale remote sensing image color balancing method based on geographic location matching, characterized in that, include: Acquire satellite images covering the area to be color-matched, perform uniform lighting processing on each satellite image and orthophoto mosaic stitching, smooth the color difference at the edge of the image to form a reference image with consistent tone and rich color. Normalization processing, including grayscale normalization and spectral normalization, is performed on the color-matched image. Construct a coordinate transformation relationship to achieve geographic location matching between the image to be color-matched and the reference image, and obtain the image of the image to be color-matched within the corresponding coordinate range on the reference image, which is marked as the reference POI image; Transform the reference POI image and the image to be color-matched from the time domain to the frequency domain; In the frequency domain, a three-level multi-scale decomposition is performed on the reference POI image and the image to be color-balanced. The first level of decomposition is based on the original image, and the second and third levels are performed on the low-frequency components of the previous level decomposition. Calculate the color feature vector for the reference POI image; Based on the color feature vector of the reference POI image, a color mapping relationship is constructed between the image to be color-matched and the reference base image, and color mapping is performed on the image to be color-matched. Based on the color mapping and signal decomposition results, a multi-scale restoration strategy is adopted to reconstruct the image step by step from bottom to top, restoring an image with consistent color tone and rich details after color uniformity.

2. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, The specific steps for acquiring satellite images covering the area to be color-matched, performing uniform illumination processing on each satellite image and orthorectifying and mosaicking the images, and smoothing color differences at adjacent edges to form a reference image with consistent tone and rich colors include: Acquire multiple first-level remote sensing images that have undergone orthorectification and cover the area to be uniformized, and perform format conversion, bit conversion, coordinate reference unification, spatial resolution adjustment, and dimension unification. Each image is processed to achieve uniform lighting, ensuring consistent color tone and moderate brightness within each image. The images after uniform lighting processing are mosaicked according to geographical location; Set the pixel width of the transition area between adjacent scenes, and use feathering to eliminate the color difference at the junction of scenes, so as to form a reference base image with consistent tone, moderate brightness and geographic reference information.

3. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, In the normalization process of the image to be color-matched, which includes gray-level normalization and spectral normalization, the specific steps of gray-level normalization include: The gray levels of the reference image are selected as the reference gray levels. The gray levels of the image to be color-normalized are then transformed to the range of the reference image gray levels. The formula for gray level normalization is: In the formula, , , , These are the maximum and minimum gray values ​​of the image to be normalized and the maximum and minimum gray values ​​of the reference image, respectively. SRC is the DN value before normalization, and DST is the DN value after normalization.

4. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, In the normalization process of the image to be color-matched, which includes grayscale normalization and spectral normalization, the specific steps of spectral normalization include: By calculating the spectral matching factor, the radiative differences between images of the same radiance formed on different sensors due to differences in the spectral responses of different remote sensors are eliminated. The formula for calculating the spectral matching factor is as follows: In the formula: , Let be the normalized spectral response function of the image to be color-matched and the reference image. , , , They are respectively , The upper and lower boundaries of the effective spectral range; The formula for image correction using the spectral matching factor is as follows: Where DN is the DN value of the image before correction. This is the corrected image DN value.

5. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, The specific steps for establishing coordinate transformation relationships to achieve geographic location matching between the image to be color-matched and the reference image, obtaining the image of the image to be color-matched within the corresponding coordinate range on the reference image, and marking it as the reference POI image include: The georeference information of the reference base image and the image to be color-matched is analyzed to obtain the ellipsoidal reference and affine six parameters of the reference base image and the image to be color-matched, respectively. Select the four corner points on the image to be color-matched as image control points; Based on the image-space coordinates of the four corner points of the image to be color-matched and their affine six parameters, calculate the object-space coordinates of the four corner points under the ellipsoid reference of the image to be color-matched. Based on the Bursa seven parameters, the object coordinates of the four corner points under the ellipsoid reference of the image to be uniformly colored are transformed to the object coordinates under the ellipsoid reference corresponding to the reference image. Based on the affine six parameters of the reference image, the object coordinates under the ellipsoidal reference corresponding to the transformed reference image are transformed to the image coordinates of the reference image. The image coordinates are the position of the image to be color-matched on the reference image. After obtaining the row and column information through coordinate matching, the POI image is obtained by position cropping.

6. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 5, characterized in that, Based on the image-space coordinates of the four corner points of the image to be uniformized and their affine six parameters, the calculation formula for calculating the object-space coordinates of the four corner points under the ellipsoidal reference of the image to be uniformized is as follows: In the formula, GT[0], GT[1], GT[2], GT[3], GT[4], and GT[5] are the six parameters of the affine transformation, col and row are the image coordinates on the image, and Xgeo and Ygeo are the object coordinates corresponding to the image coordinates.

7. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 5, characterized in that, The formula for coordinate transformation in the step of converting the object coordinates of the four corner points under the ellipsoidal reference of the image to be uniformly colored to the object coordinates under the ellipsoidal reference of the reference image based on the Bursa seven parameters is as follows: In the formula, [X1 Y1 Z1] and [X2 Y2 Z2] are the coordinates of the same point in the two coordinate systems, respectively, and [X0 Y0 Z0] is the translation parameter between the two coordinate systems. is the rotation parameter between the two coordinate systems, and m is the scale parameter.

8. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, The specific steps for converting the reference POI image and the image to be color-balanced from the time domain to the frequency domain include: Using a two-dimensional Fourier transform, the reference POI image and the image to be color-matched are transformed from the time domain to the frequency domain, and then spectral normalization and dynamic range compression are performed.

9. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, In the frequency domain, a three-level multi-scale decomposition is performed on the reference POI image and the image to be color-balanced. The first-level decomposition is based on the original image, and the second and third-level decompositions are performed on the low-frequency components of the previous level decomposition. Downsampling was used for each level of decomposition of the uniform color image, meaning that the size of the decomposition result of the next level is only 1 / 2 of that of the previous level, and the size of the three-level decomposition is only 1 / 8 of the original image.

10. The multi-scale remote sensing image color balancing method based on geographic location matching as described in claim 1, characterized in that, Based on color mapping and signal decomposition results, a multi-scale restoration strategy is employed to reconstruct the image from the bottom up, level by level. The specific steps for restoring an image with consistent color tone and rich detail after color homogenization include: The reference image is subjected to three-level signal decomposition. The low-frequency component Ls3 obtained from the third-level decomposition is subjected to SVD decomposition to obtain the feature vector UV and the feature value E, and then the color transfer matrix T is calculated. For the third-level low-frequency component L3 of the color-balanced image, color mapping is performed based on the color transfer matrix T to obtain the third-level low-frequency correction result. ; The high-frequency component H3 obtained from the third-level decomposition of the color-neutralized image and the low-frequency correction result obtained after color mapping After merging, we get Then, it is compared with the low-frequency component L2 of the second layer of the image to be uniformly colored to obtain the detail restoration factor of the third layer. Based on the results of color mapping, the detail restoration factor of the third layer, and the high-frequency components obtained from the third layer decomposition, the restoration result of the third layer is obtained. ; Upsampling is performed on the restored image I3 of the third layer to obtain the restored result of the low-frequency components of the second layer. ; The high-frequency component H2 obtained from the second-level decomposition is combined with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then, it is compared with the low-frequency component L1 of the first layer to obtain the detail restoration factor of the second layer; Based on the results of low-frequency component restoration The detail reduction factor of the second layer and the high-frequency components obtained from the second layer decomposition yield the restoration result of the second layer. ; The result of the second layer restoration Upsampling is performed to obtain the restored result of the first layer of low-frequency components. ; The high-frequency component H1 obtained from the first-level decomposition is combined with the low-frequency correction result obtained from the second-level restoration. After merging, we get Then compare it with the image I to be uniformly colored to obtain the detail restoration factor of the first layer; Based on the low-frequency component restoration results and the high-frequency components obtained from the first-level decomposition, the restoration result of the first level is obtained. ; The result of the first layer restoration Upsampling is performed to obtain the low-frequency component results at the original image size. ; Based on the low-frequency components of the first layer of restoration at the original image size, calculate the high-frequency components H0 of the image to be color-matched at the original image size. High-frequency components based on the original image size and the restored low-frequency components The result after merging is a uniformly colored product.

Images