Satellite remote sensing-based multispectral and hyperspectral imaging proportion balance optimization method
By using super-resolution reconstruction and hierarchical fusion processing, the problem of spatial resolution inconsistency when combining multispectral and hyperspectral imaging was solved, achieving optimal image resolution and information preservation, and improving the quality and application effectiveness of remote sensing images.
Patent Information
- Application Number
- CN202511254506.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-09-04
- Publication Date
- 2025-12-05
- Estimated Expiration
- 2045-09-04
AI Technical Summary
The inconsistent spatial resolution of multi-source satellite sensors leads to problems in detail display in the images after data fusion when multispectral and hyperspectral imaging are combined.
Through super-resolution reconstruction and hierarchical fusion processing, including image preprocessing, super-resolution reconstruction algorithms to improve spatial resolution, image fusion and optimization processing, a proportion balance optimization model is established to optimize image data processing time and imaging cost.
It achieves optimal allocation of imaging proportion under limited resources, improves the visual effect of images and the preservation of key information, and provides clearer and more accurate remote sensing images, supporting applications such as environmental protection, resource management and disaster early warning.
Smart Images

Figure CN120746843B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of balance optimization, more specifically, the present application relates to a multispectral and hyperspectral imaging proportion balance optimization method based on satellite remote sensing. BACKGROUND
[0002] With global environmental changes and frequent natural disasters, quickly and accurately obtaining surface information has become the key to coping with these challenges. Multispectral remote sensing has great potential in monitoring forest cutting, urban expansion, soil erosion, etc. due to its high spatial resolution. And hyperspectral remote sensing plays an irreplaceable role in crop disease and pest monitoring, mineral resource exploration, water pollution detection, etc. due to its unique spectral recognition ability. Multispectral remote sensing technology observes the target through several discrete bands, while hyperspectral remote sensing can provide tens or even hundreds of narrow bands for each pixel after dispersion, realizing continuous spectral coverage. The image obtained by hyperspectral remote sensing technology contains rich spatial, radiation and spectral triple information, which is beneficial to the analysis of spectral characteristics to study the ground objects and realize the fine classification and identification of ground objects. However, although the hyperspectral image has rich spectral information, the spatial resolution is relatively low, which limits its application field. On the contrary, multispectral image has high spatial resolution. However, the use of single multispectral or hyperspectral remote sensing data often cannot meet the complex and changeable application requirements.
[0003] In order to realize the comprehensive and accurate perception of surface information, researchers began to explore the proportion balance optimization strategy of multispectral and hyperspectral imaging. This strategy aims to combine the advantages of both, not only to ensure the integrity of spectral information, but also to improve the spatial resolution, so as to generate high-quality remote sensing images with rich spectral characteristics and clear spatial structure. This not only helps to improve the accuracy of land cover classification, but also enhances the in-depth understanding of the composition of surface materials, providing strong data support for environmental protection, resource management and disaster warning, etc.
[0004] For example, the invention patent announcement CN118656786B discloses a method and system for fusion of hyperspectral data of iron ore, relating to the field of remote sensing data fusion technology. Specifically, it relates to a method and system for fusion of hyperspectral data of iron ore. This method includes: acquiring hyperspectral and multispectral data of the target area of the iron ore; performing multiple curve fitting operations on the hyperspectral and multispectral data respectively, determining the difference coefficient of the multispectral fitted curve after each curve fitting; obtaining the curve similarity between the multispectral fitted curve and the hyperspectral fitted curve; obtaining the final multispectral fitted curve and hyperspectral fitted curve based on the curve similarity; obtaining the local difference factor of the hyperspectral fitted curve in each band; obtaining the weight of the hyperspectral data in each band; and fusing the reflectance of the hyperspectral data and multispectral data in each band to obtain the fusion result of the hyperspectral data of iron ore. This improves the accuracy of hyperspectral data fusion for iron ore.
[0005] For example, the invention patent announcement CN118395133B discloses a multispectral hair growth and hair regrowth data acquisition and analysis system, belonging to the field of data acquisition and analysis technology. It collects users' multispectral usage data and hair regrowth data, and then uses deep learning to learn from these existing data. This allows for analysis of the effects of different users using different multispectral methods based on historical data, providing users with effective references by optimizing multispectral usage strategies and improving the efficiency of hair growth devices. In addition, to address the problems of deep learning in existing technologies, such as being prone to getting trapped in local optima and slow convergence speed, a dual-search information interaction algorithm is also provided to improve the effect and efficiency of deep learning.
[0006] The disclosed technical solutions mentioned above have at least the following technical problems: the spatial resolution of multi-source satellite sensors is inconsistent, which leads to problems in detail display in the fused image when combining multispectral and hyperspectral imaging due to the difference in spatial resolution. To address these problems, this invention proposes a solution. Summary of the Invention
[0007] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method for balancing and optimizing the proportion of multispectral and hyperspectral imaging based on satellite remote sensing. This method addresses the issue of detail display in images after data fusion through super-resolution reconstruction and layered fusion processing.
[0008] To achieve the above objectives, the present invention provides the following technical solution:
[0009] A method for optimizing the proportion of multispectral and hyperspectral imaging based on satellite remote sensing includes the following steps: acquiring a first image dataset, which includes low-resolution multispectral images and hyperspectral images; preprocessing the first image dataset, selecting low-resolution multispectral images from the preprocessed images, and using a super-resolution reconstruction algorithm to improve their spatial resolution to generate reconstructed multispectral images; fusing the reconstructed multispectral images with the preprocessed hyperspectral images, and optimizing the fusion result to generate an image with the optimal resolution; and extracting second image data based on the image with the optimal resolution to establish a proportion balance optimization model.
[0010] In a preferred embodiment, the preprocessing of the first image dataset includes radiometric correction of the images, specifically: acquiring the original image data collected by the satellite sensor and its corresponding calibration parameter file, wherein the calibration parameter file includes the gain and bias calibration coefficients for each band; converting the DN values of each pixel in the original image into radiance values according to the calibration parameter file; converting the radiance values into apparent reflectance; and outputting the radiometrically corrected multispectral and hyperspectral image data.
[0011] In a preferred embodiment, the preprocessing of the first image dataset further includes geometric correction of the images, specifically: acquiring map data matching the image coverage area and data sources with known geographic coordinates; selecting ground control points based on the correspondence between ground feature points on the map data and the image, and recording their coordinates in the image coordinate system and the geographic coordinate system; calculating the coefficients of a quadratic polynomial based on GCP coordinates using a remote sensing image processing platform; and resampling the radiometrically corrected image using transformation coefficients to achieve pixel-level geographic coordinate mapping.
[0012] In a preferred embodiment, the step of selecting low-resolution multispectral images from the preprocessed images and using a super-resolution reconstruction algorithm to improve their spatial resolution to generate reconstructed multispectral images specifically involves: extracting multispectral images from the preprocessed remote sensing image data; setting a spatial resolution threshold, which is determined based on the standard resolution of similar satellite sensors; traversing all spectral images and selecting low-resolution images with spatial resolution below the threshold; and training the low-resolution images by constructing a training model to obtain high-resolution multispectral images.
[0013] In a preferred embodiment, the step of training the low-resolution image by constructing a training model further includes judging the model training effect, specifically: during the training process, calculating the Bach distance between the generated high-resolution multispectral image and the real high-resolution reference image for each band histogram; and evaluating the training effect based on the changing trend of the Bach distance.
[0014] In a preferred embodiment, the fusion of the reconstructed multispectral image and the preprocessed hyperspectral image specifically involves: converting the multispectral image and hyperspectral image after super-resolution reconstruction into grayscale images; performing a two-dimensional Fourier transform on the grayscale images to obtain a complex matrix; extracting the low-frequency components of the complex matrix as a low-frequency layer using a low-pass filter and extracting the high-frequency components as a high-frequency layer using a high-pass filter; low-frequency layer fusion: performing a weighted average of the low-frequency layers of the multispectral and hyperspectral images, with weights dynamically allocated based on the low-frequency information representation capability of the image scene; high-frequency layer fusion: extracting edge features and texture features of the high-frequency layers of the two images based on edge detection and texture analysis algorithms; constructing a joint feature descriptor and calculating the feature similarity between the two image descriptors; dynamically allocating weights based on the similarity matching results and fusing the high-frequency features of the two images; and performing an inverse discrete wavelet transform on the fused low-frequency and high-frequency layers to generate the final fused image.
[0015] In a preferred embodiment, the optimization of the fusion result to generate an optimal resolution image specifically involves: converting the fused image into a grayscale image and dividing it into pixel blocks of predefined sizes; performing adaptive histogram equalization on each pixel block: statistically analyzing the grayscale distribution within the pixel block, and when the number of pixels at a certain grayscale level exceeds a contrast threshold, uniformly distributing the excess pixels to other grayscale levels; performing wavelet transform on the equalized image to decompose it into low-frequency sub-bands and high-frequency sub-bands; performing threshold denoising on the high-frequency sub-bands: if the absolute value of the wavelet coefficients is less than the threshold, setting them to zero; if they are greater than the threshold, applying a shrinkage function; and performing inverse wavelet transform on the thresholded sub-bands to generate an optimal resolution image.
[0016] In a preferred embodiment, the step of extracting second image data based on the optimal resolution image and establishing a proportion balance optimization model specifically involves: obtaining the data processing time, imaging cost, and image fusion quality evaluation coefficient of the optimal resolution image; establishing a proportion balance optimization model with data processing time and imaging cost as constraints and the image fusion quality evaluation coefficient as the objective function; and solving the optimization model to obtain the parameter combination that maximizes the objective function.
[0017] The technical effects and advantages of this invention, based on the optimization method for balancing the proportion of multispectral and hyperspectral imaging from satellite remote sensing, are as follows:
[0018] 1. This invention achieves the optimal allocation of imaging proportions under limited resources by accurately calculating data processing time and imaging costs, combined with image fusion quality assessment, providing strong support for the widespread application of satellite remote sensing technology.
[0019] 2. This invention comprehensively considers the image quality assessment coefficient and the information fusion preservation coefficient, ensuring that key information is preserved while improving the visual effect of the image during the fusion process. This provides users with clearer and more accurate remote sensing images, helping to improve the accuracy of subsequent analysis and decision-making. Attached Figure Description
[0020] Figure 1 This is a flowchart illustrating the method for balancing and optimizing the proportion of multispectral and hyperspectral imaging based on satellite remote sensing, as described in this invention. Detailed Implementation
[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0022] Example 1, Figure 1 This invention presents a method for balancing and optimizing the proportion of multispectral and hyperspectral imaging based on satellite remote sensing, comprising the following steps:
[0023] S1, acquire a first image dataset, which includes low-resolution multispectral images and hyperspectral images;
[0024] S2, preprocess the first image dataset, select low-resolution multispectral images from the preprocessed images, and use a super-resolution reconstruction algorithm to improve their spatial resolution to generate reconstructed multispectral images.
[0025] The preprocessing of the first image dataset includes radiometric correction of the first image data, specifically:
[0026] Acquire raw image data collected by satellite sensors and their corresponding calibration parameter files, wherein the calibration parameter files include the gain and offset calibration coefficients for each band;
[0027] The DN value of each pixel in the image data is converted into a radiance value based on the calibration parameter file;
[0028] The radiance value is converted into apparent reflectance to obtain radiometrically corrected multispectral and hyperspectral image data.
[0029] The preprocessing of the first image dataset further includes geometric correction of the first image data, specifically:
[0030] Collect Landsat satellite imagery data, as well as high-precision map data of the image coverage area and other data sources with known geographic coordinates;
[0031] Based on the map data, identify ground feature points as ground control points (GCPs), mark the locations of the control points, and record the coordinate values in the image coordinate system and the geographic coordinate system;
[0032] The image coordinates and geographic coordinates of the selected ground control points are input into the ENVI software, and the coefficients of the quadratic polynomial are calculated based on the software's least squares function.
[0033] Based on the coefficients of the calculated quadratic polynomial, the original image is resampled using bilinear interpolation to map each pixel to the correct geographic coordinates.
[0034] Feature points refer to turning points, intersections, bends, and the center of independent features on the outline of a feature. These points can reflect the unique shape and structure of the feature, such as the intersection of a river and a road, or the corner of a large building.
[0035] Radiance refers to the brightness level of an object's radiated energy per unit area and unit solid angle. It is the energy radiated by a surface radiation source per unit time along the direction *n* perpendicular to the surface element's normal, per unit area and unit solid angle; that is, the radiant flux of the radiation source per unit projected area and unit solid angle. The SI unit of radiance is watts per (steradian meter²).
[0036] Apparent reflectance refers to the reflectance at the top of the atmosphere and is one of the results of radiometric calibration. It can also be called the apparent reflectance at the top of the atmosphere or apparent reflectance.
[0037] The formula for calculating apparent reflectance is: Apparent reflectance = Surface reflectance + Atmospheric reflectance;
[0038] Apparent reflectance has wide applications in remote sensing. It is an important parameter in satellite remote sensing data, used for retrieving surface parameters, monitoring surface changes, and classifying surfaces. At the same time, apparent reflectance is also an important reference for atmospheric correction. Through atmospheric correction, we can obtain the true reflectance data of the Earth's surface, providing more accurate basic data for subsequent remote sensing applications.
[0039] The formula for calculating the radiance value is as follows:
[0040]
[0041] The apparent reflectance is specifically:
[0042]
[0043] in, This is the radiance value. For gain scaling coefficients, The digital quantization values of the pixels in the original image. The bias coefficient, Apparent reflectance, The distance between the Earth and the Sun. This represents the solar irradiance at the top of the atmosphere. This is the solar zenith angle.
[0044] The process of selecting low-resolution multispectral images from the preprocessed images and using a super-resolution reconstruction algorithm to improve their spatial resolution to generate reconstructed multispectral images involves the following steps:
[0045] Obtain multispectral images from preprocessed remote sensing image data;
[0046] Set the spatial resolution threshold based on the resolution standards of similar satellite sensors;
[0047] Iterate through all preprocessed multispectral image data, compare the spatial resolution of each image with the set spatial resolution threshold, and if the spatial resolution of the image is lower than the threshold, it is determined to be a low-resolution multispectral image and the low-resolution multispectral images are filtered out.
[0048] Iterate through all spectral images and filter out low-resolution images with spatial resolution below a threshold;
[0049] High-resolution multispectral images are obtained by training a model on low-resolution images.
[0050] In this embodiment, the step of training the low-resolution image by constructing a training model specifically involves:
[0051] Collect multispectral image samples of different scenes and resolutions and their corresponding high-resolution reference images, and divide them into training set and validation set;
[0052] A GAN model consisting of a generator and a discriminator is constructed. Low-resolution multispectral images are taken as input, and through convolutional and deconvolutional neural network layers, the features and structural information of the images are learned to generate high-resolution multispectral images.
[0053] The training set is used to train the GAN model, and the training effect of the model is judged based on the similarity between the generated high-resolution multispectral image and the real high-resolution reference image.
[0054] The trained generator is obtained by adjusting the training parameters of the GAN model based on the validation set.
[0055] The selected low-resolution multispectral images are input into the trained generator to generate the corresponding high-resolution multispectral images.
[0056] The step of training a low-resolution image by constructing a training model also includes judging the model training effect, specifically:
[0057] During training, the Bach distance between the generated high-resolution multispectral image and the real high-resolution reference image for each band histogram is calculated.
[0058] The training effect was evaluated based on the changing trend of the Bach distance, specifically as follows:
[0059] During training, the Bach distance between the generated high-resolution multispectral image and the real high-resolution reference image for each band is calculated. The training effect is evaluated based on the trend of the Bach distance. If the Bach distance gradually decreases, the similarity between the generated high-resolution multispectral image and the real high-resolution reference image is higher, and the training effect of the model is better.
[0060] The basic components of a GAN model: generator and discriminator.
[0061] Generator
[0062] Task: Generate samples that are as close as possible to real data.
[0063] Input: A random noise vector.
[0064] Output: The generated samples, which can be images, audio, text, etc.
[0065] Objective: The generated samples should be able to fool the discriminator, making it believe that the generated data is real.
[0066] Discriminator
[0067] Task: Distinguish between real data and fake data generated by the generator.
[0068] Input: A data sample, which may be real or generated.
[0069] Output: A probability value representing the probability that the sample is real data.
[0070] Objective: To maximize the probability of correctly classifying real samples and generated samples.
[0071] With the generator fixed, train the discriminator: The discriminator's task here is to learn how to better distinguish between real and generated data. This is done by providing the discriminator with both real data and fake data generated by the generator, calculating its loss function (typically cross-entropy loss), and then updating the discriminator's parameters to improve its discriminative ability.
[0072] With a fixed discriminator, the generator is trained: The generator's task is to learn how to generate data that can fool the discriminator. This is done by providing the generator with a random noise vector, calculating the discriminator's probability of judging the generated data, then calculating the generator's loss function (typically minimizing the discriminator's probability of judging the generated data), and updating the generator's parameters to improve its generation capabilities.
[0073] S3 fuses the reconstructed multispectral image with the preprocessed hyperspectral image and optimizes the fusion result to generate an image with the best resolution.
[0074] The core idea of layered fusion processing is to decompose complex data or image structures into simpler components, which are processed at different levels. Each level focuses on specific information or features, thereby achieving more efficient information extraction or image fusion. This approach not only improves processing accuracy but also enhances the system's scalability and robustness.
[0075] The process of fusing the reconstructed multispectral image with the preprocessed hyperspectral image specifically involves:
[0076] Convert the super-resolution reconstructed multispectral and hyperspectral images into grayscale images;
[0077] Performing a two-dimensional Fourier transform on a grayscale image converts it from the spatial domain to the frequency domain, resulting in a complex matrix. Each element of the complex matrix represents the amplitude and phase information at the corresponding frequency.
[0078] Set up a circular low-pass filter, where the frequency components within the filter radius are treated as low-frequency information, and the frequency components outside the radius are treated as high-frequency information.
[0079] A low-pass filter is applied to the complex matrix after Fourier transform to separate the low-frequency component;
[0080] Set up a high-pass filter and separate the high-frequency component based on the high-pass filter;
[0081] The separated low-frequency and high-frequency components are respectively used as the low-frequency layer and the high-frequency layer;
[0082] In the low-frequency layer fusion, a weighted average method is used to assign weights according to the characteristics of multispectral and hyperspectral images in low-frequency information representation, and generate the fused low-frequency layer image.
[0083] During high-frequency layer fusion, edge features of the high-frequency layers of multispectral and hyperspectral images are extracted based on the Canny edge detection algorithm, and texture features of the high-frequency layers of multispectral and hyperspectral images are extracted based on the gray-level co-occurrence matrix.
[0084] A feature descriptor is established based on the extracted edge and texture features;
[0085] The feature descriptors of the multispectral image and the hyperspectral image are compared one by one. The Euclidean distance between the feature descriptors of the multispectral image and the hyperspectral image is calculated. The feature pair with the closest distance is used as the matching feature to obtain the similar feature parts in the high frequency layer of the multispectral and hyperspectral images.
[0086] Weights are assigned to the high-frequency features of the multispectral and hyperspectral images based on similar feature components in the high-frequency layer. The features are then fused according to the assigned weights to generate the fused high-frequency layer image.
[0087] The fused low-frequency and high-frequency images are subjected to inverse discrete wavelet transform, and the subbands of different frequencies are recombined into a complete fused image, resulting in fused multispectral and hyperspectral images.
[0088] The derivation of the two-dimensional Fourier transform is based on the theory of the one-dimensional Fourier transform. Suppose we have a two-dimensional signal f(x,y), and we want to find its frequency representation. Using the concept of Fourier series, f(x,y) can be represented as a superposition of a series of complex exponential functions. For a two-dimensional signal, we use two frequency variables, u and v, corresponding to the frequencies in the x and y directions, respectively.
[0089] The optimization process for the fusion result to generate an image with the best resolution is specifically as follows:
[0090] The fused image is converted into a grayscale image, and the grayscale image is divided into several sub-regions, each sub-region being a 16×16 pixel block. Histogram equalization is then performed on each pixel block.
[0091] Set a contrast threshold. When the number of pixels at a gray level exceeds the contrast threshold, the excess part will be evenly distributed to other gray levels to obtain an adaptively enhanced image.
[0092] Wavelet transform is applied to the adaptively enhanced image to decompose it into sub-bands of different frequencies;
[0093] Set a wavelet coefficient threshold. For wavelet coefficients in the high-frequency subband, when the absolute value of the wavelet coefficient is less than the wavelet coefficient threshold, the wavelet coefficient is set to zero; when the absolute value of the wavelet coefficient is greater than the wavelet coefficient threshold, the wavelet coefficient is shrunk.
[0094] After thresholding, the processed wavelet coefficients are recombined into a denoised image using inverse wavelet transform, resulting in an image with optimal resolution.
[0095] The wavelet coefficients of the high-frequency subbands are used in wavelet transform to decompose a signal into multiple subbands, each corresponding to a different frequency range. The high-frequency subbands typically contain the high-frequency components of the signal, which often correspond to the signal's details or edge information.
[0096] S4. Extract second image data based on the optimal resolution image and establish a proportion balance optimization model.
[0097] The step of extracting second image data based on the optimal resolution image and establishing a proportion balance optimization model is as follows:
[0098] To obtain the data processing time, imaging cost, and image fusion quality evaluation data of the optimal resolution image, and with data processing time and imaging cost as constraints, and image fusion quality evaluation data as the objective function, a proportion balance optimization model is established.
[0099] The image fusion quality assessment data includes quality assessment coefficients and information fusion preservation coefficients;
[0100] The constraints of the proportion balance optimization model are data processing time and imaging cost;
[0101] The specific method for obtaining the data processing time is as follows:
[0102] When performing multispectral and hyperspectral imaging data processing (including radiometric correction, geometric correction, super-resolution reconstruction, image fusion, and post-processing optimization steps), the time consumed by each processing step is recorded using the timing function of the computer system.
[0103] Imaging costs include satellite payload costs, data storage costs, data transmission costs, and data processing costs.
[0104] The quality assessment coefficient is a key indicator in the image fusion quality evaluation system. It quantitatively assesses the quality of the fused image from multiple dimensions, comprehensively reflecting the differences between the fused image and the original image in terms of visual effects, information integrity, and other aspects. A high-quality fused image typically has a high quality assessment coefficient, meaning that the image performs well in visual characteristics such as sharpness and contrast, clearly presenting various details, while also exhibiting natural and realistic colors that conform to human visual habits. Conversely, a low quality assessment coefficient indicates that the fused image may have problems such as blurriness or distortion, affecting its effective application.
[0105] The specific method for obtaining the quality assessment coefficient is as follows:
[0106] Obtain the optimal resolution image and the original image;
[0107] The gradient magnitude of each pixel in the fused image is calculated based on the Sobel operator.
[0108] The average gradient is calculated based on the gradient magnitude of each pixel.
[0109] The difference between the pixel value of each pixel in the original image and the optimal resolution image is calculated based on the square operation, and the ratio of the sum of all squared differences to the total number of pixels in the image is used as the mean square error.
[0110] The quality assessment coefficient is obtained by combining the mean square error and the average gradient with the maximum value of the image pixels based on a preset quality assessment coefficient calculation formula.
[0111] The formula for calculating the average gradient is:
[0112]
[0113] The formula for calculating the quality assessment coefficient is as follows:
[0114]
[0115] in, For the average gradient, The number of rows in the image. The number of columns in the image. This is the quality assessment coefficient. The gradient in the horizontal direction of the image. The gradient in the vertical direction of the image. The maximum value of the image pixels. These are the original image pixel values. The pixel values represent the optimal resolution image.
[0116] The quality assessment coefficient has the following benefits for balancing the proportion of multispectral and hyperspectral imaging in satellite remote sensing:
[0117] Improving the accuracy of image quality assessment: The quality assessment coefficient integrates multiple indicators such as mean square error, average gradient, and maximum image pixel value, enabling a more comprehensive and accurate reflection of image quality. In the balanced evaluation of multispectral and hyperspectral imaging, this coefficient ensures that the selected imaging method achieves optimal performance in terms of detail preservation and sharpness enhancement.
[0118] Optimizing the choice of imaging modality: By calculating quality assessment coefficients, the imaging effects of different imaging modalities (multispectral and hyperspectral) can be quantitatively compared. This helps in selecting the most suitable imaging modality based on specific needs and scenarios in practical applications. For example, hyperspectral imaging can be chosen in scenarios requiring high resolution and detailed spectral information; while multispectral imaging can be chosen in scenarios where spectral information requirements are not high, but overall image clarity and processing speed are more important.
[0119] Guiding the improvement and optimization of imaging technology: Quality assessment coefficients can not only be used to evaluate the quality of existing imaging technologies, but also serve as an important basis for guiding the improvement and optimization of imaging technologies. For example, by analyzing the relationship between quality assessment coefficients and different imaging parameters, key factors affecting image quality can be identified, thereby enabling targeted optimization and improvement of the imaging system.
[0120] Promoting the integration and development of multispectral and hyperspectral imaging technologies: In the process of integrating and developing multispectral and hyperspectral imaging technologies, the quality assessment coefficient can serve as an important indicator for measuring the fusion effect. By calculating the quality assessment coefficient of the fused image, the quality of the fusion technology can be evaluated, and the continuous innovation and development of related technologies can be promoted.
[0121] Enhancing the effectiveness of remote sensing applications: In remote sensing applications, the balance between multispectral and hyperspectral imaging technologies is crucial for improving application effectiveness. Quality assessment coefficients provide a scientific basis for evaluating this balance, ensuring that remote sensing applications achieve optimal results in different scenarios and with varying needs. For example, in fields such as environmental monitoring, agricultural yield estimation, and geological surveys, by rationally selecting the proportion of multispectral and hyperspectral imaging technologies, remote sensing information can be extracted and utilized more effectively, providing more accurate and reliable data for relevant decision-making.
[0122] The information preservation coefficient is a parameter used to control the degree of information retention during the information fusion process. Taking a readjustment structure as an example, in this structure, each sub-filter sends a portion of its information to the main filter during fusion with the main filter, while retaining a portion of its information internally. This proportion of retained information is the information preservation coefficient. By adjusting this coefficient, a balance can be achieved between the accuracy of the fusion result and the autonomy of each sub-filter.
[0123] The specific method for obtaining the information fusion preservation coefficient is as follows:
[0124] Obtain the grayscale data of the optimal resolution image and the original image;
[0125] Iterate through every pixel of the optimal resolution image, obtain the total number of pixels, and calculate the probability of each grayscale value appearing.
[0126] Calculate the information entropy of the optimal resolution image based on the probability of each gray value appearing.
[0127] Calculate the mean and variance of gray levels in local regions for the optimal resolution image and the original image, respectively.
[0128] The ratio of the grayscale mean to variance of the original image to the ratio of the grayscale mean to variance of the optimal resolution image is used as the information fusion rate.
[0129] The information fusion preservation coefficient is obtained by combining the information entropy of the optimal resolution image with the information fusion rate and based on a preset formula for calculating the information fusion preservation coefficient.
[0130] The probability of each grayscale value appearing is calculated using the following formula:
[0131]
[0132] The specific formula for calculating the information entropy is as follows:
[0133]
[0134] The specific formula for calculating the information fusion preservation coefficient is as follows:
[0135]
[0136] in, Let i be the probability of grayscale value i appearing in the image. This represents the number of pixels in the image where the grayscale value i appears. The total number of pixels in the optimal resolution image. For information entropy, The gray levels of the image. To preserve the coefficients for information fusion, For the original image in a local region The average gray level, For the original image in a local region gray variance For optimal resolution images in local regions The average gray level, For optimal resolution images in local regions gray variance The information entropy of the original image, The information entropy of the optimal resolution image.
[0137] The benefits of information fusion preservation coefficients in balancing the proportions of multispectral and hyperspectral imaging:
[0138] Optimized spectral information preservation: The information fusion preservation coefficient ensures that the integrity of spectral information is preserved to the greatest extent when fusing multispectral and hyperspectral images. This helps to accurately reflect different features and properties of the land surface in the fused image.
[0139] Balancing spatial resolution: By adjusting the information fusion preservation coefficient, spatial resolution can be balanced while preserving spectral information. This helps to clearly present surface details and edge information in the fused image.
[0140] Improving image quality: The application of information fusion preservation coefficients helps improve the overall quality of the fused image. By optimizing the balance between spectral information and spatial resolution, clearer, more accurate, and information-rich remote sensing images can be obtained.
[0141] Supporting diverse applications: The fused imagery has broad application value in various fields such as urban planning, environmental monitoring, disaster early warning, and management. The application of the information fusion preservation coefficient ensures that the needs of these application areas for remote sensing imagery are met, providing strong support for scientific decision-making and resource management.
[0142] The aforementioned proportion balance optimization model is as follows:
[0143]
[0144] The constraints are as follows:
[0145]
[0146]
[0147] in, The image fusion quality evaluation coefficient is used. This is the quality assessment coefficient. To preserve the coefficients for information fusion, These are the preset quality assessment coefficients and weighting coefficients. To maintain the preset information fusion coefficient weighting coefficient, For the total imaging cost, The proportion of multispectral imaging, The proportion of hyperspectral imaging, For the imaging cost of multispectral imaging, To reduce the imaging cost of hyperspectral imaging, To the upper limit of acceptable imaging costs, Total data processing time For multispectral imaging data processing time, For hyperspectral imaging data processing time, This represents the upper limit of acceptable data processing time.
[0148] Solve the above optimization model to find the optimal multispectral imaging ratio x and hyperspectral imaging ratio y that make the objective function fx reach its optimal value, while satisfying the constraints of data processing time and imaging cost.
[0149] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.
[0150] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.
[0151] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0152] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.
[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
[0154] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A satellite remote sensing-based multispectral and hyperspectral imaging proportion balancing optimization method, characterized in that, The method comprises the following steps: Collecting a first image data set, the first image data set comprising a low-resolution multispectral image and a hyperspectral image; Preprocessing the first image data set, screening a low-resolution multispectral image from the preprocessed image, improving the spatial resolution of the low-resolution multispectral image by using a super-resolution reconstruction algorithm, and generating a reconstructed multispectral image; Fusing the reconstructed multispectral image with the preprocessed hyperspectral image, and optimizing the fusion result to generate a best-resolution image; Based on the best-resolution image, extracting second image data, and establishing a proportion balance optimization model, specifically: Obtaining data processing time, imaging cost, and image fusion quality evaluation coefficient of the best-resolution image; Establishing the proportion balance optimization model with the data processing time and the imaging cost as constraint conditions and the image fusion quality evaluation coefficient as an objective function; the image fusion quality evaluation data comprises an information fusion retention coefficient, and the specific obtaining method is as follows: Obtaining the gray level data of the best-resolution image and the original image; Traversing each pixel of the best-resolution image to obtain the total number of pixels and calculate the probability of each gray value appearing; Calculating the information entropy of the best-resolution image according to the probability of each gray value appearing; Calculating the gray mean and variance of the local area of the best-resolution image and the original image respectively; Taking the ratio of the gray mean and variance of the original image to the ratio of the gray mean and variance of the best-resolution image as the information fusion rate; Combining the information entropy of the best-resolution image with the information fusion rate to obtain the information fusion retention coefficient based on a preset information fusion retention coefficient calculation formula; Solving the optimization model to obtain a parameter combination that maximizes the objective function.
2. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 1, characterized in that, The preprocessing of the first image data set comprises radiation correction of the image, specifically: Obtaining original image data collected by a satellite sensor and a corresponding calibration parameter file, the calibration parameter file comprising gain and bias calibration coefficients of each waveband; Converting the DN value of each pixel of the original image into a radiation brightness value according to the calibration parameter file; Converting the radiation brightness value into apparent reflectivity, and outputting the radiation-corrected multispectral and hyperspectral image data.
3. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 2, characterized in that, The preprocessing of the first image data set also comprises geometric correction of the image, specifically: Obtaining map data matched with the image coverage area and a data source with known geographic coordinates; Selecting ground control points according to the correspondence of ground feature points on the map data and the image, and recording the coordinates of the ground control points in the image coordinate system and the geographic coordinate system; Calculating the coefficients of a quadratic polynomial based on the GCP coordinates through a remote sensing image processing platform; Resampling the radiation-corrected image by using the transformation coefficients to realize pixel-level geographic coordinate mapping.
4. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 3, characterized in that, The screening of the low-resolution multispectral image from the preprocessed image, the improvement of the spatial resolution of the low-resolution multispectral image by using a super-resolution reconstruction algorithm, and the generation of the reconstructed multispectral image are specifically as follows: Extracting a multispectral image from the preprocessed remote sensing image data; Setting a spatial resolution threshold, the threshold being determined based on the standard resolution of the same type of satellite sensor; Traversing all spectral images to screen low-resolution images with a spatial resolution lower than the threshold; The low-resolution image is trained by constructing a training model to obtain a high-resolution multispectral image.
5. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 4, characterized in that, The training of the low-resolution image by constructing the training model further includes judging the training effect, specifically: During the training process, the Bhattacharyya distance of each waveband histogram of the generated high-resolution multispectral image and the real high-resolution reference image is calculated; The training effect is evaluated according to the change trend of the Bhattacharyya distance.
6. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 5, characterized in that, The reconstructed multispectral image is fused with the preprocessed hyperspectral image, specifically: The multispectral image and the hyperspectral image after super-resolution reconstruction are converted into grayscale images; The grayscale images are subjected to two-dimensional Fourier transform to obtain a complex matrix; The low-frequency components of the complex matrix are extracted as a low-frequency layer by a low-pass filter, and the high-frequency components are extracted as a high-frequency layer by a high-pass filter; Low-frequency layer fusion: The low-frequency layers of the multispectral and hyperspectral images are subjected to weighted averaging, and the weights are dynamically allocated according to the low-frequency information expression ability of the image scene; High-frequency layer fusion: Based on an edge detection algorithm and a texture analysis algorithm, the edge features and texture features of the high-frequency layers of the two images are extracted respectively; A joint feature descriptor is constructed, and the feature similarity between the descriptors of the two images is calculated; The weights are dynamically allocated according to the similarity matching result, and the high-frequency features of the two images are fused; The fused low-frequency layer and high-frequency layer are subjected to inverse discrete wavelet transform to generate a final fused image.
7. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 6, characterized in that, The fusion result is optimized to generate an image with the best resolution, specifically: The fused image is converted into a grayscale image and divided into pixel blocks of a predefined size; Adaptive histogram equalization is performed on each pixel block: the distribution of the grayscale levels in the pixel block is counted, and when the number of pixels of a certain grayscale level exceeds a contrast threshold, the excess pixels are uniformly distributed to other grayscale levels; Wavelet transform is performed on the equalized image to decompose it into low-frequency subbands and high-frequency subbands; Threshold denoising is performed on the high-frequency subbands: if the absolute value of the wavelet coefficient is less than the threshold, it is set to zero, and if it is greater than the threshold, a shrinkage function is applied for processing; Inverse wavelet transform is performed on the subbands after threshold processing to generate an image with the best resolution.
8. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 7, characterized in that, The proportion balance optimization model, specifically: The constraint condition is: wherein, is an image fusion quality evaluation coefficient, is a quality evaluation coefficient, is an information fusion retention coefficient, is a preset quality evaluation coefficient weight coefficient, is a preset information fusion retention coefficient weight coefficient, is a total imaging cost, is a proportion of multispectral imaging, is a proportion of hyperspectral imaging, is an imaging cost of multispectral imaging, is an imaging cost of hyperspectral imaging, is an acceptable upper limit of imaging cost, is a total data processing time, is a data processing time of multispectral imaging, is a data processing time of hyperspectral imaging, is an acceptable upper limit of data processing time.
9. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 8, characterized in that, The radiance value, specifically: The apparent reflectance, specifically: wherein, is a radiance value, is a gain scaling factor, is a digital quantized value of a pixel in the original image, is a bias coefficient, is an apparent reflectance, is a distance from the sun, is a solar irradiance at the top of the atmosphere, is a solar zenith angle.
10. The satellite remote sensing based multispectral and hyperspectral imaging proportion balance optimization method according to claim 9, characterized in that, The calculation formula of the quality evaluation coefficient is: wherein, is a quality assessment coefficient, is an average gradient, is a number of rows of the image, is a number of columns of the image, is a maximum value of pixels of the image, is an original image pixel value, is a best resolution image pixel value.
Citation Information
Patent Citations
A multi-spectral hair growth and hair regeneration data acquisition and analysis system
CN118395133B
A method and system for fusion of iron ore hyperspectral data
CN118656786B
Water quality multi-source remote sensing data rapid inversion method based on spectrum library
CN117434034A
Multispectral remote sensing image super-resolution reconstruction method and system based on knowledge distillation
CN118537225A
Multispectral image and hyperspectral image fusion method and system
CN120339016A
