Hyperspectral-to-multispectral image conversion method and system based on self-derived model

Abstract

The application discloses a hyperspectral-to-multispectral image conversion method and system based on a self-derived model, and belongs to the technical field of remote sensing image processing, and solves the problem of strong dependence on external high-resolution information in the prior art. A single low spatial resolution hyperspectral image data cube is acquired, and a plurality of band groups and a single-band low-resolution observation image are obtained; based on the band groups and the single-band low-resolution observation image, a cross-band joint degradation model and a variational energy functional are constructed; a variational method is used to convert a minimization problem of the variational energy functional into Euler-Lagrange equations; the Euler-Lagrange equations are solved to obtain preliminary high-resolution reconstruction images; and characteristic components of the plurality of preliminary high-resolution reconstruction images are extracted and synthetically reconstructed to obtain a high-resolution multispectral image. The application realizes effective enhancement of spatial details of a hyperspectral image without depending on external high-resolution images as prior information or input.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, and specifically relates to a method and system for hyperspectral to multispectral image conversion based on a self-derived model. Background Technology

[0002] Hyperspectral imagery possesses continuous and dense spectral sampling capabilities, providing abundant spectral information for ground feature identification, fine classification, and material inversion. However, limited by factors such as payload aperture, exposure time, and signal-to-noise ratio, the spatial resolution of hyperspectral imagery is usually lower than that of multispectral imagery from the same period, exhibiting a "fine spectral detail but spatial blur."

[0003] To improve the spatial resolution of hyperspectral images, existing methods often involve fusing them with external high spatial resolution panchromatic or multispectral images, or learning super-resolution mappings based on external high-resolution training data. When external high-resolution data is available and has good spatiotemporal matching and radiometric consistency, these methods can achieve certain results; however, when external data is limited, imaging conditions differ significantly, or registration errors are large, the reconstruction quality and spectral consistency are easily affected.

[0004] Therefore, there is an urgent need for an image conversion method that can enhance multispectral spatial details and maintain spectral consistency by utilizing only the cross-band correlation and structural consistency within a single low spatial resolution hyperspectral image data cube, without relying on external high-resolution images as prior or input data. Summary of the Invention

[0005] In view of the above analysis, the present invention aims to provide a hyperspectral to multispectral image conversion method and system based on a self-derived model, so as to solve the problem of strong dependence on external high-resolution information in the prior art.

[0006] The objective of this invention is achieved as follows:

[0007] A hyperspectral to multispectral image conversion method based on a self-derived model includes: acquiring a single low-spatial-resolution hyperspectral image data cube, and grouping the hyperspectral image data cube along the spectral dimension to obtain multiple band groups, and single-band low-resolution observation images of the band groups; constructing a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model based on the band groups and the corresponding single-band low-resolution observation images, wherein the variational energy functional includes at least two of the following: a data fidelity term, a spatial regularization term, a spectral consistency constraint term, and a band group cooperative regularization term; Using the variational method, the minimization problem of the variational energy functional is transformed into an equivalent Euler-Lagrange equation. When the spatial regularization term and / or the band group co-regularization term contain second-order derivative operators, the Euler-Lagrange equation manifests as a fourth-order partial differential equation. The Euler-Lagrange equation is iteratively solved to obtain preliminary high-resolution reconstructed images corresponding to single-band low-resolution observation images, and a set of preliminary high-resolution reconstruction results corresponding to each band group is output based on the band group. Feature components of multiple preliminary high-resolution reconstructed images are extracted and synthesized to reconstruct a high-resolution multispectral image.

[0008] In a preferred embodiment of the present invention, the hyperspectral image data cube is grouped along the spectral dimension to obtain multiple band groups, including: dividing the data into multiple basic band intervals based on the physical properties of spectral wavelengths; calculating the Pearson correlation coefficient matrix between any two basic band intervals, wherein the correlation coefficient matrix is ​​used to characterize the degree of linear correlation between any two bands; clustering the multiple bands using cluster analysis based on the Pearson correlation coefficient matrix, and evaluating the grouping results using information criteria to determine the number of band groups; and performing consistency constraint processing on the grouping results to obtain band grouping results; wherein the consistency constraint processing includes at least: evaluating the correlation between bands within each band group, and selecting and retaining band groups whose correlation meets preset conditions based on the evaluation results; and performing structural adjustment processing on the selected band groups, merging band groups that do not meet preset size conditions into the band group that is most similar to the selected band group in spectral properties.

[0009] A preferred embodiment of the present invention provides a hyperspectral to multispectral image conversion method based on a self-derived model, satisfying one or more of the following:

[0010] Before grouping the hyperspectral image data cube along the spectral dimension by band, the process further includes: preprocessing the hyperspectral image data cube; the preprocessing includes: radiometric calibration and atmospheric correction, spectral band selection, geometric correction and image registration, and spectral smoothing and noise removal;

[0011] The basic band range includes at least: the visible near-infrared band range, the first type of short-wave infrared band range, and the second type of short-wave infrared band range;

[0012] The information criteria are Akaike information criteria, and ,in, This represents the likelihood function when the number of groups is K; This indicates the number of parameters in the model when the number of groups is K;

[0013] A band group that meets the preset conditions refers to a band group whose normalized Pearson correlation coefficient is greater than 0.7.

[0014] In a preferred embodiment of the present invention, the cross-band joint degradation model is used to describe the mapping relationship between the high-resolution image components of each band to be reconstructed and the corresponding low-resolution observation images of a single band under the degradation effects of spatial blurring, spatial-spectral coupling downsampling, and geometric registration. The expression of the cross-band joint degradation model is as follows:

[0015] ;

[0016] in, Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first The spatial ambiguity matrix for each band is used to describe the spatial degradation characteristics of the imaging system and is modeled as a wavelength-dependent Gaussian kernel function. ,in, Indicated by fuzzy scale parameter The parameterized two-dimensional Gaussian degeneracy operator, With the center wavelength of this band group Related; This represents the geometric registration matrix, used to compensate for spatial offsets between images of different band groups; Indicates the first Additive mixed noise matrix for each band.

[0017] In a preferred embodiment of the present invention, the expression of the variational energy functional is:

[0018] ;

[0019] in, This represents the set of high-resolution image results to be reconstructed. Indicates data fidelity item, Indicates the weight of the data fidelity item; For spatial regularization, Indicates spatial smoothing weights; This represents the spectral consistency constraint term. Represents the weight of the spectral consistency constraint term; This indicates the band group coordinating regularization term. Indicates the band group collaborative weight;

[0020] Wherein, the data fidelity item satisfies:

[0021] ;

[0022] in, Indicates the total number of bands; Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first Spatial ambiguity matrix of each band; Represents the geometric registration matrix; For the first The adaptive weight matrix for the next iteration, where ⊙ represents element-wise multiplication. It is an L1 norm;

[0023] The space regularization term satisfies:

[0024] ;

[0025] in, Indicates the total number of bands; Represents the image spatial domain; Represents the second derivative operator; This represents the adaptive weighting function; Indicates the first The spatial location of each band The corresponding high-resolution image components; It is an L1 norm;

[0026] The spectral consistency constraint term satisfies:

[0027] ;

[0028] in, Indicates the total number of bands; Represents the image spatial domain; Represents pixels The neighborhood set; Indicates the first Each band in pixels High-resolution pixel values ​​at that location Indicates the first Each band in the neighborhood pixels High-resolution pixel values ​​at that location For robust penalty functions, For pixel pairs scalar weights, and the Pixels from low-resolution hyperspectral images and The spectral angular similarity is calculated and normalized to obtain the result;

[0029] The expression for the band group cooperative regularization term is:

[0030] ;

[0031] in, Indicates the number of band groups; Indicates the first The set of band indices contained in a band group Indicates the first Group-guided images of each band group; For collaborative weights; This represents the L2 norm.

[0032] In a preferred embodiment of the present invention, the step of employing a variational method to transform the minimization problem of the variational energy functional into an equivalent Euler-Lagrange equation includes:

[0033] Variational analysis is performed on the variational energy functional, and first-order variational values ​​are obtained for the data fidelity term, the spatial regularization term, and the band group cooperative regularization term, respectively. The corresponding Euler-Lagrange equations are then constructed based on the extremum conditions.

[0034] The data fidelity term, under the action of the degenerate operator, generates a corresponding gradient term through first-order variation; the space regularization term adopts a regularization form containing a second-order derivative operator, and its first-order variation result is introduced into the further differentiation of the second-order derivative operator, thereby forming an equivalent fourth-order differential operator term in the Euler-Lagrange equation.

[0035] In a preferred embodiment of the present invention, the step of iteratively solving the Euler-Lagrange equation to obtain a preliminary high-resolution reconstructed image corresponding to a single-band low-resolution observation image includes:

[0036] By introducing auxiliary variables, the minimization problem corresponding to the Euler-Lagrange equation is transformed into an equivalent constrained optimization problem, thereby achieving the separation of data fidelity terms, spatial regularization terms, spectral consistency constraint terms, and band group cooperative regularization terms in the variational energy functional.

[0037] The constrained optimization problem is solved using the split Bragman algorithm, wherein in each split Bragman iteration: the high-resolution image to be reconstructed corresponding to the band group, the auxiliary variables used to accommodate the constraint variables such as residual terms and second derivative terms, and the Bragman variables used to constrain the residual accumulation term are updated alternately.

[0038] After the split Bregman iteration reaches the preset convergence condition, the preliminary high-resolution reconstructed image corresponding to the band group is reconstructed, and the set of preliminary high-resolution reconstruction results corresponding to each band group is output according to the band grouping results.

[0039] In a preferred embodiment of the present invention, the step of extracting and synthesizing feature components from multiple preliminary high-resolution reconstructed images to obtain a high-resolution multispectral image includes:

[0040] Spectral dimension reduction is performed on the cubes composed of multiple preliminary high-resolution reconstructed images to extract the first P important feature components, where P satisfies that the cumulative energy contribution rate of the important feature components is not less than a preset threshold.

[0041] Calculate the spectral angles of each important feature component with typical land features in the standard spectral library, retain the important feature components with high matching degree, and obtain the original candidate components;

[0042] Calculate the correlation coefficient between the original candidate components, and when the correlation coefficient is greater than a preset threshold, delete at least one of the original candidate components according to a preset priority rule to obtain candidate components;

[0043] Within the image spatial domain corresponding to the hyperspectral image data cube, each candidate component is subjected to unsupervised clustering or segmentation to generate multiple pseudo-class sample regions. Based on the pseudo-class sample regions, the inter-class separability index of each candidate component is calculated, and the candidate component with the highest inter-class separability index is selected to obtain the feature component set.

[0044] A weighted linear method is used to sum the candidate components in the feature component set to obtain a multispectral band set, which serves as the high-resolution multispectral image.

[0045] A preferred embodiment of the present invention provides a hyperspectral to multispectral image conversion method based on a self-derived model, which satisfies one or more of the following:

[0046] Dimensionality reduction of the spectrum is performed using minimum noise separation and principal component analysis transformation.

[0047] The first P important feature components refer to the first P transformed components sorted from high to low according to their energy contribution rate after dimensionality reduction transformation;

[0048] Important feature components with high matching degree refer to important feature components with spectral angle less than 0.1 radians;

[0049] The preset threshold includes 0.8;

[0050] The priority rules include: when the correlation coefficient is greater than the preset threshold, the component with the lower energy contribution rate is deleted; if the difference in energy contribution rate is less than the preset difference, the component with the larger spectral angle is deleted; if it is still impossible to distinguish, the component with the higher signal-to-noise ratio is retained first.

[0051] The feature component set C is a three-dimensional real tensor with a spatial dimension of sM×sN×Qsel, where sM×sN is the spatial dimension of the image and Qsel is the number of filtered feature components.

[0052] The weighted summation yields a multispectral band set (PMS) consisting of K bands. The data cube of the multispectral band set PMS is a three-dimensional real tensor with a spatial dimension of sM×sN×K. Here, s is the scaling factor for spatial resolution, M and N correspond to the spatial dimensions of the single low spatial resolution hyperspectral image, and K corresponds to the number of spectral bands.

[0053] Based on a pre-constructed comprehensive quality evaluation system that integrates spatial and spectral characteristics, the high-resolution multispectral image is quantitatively evaluated from two dimensions: spatial quality and spectral quality, generating a comprehensive quality evaluation report. The evaluation indicators for spatial quality include at least one or more of peak signal-to-noise ratio, structural similarity index, and average gradient; the evaluation indicators for spectral quality include at least one or more of average spectral angle and relative dimensionless global error.

[0054] A hyperspectral to multispectral image conversion system based on a self-derived model includes:

[0055] The partitioning unit is used to acquire a single low spatial resolution hyperspectral image data cube, and to group the hyperspectral image data cube along the spectral dimension into multiple band groups, as well as single-band low resolution observation images of the band groups.

[0056] The establishment unit is used to construct a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model based on the band group and the corresponding single-band low-resolution observation image. The variational energy functional includes at least two of the following: a data fidelity term, a spatial regularization term, a spectral consistency constraint term, and a band group cooperative regularization term.

[0057] The processing unit is configured to use a variational method to transform the minimization problem of the variational energy functional into an equivalent Euler-Lagrange equation, and to iteratively solve the Euler-Lagrange equation to obtain a preliminary high-resolution reconstructed image corresponding to a single-band low-resolution observation image, and to output a set of preliminary high-resolution reconstruction results corresponding to each band group according to the band group; wherein, when the spatial regularization term and / or the band group co-regularization term contain a second-order derivative operator, the Euler-Lagrange equation is expressed as a fourth-order partial differential equation;

[0058] The reconstruction unit is used to extract and synthesize feature components from multiple preliminary high-resolution reconstructed images to obtain a high-resolution multispectral image.

[0059] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0060] a) The hyperspectral to multispectral image conversion method based on a self-derived model provided in this invention utilizes a single low spatial resolution hyperspectral image data cube. By performing band grouping operations, a cross-band joint degradation model is constructed. A variational energy functional is introduced, incorporating at least two constraints from data fidelity, spatial regularization, spectral consistency constraints, and band group co-regulation. Without relying on external high-resolution images as priors or inputs, the method fully exploits the spectral redundancy and spatial structure consistency within the hyperspectral image, achieving stable inversion of the high-resolution potential image. Simultaneously, through iterative solutions to higher-order Euler-Lagrange equations and feature fusion of multiple preliminary reconstruction results, the spatial details of the hyperspectral image are effectively enhanced, noise and spectral distortion are suppressed, and the spatial clarity and structural consistency of the reconstructed image are improved. Thus, through iterative solutions, a preliminary high-resolution reconstructed image can be determined, and then through extraction and synthetic reconstruction, a high-resolution multispectral image can be obtained. In this conversion process, the hyperspectral data itself is regarded as a dense set of observations containing multiple perspectives and complementary spectral responses of the same scene. By constructing a unified variational optimization model that integrates spectral dimension fidelity constraints, spatial dimension structural priors, and band group co-regularization, a self-derived reconstruction from low-resolution hyperspectral images to high-resolution, high-discrimination condensed multispectral images is achieved within the framework of the physical imaging inverse problem. Thus, the conversion method of this invention can effectively enhance the spatial details of hyperspectral images by utilizing only the cross-band correlation and structural consistency within the data cube of a single low spatial resolution hyperspectral image, while better maintaining spectral consistency and physical rationality during the enhancement process. This provides a low-complexity and robust technical approach for the stable and high-precision application of hyperspectral data.

[0061] (b) The hyperspectral to multispectral image conversion method based on a self-derived model provided in this invention deeply integrates computational imaging theory, the principle of internal information synergy, and a variational method-based partial differential equation solution framework, realizing a novel hyperspectral image enhancement paradigm of "single-source input - multi-dimensional decoupling - self-derived reconstruction." It innovatively proposes a "band grouping collaborative regularization" method, addressing and utilizing the inherent "multi-view" complementarity between bands within hyperspectral data from a physical model perspective. Thus, through a unified multi-constraint variational energy functional, the core contradiction between spatial detail recovery and spectral feature fidelity is systematically resolved. Compared to traditional fusion methods, this invention significantly improves spatial resolution while effectively reducing the risk of spectral distortion and enhancing the physical consistency and invertibility of derived spectra, exhibiting more stable performance in evaluation metrics such as mean spectral angle (SAM) and relative dimensionless global error (ERGAS).

[0062] c) The hyperspectral to multispectral image conversion method based on a self-derived model provided in this invention employs a unified modeling and collaborative optimization process, eliminating the need for any external high-resolution data sources or complex multi-source data registration and normalization preprocessing. The entire conversion method is based on a rigorous mathematical variational framework, possessing a clear theoretical foundation and interpretability. By employing efficient parallel numerical solvers such as the split Bregman algorithm, this process can run stably and efficiently on general-purpose or high-performance computing platforms. This strong self-containment frees it from dependence on specific satellite constellations or transit times, making it suitable for single-image hyperspectral image processing tasks under different scenarios and sensor conditions, thereby improving processing flexibility and engineering adaptability, and enhancing the stability and interpretability of the results.

[0063] d) The hyperspectral to multispectral image conversion method based on a self-derived model provided in this invention constructs a complete and automated processing chain from raw low-resolution hyperspectral data input to high-quality, high-resolution condensed multispectral product output. It systematically integrates data preprocessing, model building, collaborative reconstruction, and intelligent condensation modules, allowing users to obtain high-value images directly usable for analysis without requiring extensive expertise in remote sensing image processing. This significantly lowers the barrier to entry for hyperspectral technology, providing analysts in various application fields with a stable, objective, and efficient image enhancement tool, and helping to promote the transition of hyperspectral remote sensing technology from typical demonstrations to large-scale operational applications.

[0064] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings.

[0066] Figure 1 A flowchart of a hyperspectral to multispectral image conversion method based on a self-derived model provided by the present invention;

[0067] Figure 2 A flowchart of a band group division method provided by the present invention;

[0068] Figure 3This invention provides a schematic diagram illustrating the principle of constructing a cross-band joint degradation and multi-constraint energy functional model.

[0069] Figure 4 A flowchart for reconstructing a preliminary high-resolution multispectral image is provided by the present invention;

[0070] Figure 5 This invention provides a principle block diagram for collaborative optimization in solving constrained optimization problems.

[0071] Figure 6 A flowchart for determining high-resolution multispectral images provided by the present invention;

[0072] Figure 7 This is a schematic diagram of the structure of a hyperspectral to multispectral image conversion system based on a self-derived model provided by the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] To facilitate understanding of the embodiments of this application, further explanation and description will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of this application. In the drawings, the dimensions and relative dimensions of components may be exaggerated for clarity and / or descriptive purposes. When exemplary embodiments can be implemented differently, a specific process sequence may be performed in a different order than described. For example, two consecutively described processes may be performed substantially simultaneously or in the reverse order of their description. Furthermore, the same reference numerals denote the same components.

[0075] The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used herein, unless the context clearly indicates otherwise, the singular forms “a” and “the” are intended to include the plural forms as well. Furthermore, when the terms “comprising” and / or “including” and variations thereof are used in this specification, it indicates the presence of the stated features, integrals, steps, operations, parts, components, and / or groups thereof, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, parts, components, and / or groups thereof. It should also be noted that, as used herein, the terms “substantially,” “about,” and other similar terms are used as approximate terms rather than as terms of degree, thus explaining the inherent biases in measurements, calculated values, and / or provided values ​​that will be recognized by those skilled in the art.

[0076] As described in the background section, the core technical challenge that has long remained unresolved in this field is: how to extract information potential from a single low-resolution hyperspectral image without relying on any external high-resolution data source, and to overcome the dual bottlenecks of spatial resolution and spectral information utilization efficiency.

[0077] Research has found that traditional interpolation methods cannot create realistic details beyond the Nyquist frequency; while mainstream deep learning methods, without external prior training, struggle to stably reconstruct physically reliable high-frequency information from highly ill-conditioned underdetermined problems. Although computational imaging theory and sparse representation offer new insights, existing solutions mostly target single-dimensional degradation (such as spatial blur only), lacking effective internal collaborative reconstruction mechanisms for the coupled degradation models of "high-dimensional spectrum-low-dimensional space" in hyperspectral images.

[0078] Against this backdrop, there is an urgent need to break through the traditional predicament of "hardware dependence-data coupling-illness solution" and create a new paradigm of computational imaging that is "data-endogenous, self-supervised, and cross-band collaborative".

[0079] Based on this, the present invention provides a hyperspectral to multispectral image conversion method based on a self-derived model. Utilizing a single low-spatial-resolution hyperspectral image data cube, a cross-band joint degradation model is constructed by performing band grouping operations. A variational energy functional is introduced, incorporating at least two constraints from data fidelity, spatial regularization, spectral consistency constraints, and band group co-regulation. Without relying on external high-resolution images as priors or inputs, the method fully exploits the spectral redundancy and spatial structural consistency within the hyperspectral image, achieving stable inversion of the high-resolution potential image. Simultaneously, through iterative solutions to higher-order Euler-Lagrange equations and feature fusion of multiple preliminary reconstruction results, the spatial details of the hyperspectral image are effectively enhanced, noise and spectral distortion are suppressed, and the spatial clarity and structural consistency of the reconstructed image are improved. Thus, through iterative solutions, a preliminary high-resolution reconstructed image can be determined, and then, through extraction and synthetic reconstruction, a high-resolution multispectral image can be obtained.

[0080] In this transformation process, the hyperspectral data itself is regarded as a dense set of observations containing multiple perspectives and complementary spectral responses of the same scene. By constructing a unified variational optimization model that integrates spectral dimension fidelity constraints, spatial dimension structural priors and band group co-regularization, the self-derived reconstruction from low-resolution hyperspectral images to high-resolution, high-discrimination condensed multispectral images is achieved within the framework of the physical imaging inverse problem.

[0081] Thus, the conversion method of the present invention can effectively enhance the spatial details of hyperspectral images by utilizing only the cross-band correlation and structural consistency within the cube of a single low spatial resolution hyperspectral image data, and better maintain spectral consistency and physical rationality during the enhancement process, thereby providing a low-complexity and robust technical approach for the stable and high-precision application of hyperspectral data.

[0082] A specific embodiment of the present invention, such as Figure 1 The flowchart shown illustrates a hyperspectral to multispectral image conversion method based on a self-derived model, comprising:

[0083] S101, acquire a single low spatial resolution hyperspectral image data cube, and group the hyperspectral image data cube along the spectral dimension to obtain multiple band groups, as well as single-band low resolution observation images of the band groups.

[0084] In some embodiments, a single low spatial resolution hyperspectral image contains tens to hundreds of consecutive bands. In this embodiment, the data cube is re-characterized as an internal set of observations of the same ground scene with natural multi-view complementarity.

[0085] In this way, after acquiring a single low spatial resolution hyperspectral image data cube, the entire band can be adaptively divided into several band groups based on one or more of the physical characteristics of the spectral response (such as absorption features, atmospheric windows) or statistical correlation.

[0086] The images within each band group can be viewed as an observation of the spatial structure of the scene from different "spectral perspectives," providing an information basis for subsequent collaborative reconstruction without introducing external data.

[0087] Specifically, according to the preset grouping strategy, all spectral bands are divided into multiple non-overlapping band groups, thereby realizing the division of all spectral bands into several band groups.

[0088] Through the above processing, multiple band groups and single-band low-resolution observation images corresponding to each band group can be obtained from a single low-spatial-resolution hyperspectral image data cube, providing basic data support for subsequent image reconstruction, fusion or super-resolution processing.

[0089] In other words, step S101 aims to achieve hyperspectral data input and internal multi-view characterization.

[0090] In one example, a single low-spatial-resolution hyperspectral image data cube can be acquired by airborne, spaceborne, or ground-based hyperspectral imaging equipment, or it can be derived from an existing hyperspectral image dataset. This hyperspectral image data cube has low spatial resolution in the spatial dimension and contains multiple continuous or quasi-continuous spectral bands in the spectral dimension.

[0091] For example: a cube of single low spatial resolution hyperspectral image data Where M is the length of the spatial dimension, N is the width of the spatial dimension, and B is the number of spectral bands (for example, if an AVIRIS sensor is used, B=188; if an AVIRIS-NG sensor is used, B=224).

[0092] In other words, this embodiment groups these B spectral bands (or simply bands) into different band groups, thus creating a cube of single low spatial resolution hyperspectral image data. The division yields the first Low-resolution observation images in each band .

[0093] See Figure 2 The flowchart shown is a method for dividing band groups provided by the present invention. Figure 2 As shown, the following can be executed:

[0094] S201, based on the physical properties of spectral wavelengths, divides and forms multiple basic band intervals.

[0095] In some embodiments, the data cube is a multidimensional data structure containing spatial and spectral dimensions, wherein the spectral dimension consists of multiple continuous or discrete spectral bands. Different spectral bands correspond to different wavelength ranges, and their positions and physical properties (such as visible light, near-infrared, and short-wave infrared) in the electromagnetic spectrum differ significantly. By analyzing the wavelength ranges and physical characteristics corresponding to each band, the multiple bands in the data cube are initially divided, and bands with similar wavelength ranges and similar physical response characteristics are grouped into the same basic band range.

[0096] The initial division helps reduce the complexity of subsequent spectral processing and provides a structured band basis for subsequent feature extraction, band selection or spectral analysis, while ensuring the physical rationality of the grouping.

[0097] In practice, the division of the basic band interval can be preset or dynamically adjusted according to the application scenario requirements, such as based on the sensor's spectral response characteristics, the spectral absorption characteristics of the target substance, or predefined spectral segmentation rules.

[0098] In one example, the basic band range includes: the visible near-infrared band range (e.g., VNIR: 400-1000nm, corresponding to the dominant ground object reflection, suitable for preliminary identification of vegetation and water bodies), the first type of shortwave infrared band range (e.g., SWIR-I: 1000-1800nm, containing mineral and vegetation moisture absorption characteristics, suitable for fine classification of ground objects), and the second type of shortwave infrared band range (e.g., SWIR-II: 1800-2500nm, containing absorption bands related to rock and soil characteristics, suitable for geological exploration scenarios).

[0099] It should be noted that the division of the basic band interval can also be adjusted according to the needs of the application scenario.

[0100] S202, calculate the Pearson correlation coefficient matrix between any two basic band intervals, the correlation coefficient matrix being used to characterize the degree of linear correlation between any two bands.

[0101] In some embodiments, each basic band interval corresponds to a set of bands. By performing statistical analysis on the band characteristic data within each basic band interval, the Pearson correlation coefficient between any two basic band intervals can be calculated, thereby constructing a Pearson correlation coefficient matrix.

[0102] Specifically, the Pearson correlation coefficient is used to measure the consistency of the changing trends of data between two band intervals, and its value ranges from -1 to 1. The larger the absolute value of the correlation coefficient, the stronger the linear correlation between the two bands. When the correlation coefficient is close to 1, it indicates a positive correlation between the two bands; when the correlation coefficient is close to -1, it indicates a negative correlation; and when the correlation coefficient is close to 0, it indicates a weak linear correlation between the two bands.

[0103] By constructing a correlation coefficient matrix, the correlation between each basic band interval can be intuitively reflected, providing a basis for subsequent feature selection, band redundancy analysis, or pattern recognition, thereby improving the accuracy and stability of data analysis.

[0104] S203, based on the Pearson correlation coefficient matrix, cluster analysis is used to cluster multiple bands, and the grouping results are evaluated using information criteria to determine the number of band groups.

[0105] In some embodiments, cluster analysis is performed on multiple bands based on the band correlation matrix to form multiple band groups. During the clustering process, information criteria are introduced to evaluate the clustering results corresponding to different numbers of groups, and the number of band groups is adaptively determined based on the evaluation results, thereby avoiding reliance on manual experience to set the number of groups.

[0106] In one example, the information criterion is the Akaike information criterion, and the optimal number of groups K is adaptively determined using the Akaike information criterion (AIC).

[0107] in, ,in, This represents the likelihood function for a group number of K. The larger the value, the better the model fits the band data in the current group; This represents the number of parameters in the model when the number of groups is K. The more parameters there are, the higher the model complexity. In one example, the value of K ranges from 3 to 8.

[0108] In this embodiment, through The model with excessively high complexity is penalized to avoid overfitting.

[0109] S204, perform consistency constraint processing on the grouping results to obtain band grouping results; wherein, the consistency constraint processing includes at least: evaluating the correlation between bands within each band group, and selecting and retaining band groups whose correlation meets preset conditions based on the evaluation results; performing structural adjustment processing on the selected band groups, and merging band groups that do not meet preset size conditions into the band group that is most similar to the band group in spectral properties.

[0110] In some embodiments, the correlation between bands within each band group is evaluated based on the obtained band grouping results. The band groups are then screened based on the evaluation results, and band groups whose band correlation within the group meets preset conditions are retained, so as to enhance the consistency and reliability of the grouping results.

[0111] Furthermore, the structural characteristics of the selected band groups are further examined. When there are outlier band groups that contain fewer bands and have significant differences in correlation or spectral properties compared to other band groups, they are merged into the band group that is most similar in spectral properties or statistical characteristics to avoid adverse effects of abnormal grouping on the overall structure.

[0112] In one example, a band group that meets the preset criteria is a band group with a normalized Pearson correlation coefficient greater than 0.7.

[0113] More specifically, the normalized mean of the correlation coefficients for all bands within each group is calculated, and groups with a value greater than 0.7 are retained. In particular, if there are "outliers" containing only 1-2 bands, they are merged into the band group with the closest spectral distance to ensure strong correlations among the bands within each retained group.

[0114] The scheme in the above example realizes a grouping strategy that integrates "physical prior and adaptive clustering". Physical constraints are introduced during the grouping process to ensure the physical consistency of the grouping results. At the same time, the adaptive clustering method is used to dynamically adjust the grouping method according to the data characteristics. Thus, while ensuring the rationality of the grouping, the adaptability, stability and robustness of the grouping results are improved, making it suitable for complex or dynamically changing application scenarios.

[0115] In other words, after the above processing, the band groups in the final output band grouping results have good consistency in terms of physical properties and statistical correlation.

[0116] In some embodiments, a standard preprocessing procedure can also be performed on a single low spatial resolution hyperspectral image data cube to improve data quality.

[0117] More specifically, before grouping the hyperspectral image data cube along the spectral dimension, the method further includes: preprocessing the hyperspectral image data cube; the preprocessing includes: radiometric calibration and atmospheric correction, spectral band selection, geometric correction and image registration, and spectral smoothing and noise removal.

[0118] Among them, radiation calibration and atmospheric correction refer to converting the raw digital quantization values ​​(i.e., DN values) collected by the sensor into surface radiance or reflectance data to eliminate systematic errors caused by differences in sensor response; then, atmospheric correction is performed on the radiance values ​​to eliminate the effects of atmospheric scattering and absorption, so as to obtain the true surface reflectance data.

[0119] Spectral band screening refers to the automatic identification and removal of invalid bands (such as the 1350-1450nm and 1800-1950nm water vapor absorption bands) that are subject to severe atmospheric absorption or have excessively low signal-to-noise ratios. The removal criteria employ a dual threshold judgment, for example: or in: This is the signal-to-noise ratio threshold (typical value 20). These are minimum radiance thresholds, which can be adaptively set according to the sensor type and imaging scenario.

[0120] Spectral smoothing and noise removal: The spectral curve of each pixel in the image after geometric correction and registration is smoothed by Savitzky-Golay filtering; then, MNF transformation is performed on the smoothed image data to separate and remove noise components, resulting in a final high-quality data cube.

[0121] In this case, the remaining preprocessed data is... Each band is divided into K band groups with clearly defined physical meanings. .

[0122] Geometric correction and image registration refers to performing geometric correction on images after band selection to eliminate geometric distortions caused by platform attitude changes and terrain undulations; and registering the image with bands within the same cube to give the image accurate geographic coordinates.

[0123] Spectral smoothing and noise removal refers to applying Savitzky-Golay filtering to smooth the spectral curve of each pixel in the image after geometric correction and registration; and then performing MNF transformation on the smoothed image data to separate and remove noise components, resulting in a final high-quality data cube.

[0124] S102, Based on the band group and the corresponding single-band low-resolution observation image, construct a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model. The variational energy functional includes at least two of the following: a data fidelity term, a spatial regularization term, a spectral consistency constraint term, and a band group co-regularization term.

[0125] In some embodiments, by establishing a unified joint degradation model that couples spatial and spectral dimensions, the observed low-resolution hyperspectral image (i.e., a hyperspectral image corresponding to a single low spatial resolution image) can be formalized into the result of degradation mapping of the high-resolution image component to be determined through spatial blurring, downsampling and geometric registration, and then superimposed with noise.

[0126] This cross-band joint degradation model considers both the shared spatial degradation characteristics between different bands and the correlation between bands, so that the degradation process of different bands can be described under a unified model framework, thereby avoiding the information fragmentation problem caused by independent modeling of each band.

[0127] Furthermore, to solve the inverse problem corresponding to the cross-band joint degradation model, a corresponding variational energy functional (i.e., objective function) is constructed. This variational energy functional uses the multi-band high-resolution images to be reconstructed as optimization variables, and achieves joint image reconstruction by minimizing the variational energy functional.

[0128] Preferably, the variational energy functional includes at least two of the following terms, and these terms can be combined and / or weighted according to specific application requirements:

[0129] The data fidelity term is used to constrain the reconstructed image to be consistent with the corresponding single-band low-resolution observation image in terms of statistical significance or error significance after being processed by the cross-band joint degradation model.

[0130] Spatial regularization terms are used to characterize the prior properties of the reconstructed image in the spatial domain, in order to suppress noise and preserve image structural information, thereby enhancing the spatial smoothness or edge preservation ability of the reconstructed image.

[0131] The spectral consistency constraint is used to constrain the spectral response of the same pixel location in different bands to meet certain continuity or consistency assumptions.

[0132] The band group cooperative regularization term is the key to the "internal information mining" of this invention. It forces all bands within the same physical group to share and optimize common spatial structure details during the reconstruction process, thereby using the complementarity within the hyperspectral data to constrain ill-conditioned reconstruction problems.

[0133] Through the above technical solution, the present invention can fully utilize single-band low-resolution observation images while introducing prior constraints at the cross-band and band group levels to achieve high-quality joint reconstruction of multi-band images, effectively improving spatial resolution and maintaining good spectral consistency.

[0134] It should be noted that this invention introduces a cross-band joint degradation modeling mechanism during multi-band image processing, explicitly establishing correlation constraints between bands at the model level, and constructing corresponding variational energy functions accordingly, enabling the reconstruction process of each band to proceed within the same optimization framework. Compared to existing band-by-band independent modeling methods, this invention can utilize complementary information between different bands during the optimization process to jointly suppress degradation errors, thereby reducing the sensitivity of the optimization results to noise and outliers in a single band, and improving the stability and consistency of the overall reconstruction results.

[0135] In some embodiments, the cross-band joint degradation model is used to describe the mapping relationship between the high-resolution image components of each band (or band group) to be reconstructed and the corresponding low-resolution observation images of a single band under the degradation effects of spatial blurring, spatial-spectral coupling downsampling, and geometric registration. The expression of the cross-band joint degradation model is as follows:

[0136] ;

[0137] in, Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first The spatial ambiguity matrix (or point spread function matrix) for each band is used to describe the spatial degradation characteristics of the imaging system and is modeled as a wavelength-dependent Gaussian kernel function. ,in, Indicated by fuzzy scale parameter The parameterized two-dimensional Gaussian degeneracy operator, With the center wavelength of this band group Related, among which, center wavelength It is determined by the average value or weighting center of the wavelengths of each band within the band group; This represents the geometric registration matrix, used to compensate for spatial offsets between images of different band groups; Indicates the first Additive mixed noise matrix for each band.

[0138] Correspondingly, the expression for the variational energy functional is:

[0139] ;

[0140] in, This represents the set of high-resolution image results to be reconstructed. This represents the data fidelity term, used to measure the difference between degraded observations and reconstructed results after degradation mapping. Indicates the weight of the data fidelity item; This is a spatial regularization term used to constrain the spatial smoothness and structure preservation of the reconstructed image. Indicates spatial smoothing weights; This represents a spectral consistency constraint term used to maintain spectral correlations across bands / band groups. Represents the weight of the spectral consistency constraint term; This represents a band group cooperative regularization term, used to enhance cooperative prior information within the same band group and cross-group information complementarity. This indicates the collaborative weight of the band group.

[0141] It should be noted that the functional constructed in this embodiment is not a simple superposition of conventional regularization models, but rather a targeted design based on systematic analysis to address multi-level constraints such as spatial structure preservation, spectral curve fidelity, and inter-band synergy. The aforementioned regularization terms are coupled and synergistic during the optimization process, jointly achieving comprehensive constraints on the reconstruction results at multiple scales: space, spectrum, and band groups.

[0142] In some embodiments, the data fidelity item satisfies:

[0143] ;

[0144] in, Indicates the total number of bands; Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first Spatial ambiguity matrix of each band; Represents the geometric registration matrix; For the first The adaptive weight matrix for the next iteration, where ⊙ represents element-wise multiplication. It is an L1 norm.

[0145] The space regularization term satisfies:

[0146] ;

[0147] in, Indicates the total number of bands; Represents the image spatial domain; Represents the second derivative operator; This represents the adaptive weighting function; Indicates the first The spatial location of each band The corresponding high-resolution image components; It is an L1 norm.

[0148] It should be noted that, in order to facilitate the first-order variational analysis of the variational energy functional containing the L1 norm and to derive a solvable Euler-Lagrange equation, an iteratively reweighted strategy is employed in some embodiments:

[0149] In the iteration, the adaptive weight matrix / weight function is fixed, and the L1 norm terms are equivalently transformed into a weighted L2 norm form for solution. The weights are updated in the next iteration based on the current residual and second derivative response. Therefore, the L1 norm is used at the model definition level to enhance robustness to noise and outliers, while the equivalent "weighted L2 form" can be used at the numerical solution and derivation level to obtain a differentiable variational expression.

[0150] The spectral consistency constraint term satisfies:

[0151] ;

[0152] in, Indicates the total number of bands; Represents the image spatial domain; Represents pixels The neighborhood set; Indicates the first Each band in pixels High-resolution pixel values ​​at that location Indicates the first Each band in the neighborhood pixels High-resolution pixel values ​​at that location For robust penalty functions, For pixel pairs scalar weights, and the Pixels from low-resolution hyperspectral images and The spectral angular similarity was calculated and normalized to obtain the result.

[0153] The expression for the band group cooperative regularization term is:

[0154] ;

[0155] in, Indicates the number of band groups; Indicates the first The set of band indices contained in a band group Indicates the first Group-guided images of each band group; For collaborative weights; This represents the L2 norm.

[0156] In some embodiments, the band group co-regulation term employs the L2 norm to stabilize the overall difference between the group-guided image and the bands within the group; when stronger robustness is required, it can be replaced with the L1 norm form.

[0157] In some embodiments, group guide images Can be determined by a set of band indices The reconstruction results of each band group were obtained by weighted fusion based on signal-to-noise ratio:

[0158] .

[0159] in, This represents the signal-to-noise ratio of the b-th band group, used to reflect the reliability or observation quality of the reconstruction results for that band group; This represents the signal-to-noise ratio of the b'-th band group; The fusion coefficient (weight) of the b-th band group within set g is given by... Normalization yields the result.

[0160] It should be noted that b represents the index of a specific band group currently involved in the calculation; b' represents the index of the set. The summation index used when performing summation traversal.

[0161] See Figure 3 The diagram shown illustrates the principle of constructing a cross-band joint degradation and multi-constraint energy functional model provided by the present invention. Figure 3 As shown, it includes: physical degradation processes and energy functional processes, wherein:

[0162] Physical degradation refers to the process through spatial fuzzy matrices Spatial-spectral coupled downsampling matrix Geometric registration matrix and additive mixed noise matrix The set of high-resolution image results to be reconstructed Mapping to low-resolution observation images This enables the mapping of high-resolution images to low-resolution observation images.

[0163] Correspondingly, the energy functional process includes: based on data fidelity terms Space regularization term F space Spectral consistency constraint and band group coordinating regularization terms The unified variational energy functional construction equation is obtained, and then the set of high-resolution image results is determined. .

[0164] S103, using the variational method, the minimization problem of the variational energy functional is transformed into an equivalent Euler-Lagrange equation; wherein, when the spatial regularization term and / or the band group cooperative regularization term contain second-order derivative operators, the Euler-Lagrange equation is expressed as a fourth-order partial differential equation.

[0165] In some embodiments, after the variational energy functional is constructed, the variational method is used to minimize the variational energy functional.

[0166] Specifically, by performing a first-order variation on the variational energy functional, the problem of minimizing the variational energy functional is transformed into the equivalent Euler-Lagrange equation, thereby transforming the original optimization problem into a partial differential equation solution problem.

[0167] Furthermore, when the spatial regularization term and / or the band group co-regularization term contain second-order derivative operators, such as the Laplace operator or the biLaplace operator, the equivalently determined Euler-Lagrange equations are expressed as fourth-order partial differential equations.

[0168] In this way, the present invention uses the variational method to transform the complex constrained optimization problem into a high-order partial differential equation model with clear meaning. This not only effectively integrates spatial information and band group collaborative information, but also improves the performance of image reconstruction results in terms of structure preservation and smoothness, thereby achieving high-quality processing of multi-band images.

[0169] In one embodiment, step S103 may include: performing variational analysis on the variational energy functional, obtaining first-order variational values ​​for the data fidelity term, the spatial regularization term, and the band group co-regularization term, and constructing the corresponding Euler-Lagrange equation based on the extremum condition; wherein, the data fidelity term, under the action of the degeneracy operator, generates a corresponding gradient term through first-order variation; the spatial regularization term adopts a regularization form containing a second-order derivative operator, and its first-order variational result is introduced into the further differentiation of the second-order derivative operator, thereby forming an equivalent fourth-order differential operator term in the Euler-Lagrange equation.

[0170] For example, for any band Define the corresponding sub-energy functional. During the iteration process, fix the adaptive weights obtained by updating the residuals and second derivative responses of the previous iteration. Represent the data fidelity term and the space regularization term in the form of a weighted L2 norm, which facilitates variational derivation and numerical solution.

[0171] ;

[0172] in, Indicates the first Low-resolution observation images in one band; Represents a linear degeneracy operator. ; This represents an adaptive weighting function (e.g., a spatial weighting function). For the Laplace operator; Indicates the weight of the data fidelity item; Indicates the first Each band corresponds to a high-resolution image component. This represents the L2 norm (where it is used as an equivalent representation of the L1 norm term in iterative reweighted solutions). This represents the spatial smoothing weights. Next, the sub-energy functional... Variational analysis is performed by introducing arbitrary perturbation functions. ,make and in If we set the first-order variational to zero, then:

[0173] The variables of data fidelity items are: The corresponding second-order derivative regularization term is:

[0174] ;

[0175] Next, regarding Performing integration by parts twice, we get:

[0176] ;

[0177] The Euler-Lagrange equations are:

[0178] ;

[0179] because Approximate constant, then .

[0180] In other words, by employing the variational method, a set of equivalent adaptive fourth-order partial differential equations is derived from the energy functional minimization problem described above. The SplitBregman algorithm is then used to systematically and efficiently solve these fourth-order partial differential equations numerically.

[0181] S104, the Euler-Lagrange equation is solved iteratively to obtain the preliminary high-resolution reconstructed image corresponding to the single-band low-resolution observation image, and the set of preliminary high-resolution reconstruction results corresponding to each band group is output according to the band group.

[0182] In some embodiments, the spatial subproblem is solved by utilizing an adaptive higher-order PDE update term (including a nonlinear diffusion term for edge preservation and an equivalent fourth-order structure preservation term generated by second-order regularization) introduced by the first-order variation of the data fidelity gradient term and the spatial regularization term, and optionally by introducing a gradient-based detail enhancement term to improve texture and edge sharpness, thereby updating the spatial details of the high-resolution image.

[0183] Solving the spectral subproblem involves optimizing the spectral vector of each pixel under the joint constraints of spectral fidelity and band group synergy terms. This ensures the physical authenticity of the pixel and enhances the structural consistency of bands within a band group. The iterative process continues until the reconstruction results converge, outputting a preliminary high-resolution reconstruction set for each band group. This set is then used for feature extraction and synthetic reconstruction in subsequent steps to obtain the final high-resolution multispectral image.

[0184] See Figure 4 The flowchart shown below illustrates a method for reconstructing preliminary high-resolution multispectral images provided by the present invention, such as... Figure 4 As shown, it includes:

[0185] S401, by introducing auxiliary variables, the minimization problem corresponding to the Euler-Lagrange equation is transformed into an equivalent constrained optimization problem, so as to achieve the separation of data fidelity terms, spatial regularization terms, spectral consistency constraint terms, and band group cooperative regularization terms in the variational energy functional.

[0186] In some embodiments, to facilitate numerical solutions to optimization problems with multiple regularization terms, the split Bregman algorithm can be used to iteratively solve the constrained optimization problem. The split Bregman algorithm decomposes the original problem into several subproblems through variable splitting, and updates the main variables, auxiliary variables, and Bregman variables through alternating iterations to gradually satisfy the constraints and achieve convergence.

[0187] S402, the split Bregman algorithm is used to solve the constrained optimization problem, wherein in each split Bregman iteration: the high-resolution image to be reconstructed corresponding to the band group, the auxiliary variables used to accept the constraint variables such as residual terms and second derivative terms, and the Bregman variables used to constrain the residual accumulation term are updated alternately.

[0188] In some embodiments, the above alternating updates can be achieved by constructing an augmented Lagrangian function: updating the main variable while keeping the auxiliary and Bregman variables fixed; updating the auxiliary variable while keeping the main variable fixed; and then updating the Bregman variable based on the current constraint residuals to penalize the degree of constraint violation and drive iterative convergence.

[0189] In a further embodiment, when the changes in the main variable and auxiliary variable are less than a preset threshold, or when the objective function value meets a preset convergence condition, the iteration process is terminated, and the current iterative solution is taken as the output of the constrained optimization problem. Here, the main variable is the reconstruction variable of the initial high-resolution multispectral image, the auxiliary variable is the splitting variable used for regularization, and the Bregman variable is the variable corresponding to the constraint residual.

[0190] S403, after the split Bregman iteration reaches the preset convergence condition, the preliminary high-resolution reconstructed image corresponding to the band group is reconstructed, and the set of preliminary high-resolution reconstructed results corresponding to each band group is output according to the band grouping results.

[0191] In some embodiments, after iterative convergence, it indicates that reconstruction has been completed, thereby obtaining the preliminary high-resolution reconstructed image corresponding to the band group, and outputting the preliminary high-resolution reconstruction result set corresponding to each band group according to the band grouping result, and outputting the preliminary high-resolution multispectral image.

[0192] It should be noted that more details about the split Bregman algorithm can be found in the existing examples.

[0193] In one specific embodiment, see Figure 5 The diagram shown is a principle block diagram of a collaborative optimization solution to a constrained optimization problem provided by the present invention. Figure 5 As shown, the following can be executed:

[0194] S501, initialize the high-resolution image U to be reconstructed, as well as the corresponding weight parameter λ and convergence threshold.

[0195] S502, set the iteration count m=0.

[0196] S503, execute the iterative loop to solve for the JOIN space regularization term respectively. The corresponding subproblem involves updating the spatial constraint variables and solving for the spectral consistency constraint term. The spectral constraint variables are updated by updating the subproblems corresponding to the updated spectral vectors, and based on the band group cooperative regularization term. Update the group guide image Mg and calculate the relevant residuals.

[0197] S504, determine whether the change in residual or objective function is less than the convergence threshold; if yes, proceed to step S505 and output the preliminary high-resolution reconstructed image; if no, return to step S503.

[0198] S104, extract and synthesize the feature components of the multiple preliminary high-resolution reconstructed images to obtain a high-resolution multispectral image.

[0199] In some embodiments, the preliminary high-resolution multispectral image can be subjected to spectral dimension condensation processing to obtain a set of feature components with strong ground feature differentiation capabilities through dimensionality reduction and screening, and then further synthesized to generate a multispectral product, namely a high-resolution multispectral image.

[0200] See Figure 6 The flowchart shown in this invention provides a method for determining high-resolution multispectral images, such as... Figure 6 As shown, it includes:

[0201] S601, spectral dimension reduction processing is performed on the cubes composed of multiple preliminary high-resolution reconstructed images to extract the first P important feature components, wherein P satisfies that the cumulative energy contribution rate of the important feature components is not less than a preset threshold.

[0202] In some embodiments, spectral dimension reduction is performed using minimum noise separation and principal component analysis transformation.

[0203] More specifically, the Minimum Noise Separation (MNF) transform is selected, and the signal and noise are effectively separated through two PCA transforms. Furthermore, the cumulative signal-to-noise ratio contribution rate method is used to select the top P MNF components (i.e., the top P important feature components refer to the top P transformed components sorted from high to low energy contribution rate after dimensionality reduction transform), so that their cumulative signal-to-noise ratio contribution is greater than 95% of the total signal-to-noise ratio.

[0204] It should be noted that if the spectral properties of ground features satisfy a linear mixture model and the goal is to obtain endmembers, independent component analysis (ICA) can be used.

[0205] S602, calculate the spectral angle of each important feature component with that of typical land features in the standard spectral library, retain the important feature components with high matching degree, and obtain the original candidate components.

[0206] In some embodiments, a multi-criteria screening strategy is used to select the most discriminative features from the dimensionality-reduced components, that is, to calculate the spectral angle (SAM) of each component with typical land features in standard spectral libraries (such as USGS, JHU) and retain the components with high matching degree.

[0207] For example, important feature components with high matching degree refer to important feature components with spectral angle less than 0.1 radians.

[0208] S603, calculate the correlation coefficient between the original candidate components, and in response to the correlation coefficient being greater than a preset threshold, delete at least one of the original candidate components according to a preset priority rule to obtain candidate components.

[0209] The priority rules include: prioritizing the retention of components with higher energy contribution rates; if the difference in energy contribution rates between the two is less than a preset difference, prioritizing the retention of components with higher matching degree (smaller spectral angle) to the standard spectral library; if they still cannot be distinguished, prioritizing the retention of components with higher signal-to-noise ratios, or deleting those with lower signal-to-noise ratios.

[0210] In some embodiments, the correlation coefficients between the original candidate components are further determined, and at least one original candidate component is removed by means of a preset threshold.

[0211] For example, the preset threshold includes 0.8, that is, if the correlation coefficient is greater than 0.8, one of the original candidate components is removed based on the signal-to-noise ratio or physical meaning.

[0212] S604, within the image spatial domain corresponding to the hyperspectral image data cube, unsupervised clustering or segmentation is performed on each candidate component to generate multiple pseudo-class sample regions. Based on the pseudo-class sample regions, the inter-class separability index of each candidate component is calculated, and the candidate component with the highest inter-class separability index is selected to obtain the feature component set.

[0213] In some embodiments, the inter-class separation index can be characterized by the Jeffries–Matusita (JM) distance or transform divergence (TD) between samples. S605, a weighted linear method is used to sum the candidate components in the feature component set to obtain a multispectral band set, which serves as the high-resolution multispectral image.

[0214] Accordingly, the weighted summation yields a multispectral band set PMS (e.g., a data cube of the multispectral band set PMS) consisting of K bands. The multispectral band set PMS is a three-dimensional real tensor with a spatial dimension of sM×sN×K. Here, s is the scaling factor of the spatial resolution, M and N correspond to the spatial dimensions of the single low spatial resolution hyperspectral image, and K corresponds to the number of spectral bands.

[0215] More specifically, for the selected Qsel feature component set C, where C is a three-dimensional real tensor with spatial dimensions of sM×sN×Qsel, then sM×sN is the spatial dimension of the image, and Qsel is the number of selected feature components. C is then synthesized using a weighted linear method, obtaining a multispectral band set of K bands through weighted summation, with spatial dimensions of sM×sN×K.

[0216] By performing synthesis, it is possible to maximize the total amount of information in the output band set while ensuring good independence between bands and further improving the separability for the target application task.

[0217] In some embodiments, the hyperspectral to multispectral image conversion method based on a self-derived model may further include: quantitatively evaluating the PMS data cube from two dimensions, spatial quality and spectral quality, based on a pre-constructed comprehensive quality evaluation system, and generating a quality evaluation report.

[0218] Spatial quality evaluation metrics may include, but are not limited to, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and average gradient (AG); spectral quality evaluation metrics may include, but are not limited to, average spectral angle (SAM) and relative dimensionless global error (ERGAS).

[0219] For example, the high-resolution multispectral data cube PMS output by the method of the present invention can be quantitatively evaluated from two dimensions, spatial quality and spectral quality, based on a pre-constructed comprehensive quality evaluation system of coverage spatial characteristics and spectral characteristics, and a comprehensive quality evaluation report can be generated.

[0220] The spectral quality evaluation includes at least one or more of the following: average spectral angle and relative dimensionless global error.

[0221] Specifically, the quality evaluation indicators may include, but are not limited to, peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), average gradient (AG), average spectral angle (SAM), and relative dimensionless global error (ERGAS), and a quality evaluation report may be generated based on the indicators to characterize the spatial detail performance and spectral consistency of the PMS.

[0222] When the above exemplary scheme is adopted, the present invention can reconstruct and output high-resolution multispectral images by utilizing the internal cross-band correlation and structural consistency constraints of a single low spatial resolution hyperspectral image data cube without relying on external high-resolution images as input or priors. This provides a multispectral data foundation with high spatial detail and spectral consistency for subsequent applications.

[0223] The above description, in conjunction with embodiments, illustrates a hyperspectral to multispectral image conversion method based on a self-derived model. To facilitate understanding and implementation by those skilled in the art, the system structure corresponding to the method is further described below.

[0224] See Figure 7 The diagram shown is a structural schematic of a hyperspectral to multispectral image conversion system based on a self-derived model provided by the present invention. Figure 7 As shown, the hyperspectral to multispectral image conversion system 700 based on a self-derived model may include:

[0225] The partitioning unit 710 is used to acquire a single low spatial resolution hyperspectral image data cube, and to group the hyperspectral image data cube along the spectral dimension into multiple band groups, as well as single-band low resolution observation images of the band groups.

[0226] Establishment unit 720 is used to construct a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model based on the band group and the corresponding single-band low-resolution observation image. The variational energy functional includes at least two of the following: data fidelity term, spatial regularization term, spectral consistency constraint term, and band group cooperative regularization term.

[0227] Processing unit 730 is used to transform the minimization problem of the variational energy functional into an equivalent Euler-Lagrange equation using a variational method, and to iteratively solve the Euler-Lagrange equation to obtain a preliminary high-resolution reconstructed image corresponding to a single-band low-resolution observation image, and to output a set of preliminary high-resolution reconstruction results corresponding to each band group according to the band group; wherein, when the spatial regularization term and / or the band group co-regularization term contain a second-order derivative operator, the Euler-Lagrange equation is expressed as a fourth-order partial differential equation;

[0228] The reconstruction unit 740 is used to extract and synthesize the feature components of multiple preliminary high-resolution reconstructed images to obtain high-resolution multispectral images.

[0229] For further details regarding the division unit 710, the establishment unit 720, the processing unit 730, and the reconstruction unit 740, please refer to the aforementioned examples.

[0230] It is understandable that the above division of units is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, the above units can be implemented by the processor calling software.

[0231] In one embodiment, a hyperspectral to multispectral image conversion system based on a self-derived model can be deployed in a satellite's on-orbit intelligent processing unit or a ground-based high-performance data processing center. The on-orbit unit can employ an embedded system integrating high-end aerospace-grade GPUs (such as NVIDIA Jetson Orin NX) to achieve real-time self-enhancing processing of raw data before it is deployed to the ground; the ground center can leverage GPU clusters to achieve batch regeneration and value enhancement of massive amounts of historical archived data.

[0232] This invention also provides an electronic device for implementing a hyperspectral to multispectral image conversion method based on a self-derived model.

[0233] The electronic device includes a memory and a processor, as well as a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the hyperspectral to multispectral image conversion method based on the self-derived model as described in any of the preceding claims.

[0234] It should be noted that the computer system of the electronic device shown below is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0235] Electronic devices include memory, a processor, and a transceiver. The processor is coupled to the memory and transceiver. The memory can be located inside or outside the terminal. The memory, processor, and transceiver can be connected via a communication bus. The transceiver is used to communicate with other devices or communication networks.

[0236] Optionally, the transceiver can be a transmitter. The memory stores a computer program that can run on the processor, and when the processor runs the computer program, the transceiver performs the steps in the hyperspectral to multispectral image conversion method based on a self-derived model provided in the above embodiments.

[0237] It should be understood that in the embodiments of this application, the processor can be a central processing unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0238] It should also be understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. Non-volatile memory can be ROM, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DRRAM).

[0239] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A hyperspectral to multispectral image conversion method based on a self-derived model, characterized in that, include: A single low spatial resolution hyperspectral image data cube is acquired, and the hyperspectral image data cube is grouped along the spectral dimension to obtain multiple band groups, as well as single-band low resolution observation images of the band groups. Based on the band group and the corresponding single-band low-resolution observation images, a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model are constructed. The variational energy functional includes at least two of the following: a data fidelity term, a spatial regularization term, a spectral consistency constraint term, and a band group co-regulation term. The cross-band joint degradation model describes the mapping relationship between the high-resolution image components of each band to be reconstructed and the corresponding single-band low-resolution observation images generated under the degradation effects of spatial blurring, spatial-spectral coupling downsampling, and geometric registration. The expression of the cross-band joint degradation model is: ;in, Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first The spatial ambiguity matrix for each band is used to describe the spatial degradation characteristics of the imaging system and is modeled as a wavelength-dependent Gaussian kernel function. ,in, Indicated by fuzzy scale parameter The parameterized two-dimensional Gaussian degeneracy operator, With the center wavelength of this band group Related; This represents the geometric registration matrix, used to compensate for spatial offsets between images of different band groups; Indicates the first Additive mixed noise matrix for each band; Using the variational method, the minimization problem of the variational energy functional is transformed into an equivalent Euler-Lagrange equation; wherein, when the spatial regularization term and / or the band group cooperative regularization term contain second-order derivative operators, the Euler-Lagrange equation is expressed as a fourth-order partial differential equation. The Euler-Lagrange equation is solved iteratively to obtain the preliminary high-resolution reconstructed image corresponding to the single-band low-resolution observation image, and the set of preliminary high-resolution reconstruction results corresponding to each band group is output according to the band group. Feature components of multiple preliminary high-resolution reconstructed images are extracted and synthesized to obtain high-resolution multispectral images.

2. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 1, characterized in that, The hyperspectral image data cube is grouped along the spectral dimension to obtain multiple band groups, including: Based on the physical properties of spectral wavelengths, multiple basic waveband intervals are formed; Calculate the Pearson correlation coefficient matrix between any two basic band intervals, and the correlation coefficient matrix is ​​used to characterize the degree of linear correlation between any two bands; Based on the Pearson correlation coefficient matrix, cluster analysis is used to cluster multiple bands, and the grouping results are evaluated using information criteria to determine the number of band groups. The grouping results are subjected to consistency constraint processing to obtain the band grouping results; wherein, the consistency constraint processing includes at least: evaluating the correlation between bands within each band group, and selecting and retaining band groups whose correlation meets the preset conditions based on the evaluation results; performing structural adjustment processing on the selected band groups, and merging band groups that do not meet the preset size conditions into the band group that is most similar to the band group in spectral properties.

3. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 2, characterized in that, Meet one or more of the following conditions: Before grouping the hyperspectral image data cube along the spectral dimension by band, the process further includes: preprocessing the hyperspectral image data cube; the preprocessing includes: radiometric calibration and atmospheric correction, spectral band selection, geometric correction and image registration, and spectral smoothing and noise removal; The basic band range includes at least: the visible near-infrared band range, the first type of short-wave infrared band range, and the second type of short-wave infrared band range; The information criteria are Akaike information criteria, and ,in, This represents the likelihood function when the number of groups is K; This indicates the number of parameters in the model when the number of groups is K; A band group that meets the preset conditions refers to a band group whose normalized Pearson correlation coefficient is greater than 0.

7.

4. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 1, characterized in that, The expression for the variational energy functional is: ； in, This represents the set of high-resolution image results to be reconstructed. Indicates data fidelity item, Indicates the weight of the data fidelity item; For spatial regularization, Indicates spatial smoothing weights; This represents the spectral consistency constraint term. Represents the weight of the spectral consistency constraint term; This indicates the band group coordinating regularization term. Indicates the band group collaborative weight; Wherein, the data fidelity item satisfies: ； in, Indicates the total number of bands; Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first Spatial ambiguity matrix of each band; Represents the geometric registration matrix; For the first The adaptive weight matrix for the next iteration, where ⊙ represents element-wise multiplication. It is an L1 norm; The space regularization term satisfies: ； in, Indicates the total number of bands; Represents the image spatial domain; Represents the second derivative operator; This represents the adaptive weighting function; Indicates the first The spatial location of each band The corresponding high-resolution image components; It is an L1 norm; The spectral consistency constraint term satisfies: ； in, Indicates the total number of bands; Represents the image spatial domain; Represents pixels The neighborhood set; Indicates the first Each band in pixels High-resolution pixel values ​​at that location Indicates the first Each band in the neighborhood pixels High-resolution pixel values ​​at that location For robust penalty functions, For pixel pairs scalar weights, and Pixels from low-resolution hyperspectral images and The spectral angular similarity is calculated and normalized to obtain the result; The expression for the band group cooperative regularization term is: ； in, Indicates the number of band groups; Indicates the first The set of band indices contained in a band group Indicates the first Group-guided images of each band group; For collaborative weights; This represents the L2 norm.

5. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 4, characterized in that, The method employing variational methods to transform the minimization problem of the variational energy functional into an equivalent Euler-Lagrange equation includes: Variational analysis is performed on the variational energy functional, and first-order variational values ​​are obtained for the data fidelity term, the spatial regularization term, and the band group cooperative regularization term, respectively. The corresponding Euler-Lagrange equations are then constructed based on the extremum conditions. The data fidelity term, under the action of the degenerate operator, generates a corresponding gradient term through first-order variation; the space regularization term adopts a regularization form containing a second-order derivative operator, and its first-order variation result is introduced into the further differentiation of the second-order derivative operator, thereby forming an equivalent fourth-order differential operator term in the Euler-Lagrange equation.

6. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 4 or 5, characterized in that, The iterative solution of the Euler-Lagrange equation to obtain the preliminary high-resolution reconstructed image corresponding to the single-band low-resolution observation image includes: By introducing auxiliary variables, the minimization problem corresponding to the Euler-Lagrange equation is transformed into an equivalent constrained optimization problem, thereby achieving the separation of data fidelity terms, spatial regularization terms, spectral consistency constraint terms, and band group cooperative regularization terms in the variational energy functional. The constrained optimization problem is solved using the split Bragman algorithm, wherein in each split Bragman iteration: the high-resolution image to be reconstructed corresponding to the band group, the auxiliary variables used to accommodate the constraint variables such as residual terms and second derivative terms, and the Bragman variables used to constrain the residual accumulation term are updated alternately. After the split Bregman iteration reaches the preset convergence condition, the preliminary high-resolution reconstructed image corresponding to the band group is reconstructed, and the set of preliminary high-resolution reconstruction results corresponding to each band group is output according to the band grouping results.

7. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 1, characterized in that, The step of extracting and synthesizing feature components from multiple preliminary high-resolution reconstructed images to obtain a high-resolution multispectral image includes: Spectral dimension reduction is performed on the cubes composed of multiple preliminary high-resolution reconstructed images to extract the first P important feature components, where P satisfies that the cumulative energy contribution rate of the important feature components is not less than a preset threshold. Calculate the spectral angles of each important feature component with typical land features in the standard spectral library, retain the important feature components with high matching degree, and obtain the original candidate components; Calculate the correlation coefficient between the original candidate components, and when the correlation coefficient is greater than a preset threshold, delete at least one of the original candidate components according to a preset priority rule to obtain candidate components; Within the image spatial domain corresponding to the hyperspectral image data cube, each candidate component is subjected to unsupervised clustering or segmentation to generate multiple pseudo-class sample regions. Based on the pseudo-class sample regions, the inter-class separability index of each candidate component is calculated, and the candidate component with the highest inter-class separability index is selected to obtain the feature component set. A weighted linear method is used to sum the candidate components in the feature component set to obtain a multispectral band set, which serves as the high-resolution multispectral image.

8. The hyperspectral to multispectral image conversion method based on a self-derived model according to claim 7, characterized in that, Meet one or more of the following conditions: Dimensionality reduction of the spectrum is performed using minimum noise separation and principal component analysis transformation. The first P important feature components refer to the first P transformed components sorted from high to low according to their energy contribution rate after dimensionality reduction transformation; Important feature components with high matching degree refer to important feature components with spectral angle less than 0.1 radians; The preset threshold includes 0.8; The priority rules include: when the correlation coefficient is greater than the preset threshold, the component with the lower energy contribution rate is deleted; if the difference in energy contribution rate is less than the preset difference, the component with the larger spectral angle is deleted; if it is still impossible to distinguish, the component with the higher signal-to-noise ratio is retained first. The feature component set C is a three-dimensional real tensor with a spatial dimension of sM×sN×Qsel, where sM×sN is the spatial dimension of the image and Qsel is the number of filtered feature components. The weighted summation yields a multispectral band set (PMS) consisting of K bands. The data cube of the multispectral band set PMS is a three-dimensional real tensor with a spatial dimension of sM×sN×K. Here, s is the scaling factor for spatial resolution, M and N correspond to the spatial dimensions of the single low spatial resolution hyperspectral image, and K corresponds to the number of spectral bands. Based on a pre-constructed comprehensive quality evaluation system that integrates spatial and spectral characteristics, the high-resolution multispectral image is quantitatively evaluated from two dimensions: spatial quality and spectral quality, generating a comprehensive quality evaluation report. The evaluation indicators for spatial quality include at least one or more of peak signal-to-noise ratio, structural similarity index, and average gradient; the evaluation indicators for spectral quality include at least one or more of average spectral angle and relative dimensionless global error.

9. A hyperspectral to multispectral image conversion system based on a self-derived model, characterized in that, include: The partitioning unit is used to acquire a single low spatial resolution hyperspectral image data cube, and to group the hyperspectral image data cube along the spectral dimension into multiple band groups, as well as single-band low resolution observation images of the band groups. A unit is established to construct a cross-band joint degradation model and a variational energy functional corresponding to the cross-band joint degradation model based on the band group and the corresponding single-band low-resolution observation images. The variational energy functional includes at least two of the following: a data fidelity term, a spatial regularization term, a spectral consistency constraint term, and a band group co-regulation term. The cross-band joint degradation model describes the mapping relationship between the high-resolution image components of each band to be reconstructed and the corresponding single-band low-resolution observation images under the degradation effects of spatial blurring, spatial-spectral coupling downsampling, and geometric registration. The expression of the cross-band joint degradation model is: ;in, Indicates the first Low-resolution observation images in each band; The downsampling matrix representing spatial-spectral coupling; Indicates the first Each band corresponds to a high-resolution image component. Indicates the first The spatial ambiguity matrix for each band is used to describe the spatial degradation characteristics of the imaging system and is modeled as a wavelength-dependent Gaussian kernel function. ,in, Indicated by fuzzy scale parameter The parameterized two-dimensional Gaussian degeneracy operator, With the center wavelength of this band group Related; This represents the geometric registration matrix, used to compensate for spatial offsets between images of different band groups; Indicates the first Additive mixed noise matrix for each band; The processing unit is configured to use a variational method to transform the minimization problem of the variational energy functional into an equivalent Euler-Lagrange equation, and to iteratively solve the Euler-Lagrange equation to obtain a preliminary high-resolution reconstructed image corresponding to a single-band low-resolution observation image, and to output a set of preliminary high-resolution reconstruction results corresponding to each band group according to the band group; wherein, when the spatial regularization term and / or the band group co-regularization term contain a second-order derivative operator, the Euler-Lagrange equation is expressed as a fourth-order partial differential equation; The reconstruction unit is used to extract and synthesize the feature components of multiple preliminary high-resolution reconstructed images to obtain high-resolution multispectral images.

Images