Interference hyperspectral image trend item on-orbit real-time correction system, method and device

Abstract

The present application relates to the technical field of image processing, and more particularly to an on-orbit real-time correction system, method and device for trend items of interferometric hyperspectral images, which is based on the strong nonlinear mapping capability of KAN neural network and the local approximation advantage of RBF neural network, and innovatively introduces a weight generation network to dynamically allocate weights to strengthen the correction contribution of high-value wavebands, and is deployed on a star image processor to achieve accurate modeling and removal of trend items on-orbit in real time.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and particularly relates to an on-orbit real-time correction system, method and apparatus for trend terms in interferometric hyperspectral images. Background Technology

[0002] Interferometric hyperspectral images, with their spatial-spectral integration, have irreplaceable application value in fields such as remote sensing monitoring and precision agriculture. However, their imaging process is susceptible to various error factors that generate trend term noise, severely reducing the accuracy of quantitative spectral inversion and ground feature identification. In previous studies, researchers have proposed several trend term correction methods to enhance images, mainly including physical model-based trend term correction methods, differential filtering methods, polynomial fitting methods, and empirical mode decomposition methods. Among these methods, trend term correction based on physical models is only applicable to specific spectral images and lacks universality; differential filtering is simple in structure and fast in computation, but it does not completely remove trend terms; while polynomial fitting, which uses a polynomial model to fit and remove trend terms, has the advantage of low computational complexity and ease of implementation, but it has significant limitations, namely, when the trend term exhibits a strongly nonlinear distribution, it is prone to underfitting or overfitting, resulting in residual trend term components in the corrected image; Empirical Mode Decomposition (EMD), first proposed by NASA, is based on Fourier transform and decomposes any complex time series into intrinsic mode sequences and trend terms, making trend term extraction more reasonable and accurate. None of these methods are universal; they cannot correct interferograms with a large dynamic range, or they either do not completely remove trend terms, or they excessively eliminate effective information in the interferogram, introducing false information, resulting in low signal-to-noise ratios and poor spectral consistency in the corrected image. Furthermore, none of these methods are suitable for deployment on onboard processors, lacking the capability for real-time on-orbit processing, and thus cannot contribute to improving the accuracy of on-orbit target recognition and fine classification.

[0003] Existing trend term correction methods have four core problems:

[0004] (1) Insufficient nonlinear adaptability makes it difficult to cope with the strong nonlinear trend terms in complex scenarios after the superposition of multiple physical interference sources;

[0005] (2) The impact of noise differences in different frequency bands on the correction effect was not considered;

[0006] (3) It is difficult to balance global and local aspects, and it is impossible to simultaneously achieve accurate removal of trend items and preservation of true information about ground features;

[0007] (4) It does not have a solution for deploying on-board processors, does not have the ability to process in real time on orbit, and cannot help improve the accuracy of on-orbit target identification and fine classification. Summary of the Invention

[0008] In view of this, the present invention aims to provide an on-orbit real-time correction system, method and device for trend terms in interferometric hyperspectral images. By integrating the strong nonlinear mapping capability of KAN (Kolmogorov-Arnold Net) and the local approximation advantage of RBF (Radial Basis Functions Network), a dual-network collaborative architecture is constructed. An adaptive weighting mechanism is innovatively introduced under each optical path difference, and the weights are dynamically allocated based on the high and low frequency characteristics of each optical path difference to enhance the contribution of high-value bands. It can be directly deployed on the on-board processor to achieve on-orbit real-time accurate modeling and removal of trend terms in interferometric hyperspectral images, thereby improving image quality and enhancing the usability of the images.

[0009] To achieve the above objectives, the technical solution created by this invention is implemented as follows:

[0010] An on-orbit real-time correction system for trend terms in interferometric hyperspectral images includes an RBF neural network, a KAN neural network, and a weight generation network. The system inputs the interferometric hyperspectral image into the RBF neural network to obtain a first trend term component; inputs the interferometric hyperspectral image into the KAN neural network to obtain a second trend term component; inputs the interferometric hyperspectral image into the weight generation network to obtain RBF weights and KAN weights; the first and second trend term components are weighted and summed according to the RBF and KAN weights to obtain a total trend term component; and the total trend term component is subtracted from the interferometric hyperspectral image to obtain the corrected interferometric information.

[0011] Furthermore, a Gaussian function is used as the radial basis function of the RBF neural network.

[0012] Furthermore, in the KAN neural network, the B-spline function is used as the activation function; and the SiLU activation function is added to each layer of the KAN neural network for bias control.

[0013] Furthermore, in the weight generation network, after preprocessing the interferometric hyperspectral image, the preprocessed features are subjected to a one-dimensional convolution operation followed by a softmax operation to obtain RBF weights and KAN weights.

[0014] Furthermore, the preprocessing of the interferometric hyperspectral image includes adjusting the number of columns, rows, and dimensions of the interferometric hyperspectral image.

[0015] An on-orbit real-time correction method for trend terms in interferometric hyperspectral images includes:

[0016] S1: Obtain the dataset of interferometric hyperspectral images to obtain the training set;

[0017] S2: Construct an on-orbit real-time correction system for trend terms of interferometric hyperspectral images as provided in this invention;

[0018] S3: Using the training set obtained in step S1, train the real-time correction system constructed in step S2 to obtain the real-time correction model;

[0019] S4: Input the interference hyperspectral image to be processed into the real-time correction model obtained in step S3 to obtain the corrected interference information.

[0020] Furthermore, in step S3, the real-time correction system is trained using the following total loss function:

[0021] ;

[0022] in, Represents the total loss function. Indicates the mean absolute error loss. This represents the smoothness constraint loss. α represents the high-frequency constraint loss, and β and γ represent the loss weights of the three types of loss, respectively.

[0023] Furthermore, in the total loss function: the smoothness constraint loss is:

[0024] ;

[0025] Where Z represents the number of samples in each training iteration. This represents the i-th sampling point in the predicted total trend component;

[0026] The high-frequency constraint loss is:

[0027] ;

[0028] in, denoted by , where represents the weight of the high-frequency suppression term, Q represents the total number of high-frequency regions in the samples during each training iteration, S represents the high-frequency part of the samples in the frequency domain during each training iteration, f represents the frequency of the samples during each training iteration, and FFT represents the Fast Fourier Transform.

[0029] An on-orbit real-time correction device for trend terms in interferometric hyperspectral images includes an imaging module, an image acquisition module, and an image processing module. The image acquisition module controls the imaging module to capture interferometric hyperspectral images, and the imaging module transmits the interferometric hyperspectral images to the image processing module via an interface. The image processing module processes the interferometric hyperspectral images using the on-orbit real-time correction method for trend terms in interferometric hyperspectral images provided by this invention to obtain corrected interferometric information.

[0030] Compared with the prior art, the present invention can achieve the following beneficial effects:

[0031] (1) In the on-orbit real-time correction system for trend terms of interferometric hyperspectral images created by this invention, the RBF neural network is good at local feature modeling and can capture local band changes well, but it may be insufficient when smoothing global trends; the KAN neural network has advantages in sequence modeling and cross-band dependency capture, and can better handle global trends, but it may underfit or overfit when fitting local bands; while trend term correction requires both suppression of local interference and capture of global trends, the RBF neural network and the KAN neural network have strong complementarity in these two aspects. Therefore, this invention also introduces a weight generation network to achieve effective fusion of the outputs of the RBF neural network and the KAN neural network, effectively overcoming the problems of insufficient nonlinear adaptation and lack of band heterogeneity adaptation in traditional methods. For the differences in spectral characteristics and noise of different input interferometric hyperspectral images, it automatically reduces the influence of the inferior model, dynamically adjusts the branch contribution, and improves the overall fitting accuracy, showing stable and excellent performance in the correction of trend terms of interferometric hyperspectral images; at the same time, thanks to the adaptive weights in the trained model, it can adapt to dynamic errors such as load aging and environmental changes, and is suitable for processing interferometric hyperspectral image data with different loads and characteristics.

[0032] (2) In the on-orbit real-time correction method for trend terms of interferometric hyperspectral images described in this invention, the system is trained by a total loss function including average absolute error constraints, smoothness constraints and high frequency constraints, which guides the system to be trained according to the interferometric imaging mechanism and trend term characteristics, further improving the robustness of the correction of trend terms of interferometric hyperspectral images and further improving the fitting accuracy.

[0033] (3) The on-orbit real-time correction device for trend terms of interferometric hyperspectral images created by the present invention can be well adapted to the computing power and power consumption constraints of the satellite. At the same time, the device provided by the present invention has good compatibility, that is, the device provided by the present invention can be directly deployed on the satellite platform and can be directly integrated into the existing interferometric hyperspectral payload. In addition, the device provided by the present invention is applicable to various platforms such as satellites and UAVs, and has a wide range of applications. Attached Figure Description

[0034] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments and descriptions of the invention are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings:

[0035] Figure 1 A schematic diagram of the on-orbit real-time correction system for trend terms of interferometric hyperspectral images as described in an embodiment of the present invention;

[0036] Figure 2 A schematic flowchart of the on-orbit real-time correction method for trend terms in interferometric hyperspectral images as described in an embodiment of the present invention;

[0037] Figure 3 The structure of the on-orbit real-time correction device for the trend term of the interferometric hyperspectral image described in the embodiment of the present invention. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not constitute a limitation thereof.

[0039] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0040] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on this invention. Furthermore, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, features defined with "first," "second," etc., may explicitly or implicitly include one or more of that feature. In the description of this invention, unless otherwise stated, "a plurality of" means two or more.

[0041] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art will understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0042] The invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0043] like Figure 1As shown in the embodiment of the present invention, the on-orbit real-time correction system for the trend term of an interferometric hyperspectral image includes an RBF neural network, a KAN neural network, and a weight generation network. Specifically, the interferometric hyperspectral image is input into the RBF neural network to obtain a first trend term component; the interferometric hyperspectral image is input into the KAN neural network to obtain a second trend term component; the interferometric hyperspectral image is input into the weight generation network to obtain RBF weights and KAN weights; the first and second trend term components are weighted and summed according to the RBF weights and KAN weights to obtain the total trend term component. Since the interferometric hyperspectral image is the sum of effective interferometric information and the trend term component, that is:

[0044] M = L + B;

[0045] Where M represents the interferometric hyperspectral image, L represents the effective interferometric information, and B represents the trend component. Therefore, after obtaining the sum of the total trend components through the RBF neural network, KAN neural network, and weight generation network, the total trend components are subtracted from the interferometric hyperspectral image to obtain the corrected interferometric information.

[0046] In some embodiments, a Gaussian function is used as the radial basis function of the RBF neural network. In this embodiment of the invention, a Gaussian function is used as the radial basis function of the RBF neural network, and the hidden layer activation function of the RBF neural network is:

[0047] ;

[0048] Where G represents the hidden layer activation function, z represents the input, c represents the center parameter of the hidden layer in the RBF neural network, and σ represents the width parameter of the hidden layer in the RBF neural network.

[0049] Furthermore, the output of the hidden layer of the RBF neural network is:

[0050] ;

[0051] Where out represents output. Let J represent the weight of the j-th node in each hidden layer of the RBF neural network, where J represents the total number of nodes in each hidden layer of the RBF neural network. This represents the center parameter of the j-th node in each hidden layer of the RBF neural network. After processing the interferometric hyperspectral image, the RBF neural network yields the first trend component RBF.

[0052] In some embodiments, a B-spline function is used as the activation function in the KAN neural network; a SiLU activation function is added to each layer of the KAN neural network for bias control.

[0053] In this embodiment of the invention, the activation function of each layer of the KAN neural network is represented by the following formula:

[0054] ;

[0055] in, Let x represent each layer of the KAN neural network, x represent the input, and ε represent a constant that controls the activation function, which is selected adaptively according to the actual situation. This represents the activation function, i.e.:

[0056] ;

[0057] in, Let B-spline function be the activation function of the m-th node in the KAN neural network. This represents the weight parameters of the B-spline function in the activation function of the m-th node in the KAN neural network.

[0058] The SiLU activation function is as follows:

[0059] .

[0060] At this point, the data processing procedure for each layer of the KAN neural network can be represented as:

[0061] ;

[0062] in, This represents the output of each layer of the KAN neural network. Let L represent the output of the l-th neuron in each layer of the KAN neural network, and let L represent the total number of neurons in each layer of the KAN neural network. This represents the input to the l-th neuron in each layer of the KAN neural network. After processing the interferometric hyperspectral image, the KAN neural network yields the second trend component, KAN.

[0063] In some embodiments, after preprocessing the interferometric hyperspectral image in the weight generation network, a one-dimensional convolution operation is performed on the preprocessed features, followed by a softmax operation to obtain RBF weights and KAN weights. The preprocessing process includes adjusting the number of columns, rows, and dimensions of the interferometric hyperspectral image.

[0064] In this embodiment of the invention, the weight generation network performs a one-dimensional convolution operation (Conv1D) on the reshaped interferometric hyperspectral image after adjusting the number of columns, rows, and dimensions, followed by a softmax operation to obtain the RBF weights μ and KAN weights λ, i.e.:

[0065] [μ,λ]=softmax(Conv1D(Reshape(x)));

[0066] The sum of the RBF weight μ and the KAN weight λ is 1, i.e., μ + λ = 1.

[0067] The first trend component RBF and the second trend component KAN are weighted and summed according to the RBF weight μ and the KAN weight λ to obtain the total trend component, i.e.:

[0068] B = μ × RBF + λ × KAN;

[0069] Subtracting the total trend component B from the interferometric hyperspectral image M yields the corrected interferometric information L, i.e.:

[0070] L=MB.

[0071] This invention also provides an on-orbit real-time correction method for trend terms in interferometric hyperspectral images, such as... Figure 2 As shown, it includes:

[0072] S1: Obtain the dataset of interferometric hyperspectral images to obtain the training set. In this embodiment of the invention, training is performed based on the actual on-orbit data of the interferometric hyperspectral imager carried by Chang'e-1 and the Environment and Disaster Reduction Satellite-2.

[0073] S2: Construct an on-orbit real-time correction system for trend terms of interferometric hyperspectral images as provided in this invention.

[0074] S3: Using the training set obtained in step S1, train the real-time correction system constructed in step S2 to obtain the real-time correction model.

[0075] In some embodiments, the real-time correction system is trained using the following total loss function:

[0076] ;

[0077] in, Represents the total loss function. Indicates the mean absolute error loss. This represents the smoothness constraint loss. α represents the high-frequency constraint loss, and β and γ represent the loss weights of the three types of loss, respectively.

[0078] The mean absolute error loss is:

[0079] ;

[0080] Where Z represents the number of samples in each training iteration. z represents the i-th sampling point in the predicted total trend component. iThis represents the i-th sampling point in the true total trend component.

[0081] The smoothness constraint loss is:

[0082] ;

[0083] The high-frequency constraint loss is:

[0084] ;

[0085] in, denoted by , where represents the weight of the high-frequency suppression term, Q represents the total number of high-frequency regions in the samples during each training iteration, S represents the high-frequency part of the samples in the frequency domain during each training iteration, f represents the frequency of the samples during each training iteration, and FFT represents the Fast Fourier Transform.

[0086] For high-frequency constraints, instead of applying the same penalty weight to all pixels, an adaptive adjustment function is used for different signal-to-noise ratio (SNR) data. After calculating the image SNR using the standard SNR calculation formula, the weight of the high-frequency suppression term is adjusted according to the SNR. A high SNR curve reduces high-frequency suppression, while a low SNR curve enhances high-frequency suppression.

[0087] S4: Input the interference hyperspectral image to be processed into the real-time correction model obtained in step S3 to obtain the corrected interference information.

[0088] This invention also provides an on-orbit real-time correction device for trend terms in interferometric hyperspectral images, such as... Figure 3 As shown, it includes an imaging module, an image acquisition module, and an image processing module. Specifically: the image acquisition module controls the imaging module to capture interferometric hyperspectral images; the imaging module transmits the interferometric hyperspectral images to the image processing module via an interface; the image processing module processes the interferometric hyperspectral images using the on-orbit real-time correction method for the trend term of interferometric hyperspectral images provided by this invention to obtain corrected interferometric information.

[0089] In this embodiment of the invention, the imaging module employs a global exposure CMOS detector, the image acquisition module employs an imaging control FPGA, and the image processing module employs an image processing CPU deployed with the on-orbit real-time correction method for the trend term of the interferometric hyperspectral image provided by this invention. The imaging control FPGA controls the CMOS detector to capture images and acquires integrated image data, which is then sent to the image processing CPU via a RAPID IO interface or a GTX interface. The image processing CPU performs trend term correction on the received images to obtain the corrected interferometric information.

[0090] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this invention can be achieved, and this is not limited herein.

[0091] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An on-orbit real-time correction system for trend terms in interferometric hyperspectral images, characterized in that, This includes RBF neural networks, KAN neural networks, and weight generation networks; among them, The interferometric hyperspectral image is input into the RBF neural network to obtain the first trend term component; The interferometric hyperspectral image is input into the KAN neural network to obtain the second trend term component; The interferometric hyperspectral image is input into the weight generation network to obtain RBF weights and KAN weights; the first trend component and the second trend component are weighted and summed according to the RBF weights and KAN weights to obtain the total trend component. The corrected interference information is obtained by subtracting the total trend term component from the interference hyperspectral image.

2. The on-orbit real-time correction system for trend terms of interferometric hyperspectral images according to claim 1, characterized in that, A Gaussian function is used as the radial basis function of the RBF neural network.

3. The on-orbit real-time correction system for trend terms of interferometric hyperspectral images according to claim 1, characterized in that, In the KAN neural network, the B-spline function is used as the activation function; and the SiLU activation function is added to each layer of the KAN neural network for bias control.

4. The on-orbit real-time correction system for trend terms of interferometric hyperspectral images according to claim 1, characterized in that, In the weight generation network, after preprocessing the interferometric hyperspectral image, a one-dimensional convolution operation is performed on the preprocessed features, followed by a softmax operation to obtain the RBF weights and the KAN weights.

5. The on-orbit real-time correction system for trend terms of interferometric hyperspectral images according to claim 1, characterized in that, The preprocessing of the interferometric hyperspectral image includes: The number of columns, rows, and dimensions of the interferometric hyperspectral image are adjusted.

6. A method for real-time on-orbit correction of trend terms in interferometric hyperspectral images, characterized in that, include: S1: Obtain the dataset of interferometric hyperspectral images to obtain the training set; S2: Construct an on-orbit real-time correction system for trend terms of interferometric hyperspectral images as described in any one of claims 1 to 5; S3: Using the training set obtained in step S1, train the real-time correction system constructed in step S2 to obtain the real-time correction model; S4: Input the interference hyperspectral image to be processed into the real-time correction model obtained in step S3 to obtain the corrected interference information.

7. The on-orbit real-time correction method for trend terms in interferometric hyperspectral images according to claim 6, characterized in that, In step S3, the real-time correction system is trained using the following total loss function: ； in, Represents the total loss function, Indicates the mean absolute error loss. This represents the smoothness constraint loss. α represents the high-frequency constraint loss, and β and γ represent the loss weights of the three types of loss, respectively.

8. The on-orbit real-time correction method for trend terms in interferometric hyperspectral images according to claim 7, characterized in that, In the total loss function: The smoothness constraint loss is: ； Where Z represents the number of samples in each training iteration. This represents the i-th sampling point in the predicted total trend component; The high-frequency constraint loss is: ； in, denoted by , where represents the weight of the high-frequency suppression term, Q represents the total number of high-frequency regions in the samples during each training iteration, S represents the high-frequency part of the samples in the frequency domain during each training iteration, f represents the frequency of the samples during each training iteration, and FFT represents the Fast Fourier Transform.

9. An on-orbit real-time correction device for trend terms in interferometric hyperspectral images, characterized in that, It includes an imaging module, an image acquisition module, and an image processing module; among which: The image acquisition module controls the imaging module to capture interferometric hyperspectral images, and the imaging module transmits the interferometric hyperspectral images to the image processing module through an interface; The image processing module uses the on-orbit real-time correction method for the trend term of the interferometric hyperspectral image as described in claim 8 to process the interferometric hyperspectral image and obtain the corrected interferometric information.

Images