Hyperspectral change detection method for ground object properties based on multi-task learning
Patent Information
- Application Number
- CN202610810511.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-05
- Publication Date
- 2026-10-09
AI Technical Summary
[0005]因此，双时相高光谱图像受观测条件差异影响而产生伪变化响应、地物属性信息与观测条件信息耦合严重、变化检测精度和稳定性不足，成为当前高光谱变化检测领域亟待解决的技术问题
[0077]一种基于多任务学习的高光谱地物属性变化检测方法，通过构建观测条件编码分支和辐射一致性校正分支，能够实现观测条件信息与地物属性信息的分离建模，从而减弱双时相高光谱图像中由观测条件差异引起的伪变化响应，提高真实变化区域与不变区域的区分能力。
Smart Images
Figure CN122888404A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of intelligent interpretation of remote sensing images and hyperspectral image processing technology, specifically relating to a method for detecting changes in hyperspectral ground property attributes based on multi-task learning. Background Technology
[0002] Hyperspectral remote sensing images, characterized by a large number of bands, high spectral resolution, and the ability to finely characterize differences in the spectral response of ground features, have significant application value in land use monitoring, ecological environment assessment, disaster identification, mining area supervision, and urban dynamic analysis. Change detection technology using dual-temporal hyperspectral images can identify changes in the type, state, or spatial distribution of ground features by comparing hyperspectral data acquired from the same area at different times, and has therefore become an important research direction in hyperspectral remote sensing applications.
[0003] However, in practical applications, dual-temporal hyperspectral images are usually not acquired under completely consistent observation conditions. They are easily affected by factors such as variations in solar altitude angle, atmospheric conditions, sensor response differences, observation geometry differences, and imaging time differences, causing the same unchanged ground feature to exhibit significantly different spectral responses between the two temporal phases. This phenomenon typically manifests as radiometric inconsistency. Radiometric inconsistency can lead to spurious changes, thereby interfering with the extraction of change information and reducing the accuracy and stability of change detection results.
[0004] On the other hand, the number of existing hyperspectral change detection datasets is relatively limited, and real dual-temporal data is difficult to systematically cover radiation inconsistency scenarios under different atmospheric parameters and observation geometric parameters under controlled conditions, making it difficult for models to fully learn and evaluate radiation inconsistencies. To simulate radiative transfer processes under different atmospheric conditions, existing technologies include atmospheric radiative transfer models, such as MODTRAN, which can simulate spectral radiative transfer processes based on given parameters and are suitable for constructing controlled radiation inconsistency samples.
[0005] Therefore, the pseudo-change response caused by the difference in observation conditions in dual-temporal hyperspectral images, the serious coupling between ground object attribute information and observation condition information, and the insufficient accuracy and stability of change detection have become technical problems that urgently need to be solved in the current field of hyperspectral change detection. Summary of the Invention
[0006] To overcome the problems of pseudo-change responses caused by differences in observation conditions in dual-temporal hyperspectral images, severe coupling between ground property information and observation condition information, and insufficient accuracy and stability of change detection, this invention proposes a hyperspectral ground property change detection method based on multi-task learning.
[0007] The technical solution adopted by this invention to solve its technical problem is:
[0008] A method for detecting changes in hyperspectral ground cover attributes based on multi-task learning includes the following steps:
[0009] Step S1, Acquisition of dual-temporal hyperspectral data and construction of dual-temporal simulated hyperspectral images:
[0010] Acquire labeled dual-temporal hyperspectral images, preprocess the dual-temporal hyperspectral images, set the atmospheric parameters and observation geometric parameters corresponding to the first and second temporal phases, and construct dual-temporal simulated hyperspectral images.
[0011] Step S2, Extraction of dual-temporal observation condition features:
[0012] Based on the dual-temporal simulated hyperspectral images, the dual-temporal observation condition features are extracted using an observation condition encoder.
[0013] Step S3, Radiation Consistency Correction and Change Detection:
[0014] Using dual-temporal simulated hyperspectral images, we extract dual-temporal multi-scale basic features; we perform feature interaction on the dual-temporal multi-scale basic features to construct land cover attribute features after radiometric consistency correction of the hyperspectral image, and output the detection results of hyperspectral land cover attribute changes.
[0015] The above-mentioned hyperspectral ground property change detection method, step S1, further includes:
[0016] Step 1: Acquisition of labeled dual-temporal hyperspectral images:
[0017] We used publicly available hyperspectral ground feature change detection data, and used the bi-temporal images and their change labels as labeled bi-temporal hyperspectral images.
[0018] The second step is preprocessing of the dual-temporal hyperspectral images:
[0019] The labeled dual-temporal hyperspectral images are preprocessed to obtain the basic dual-temporal hyperspectral samples for input to the network.
[0020] The preprocessing includes image registration, invalid band removal, normalization, and block cropping.
[0021] Step 3, parameter settings:
[0022] Set the atmospheric parameters and observation geometric parameters corresponding to the first and second time phases.
[0023] The atmospheric parameters include aerosol type, visibility, and water vapor content, while the observation geometric parameters include solar zenith angle, observation zenith angle, and observation azimuth angle.
[0024] Step 4: Construction of dual-temporal simulated hyperspectral images under different radiation inconsistencies:
[0025] Based on the basic dual-temporal hyperspectral samples and set parameters, the entrance pupil radiance corresponding to the ground reflectance information of the atmospheric radiative transfer simulation model is generated, and dual-temporal simulated hyperspectral images under different radiative inconsistencies are constructed.
[0026] Among them, the dual-temporal simulated hyperspectral images under different radiation inconsistency conditions include a first-temporal hyperspectral image and a second-temporal hyperspectral image.
[0027] The above-mentioned hyperspectral ground property change detection method, step S2, further includes:
[0028] The first and second temporal hyperspectral images obtained from the simulation are input into the corresponding observation condition encoders to extract the dual-temporal observation condition features.
[0029] Among them, the dual-temporal observation condition characteristics include the first-temporal observation condition characteristics. Characteristics of the second phase observation conditions The calculation formula is as follows:
[0030]
[0031]
[0032] In the above formula, Indicates the observation condition encoder. This represents the simulated hyperspectral image of the first time phase. This represents the simulated hyperspectral image of the second time phase.
[0033] The above-mentioned hyperspectral ground property change detection method, step S3, further includes:
[0034] Step 1: Extraction of basic features across two temporal phases and multiple scales:
[0035] Dual-temporal simulated hyperspectral images and Inputting a backbone feature extraction network with shared parameters yields dual-temporal, multi-scale basic features. and ,in, and , .
[0036] The backbone feature extraction network is a spectral-spatial joint feature extraction network, which is used to simultaneously extract the spectral and spatial features of hyperspectral images.
[0037] The second step, feature interaction:
[0038] By performing multi-level feature interaction on the dual-temporal multi-scale basic features, we obtain the fused shallow spatial detail features and deep semantic features.
[0039] The third step is progressive upsampling:
[0040] Progressive upsampling is performed on the fused shallow spatial detail features and deep semantic features to construct the land cover attribute features of the hyperspectral image after radiometric consistency correction. and .
[0041] The calculation formula for the radiometrically corrected land feature attributes is as follows:
[0042]
[0043] In the above formula, This indicates a shared backbone feature extraction network. This represents a multi-level feature interaction process. This indicates a progressive upsampling process. This represents the radiometrically corrected land feature characteristics corresponding to the t-th time.
[0044] Step 4: Output of hyperspectral ground property change detection results:
[0045] The radiometrically corrected land cover attribute features from the hyperspectral image are input into the land cover attribute change detection module. The land cover attribute change detection module outputs the hyperspectral land cover attribute change detection results.
[0046] The feature attribute change detection module includes a feature mixing unit HS-Mixblock, a feature interaction unit HS-MAB, an upsampling unit Upsample, and a classification unit Classifier.
[0047] The feature mixing unit introduces dual-temporal original features, difference features, and product features; the attention interaction unit introduces a spectral-space joint attention mechanism.
[0048] The above-mentioned method for detecting changes in hyperspectral land cover attributes also includes step S4, model constraints and iteration:
[0049] A loss function is constructed, and the loss functions are jointly optimized to obtain the total loss function. The total loss function is used to constrain the model, and the model is iterated to obtain the hyperspectral ground property change detection model.
[0050] The loss function includes change detection loss, feature separation loss, hyperspectral reconstruction loss, and invariant region consistency loss.
[0051] The above-mentioned hyperspectral ground property change detection method, step S4, further includes:
[0052] Step 1: Constructing the change detection loss:
[0053] Based on the hyperspectral changes in ground cover attributes and the actual change labels, a change detection loss is constructed. The calculation formula is as follows:
[0054]
[0055] In the formula, For the change detection results, To correspond to the actual changes in the labels, For Binary CrossEntropy Loss, specifically, Where N is the number of pixels participating in the training. This represents the actual change label of the i-th pixel. This represents the prediction result for the i-th pixel.
[0056] The second step is to construct the feature separation loss:
[0057] Based on the ground feature attributes and observation conditions after radiometric consistency correction, a feature separation loss is constructed. The calculation formula is as follows:
[0058]
[0059] In the above formula, and These represent the land cover attributes after radiometric consistency correction in both temporal phases. These represent the observation conditions characteristics of the first time phase and the observation conditions characteristics of the second time phase, respectively.
[0060] Step 3: Construction of hyperspectral reconstruction loss:
[0061] First, a hyperspectral reconstruction branch is constructed. The dual-temporal observation condition characteristics and the ground property characteristics after radiometric consistency correction are input into the hyperspectral reconstruction decoder to reconstruct the dual-temporal simulated hyperspectral image. The hyperspectral reconstruction branch is constructed to obtain the reconstructed dual-temporal simulated hyperspectral image.
[0062] Secondly, based on the reconstructed dual-temporal simulated hyperspectral images, a hyperspectral reconstruction loss is constructed. The calculation formula is as follows:
[0063]
[0064] In the above formula, and This represents a dual-temporal simulated hyperspectral image. and This represents the reconstruction of a dual-temporal simulated hyperspectral image.
[0065] Step 4: Constructing consistency constraints for invariant regions:
[0066] Land feature attributes after radiometric consistency correction for two temporal phases and Apply invariant region consistency constraints and construct invariant region consistency loss. The calculation formula is as follows:
[0067]
[0068] In the above formula, i represents the spatial position within the invariant region. This indicates the number of pixels in the invariant region. and and represent the feature values of the land cover attributes after radiometric consistency correction in both time phases at location i.
[0069] Step 5, Joint optimization of the loss function:
[0070] The change detection loss, feature separation loss, hyperspectral reconstruction loss, and invariant region consistency loss are jointly optimized to obtain the total loss function. The calculation formula is as follows:
[0071]
[0072] In the above formula, Indicates the change detection loss. This represents the consistency loss in invariant regions. Indicates feature separation loss, Indicates the loss in hyperspectral reconstruction. , , , These are the loss weight coefficients for change detection loss, invariant region consistency loss, feature separation loss, and hyperspectral reconstruction loss, respectively.
[0073] Step 6, Constraints and Training Iterations:
[0074] By constraining the model with the total loss function and training iteratively, a hyperspectral ground property change detection model is obtained.
[0075] The hyperspectral land cover attribute change detection model is used to detect the changes in the dual-temporal hyperspectral land cover attribute images and output the corresponding change map.
[0076] The beneficial effects of this invention are:
[0077] A hyperspectral land cover attribute change detection method based on multi-task learning can achieve separate modeling of observation condition information and land cover attribute information by constructing an observation condition coding branch and a radiometric consistency correction branch. This reduces the pseudo-change response caused by the difference in observation conditions in dual-temporal hyperspectral images and improves the ability to distinguish between real change areas and unchanged areas.
[0078] A hyperspectral land cover attribute change detection method based on multi-task learning can obtain land cover attribute features that are more robust to changes in observation conditions by using a shared parameter backbone feature extraction network, multi-level feature interaction and progressive upsampling mechanism, thereby improving the stability and discrimination ability of the intrinsic attribute representation of land cover in dual-temporal hyperspectral images.
[0079] A hyperspectral land cover attribute change detection method based on multi-task learning is proposed. The method takes land cover attribute change detection as the main task and radiometric consistency correction as the auxiliary task. It combines invariant region consistency constraints, feature separation constraints and hyperspectral reconstruction constraints for multi-task collaborative optimization. This method can enhance the ability to model dual-temporal difference features while preserving effective land cover attribute information, thereby improving the accuracy, stability and robustness of land cover attribute change detection results. Attached Figure Description
[0080] Figure 1 This is a flowchart of the hyperspectral ground property change detection process based on multi-task learning, according to Embodiment 1 of the present invention.
[0081] Figure 2 This is a schematic diagram of the training sample construction process in Embodiment 1 of the present invention;
[0082] Figure 3 This is a network architecture diagram of the hyperspectral ground property change detection method according to Embodiment 1 of the present invention;
[0083] Figure 4 This is a schematic diagram of a data example from Embodiment 1 of the present invention; (a) and (b) are hyperspectral data of an irrigated farmland in Hermiston, Umartilla County, Oregon, USA, collected on May 1, 2004 and May 8, 2007, respectively; (c) is the true value of ground variation;
[0084] Figure 5 This is a schematic diagram of a data example after simulation in Embodiment 1 of the present invention; (a) and (b) are hyperspectral data of an irrigated farmland in Hermiston, Umartilla County, Oregon, USA, collected on May 1, 2004 and May 8, 2007, respectively; (c) is the true value of ground variation;
[0085] Figure 6 This is the spectral curve of a certain pixel after simulation in Embodiment 1 of the present invention;
[0086] Figure 7This is a visualization diagram comparing the change detection results with the actual annotations in Embodiment 1 of the present invention. In this diagram, the TP region and the TN region are displayed in white and black, respectively, and the FP region and the FN region are highlighted in blue and green, respectively. Detailed Implementation
[0087] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.
[0088] Example 1
[0089] A hyperspectral ground property change detection method based on multi-task learning comprises four parts: data acquisition and radiometric inconsistency sample construction, observation condition decoupling and hyperspectral reconstruction branch, radiometric consistency correction and change detection branch, and multi-loss joint training. Figure 1 The flowchart includes the following steps:
[0090] S1. Acquisition and Preprocessing of Dual-Temporal Hyperspectral Data
[0091] The first step is to acquire labeled dual-temporal hyperspectral image data. Preferably, dual-temporal images and their change labels from a publicly available hyperspectral change detection dataset can be used as the base data. This example uses hyperspectral reflectance data from an irrigated farmland in Hermiston, Umartilla County, Oregon, USA, collected on May 1, 2004, and May 8, 2007.
[0092] The second step involves preprocessing the dual-temporal hyperspectral images. This includes image registration, invalid band removal, normalization, and necessary cropping to obtain the basic dual-temporal hyperspectral samples for input to the network. Let the preprocessed first-temporal hyperspectral image and the second-temporal hyperspectral image be: and ,in, and These all represent hyperspectral image data of the same region acquired at different time points. For example... Figure 4 As shown.
[0093] The third step involves setting the atmospheric parameters, observation geometric parameters, and / or sensor-related parameters corresponding to the first and second time phases. These parameters may include aerosol type, visibility, water vapor content, solar zenith angle, observation zenith angle, and observation azimuth angle. For example... Figure 2 As shown, in one specific embodiment, to simulate the imaging differences of dual-temporal hyperspectral images under different radiation conditions, the first temporal parameters can be set as follows: visibility 40 km, water vapor content 0.12 g / cm², solar zenith angle 70°, and the atmospheric model is a mid-latitude winter; the second temporal parameters can be set as follows: visibility 10 km, water vapor content 1.362 g / cm², solar zenith angle 40°, and the atmospheric model is a mid-latitude summer.
[0094] By setting different parameter combinations, dual-temporal simulated hyperspectral images with different degrees of radiometric inconsistency can be generated, thereby providing controlled samples for subsequent change detection model training.
[0095] It should be noted that the above parameter settings are only examples, and those skilled in the art can adjust the corresponding parameter values according to specific application scenarios.
[0096] Fourthly, the atmospheric radiative transfer simulation model generates the corresponding entrance pupil radiance based on the observed geometric parameters, atmospheric parameters, and ground reflectance information from the selected public dataset, thereby constructing dual-temporal simulated hyperspectral images under different radiative inconsistencies. Preferably, the atmospheric radiative transfer simulation model adopts the MODTRAN model. The simulation sample construction process can be represented as follows: ; in, This represents the simulation process of atmospheric radiative transfer. and This represents different combinations of atmospheric parameters. and This represents a simulated two-phase hyperspectral image exhibiting radiometric inconsistencies. For example... Figure 5 , 6 As shown.
[0097] In this embodiment, the variation labels corresponding to the original data are preferably reused as the supervision labels of the simulation samples, so as to construct training samples under various radiation inconsistency conditions without increasing the cost of manual annotation.
[0098] S2. Using simulated two-phase data as input, perform temporal private observation condition feature extraction and hyperspectral reconstruction.
[0099] To characterize temporal private observation condition information in dual-temporal hyperspectral images and enhance the separability between observation condition information and ground feature attribute information, this example sets up an observation condition decoupling and hyperspectral reconstruction branch, such as... Figure 3 As shown, the specific steps include the following:
[0100] The first step is to decouple the two-phase observation conditions. The first-phase hyperspectral image and the second-phase hyperspectral image obtained from the simulation are input into the corresponding observation condition encoders to extract the features of the two-phase observation conditions. , ,in, This refers to the observation condition encoder. The observation condition encoder is used to extract dual-temporal private observation condition information, thereby providing a basis for separating temporal observation condition features from subsequent ground feature attribute features.
[0101] The second step involves constructing a hyperspectral reconstruction branch. The subsequently radiometrically corrected land cover attribute features, along with their corresponding observation condition features, are input into the hyperspectral reconstruction decoder to reconstruct the simulated dual-temporal hyperspectral image. This reconstruction process ensures that the radiometrically corrected land cover attribute features retain valid information about the land cover content and constrains radiometric inconsistencies caused by differences in observation conditions.
[0102] S3. Perform radiation consistency correction and change detection using simulated two-phase data as input.
[0103] To extract land cover attribute information from dual-temporal hyperspectral images that is related to the intrinsic properties of land cover and robust to radiometric inconsistencies, and to perform change detection based on this information, this embodiment sets up a radiometric consistency correction and change detection branch, such as... Figure 3 As shown, the specific steps include the following:
[0104] The first step is to simulate dual-temporal hyperspectral images. and Inputting a backbone feature extraction network with shared parameters yields dual-temporal, multi-scale basic features. and , and , The backbone feature extraction network is preferably a spectral-spatial joint feature extraction network, used to simultaneously extract the spectral and spatial features of the hyperspectral image.
[0105] The second step involves multi-level feature interaction on the dual-temporal, multi-scale basic features to integrate shallow spatial detail information and deep semantic information, thereby enhancing the discriminative ability and robustness of content features.
[0106] The third step involves progressive upsampling. This progressive upsampling restores the spatial resolution required for change detection, thereby constructing the ground feature characteristics of the hyperspectral image after radiometric consistency correction. and The radiometrically corrected land cover attribute characteristics can be expressed as:
[0107]
[0108] in This indicates a shared backbone feature extraction network. This represents a multi-level feature interaction process. This indicates a progressive upsampling process. This represents the radiometrically corrected land feature characteristics corresponding to the t-th time.
[0109] The fourth step is to input the data into the land cover attribute change detection module. This involves obtaining the land cover attribute features after radiometric consistency correction for both temporal phases. and Afterwards, and The input is a land cover attribute change detection module. This module includes a feature mixing unit (HS-Mixblock), a feature interaction unit (HS-MAB), an upsampling unit, and a classification unit. It enhances the ability to model differences between land cover attribute features after radiometric consistency correction across two temporal phases and outputs the final change detection results. Specifically, the feature mixing unit can jointly introduce original features from both temporal phases, difference features, and product features to enhance the representation of differences and consistency relationships in hyperspectral changes. Simultaneously, the attention interaction unit introduces a spectral-space joint attention mechanism to enhance the network's ability to focus on the importance of hyperspectral dimensions and the spatial distribution of change regions.
[0110] Step 5: Output the change detection results. Let the final change detection result be... The corresponding real change label is .
[0111] S4. Loss Function Design and Joint Training
[0112] To ensure that the network proposed in this invention can simultaneously achieve change detection, separation of radiometric and content information, and hyperspectral reconstruction constraints, this embodiment employs a multi-loss joint training method, specifically including the following steps:
[0113] The first step is to construct the change detection loss. To monitor the change detection results, this embodiment constructs a change detection loss between the change detection results and the actual change labels, using the BCE loss as follows:
[0114]
[0115] The second step is to construct the feature separation loss. To enhance the separation capability between the radiometrically corrected land feature attributes and the observation condition features, this embodiment further introduces a feature separation loss to reduce the correlation between the radiometrically corrected land feature attributes and the observation condition features. Preferably, the feature separation loss can be expressed as:
[0116]
[0117] The third step is to establish the hyperspectral reconstruction loss. Let the reconstruction results be as follows: and The hyperspectral reconstruction loss can be expressed in the following form:
[0118]
[0119] in, and This represents a simulated two-phase hyperspectral image. and This represents the reconstructed hyperspectral image. In other embodiments, spectral angle constraints, band weighting constraints, or spectral correlation constraints may be further introduced to enhance the hyperspectral fidelity of the reconstruction.
[0120] The fourth step is to establish consistency constraints for invariant regions. To suppress spurious differences introduced by radiometric inconsistencies in invariant regions, the land cover attribute characteristics after radiometric consistency correction in both temporal phases are... and Apply an invariant region consistency constraint. Let the invariant region mask be... Then the consistency loss can be expressed as:
[0121]
[0122] Where i represents the spatial location within the invariant region. This indicates the number of pixels in the invariant region. and These represent the location of ground feature attributes after radiometric consistency correction in both temporal phases. By using the feature values at the location, this loss constraint can make the radiometrically consistent ground property characteristics corresponding to the invariant region in the dual-temporal hyperspectral image tend to be consistent, thereby reducing the pseudo-change response caused by radiometric inconsistency in the invariant region.
[0123] Fifth, jointly optimize multiple loss functions. The change detection loss, consistency loss, feature separation loss, and reconstruction loss are jointly optimized to obtain the total loss function for the entire network:
[0124]
[0125] in, Indicates the change detection loss. This represents the consistency loss in invariant regions. Indicates feature separation loss, Indicates the loss in hyperspectral reconstruction. , , , This is the loss weighting coefficient.
[0126] Step six yields the final hyperspectral change detection model. By jointly optimizing the overall loss function, a hyperspectral change detection model with strong robustness under radiometric inconsistency conditions can be obtained. After training, this model is used to detect changes in the dual-temporal hyperspectral images to be detected, outputting the corresponding change map, as shown below. Figure 7 As shown.
[0127] Example 2
[0128] A multi-task learning-based hyperspectral land cover attribute change detection method extracts observation condition features from dual-temporal hyperspectral images and constructs radiometrically consistent (RTC) land cover attribute features through a shared-parameter backbone feature extraction network, multi-level feature interaction, and progressive upsampling. Furthermore, with land cover attribute change detection as the primary task and RTC as an auxiliary task, the entire network is collaboratively optimized through multi-task optimization, incorporating invariant region consistency constraints, hyperspectral reconstruction constraints, and feature separation constraints. This achieves joint modeling of observation condition information and intrinsic land cover attribute information, improving the accuracy, stability, and robustness of dual-temporal hyperspectral land cover attribute change detection. The method includes the following steps:
[0129] S1. Acquire the first and second hyperspectral images and corresponding change labels of the same area collected at different time phases, and preprocess the first and second hyperspectral images. The preprocessing includes at least registration, invalid band removal, and normalization to obtain preprocessed dual-temporal hyperspectral image data.
[0130] S2. Set the parameter combination used to characterize the radiation inconsistency conditions, including aerosol type, visibility, water vapor content, solar zenith angle, observation zenith angle, and observation azimuth angle.
[0131] S3. Based on the preprocessed dual-temporal hyperspectral image data and the parameter combination, the atmospheric radiative transfer simulation model is used to simulate the radiative transfer of the dual-temporal hyperspectral images, generating simulated dual-temporal hyperspectral sample pairs with different radiative levels, and reusing the original change labels as corresponding supervision information.
[0132] The atmospheric radiative transfer simulation model is preferably the MODTRAN model.
[0133] S4. Input the simulated dual-temporal hyperspectral sample pairs into the corresponding observation condition encoders to extract the observation condition features corresponding to each time, which are used to characterize the radiometric inconsistency information caused by the difference in observation conditions between the dual-temporal hyperspectral images.
[0134] S5. Input the simulated dual-temporal hyperspectral sample pairs into the backbone feature extraction network with shared parameters to extract the multi-scale basic features of the dual-temporal hyperspectral images.
[0135] The backbone feature extraction network with shared parameters is a spectral-spatial joint feature extraction network, which is used to simultaneously extract the spectral and spatial features of hyperspectral images.
[0136] S6. Perform multi-level feature interaction on the multi-scale basic features to fuse spectral and spatial information at different levels, and obtain the land cover attribute features after radiometric consistency correction in the two-temporal phases through progressive upsampling. The land cover attribute features after radiometric consistency correction are used to characterize the land cover attribute information in the two-temporal hyperspectral images that is related to the intrinsic attributes of land cover and is robust to radiometric inconsistency.
[0137] The multi-level feature interaction includes the fusion, enhancement and information transmission between features at different levels, and the progressive upsampling is used to restore the spatial resolution required for change detection, so as to improve the ability to locate the boundary of the change area.
[0138] S7. Based on the ground property characteristics after radiometric consistency correction of the two-temporal images, an invariant region consistency constraint is constructed to make the ground property characteristics corresponding to the invariant region of the two-temporal hyperspectral images tend to be consistent, so as to suppress the pseudo-change response caused by radiometric inconsistency.
[0139] The invariant region consistency constraint applies to the invariant region of the dual-temporal hyperspectral image to reduce the feature shift caused by radiation inconsistency in the invariant region.
[0140] S8. Input the radiometrically corrected ground feature attribute features and the corresponding observation condition features of the dual-temporal model into the hyperspectral reconstruction decoder to reconstruct the simulated dual-temporal hyperspectral sample pair, so as to constrain the radiometrically corrected ground feature attribute features to retain the effective content information of the ground features, and constrain the observation condition features to carry radiometric inconsistency information.
[0141] The hyperspectral reconstruction decoder recovers the corresponding temporal hyperspectral image based on the radiometrically corrected land cover attribute features and observation condition features, thereby enhancing the separability and interpretability between the radiometrically corrected land cover attribute features and observation condition features.
[0142] S9. Input the radiometrically corrected land feature attributes of the two time phases into the land feature attribute change detection module. The land feature attribute change detection module performs feature mixing, attention interaction and classification prediction on the radiometrically corrected land feature attributes of the two time phases, and outputs the corresponding change detection results.
[0143] The land feature attribute change detection module includes one or more of the following: feature mixing unit, feature interaction unit, upsampling unit, and classification unit, used to model the differences between land feature attribute features after radiometric consistency correction in two time phases and to identify change areas.
[0144] S10. Based on the change detection results, invariant region consistency constraints, feature separation constraints, and hyperspectral reconstruction constraints, the entire network is jointly trained to obtain a hyperspectral change detection model under radiometric inconsistency conditions; and the trained model is used to detect changes in the dual-temporal hyperspectral image to be detected.
[0145] Example 3
[0146] A method for detecting changes in hyperspectral ground cover attributes based on multi-task learning includes the following steps:
[0147] S1. Acquire dual-temporal hyperspectral images of the same region collected at different times and their corresponding change labels, and perform preprocessing.
[0148] Preprocessing includes one or more of the following: registration, invalid band removal, normalization, and clipping.
[0149] S2. Input the first hyperspectral image and the second hyperspectral image into the corresponding observation condition encoder respectively, and extract the corresponding observation condition features at each time to characterize the radiometric inconsistency information caused by the difference in observation conditions between the two-time-phase hyperspectral images.
[0150] The observation condition encoder is used to extract temporal private observation condition features caused by differences in atmospheric parameters, solar zenith angle, observation zenith angle, or observation azimuth angle.
[0151] S3. Input the dual-temporal hyperspectral image into the shared parameter backbone feature extraction network to extract the multi-scale basic features of the dual-temporal hyperspectral image.
[0152] The shared parameter backbone feature extraction network is a spectral-spatial joint feature extraction network, used to simultaneously extract the spectral and spatial features of hyperspectral images.
[0153] The shared parameter backbone feature extraction network can be any one of two-dimensional convolutional networks, three-dimensional convolutional networks, two-dimensional and three-dimensional hybrid convolutional networks, or spectral spatial feature extraction networks based on attention mechanisms.
[0154] S4. Perform multi-level feature interaction on the multi-scale basic features, and obtain the ground object attribute features after radiometric consistency correction in the two-temporal phases through progressive upsampling. The ground object attribute features are used to represent stable attribute information in the two-temporal hyperspectral images that are related to the intrinsic attributes of ground objects and are robust to changes in observation conditions.
[0155] Among them, multi-level feature interaction is used to fuse spectral and spatial information at different levels, and progressive upsampling is used to restore the spatial resolution required for change detection and maintain the spectral consistency of land cover attribute features after radiometric consistency correction.
[0156] S5. Based on the ground property characteristics after radiometric consistency correction in the two time phases, an invariant regional consistency constraint is constructed to suppress the pseudo-change response caused by radiometric inconsistency.
[0157] S6. Input the ground feature attribute features after radiometric consistency correction of the two temporal phases and the corresponding observation condition features into the hyperspectral reconstruction decoder to reconstruct the two temporal hyperspectral images, so as to constrain the ground feature attribute features to retain the effective attribute information of the ground features and constrain the observation condition features to carry radiometric inconsistency information.
[0158] The hyperspectral reconstruction decoder is used to recover dual-temporal hyperspectral images based on the radiometrically consistent ground property features and the corresponding observation condition features, so as to enhance the separability and interpretability between ground property features and observation condition features.
[0159] S7. Input the ground feature attribute features after radiometric consistency correction of the two time phases into the ground feature attribute change detection module, and output the corresponding change detection results.
[0160] The land cover attribute change detection module includes one or more of the following: dual-temporal feature mixing unit (HS-Mixblock), feature interaction unit (HS-MAB), upsampling unit (Upsample), and classification unit (Classifier), which are used to model the differences between land cover attribute features after radiometric consistency correction in the two temporal phases and to identify change areas.
[0161] S8. Based on the change detection loss, the invariant region consistency constraint loss, the feature separation loss, and the hyperspectral reconstruction loss, the entire network is jointly trained to obtain a hyperspectral land cover attribute change detection model.
Claims
1. A method for detecting changes in hyperspectral ground cover attributes based on multi-task learning, characterized in that, Includes the following steps: Step S1, Acquisition of dual-temporal hyperspectral data and construction of dual-temporal simulated hyperspectral images: Acquire labeled dual-temporal hyperspectral images, preprocess the labeled dual-temporal hyperspectral images, set the atmospheric parameters and observation geometric parameters corresponding to the first and second temporal phases, and construct dual-temporal simulated hyperspectral images; Step S2, Extraction of dual-temporal observation condition features: Based on the dual-temporal simulated hyperspectral images, the dual-temporal observation condition features are extracted using an observation condition encoder; Step S3, Radiation Consistency Correction and Change Detection: Using dual-temporal simulated hyperspectral images, we extracted dual-temporal multi-scale basic features; Feature interaction is performed on the basic features of dual temporal and multi-scale data to construct the land cover attribute features after radiometric consistency correction of hyperspectral images, and output the detection results of hyperspectral land cover attribute changes.
2. The method for detecting changes in hyperspectral ground cover properties according to claim 1, characterized in that, Step S1 further includes: Step 1: Acquisition of labeled dual-temporal hyperspectral images: Publicly available hyperspectral ground property change detection data were used, with the bi-temporal images and their change labels serving as labeled bi-temporal hyperspectral images; The second step is preprocessing of the dual-temporal hyperspectral images: The labeled dual-temporal hyperspectral images are preprocessed to obtain the basic dual-temporal hyperspectral samples for input to the network; The preprocessing includes image registration, invalid band removal, normalization, and block cropping. Step 3, parameter settings: Set the atmospheric parameters and observation geometric parameters corresponding to the first and second time phases; The atmospheric parameters include aerosol type, visibility, and water vapor content, and the observation geometric parameters include solar zenith angle, observation zenith angle, and observation azimuth angle. Step 4: Construction of dual-temporal simulated hyperspectral images under different radiation inconsistencies: Based on the basic dual-temporal hyperspectral samples and set parameters, the entrance pupil radiance corresponding to the ground reflectance information of the atmospheric radiative transfer simulation model is generated, and dual-temporal simulated hyperspectral images under different radiative inconsistencies are constructed. Among them, the dual-temporal simulated hyperspectral images under different radiation inconsistency conditions include a first-temporal hyperspectral image and a second-temporal hyperspectral image.
3. The method for detecting changes in hyperspectral ground cover attributes according to claim 1, characterized in that, Step S2 further includes: The first and second temporal hyperspectral images obtained from the simulation are input into the corresponding observation condition encoders to extract the dual-temporal observation condition features. Among them, the dual-temporal observation condition characteristics include the first-temporal observation condition characteristics. Characteristics of the second phase observation conditions The calculation formula is as follows: ； In the above formula, Indicates the observation condition encoder. This represents the simulated hyperspectral image of the first time phase. This represents the simulated hyperspectral image of the second time phase.
4. The method for detecting changes in hyperspectral ground cover properties according to claim 1, characterized in that, Step S3 further includes: Step 1: Extraction of basic features across two temporal phases and multiple scales: Dual-temporal simulated hyperspectral images and Inputting a backbone feature extraction network with shared parameters yields dual-temporal, multi-scale basic features. and ,in, and , ; The backbone feature extraction network is a spectral-spatial joint feature extraction network, which is used to simultaneously extract the spectral and spatial features of hyperspectral images. The second step, feature interaction: By performing multi-level feature interaction on dual-temporal and multi-scale basic features, we obtain fused shallow spatial detail features and deep semantic features. The third step is progressive upsampling: Progressive upsampling is performed on the fused shallow spatial detail features and deep semantic features to construct the land cover attribute features of the hyperspectral image after radiometric consistency correction. and ; The calculation formula for the radiometrically corrected land feature attributes is as follows: ； In the above formula, This indicates a shared backbone feature extraction network. This represents a multi-level feature interaction process. This indicates a progressive upsampling process. This represents the radiometrically corrected land cover attribute characteristics corresponding to the t-th time. Step 4: Output of hyperspectral ground property change detection results: The hyperspectral image, after radiometric consistency correction, is input into the land cover attribute change detection module; the land cover attribute change detection module outputs the hyperspectral land cover attribute change detection results. The feature attribute change detection module includes a feature mixing unit HS-Mixblock, a feature interaction unit HS-MAB, an upsampling unit Upsample, and a classification unit Classifier. The feature mixing unit introduces dual-temporal original features, difference features, and product features; the attention interaction unit introduces a spectral-space joint attention mechanism.
5. The method for detecting changes in hyperspectral ground cover properties according to claim 1, characterized in that, It also includes step S4, model constraints and iteration: A loss function is constructed, and the loss functions are jointly optimized to obtain the total loss function. The total loss function is used to constrain the model, and the model is iterated to obtain the hyperspectral ground property change detection model. The loss function includes change detection loss, feature separation loss, hyperspectral reconstruction loss, and invariant region consistency loss.
6. The method for detecting changes in hyperspectral ground cover properties according to claim 5, characterized in that, Step S4 further includes: Step 1: Constructing the change detection loss: Based on the hyperspectral changes in ground cover attributes and the actual change labels, a change detection loss is constructed. The calculation formula is as follows: ； In the formula, For the change detection results, To correspond to the actual changes in the labels, For Binary Cross Entropy Loss, specifically, Where N is the number of pixels participating in the training. This represents the actual change label of the i-th pixel. This represents the prediction result for the i-th pixel; The second step is to construct the feature separation loss: Based on the ground feature attributes and observation conditions after radiometric consistency correction, a feature separation loss is constructed. The calculation formula is as follows: ； In the above formula, and These represent the land cover attributes after radiometric consistency correction in both temporal phases. These represent the observation conditions characteristics of the first and second time phases, respectively. Step 3: Construction of hyperspectral reconstruction loss: First, a hyperspectral reconstruction branch is constructed. The dual-temporal observation condition characteristics and the ground property characteristics after radiometric consistency correction are input into the hyperspectral reconstruction decoder to reconstruct the dual-temporal simulated hyperspectral image. The hyperspectral reconstruction branch is constructed to obtain the reconstructed dual-temporal simulated hyperspectral image. Secondly, based on the reconstructed dual-temporal simulated hyperspectral images, a hyperspectral reconstruction loss is constructed. The calculation formula is as follows: ； In the above formula, and This represents a dual-temporal simulated hyperspectral image. and This represents the reconstruction of a dual-temporal simulated hyperspectral image; Step 4: Constructing consistency constraints for invariant regions: Land feature attributes after radiometric consistency correction for two temporal phases and Apply invariant region consistency constraints and construct invariant region consistency loss. The calculation formula is as follows: ； In the above formula, i represents the spatial position within the invariant region. Indicates the number of pixels in the invariant region; and These represent the feature values of the land cover attributes at location i after radiometric consistency correction in both temporal phases; Step 5, Joint optimization of the loss function: The change detection loss, feature separation loss, hyperspectral reconstruction loss, and invariant region consistency loss are jointly optimized to obtain the total loss function. The calculation formula is as follows: ； In the above formula, Indicates the change detection loss. This represents the consistency loss in invariant regions. Indicates feature separation loss, Indicates the loss in hyperspectral reconstruction. , , , These are the loss weight coefficients for change detection loss, invariant region consistency loss, feature separation loss, and hyperspectral reconstruction loss, respectively. Step 6, Constraints and Training Iterations: By constraining the model with the total loss function and training iteratively, a hyperspectral ground property change detection model is obtained. The hyperspectral land cover attribute change detection model is used to detect the changes in the dual-temporal hyperspectral land cover attribute images and output the corresponding change map.